PS1/2 Emulator WiiStation - Another Fork of WiiSXRX released

SaulFabre

I like Yoshis and the Wii/Wii U scene.
Member
Joined
Feb 6, 2019
Messages
3,135
Trophies
1
Age
25
Location
Ecuador
Website
saulfabreg-wiivc.blogspot.com
XP
7,552
Country
Ecuador
I tested the Zstandard compression using

chdman createcd -i input.cue -o output.chd -c zstd,cdzs
I think zstd and cdzs are exactly the same compression type, lol.
So it should be
chdman createcd -i input.cue -o output.chd -c cdzs

Have you tried also this for achieve a little more compression? (ZStandard+FLAC)
chdman createcd -i input.cue -o output.chd -c cdzs,cdfl
 
  • Like
Reactions: Hyllian

_47iscool

Noticer
Member
Joined
Nov 18, 2013
Messages
652
Trophies
1
XP
1,086
Country
United States
I think zstd and cdzs are exactly the same compression type, lol.
So it should be
chdman createcd -i input.cue -o output.chd -c cdzs

Have you tried also this for achieve a little more compression? (ZStandard+FLAC)
chdman createcd -i input.cue -o output.chd -c cdzs,cdfl

I would prefer using the "cdzs" cmd argument instead of the other, due to the last three letters.
 
Last edited by _47iscool,
  • Like
Reactions: SaulFabre

Ouaz

Well-Known Member
Member
Joined
Jan 5, 2023
Messages
124
Trophies
0
Age
42
XP
243
Country
France
I think zstd and cdzs are exactly the same compression type, lol.
Oh?

I followed chdman documentation, as i saw "data" for one and "audio data" for the other, and thinking that "zstd" has to be precised (as it is not default).

"
zstd – Zstandard
Compresses data using the Zstandard algorithm. This gives very good compression and decompression performance with better compression ratios than zlib deflate

cdzs – Zstandard for CD-ROM data
Compresses audio data and subchannel data from CD-ROM sectors separately using the Zstandard algorithm. This gives very good compression and decompression performance with better compression ratios than zlib deflate

"

Ah, yes, cdzs can be used alone:
"Zstandard is not enabled by default so new CHDs will not be incompatible by default.
Use zstd for regular Zstandard codec, and cdzs for Zstandard CD codec.
"

Have you tried also this for achieve a little more compression? (ZStandard+FLAC)
chdman createcd -i input.cue -o output.chd -c cdzs,cdfl

Won't cdfl impact performance (and possibly induce crash on Wiistation) as a part of it use zlib algorithm ?

"
cdfl – Free Lossless Audio Codec/zlib deflate for CD-ROM data
Compresses audio data and subchannel data from CD-ROM sectors separately using the Free Lossless Audio Codec (FLAC) for audio data and the zlib deflate algorithm for subchannel data.

"
 
  • Like
Reactions: SaulFabre

SaulFabre

I like Yoshis and the Wii/Wii U scene.
Member
Joined
Feb 6, 2019
Messages
3,135
Trophies
1
Age
25
Location
Ecuador
Website
saulfabreg-wiivc.blogspot.com
XP
7,552
Country
Ecuador
Oh?

I followed chdman documentation, as i saw "data" for one and "audio data" for the other, and thinking that "zstd" has to be precised (as it is not default).

"
zstd – Zstandard
Compresses data using the Zstandard algorithm. This gives very good compression and decompression performance with better compression ratios than zlib deflate

cdzs – Zstandard for CD-ROM data
Compresses audio data and subchannel data from CD-ROM sectors separately using the Zstandard algorithm. This gives very good compression and decompression performance with better compression ratios than zlib deflate

"

Ah, yes, cdzs can be used alone:
"Zstandard is not enabled by default so new CHDs will not be incompatible by default.
Use zstd for regular Zstandard codec, and cdzs for Zstandard CD codec.
"



Won't cdfl impact performance (and possibly induce crash on Wiistation) as a part of it use zlib algorithm ?

"
cdfl – Free Lossless Audio Codec/zlib deflate for CD-ROM data
Compresses audio data and subchannel data from CD-ROM sectors separately using the Free Lossless Audio Codec (FLAC) for audio data and the zlib deflate algorithm for subchannel data.

"
i don't see any performance impact when using cdzs (ZStandard) and cdfl (FLAC) at the same time.
 
  • Like
Reactions: _47iscool

ssjkakaroto

Well-Known Member
Member
Joined
Dec 14, 2005
Messages
140
Trophies
1
XP
724
Country
Oh?

I followed chdman documentation, as i saw "data" for one and "audio data" for the other, and thinking that "zstd" has to be precised (as it is not default).

"
zstd – Zstandard
Compresses data using the Zstandard algorithm. This gives very good compression and decompression performance with better compression ratios than zlib deflate

cdzs – Zstandard for CD-ROM data
Compresses audio data and subchannel data from CD-ROM sectors separately using the Zstandard algorithm. This gives very good compression and decompression performance with better compression ratios than zlib deflate

"

Ah, yes, cdzs can be used alone:
"Zstandard is not enabled by default so new CHDs will not be incompatible by default.
Use zstd for regular Zstandard codec, and cdzs for Zstandard CD codec.
"



Won't cdfl impact performance (and possibly induce crash on Wiistation) as a part of it use zlib algorithm ?

"
cdfl – Free Lossless Audio Codec/zlib deflate for CD-ROM data
Compresses audio data and subchannel data from CD-ROM sectors separately using the Free Lossless Audio Codec (FLAC) for audio data and the zlib deflate algorithm for subchannel data.

"
CHDs using only zlib and FLAC were already faster than the ones using LZMA. The real issue was and is LZMA compression. zlib is fine.
Here's an example:

Code:
chdman - MAME Compressed Hunks of Data (CHD) manager 0.262 (mame0262)
Input file:   Akumajou Dracula X - Gekka no Yasoukyoku (Japan) (Rev 2) (En).chd
File Version: 5
Logical size: 613,204,416 bytes
Hunk Size:    19,584 bytes
Total Hunks:  31,312
Unit Size:    2,448 bytes
Total Units:  250,492
Compression:  cdzs (CD Zstandard), cdfl (CD FLAC)
CHD size:     371,396,948 bytes
Ratio:        60.6%
SHA1:         2d50511893883c3e07daca4ba7cb47ebc0b65c2b
Data SHA1:    d4937772d34393553da6d12728b87b5c0e60e368
Metadata:     Tag='CHT2'  Index=0  Length=93 bytes
              TRACK:1 TYPE:MODE2_RAW SUBTYPE:NONE FRAMES:231905 PREGAP:0 PGTYPE:MODE1 PGSUB:NONE POSTGAP:0.
Metadata:     Tag='CHT2'  Index=1  Length=91 bytes
              TRACK:2 TYPE:AUDIO SUBTYPE:NONE FRAMES:18582 PREGAP:150 PGTYPE:VAUDIO PGSUB:NONE POSTGAP:0.

     Hunks  Percent  Name
----------  -------  ------------------------------------
        17     0.1%  Copy from self
    28,990    92.6%  CD Zstandard
     2,305     7.4%  CD FLAC

We need to test with a disc with actual subchannel data to see which algorithm will take precedence to compress it.
 
  • Like
Reactions: SaulFabre

GustavoReis

GameCube and wii lover
Member
Joined
Dec 19, 2016
Messages
261
Trophies
0
Age
34
Location
Brazil
Website
mundowiihack.wordpress.com
XP
1,699
Country
Brazil
:yayps3:


1000021918.jpg
 

Ouaz

Well-Known Member
Member
Joined
Jan 5, 2023
Messages
124
Trophies
0
Age
42
XP
243
Country
France
CHDs using only zlib and FLAC were already faster than the ones using LZMA. The real issue was and is LZMA compression. zlib is fine.
I asked because I had crashes with chd compressed with cdzl + cdfl (still Colin Mc Rae PAL). Because of cdzl then, not cdfl ?

Have you tried also this for achieve a little more compression? (ZStandard+FLAC)
chdman createcd -i input.cue -o output.chd -c cdzs,cdfl
I tried.
And the size reduction is noticeable:

Colin Mc Rae PAL:

313Mo (cdzs only) -> 268Mo (cdzs + cdfl)

Code:
chdman - MAME Compressed Hunks of Data (CHD) manager 0.263 (mame0263)
Input file:   Colin McRae Rally (Europe) (En,Fr,De,Es) (Rev 1).chd
File Version: 5
Logical size: 511,024,896 bytes
Hunk Size:    19,584 bytes
Total Hunks:  26,094
Unit Size:    2,448 bytes
Total Units:  208,752
Compression:  cdzs (CD Zstandard), cdfl (CD FLAC)
CHD size:     275,336,932 bytes
Ratio:        53.9%
SHA1:         741067e01788665ff90b9bd10fa23fae8f7948de
Data SHA1:    cc1debc7f605e5bd864ad3350d56867e91e204a4
Metadata:     Tag='CHT2'  Index=0  Length=93 bytes
              TRACK:1 TYPE:MODE2_RAW SUBTYPE:NONE FRAMES:147897 PREGAP:0 PGTYPE:MODE1 PGSUB:NONE POSTGAP:0.
Metadata:     Tag='CHT2'  Index=1  Length=91 bytes
              TRACK:2 TYPE:AUDIO SUBTYPE:NONE FRAMES:23700 PREGAP:150 PGTYPE:VAUDIO PGSUB:NONE POSTGAP:0.
Metadata:     Tag='CHT2'  Index=2  Length=91 bytes
              TRACK:3 TYPE:AUDIO SUBTYPE:NONE FRAMES:13450 PREGAP:150 PGTYPE:VAUDIO PGSUB:NONE POSTGAP:0.
Metadata:     Tag='CHT2'  Index=3  Length=91 bytes
              TRACK:4 TYPE:AUDIO SUBTYPE:NONE FRAMES:23698 PREGAP:150 PGTYPE:VAUDIO PGSUB:NONE POSTGAP:0.

Hunks       Percent  Name
----------  -------  ------------------------------------
        53     0.2%  Copy from self
    18,489    70.9%  CD Zstandard
     7,552    28.9%  CD FLAC


I didn't test it yet, to see if crashes happen or not. (It shouldn't ? as there's no subchannel data, so no zlib anywhere ?)
 
Last edited by Ouaz,
  • Like
Reactions: SaulFabre

ssjkakaroto

Well-Known Member
Member
Joined
Dec 14, 2005
Messages
140
Trophies
1
XP
724
Country
I asked because I had crashes with chd compressed with cdzl + cdfl (still Colin Mc Rae PAL). Because of cdzl then, not cdfl ?


I tried.
And the size reduction is noticeable:

Colin Mc Rae PAL:

313Mo (cdzs only) -> 268Mo (cdzs + cdfl)

Code:
chdman - MAME Compressed Hunks of Data (CHD) manager 0.263 (mame0263)
Input file:   Colin McRae Rally (Europe) (En,Fr,De,Es) (Rev 1).chd
File Version: 5
Logical size: 511,024,896 bytes
Hunk Size:    19,584 bytes
Total Hunks:  26,094
Unit Size:    2,448 bytes
Total Units:  208,752
Compression:  cdzs (CD Zstandard), cdfl (CD FLAC)
CHD size:     275,336,932 bytes
Ratio:        53.9%
SHA1:         741067e01788665ff90b9bd10fa23fae8f7948de
Data SHA1:    cc1debc7f605e5bd864ad3350d56867e91e204a4
Metadata:     Tag='CHT2'  Index=0  Length=93 bytes
              TRACK:1 TYPE:MODE2_RAW SUBTYPE:NONE FRAMES:147897 PREGAP:0 PGTYPE:MODE1 PGSUB:NONE POSTGAP:0.
Metadata:     Tag='CHT2'  Index=1  Length=91 bytes
              TRACK:2 TYPE:AUDIO SUBTYPE:NONE FRAMES:23700 PREGAP:150 PGTYPE:VAUDIO PGSUB:NONE POSTGAP:0.
Metadata:     Tag='CHT2'  Index=2  Length=91 bytes
              TRACK:3 TYPE:AUDIO SUBTYPE:NONE FRAMES:13450 PREGAP:150 PGTYPE:VAUDIO PGSUB:NONE POSTGAP:0.
Metadata:     Tag='CHT2'  Index=3  Length=91 bytes
              TRACK:4 TYPE:AUDIO SUBTYPE:NONE FRAMES:23698 PREGAP:150 PGTYPE:VAUDIO PGSUB:NONE POSTGAP:0.

Hunks       Percent  Name
----------  -------  ------------------------------------
        53     0.2%  Copy from self
    18,489    70.9%  CD Zstandard
     7,552    28.9%  CD FLAC


I didn't test it yet, to see if crashes happen or not. (It shouldn't ? as there's no subchannel data, so no zlib anywhere ?)
If I'm not mistaken, FLAC is even lighter than Zstandard and deflate (zlib) as it can be decoded on many DAP's with hardware even more restricted than the Wii.

Do the crashes also happen with uncompressed bin/cue?
 

Ouaz

Well-Known Member
Member
Joined
Jan 5, 2023
Messages
124
Trophies
0
Age
42
XP
243
Country
France
Do the crashes also happen with uncompressed bin/cue?
Nope, all's fine with cue+bin.
(I use Cue+Bin for all games -including Colin McRae-, but I also use this game for CHD tests).

So far:

- CHD Zlib (cdzl) + LZMA (cdlz) + FLAC (cdfl): crash (chdman default compression)
- CHD Zlib (cdzl) + FLAC (cdfl) : crash
- CHD Zstandard (cdzs) : no crash \o/
- CHD Zstandard (cdzs) + FLAC (cdfl) : not fully tested yet (not played enough).

From what I understand, flac use zlib algorithm only for subchannel data. I wonder if this could induce crash.
As you said, it has to be tested with a game having subchannel data, to be 100% sure it's safe.
 
Last edited by Ouaz,

ssjkakaroto

Well-Known Member
Member
Joined
Dec 14, 2005
Messages
140
Trophies
1
XP
724
Country
Nope, all's fine with cue+bin.
(I use Cue+Bin for all games -including Colin McRae-, but I also use this game for CHD tests).

So far:

- CHD Zlib (cdzl) + LZMA (cdlz) + FLAC (cdfl): crash (chdman default compression)
- CHD Zlib (cdzl) + FLAC (cdfl) : crash
- CHD Zstandard (cdzs) : no crash \o/
- CHD Zstandard (cdzs) + FLAC (cdfl) : not fully tested yet (not played enough).

From what I understand, flac use zlib algorithm only for subchannel data. I wonder if this could induce crash.
As you said, it has to be tested with a game having subchannel data, to be 100% sure it's safe.
I don't think we should worry about subchannel data in CHD's, because chdman does not support MSD/MDF or CCD/IMG/SUB or SBI files. These are the main formats that have subchannel data. Even if you try to convert a game with BIN/CUE/SBI, the SBI file will get ignored by chdman.

Having said that, there's still some performance issue with Wiistation and CCD/IMG/SUB format which I reported a while ago. For people that are playing European games with libcrypt protection, that might be relevant.
 
  • Like
Reactions: Ouaz

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
  • No one is chatting at the moment.
    K3Nv2 @ K3Nv2: Sorry for accidentally bending over