Any thoughts on adding ZSO (LZ4) support to Nintendont?

Lv44ES_Burner

Well-Known Member
Member
Joined
Dec 11, 2020
Messages
129
Trophies
0
Age
33
Location
Perdition's flames
XP
499
Country
United States
I've been thinking it would be far more useful and cross-platform friendly for OPL (and, to be honest, any other loader on competing hardware) to better support the MAME .CHD format that's been gaining steady traction with emulators as of late. While I've read Acid_Snake's writeup there in that thread about the .ZSO format, I believe .CHD would make more sense to adapt, as that would mean users need only one storage format for their files, whether it be for PCSX2's support of CHDs or for OPL's, and the same benefits can be had of .CHD that are now had with the research done in .ZSO and using the LZ4 compression mechanism.

But I'm just a blob, what do I know?
 

_47iscool

Anti-communist
OP
Member
Joined
Nov 18, 2013
Messages
221
Trophies
0
XP
366
Country
United States
I've been thinking it would be far more useful and cross-platform friendly for OPL (and, to be honest, any other loader on competing hardware) to better support the MAME .CHD format that's been gaining steady traction with emulators as of late. While I've read Acid_Snake's writeup there in that thread about the .ZSO format, I believe .CHD would make more sense to adapt, as that would mean users need only one storage format for their files, whether it be for PCSX2's support of CHDs or for OPL's, and the same benefits can be had of .CHD that are now had with the research done in .ZSO and using the LZ4 compression mechanism.

But I'm just a blob, what do I know?
Check this comment:
https://github.com/ps2homebrew/Open-PS2-Loader/issues/225#issuecomment-1125244689

Using chd on the Wii may cause stuttering issues, which is why I said ZSO.
I know the Wii is a good bit faster than the PS2, but still.

I always thought chd was a MAME-exclusive format.
 
  • Like
Reactions: Acid_Snake

Lv44ES_Burner

Well-Known Member
Member
Joined
Dec 11, 2020
Messages
129
Trophies
0
Age
33
Location
Perdition's flames
XP
499
Country
United States
No, recently CHD was adopted as a viable format for the PCSX2 emulator (as well as Duckstation and PCSX-ReARMed) for playing and storing backups of PS2 (as well as PSOne) games by way of its own variation of lossless compression and decompression of files in archive-quality, 1:1 conversions. Best audio quality, best data throughput from what has been reported by users of the PCSX2, Duckstation and PCSX-ReARMed cores for LibRetro/Retroarch. That's why I say that. CHDv5 uses LZMA compression from 7zip for the data streams and lossless FLAC compression for the audio streams for the highest optimizations possible over using BIN+CUE+MP3/BIN+CUE+WAV alone.
 

_47iscool

Anti-communist
OP
Member
Joined
Nov 18, 2013
Messages
221
Trophies
0
XP
366
Country
United States
No, recently CHD was adopted as a viable format for the PCSX2 emulator (as well as Duckstation and PCSX-ReARMed) for playing and storing backups of PS2 (as well as PSOne) games by way of its own variation of lossless compression and decompression of files in archive-quality, 1:1 conversions. Best audio quality, best data throughput from what has been reported by users of the PCSX2, Duckstation and PCSX-ReARMed cores for LibRetro/Retroarch. That's why I say that. CHDv5 uses LZMA compression from 7zip for the data streams and lossless FLAC compression for the audio streams for the highest optimizations possible over using BIN+CUE+MP3/BIN+CUE+WAV alone.

Maybe. But PC's are much stronger than the Wii.

But it seems that chd has become more flexible now, so who knows?
 

Acid_Snake

Well-Known Member
Member
Joined
Aug 20, 2019
Messages
278
Trophies
0
Age
28
XP
851
Country
Spain
No, recently CHD was adopted as a viable format for the PCSX2 emulator (as well as Duckstation and PCSX-ReARMed) for playing and storing backups of PS2 (as well as PSOne) games by way of its own variation of lossless compression and decompression of files in archive-quality, 1:1 conversions. Best audio quality, best data throughput from what has been reported by users of the PCSX2, Duckstation and PCSX-ReARMed cores for LibRetro/Retroarch. That's why I say that. CHDv5 uses LZMA compression from 7zip for the data streams and lossless FLAC compression for the audio streams for the highest optimizations possible over using BIN+CUE+MP3/BIN+CUE+WAV alone.
The key word here is "emulator" and "PC", CHD is a very complex format with a very complex library to drive it.
The PS2 would have never been able to successfully support it. ZSO was made with lightweigth and speed in mind, it was made for the PSP, so it has the potential to work on pretty much every old DVD-based console. The LZ4 algorithm is fast enough and the CSO/ZSO readers are so well optimized today that even the poor PS1 CPU on the PS2 can handle it.

I did check Nintendont's source code and it seems both viable and simple to implement.
Nintendont already supports a custom CSO format (it doesn't use compression though but rather trims empty blocks), so adding ZSO to it should be trivial.
 
  • Like
Reactions: _47iscool and sith

Lv44ES_Burner

Well-Known Member
Member
Joined
Dec 11, 2020
Messages
129
Trophies
0
Age
33
Location
Perdition's flames
XP
499
Country
United States
That's fair enough, and I had actually forgotten you had a user account here. I thank you for the clarification, even if it means I'm a bit crestfallen in regards for using CHDs. But I am very satisfied nonetheless with the advancements made in supporting ZSOs and look forward to the future of OPL using them in tandem with the MX4SIO (hopefully DECKARD-based systems like my own SCPH-90001 will benefit in the longer run), and no doubt it can be very beneficial to Nintendont and other USB loaders on the Wii as well. Whatever other optimizations can be done in tandem with the format can only benefit less powerful machines, right?
 

niuus

Well-Known Member
Member
Joined
Mar 4, 2016
Messages
1,637
Trophies
1
XP
2,983
Country
Venezuela
I've been thinking it would be far more useful and cross-platform friendly for OPL (and, to be honest, any other loader on competing hardware) to better support the MAME .CHD format that's been gaining steady traction with emulators as of late.

Using chd on the Wii may cause stuttering issues, which is why I said ZSO.
I know the Wii is a good bit faster than the PS2, but still.

I always thought chd was a MAME-exclusive format.

Maybe. But PC's are much stronger than the Wii.

CHD is already supported in GenesisPlus GX since a couple of years. More recently, it was also added for MD+ games. It's doable.
 
  • Like
Reactions: _47iscool

Acid_Snake

Well-Known Member
Member
Joined
Aug 20, 2019
Messages
278
Trophies
0
Age
28
XP
851
Country
Spain
CHD is already supported in GenesisPlus GX since a couple of years. More recently, it was also added for MD+ games. It's doable.
I don't know if you have noticed, but there's a big difference between a genesis game on an emulator and a ps2 game on retail console.
 
  • Like
Reactions: _47iscool

niuus

Well-Known Member
Member
Joined
Mar 4, 2016
Messages
1,637
Trophies
1
XP
2,983
Country
Venezuela
I don't know if you have noticed, but there's a big difference between a genesis game on an emulator and a ps2 game on retail console.
I don't know if you have noticed, but I just said it's doable, on the Wii, which is why I specifically mention ekeeke's implementation on GenesisPlus GX, which BTW isn't for "a genesis game", it was implemented for Sega CD, which means Data+CD audio streaming.

Didn't mention PS2 examples, nor I care about PS2.

Using chd on the Wii may cause stuttering issues, which is why I said ZSO.
I know the Wii is a good bit faster than the PS2, but still.

I always thought chd was a MAME-exclusive format.
I also specifically quoted this as a way of letting the guy know CHD is not MAME exclusive. The same way CSO or Nkit is not "Nintendont-exclusive".
 

Acid_Snake

Well-Known Member
Member
Joined
Aug 20, 2019
Messages
278
Trophies
0
Age
28
XP
851
Country
Spain
I don't know if you have noticed, but I just said it's doable, on the Wii, which is why I specifically mention ekeeke's implementation on GenesisPlus GX, which BTW isn't for "a genesis game", it was implemented for Sega CD, which means Data+CD audio streaming.

Didn't mention PS2 examples, nor I care about PS2.


I also specifically quoted this as a way of letting the guy know CHD is not MAME exclusive. The same way CSO or Nkit is not "Nintendont-exclusive".
Last I checked Nintendont wasn't really swimming in loads of free RAM, so I very much doubt you can fit the CHD library in there. What you fail to understand here is the complexity behind the format and the library that drives it, it's far from a simple format with a simple reader like CSO/ZSO.
 
  • Like
Reactions: _47iscool

MikaDubbz

Well-Known Member
Member
Joined
Dec 12, 2017
Messages
2,893
Trophies
1
Age
34
XP
5,102
Country
United States
Using chd on the Wii may cause stuttering issues, which is why I said ZSO.
I know the Wii is a good bit faster than the PS2, but still.
I don't get this thread, are you experiencing stuttering of GameCube games running in Nintendont? That shouldn't happen, Nintendont isn't an emulator, is really just a native GameCube game loader that runs all GameCube games at full speeds without stuttering on the Wii (or Wii U).

Now if the goal is just to reduce the file sizes of your GameCube games on your drive or SD card, then OK, I can understand that, but if the thought is that performance will greatly improve or something, I'm kinda left wondering if I'm missing something here as Nintendont performance is truly fantastic as is.
 

Acid_Snake

Well-Known Member
Member
Joined
Aug 20, 2019
Messages
278
Trophies
0
Age
28
XP
851
Country
Spain
I don't get this thread, are you experiencing stuttering of GameCube games running in Nintendont? That shouldn't happen, Nintendont isn't an emulator, is really just a native GameCube game loader that runs all GameCube games at full speeds without stuttering on the Wii (or Wii U).

Now if the goal is just to reduce the file sizes of your GameCube games on your drive or SD card, then OK, I can understand that, but if the thought is that performance will greatly improve or something, I'm kinda left wondering if I'm missing something here as Nintendont performance is truly fantastic as is.
No you misunderstood, nobody is saying that Nintendont has stuttering right now, he's just proposing to use ZSO to avoid stuttering if a compressed format is to be used since it's the fastest and lightest format. Implementing CHD might indeed either break things or make it go slower due to the more heavy compression algorithms.
 
  • Like
Reactions: _47iscool

MikaDubbz

Well-Known Member
Member
Joined
Dec 12, 2017
Messages
2,893
Trophies
1
Age
34
XP
5,102
Country
United States
No you misunderstood, nobody is saying that Nintendont has stuttering right now, he's just proposing to use ZSO to avoid stuttering if a compressed format is to be used since it's the fastest and lightest format. Implementing CHD might indeed either break things or make it go slower due to the more heavy compression algorithms.
Ah I see, so the goal is just to decrease the size of the games on drive or sd card. I'm certainly in favor of that, though it's also not something I'm dying for lol.
 

niuus

Well-Known Member
Member
Joined
Mar 4, 2016
Messages
1,637
Trophies
1
XP
2,983
Country
Venezuela
Last I checked Nintendont wasn't really swimming in loads of free RAM, so I very much doubt you can fit the CHD library in there. What you fail to understand here is the complexity behind the format and the library that drives it, it's far from a simple format with a simple reader like CSO/ZSO.
Can't say I care about your implementation. But I sure don't blame you for not being as talented as ekeeke in understanding the complexity behind the format and the library that drives it. Nkit still covers the what CHD or ZSO does. Good luck on your endeavors!
 

Acid_Snake

Well-Known Member
Member
Joined
Aug 20, 2019
Messages
278
Trophies
0
Age
28
XP
851
Country
Spain
Can't say I care about your implementation. But I sure don't blame you for not being as talented as ekeeke in understanding the complexity behind the format and the library that drives it. Nkit still covers the what CHD or ZSO does. Good luck on your endeavors!
What endeavors? I'm not doing anything here, you're the one who should have all the luck in the world to get CHD working on Nintendont.
You speak nonsense comparing an emulator of a 16 bit machine with gamecube games on retail hardware, and you talk about talent? I might not have "enough talent" but at least I'm not stupid enough to believe that a 16 bit game on an emulator is any proof of anything in regards to the power of a 7th gen machine natively running games that are only one generation behind.
 

_47iscool

Anti-communist
OP
Member
Joined
Nov 18, 2013
Messages
221
Trophies
0
XP
366
Country
United States
I don't get this thread, are you experiencing stuttering of GameCube games running in Nintendont? That shouldn't happen, Nintendont isn't an emulator, is really just a native GameCube game loader that runs all GameCube games at full speeds without stuttering on the Wii (or Wii U).

Now if the goal is just to reduce the file sizes of your GameCube games on your drive or SD card, then OK, I can understand that, but if the thought is that performance will greatly improve or something, I'm kinda left wondering if I'm missing something here as Nintendont performance is truly fantastic as is.

No. No stuttering. It would help to save space.

About Nintendont: I know exactly what it is. I used it quite a lot in its early days. It is basically virtualization on a faster machine.
 

sion_zaphod

Ex-Galactic President; confidence trickster; etc
Member
Joined
Sep 4, 2011
Messages
469
Trophies
0
Age
43
Location
Wales
XP
1,766
Country
If this compression could be implemented in Swiss-GC how would the compression compare to that of NKIT? So far the entire GC library USA dominant without dups in NKIT format is 640 gb. If using LZ4 compression could get that down to 512 gb I'm on board if not I do not see the point.
 
General chit-chat
Help Users
  • No one is chatting at the moment.
    KenniesNewName @ KenniesNewName: You've only seen a small percent of the worst of it