Any thoughts on adding ZSO (LZ4) support to Nintendont?

[_47iscool]

Open PS2 loader beta supports this CSO variant, and it works very well. The Wii should have no problem supporting it.
https://www.psx-place.com/threads/t...loader-with-zso-support-compressed-iso.37223/

 

[Lv44ES_Burner]

I've been thinking it would be far more useful and cross-platform friendly for OPL (and, to be honest, any other loader on competing hardware) to better support the MAME .CHD format that's been gaining steady traction with emulators as of late. While I've read Acid_Snake's writeup there in that thread about the .ZSO format, I believe .CHD would make more sense to adapt, as that would mean users need only one storage format for their files, whether it be for PCSX2's support of CHDs or for OPL's, and the same benefits can be had of .CHD that are now had with the research done in .ZSO and using the LZ4 compression mechanism.

But I'm just a blob, what do I know?

 

[_47iscool]

I've been thinking it would be far more useful and cross-platform friendly for OPL (and, to be honest, any other loader on competing hardware) to better support the MAME .CHD format that's been gaining steady traction with emulators as of late. While I've read Acid_Snake's writeup there in that thread about the .ZSO format, I believe .CHD would make more sense to adapt, as that would mean users need only one storage format for their files, whether it be for PCSX2's support of CHDs or for OPL's, and the same benefits can be had of .CHD that are now had with the research done in .ZSO and using the LZ4 compression mechanism.

But I'm just a blob, what do I know?

Check this comment:
https://github.com/ps2homebrew/Open-PS2-Loader/issues/225#issuecomment-1125244689

Using chd on the Wii may cause stuttering issues, which is why I said ZSO.
I know the Wii is a good bit faster than the PS2, but still.

I always thought chd was a MAME-exclusive format.

 

[Lv44ES_Burner]

No, recently CHD was adopted as a viable format for the PCSX2 emulator (as well as Duckstation and PCSX-ReARMed) for playing and storing backups of PS2 (as well as PSOne) games by way of its own variation of lossless compression and decompression of files in archive-quality, 1:1 conversions. Best audio quality, best data throughput from what has been reported by users of the PCSX2, Duckstation and PCSX-ReARMed cores for LibRetro/Retroarch. That's why I say that. CHDv5 uses LZMA compression from 7zip for the data streams and lossless FLAC compression for the audio streams for the highest optimizations possible over using BIN+CUE+MP3/BIN+CUE+WAV alone.

 

[_47iscool]

No, recently CHD was adopted as a viable format for the PCSX2 emulator (as well as Duckstation and PCSX-ReARMed) for playing and storing backups of PS2 (as well as PSOne) games by way of its own variation of lossless compression and decompression of files in archive-quality, 1:1 conversions. Best audio quality, best data throughput from what has been reported by users of the PCSX2, Duckstation and PCSX-ReARMed cores for LibRetro/Retroarch. That's why I say that. CHDv5 uses LZMA compression from 7zip for the data streams and lossless FLAC compression for the audio streams for the highest optimizations possible over using BIN+CUE+MP3/BIN+CUE+WAV alone.

Maybe. But PC's are much stronger than the Wii.

But it seems that chd has become more flexible now, so who knows?

 

[Acid_Snake]

No, recently CHD was adopted as a viable format for the PCSX2 emulator (as well as Duckstation and PCSX-ReARMed) for playing and storing backups of PS2 (as well as PSOne) games by way of its own variation of lossless compression and decompression of files in archive-quality, 1:1 conversions. Best audio quality, best data throughput from what has been reported by users of the PCSX2, Duckstation and PCSX-ReARMed cores for LibRetro/Retroarch. That's why I say that. CHDv5 uses LZMA compression from 7zip for the data streams and lossless FLAC compression for the audio streams for the highest optimizations possible over using BIN+CUE+MP3/BIN+CUE+WAV alone.

The key word here is "emulator" and "PC", CHD is a very complex format with a very complex library to drive it.
The PS2 would have never been able to successfully support it. ZSO was made with lightweigth and speed in mind, it was made for the PSP, so it has the potential to work on pretty much every old DVD-based console. The LZ4 algorithm is fast enough and the CSO/ZSO readers are so well optimized today that even the poor PS1 CPU on the PS2 can handle it.

I did check Nintendont's source code and it seems both viable and simple to implement.
Nintendont already supports a custom CSO format (it doesn't use compression though but rather trims empty blocks), so adding ZSO to it should be trivial.

 

[Lv44ES_Burner]

That's fair enough, and I had actually forgotten you had a user account here. I thank you for the clarification, even if it means I'm a bit crestfallen in regards for using CHDs. But I am very satisfied nonetheless with the advancements made in supporting ZSOs and look forward to the future of OPL using them in tandem with the MX4SIO (hopefully DECKARD-based systems like my own SCPH-90001 will benefit in the longer run), and no doubt it can be very beneficial to Nintendont and other USB loaders on the Wii as well. Whatever other optimizations can be done in tandem with the format can only benefit less powerful machines, right?

 

[niuus]

I've been thinking it would be far more useful and cross-platform friendly for OPL (and, to be honest, any other loader on competing hardware) to better support the MAME .CHD format that's been gaining steady traction with emulators as of late.

Using chd on the Wii may cause stuttering issues, which is why I said ZSO.
I know the Wii is a good bit faster than the PS2, but still.

I always thought chd was a MAME-exclusive format.

Maybe. But PC's are much stronger than the Wii.

CHD is already supported in GenesisPlus GX since a couple of years. More recently, it was also added for MD+ games. It's doable.

 

[Acid_Snake]

CHD is already supported in GenesisPlus GX since a couple of years. More recently, it was also added for MD+ games. It's doable.

I don't know if you have noticed, but there's a big difference between a genesis game on an emulator and a ps2 game on retail console.

 

[niuus]

I don't know if you have noticed, but there's a big difference between a genesis game on an emulator and a ps2 game on retail console.

I don't know if you have noticed, but I just said it's doable, on the Wii, which is why I specifically mention ekeeke's implementation on GenesisPlus GX, which BTW isn't for "a genesis game", it was implemented for Sega CD, which means Data+CD audio streaming.

Didn't mention PS2 examples, nor I care about PS2.

Using chd on the Wii may cause stuttering issues, which is why I said ZSO.
I know the Wii is a good bit faster than the PS2, but still.

I always thought chd was a MAME-exclusive format.

I also specifically quoted this as a way of letting the guy know CHD is not MAME exclusive. The same way CSO or Nkit is not "Nintendont-exclusive".

 

[Acid_Snake]

I don't know if you have noticed, but I just said it's doable, on the Wii, which is why I specifically mention ekeeke's implementation on GenesisPlus GX, which BTW isn't for "a genesis game", it was implemented for Sega CD, which means Data+CD audio streaming.

Didn't mention PS2 examples, nor I care about PS2.


I also specifically quoted this as a way of letting the guy know CHD is not MAME exclusive. The same way CSO or Nkit is not "Nintendont-exclusive".

Last I checked Nintendont wasn't really swimming in loads of free RAM, so I very much doubt you can fit the CHD library in there. What you fail to understand here is the complexity behind the format and the library that drives it, it's far from a simple format with a simple reader like CSO/ZSO.

 

[MikaDubbz]

Using chd on the Wii may cause stuttering issues, which is why I said ZSO.
I know the Wii is a good bit faster than the PS2, but still.

I don't get this thread, are you experiencing stuttering of GameCube games running in Nintendont? That shouldn't happen, Nintendont isn't an emulator, is really just a native GameCube game loader that runs all GameCube games at full speeds without stuttering on the Wii (or Wii U).

Now if the goal is just to reduce the file sizes of your GameCube games on your drive or SD card, then OK, I can understand that, but if the thought is that performance will greatly improve or something, I'm kinda left wondering if I'm missing something here as Nintendont performance is truly fantastic as is.

 

[Acid_Snake]

I don't get this thread, are you experiencing stuttering of GameCube games running in Nintendont? That shouldn't happen, Nintendont isn't an emulator, is really just a native GameCube game loader that runs all GameCube games at full speeds without stuttering on the Wii (or Wii U).

Now if the goal is just to reduce the file sizes of your GameCube games on your drive or SD card, then OK, I can understand that, but if the thought is that performance will greatly improve or something, I'm kinda left wondering if I'm missing something here as Nintendont performance is truly fantastic as is.

No you misunderstood, nobody is saying that Nintendont has stuttering right now, he's just proposing to use ZSO to avoid stuttering if a compressed format is to be used since it's the fastest and lightest format. Implementing CHD might indeed either break things or make it go slower due to the more heavy compression algorithms.

 

[MikaDubbz]

No you misunderstood, nobody is saying that Nintendont has stuttering right now, he's just proposing to use ZSO to avoid stuttering if a compressed format is to be used since it's the fastest and lightest format. Implementing CHD might indeed either break things or make it go slower due to the more heavy compression algorithms.

Ah I see, so the goal is just to decrease the size of the games on drive or sd card. I'm certainly in favor of that, though it's also not something I'm dying for lol.

 

[niuus]

Last I checked Nintendont wasn't really swimming in loads of free RAM, so I very much doubt you can fit the CHD library in there. What you fail to understand here is the complexity behind the format and the library that drives it, it's far from a simple format with a simple reader like CSO/ZSO.

Can't say I care about your implementation. But I sure don't blame you for not being as talented as ekeeke in understanding the complexity behind the format and the library that drives it. Nkit still covers the what CHD or ZSO does. Good luck on your endeavors!

 

[Acid_Snake]

Can't say I care about your implementation. But I sure don't blame you for not being as talented as ekeeke in understanding the complexity behind the format and the library that drives it. Nkit still covers the what CHD or ZSO does. Good luck on your endeavors!

What endeavors? I'm not doing anything here, you're the one who should have all the luck in the world to get CHD working on Nintendont.
You speak nonsense comparing an emulator of a 16 bit machine with gamecube games on retail hardware, and you talk about talent? I might not have "enough talent" but at least I'm not stupid enough to believe that a 16 bit game on an emulator is any proof of anything in regards to the power of a 7th gen machine natively running games that are only one generation behind.

 

[_47iscool]

I don't get this thread, are you experiencing stuttering of GameCube games running in Nintendont? That shouldn't happen, Nintendont isn't an emulator, is really just a native GameCube game loader that runs all GameCube games at full speeds without stuttering on the Wii (or Wii U).

Now if the goal is just to reduce the file sizes of your GameCube games on your drive or SD card, then OK, I can understand that, but if the thought is that performance will greatly improve or something, I'm kinda left wondering if I'm missing something here as Nintendont performance is truly fantastic as is.

No. No stuttering. It would help to save space.

About Nintendont: I know exactly what it is. I used it quite a lot in its early days. It is basically virtualization on a faster machine.

 

[sion_zaphod]

If this compression could be implemented in Swiss-GC how would the compression compare to that of NKIT? So far the entire GC library USA dominant without dups in NKIT format is 640 gb. If using LZ4 compression could get that down to 512 gb I'm on board if not I do not see the point.

 

[_47iscool]

Experimental JISO support has been added in case you missed it:

https://github.com/FIX94/Nintendont/pull/1141