Hardware NKit :: Recover to Redump Images :: Shrink and Preserve Wii and GameCube Disc Images as Playable

[Alopexau]

Sure, it's just a name (and GC NKit-ISO is functionally equivalent to GC ISO = GCM)

Define "performance" and "compressed" - NKit-ISO isn't compressed, "just" scrubbed and decrypted (that's why it can offer the lowest sizes after compression with a general purpose compression program) though of course they can't then directly be used;

as said above there's no performance difference, for GC titles, between NKit-ISO and ISO since nothing is actually moved around the disc (unlike less accurate GC "trimmers");

NKit-GCZ requires of course more processing power to play directly since it has to be decompressed on demand, but that's not significantly different from a regular GCZ

That answered everything I needed to know, thanks. I wasn't entirely sure if the shifting around of the data into a smaller image would introduce any reading differences, particularly with Wii games.

 

[Doctorbeefy]

Can you convert gamecube isos to nkit.iso with this? Or do I need to do some other stuff before?

 

[paint3d]

I believe my answer from a week or two ago in this thread will help you...
There's a way around this. In the "NKit Processing App" there is an option you can unselect labelled "Delete Recover Match Failures." Unselecting that will leave the unverified (most likely working but with no update partition) ISO file in the following folder:
\NKit\Processed\Wii_MatchFail

I've been trying to do that but the box won't get unchecked... it's just grayed out and my game keeps giving me the same error

 

[mundodisco]

Hi all!

Some weeks ago I was totally lost about how to use NKit, but after reading and many trials-and-errors, now I can happily say I have a fairly good knowledge of the whole process (and I could recover all my all WBFS files to newer ones with all partitions and Redump recoverable!.

Just to thank everyone that helped me before (answering my questions in different forums, or by spending time and writing clear explanations in posts and blogs), now I am willing to help other users!

To start with, you can read my answers in this post: https://gbatemp.net/threads/to-stupid-for-nkit-convert-to-iso.552191/

There you have some details about what you need, and magnet link to NKit 2020.4.29 full package... (NO ROMS OR GAMES, just the program and partitions).

And after that, feel free to ask me any doubts you have (or issues you come accross). You can write in English or Spanish (and maybe a little Portuguese and Italian)!

Regards,
Gabriel (from Buenos Aires, Argentina)

 

[TwilightSlick]

Hey, I found a game that has problems with Nkit.

Namco Museum (the first one, not 50th Anniversary) (GNMEAF) on the GameCube freezes on the title screen (before "press start" comes up) when in the Nkit format. This was tested on Nintendont on Wii (nkit.iso) and the Dolphin emulator (nkit.iso & nkit.gcz) converted using the latest Nkit and the latest Redump GC DAT.

The 1:1 ISO and a CISO made via Wit both boot fine on the Wii and Dolphin.
I can't test other regions or versions since it is NA exclusive and has no known code revisions.

Just some food for thought for the upcoming version of Nkit. Can't wait for WBFS output support! With that I could kiss Wii Backup Manager goodbye (unless WBM adds support for Nkit images so I don't have to go from Nkit>ISO>WBFS).

***EDIT - FURTHER TESTING - After reading a Dolphin progress update I decided to go into nkit.dll.config, add an FST alignment exception for the game (the one that by default only applies to the GameCube Service Disc and Aging Disc), re-converted into Nkit, and the resulting image booted on both Dolphin and Nintendont!***

 

[Nonstiq]

Just looking for confirmation - nkit.iso files will load on Swiss on a GameCube, but nkit.gcz will not. Is this correct?

 

[mundodisco]

Just looking for confirmation - nkit.iso files will load on Swiss on a GameCube, but nkit.gcz will not. Is this correct?

Yes, you're right @Nonstiq . At least, I can confirm nkit.iso can be loaded on Nintendont on Wii, but not nkit.gcz (the latter must be converted with NKit app before). And as the difference in MB is minimal between both formats, I should suggest not using nkit.gcz for archival purposes (keep all in nkit.iso).

 

[Vimm]

Seems some Dolphin developers have a bone to pick with NKit. Were you aware that in the latest build of Dolphin (currently 5.0-12528) when you load an NKit game a lengthy warning pops up lambasting NKit?

 

[leseratte]

Well, all the points they have about NKit are valid - I wouldn't want to use NKit to actually play games either.

 

[wiindsurf]

Seems some Dolphin developers have a bone to pick with NKit. Were you aware that in the latest build of Dolphin (currently 5.0-12528) when you load an NKit game a lengthy warning pops up lambasting NKit?

Does anyone have a screenshot to share? Thanks!

 

[leseratte]

Here it is: https://github.com/dolphin-emu/dolphin/pull/8875

 

[enarky]

For some reason all my isos fail to convert to NKit format. Same error message on 53 files:

PROCESSING
-------------------------------------------------------------------------------
GameName [GMID]  GameCube  [MiB:1392.3]

1 Pass: [ISO] >> [To NKit]

To NKit:        .1.2.3.4.5.6.7.8.9.10 ~ 0m 42s  [MiB:    3.2]  NKit Written

    |DETAIL
    |...............................
    |Fst parsing error - Converting as bad image
    |NKit Disc [GMID]: In [3.2 MiB] (3318112 bytes), Out [3.2 MiB] (bytes 3319808)
    |NKit Disc [GMID]: Preservation Padding [.0 MiB] 1696 bytes
    |...............................


Verification Failed Crc:ABCDABCD - Failed Test Crc:DCBADCBA
Deleting Output

Summary Log Written

Converting on a headless Debian Linux using Mono. Gamecube ISOs come from a source that claims to be Redump verified, but I haven't verified them myself.

Anyone has any input what's happening?

 

[mundodisco]

@enarky ,
Try with some ISOs from a certain archive site.
Those worked fine for me. If you still have the same bad results, then check that your NKit has the partitions to recover games (NKit complete folder weighs about 14GB).
Check my posts in this thread. I have also posted a magnet -torrent link- to complete Nkit:
https://gbatemp.net/threads/to-stupid-for-nkit-convert-to-iso.552191/

Any other doubts, do not hesitate to drop me a message...
Kind regards, from Buenos Aires, Argentina
GABRIEL

 

[stanwebber]

I had initial reservations about nkit (.net related), but I have since embraced it as my primary archive format. truth be told I still employ wbfs on a physical console to play most games, but I have everything stored permanently in nkit.gcz as a compromise for ready emulation use. I have read the nkit grumblings from the dolphin team and reviewed the favorable specs on their new rvz compression format, but I am unconvinced--nkit is still attractive to me with it's versatility. the only piece missing is the direct nkit to wbfs conversion routine that was already announced on the roadmap. from initial appearances it looks like you can convert nkit.iso to nkit.rvz so I will be investigating converting my entire nkit.gcz collection to rvz. if that's the case I would expect nkit to switch from gcz to rvz processing in future releases. there hasn't been any news released in a while...i do hope development hasn't stopped.

 

[johnsanc]

I'll just chime in on this since I'm fairly close to the subject as well. I'm also not a developer for Dolphin or NKit, but I am pretty darn familiar with the history. I helped Nanook with testing and had a lot of fun with this project along the way. This project brought together some of the of brightest people about the subject who still actively collaborate to this day.

First the history of NKit:

• NKit was originally created as a repairing tool as a way to clean up all of the badly scrubbed isos and wbfs files out there to repair to a full iso that matches what is in the redump.org database. Literally every wbfs creation app has at least one bug that causes data loss. NKit can fix these as well as repair most bad scene releases and restore partitions that were scrubbed.
• The NKit FORMAT (iso/gcz) was then created mainly as a proof of concept to create a dolphin playable archival grade format to use since NASOS images were not playable (my request - hah!)
• Under the hood, the NKit format has its roots in Nintendo's RVT-H format which is why it took very little change to Dolphin to make this playable.
• There were never any fundamental changes to Dolphin to explicitly support NKit ISO/GCZ aside from enabling the ability read RVT-H.
• After people started using NKit ISO/GCZ, limitations were found that had a real world impact on the playability of SOME games (not all). Some games are coded in such a way that are simply very picky about file alignment timing around how data would be read from the disc, like Super Paper Mario. This is a gross over simplification and I'm sure someone else can correct me.
• The warning was added to Dolphin for good reason - To reduce people reporting bugs with Dolphin that were legitimately NOT DOLPHIN BUGS. I know this appears like a jab at NKit, but try to look at this from a Dolphin support perspective. Most people will not do their own research to determine the root cause of their issue and fix it themselves.

NKit is a multi-purpose app focused on fixing and validating what you have. The NKit ISO/GCZ FORMAT is a consistent archival grade format that has the benefit of being mostly playable in Dolphin due to its roots in RVT-H. Its a nice side effect. If you run into an issue playing a game in NKit ISO/GCZ format, simply convert that game to full iso.

Now for RVZ:

• RVZ was a tabled idea that never came to fruition until after the NKit ISO/GCZ format was already in the wild. This is purely speculative, but I personally believe that the collaboration that went into NKit helped fuel motivation to bring RVZ to life.
  
• RVZ has its roots in Wiimm's WIA format, which was never really adopted for any practical purpose until recently.
• RVZ takes everything learned about Wii / GC discs over the years and rolls it all into one: The compression efficiencies of WIA, the junk compression/regeneration of NKit and NASOS, the importance of file layout and timings, etc... and then layers in ZSTD compression which is gaining traction and is a modern compression algorithm
• Since RVZ was created by Dolphin developers, it should make sense why its recommended now for playability.
• Dolphin's implementation of RVZ effectively solves all the problems of the past except two:
  • 1) Consistency. As of right now there are many options for RVZ as its essentially a container. There are also parameters to ZSTD under the hood that impact the output that the user does not have direct control over. This means files created with one version of Dolphin may not match files created with a newer version of Dolphin.
  • 2) Verification. There is no internal reference or meta data in dolphin's RVZ output that indicates the source for verification purposes. This can lead to people using RVZ on basically anything Dolphin will accept, which will have mixed results.

So I know this is a lot to digest, but let me try to distill it:

• If you want the best compatibility and dolphin support, use RVZ created from FULL ISOS that match the redump.org database. You can use NKit to repair just about anything you throw at it to a full matching ISO.
• If you want consistent images for archival purposes and occasional casual playing, NKit ISO/GCZ is still a great choice. These can always be converted to any other image format, particularly if you run into an edge case as mentioned above.
• Note that Dolphins's RVZ output may change over time - so if you are picky about consistency, keep this in mind.

All that being said, please do not think of NKit ISO/GCZ and RVZ as a format war. Think of it as building upon prior learnings.

Cheers and hope that helps clear up the evolution of how things came to be the way they are now. I'm sure @nanook can add additional color around where he wants to take NKit next

 

[nanook]

Hi all. Been very busy. NKit is not dead and is still in constant development. Time is proving a challenge at the moment, but I'm still getting about 10 hours a week on it.

@johnsanc is correct with his comments. He was there from the start advising on common image issues and how to repair them. Later with requirements to ensure the app was fit for archival purposes.

The NKit FORMAT was a nice to have initially. There had been attempts to get the WIA and ISO.DEC (Nasos) format support added to Dolphin for years by others that had never happened for whatever reason. So working around what was already supported by Dolphin seemed key at that point. It was discovered thanks to discussions with JosJuice (who had been helping with technical queries around GC and Wii images) that Dolphin could already read RVT-H which was dehashed and decrypted. So it seemed like a perfect fit.

Unfortunately the NKit format brought Dolphin compatibility issues despite weeks of testing. A couple of games didn't play well. Plus it broke TAS. So after v1.4 was released we went back to the drawing board. The NKit FORMAT still remains the smallest archive grade format out there, but we're fully on board with RVZ and discussed it at length before it was implemented.

So going forward....

• NKit is to continue with image repairing/restoration.
• More formats have been added
  • WIA
  • WiiU (WUD and WUX reading and file extraction)
• More formats will be added
  • RVZ reading and writing will be added to replace the NKit Format (archival grade). Johnsanc is researching this currently
  • WBFS writing (archival grade - reading is already supported)
  • CISO writing (archival grade - reading is already supported)

If you're wondering what's happened in the interim, well NKit has had a complete rewrite and is significantly more performant. The new model aims to create a common model to allow many other systems to be added in the future. ISO 9660, PS3, PS2, PSP, PSX are currently readable and files can be extracted.

The aim is to turn this in to a toolkit for all the disparate conversions and formats. I'm hoping over time it can be fleshed out with people's requests for features and turned in to something special.

Thanks for your support.

 

[Anuskuss]

I'm just glad to finally get an update from you again! Can't wait to see the final product.

 

[RandomByte]

• • RVZ reading and writing will be added to replace the NKit Format (archival grade). Johnsanc is researching this currently

Hi, sorry I didn't get quoted part! will we get the option to compress an image into RVZ from NKit app? Like convert an nkit.gcz to .rvz?

 

[kingjinxy2]

So going forward....

• NKit is to continue with image repairing/restoration.
• More formats have been added
  • WIA
  • WiiU (WUD and WUX reading and file extraction)
• More formats will be added
  • RVZ reading and writing will be added to replace the NKit Format (archival grade). Johnsanc is researching this currently
  • WBFS writing (archival grade - reading is already supported)
  • CISO writing (archival grade - reading is already supported)

If you're wondering what's happened in the interim, well NKit has had a complete rewrite and is significantly more performant. The new model aims to create a common model to allow many other systems to be added in the future. ISO 9660, PS3, PS2, PSP, PSX are currently readable and files can be extracted.

The aim is to turn this in to a toolkit for all the disparate conversions and formats. I'm hoping over time it can be fleshed out with people's requests for features and turned in to something special.

Thanks for your support.

All of this sounds amazing. I wish you the best of luck!

 

[PawsofHorror]

Is there any news on the Android version? Got lots of files which I want to convert from nkit to iso that match redump.

Android is so good for emulation but is missing so many usefull tools which only work for pc and I got no pc