Don't use no-intro Dat-Files!

godreborn

Welcome to the Machine
Member
Joined
Oct 10, 2009
Messages
38,471
Trophies
3
XP
29,138
Country
United States
still won't work. they're different format. I've seen how gba games are formatted, and they're just a bunch of numbers without an extension iirc. they're compiled as well, and the format is then encrypted.
 

godreborn

Welcome to the Machine
Member
Joined
Oct 10, 2009
Messages
38,471
Trophies
3
XP
29,138
Country
United States
because they're not the same format. that's why we have scripts for these things. you can't take a rom and just install it on to the wii u. it's impossible. it has to be converted first. plus, clrmame pro typically zips this stuff, which is also a format wup installer will not recognize.
 

Korschan

Member
OP
Newcomer
Joined
Feb 28, 2022
Messages
15
Trophies
0
Age
36
Location
Deutschöla
XP
77
Country
Denmark
because they're not the same format. that's why we have scripts for these things. you can't take a rom and just install it on to the wii u. it's impossible. it has to be converted first. plus, clrmame pro typically zips this stuff, which is also a format wup installer will not recognize.
I am not a dumbhead. As I mentioned before I surely unzip my before before trying to install.

Whats now true? When I download the files with nusgrubber I just can install them on my wiiu.
Sure, I have to unzip them when using clrmamepro - but I though they are just renamed, you are now talking about new format so I have to convert them?

What for scripts are you talking about?
 

The Real Jdbye

*is birb*
Member
Joined
Mar 17, 2010
Messages
23,338
Trophies
4
Location
Space
XP
13,916
Country
Norway
I used clrmamepro to manage my files while using no-intro dat files.
Renamed everything fine, got a lot of roms - but no installer on wiiu will work with these files.
And they are not named like on the NUS itself - try yourself! Use nusspli or NUSGrabber to download these files and BAM(!) no-intro will rename your files to non-usable files B-)

And people like Modern Vintage Gamer from YouTube always make me new hope - "for preservation, the wii u files have been also archived on archive.com" which is absolutly not true. I should make a videos about him: "MVGs YouTube Videos - Mistakes were made". lol.
I'm pretty sure on NUS they're just named with strings of hexadecimal letters and numbers and NUS downloader tools rename them to work with installers. And the dat probably just has them with their original NUS filenames. No clue why the naming is different on the servers compared to what installers expect though. It was the same on the Wii - the files on NUS were separate .app, .title, .tmd files and the NUS downloaders had to pack those into WADs in order to be usable with installers. I guess it's because you're not meant to be installing those files that way and when installed through the eShop they use a different installation method calling different install functions that install from NUS directly and work with that sort of file naming. Whereas installing from files was intended for dev usage and the files generated by dev tools probably already have the correct .app/.h3 naming used for installation that way.

I'd guess there has to be some "simple" way to figure out how the files need to be renamed for them to be installable the same way NUS downloaders originally figure it out. The TMD contains all the needed info (4 byte/8 hexadecimal character "content ID", 2-byte index, 2-byte type), using that info the files should be renamed from content ID -> index.type. I think that's how the NUS downloaders do it. But not sure about what type values correspond to what file extensions.

MVG makes significant mistakes in every video about homebrew that anybody with some knowledge about the scene could spot. But for his target audience, his videos serve their purpose well. They're not targeted at you or me, they're targeted at the general audience that doesn't care about the fine details, and probably wouldn't bother to watch a video that went more in depth, and if his videos can get those people interested in homebrew/hacking then that's a good thing.
Apparently, he's even admitted to making mistakes in videos on purpose to encourage discussion in the comments. Sounds like a cop-out excuse to me.

Edit: Had a look at this code https://github.com/ajd4096/nuspy/blob/master/nuspy/nuspy.py#L276
It appears the numbered files without a file extension from CDN are simply .app files. So you can add the .app extension to them and they should be correct without further renaming.
tmd needs to be renamed to title.tmd
cert needs to be renamed to title.cert
title.tik should be present. It's not in the dat, because it's not available on CDN. You normally get it by purchasing a game. They may have been deleted when you ran clrmamepro since they're not in the dat. But for eShop only titles (which is all this dat contains AFAIK) we just generate a fake ticket which you can do using a tool like https://github.com/dojafoja/Kii-U-Generator so nothing special is needed to regenerate the .tiks.
The .h3 files are not in the No-Intro database. Unsure why they were not included, as they are available on NUS with the .h3 extension. It's possible running clrmamepro would've deleted these files, since they're not in the dat. Or perhaps it just ignores them? You'll have to tell me.
They contain SHA-1 hashes to verify the data. I'm not sure if they're absolutely needed for WUPInstaller to install the files. The WUPInstaller code doesn't check for them but the internal system functions for installation could. If they've been deleted, you could try installing without them after renaming everything else.
I couldn't find a tool to generate them. Doing that would negate any benefits having those files would have since if the files are bad, the SHA-1 hashes would also be generated based on the bad data, so the installer wouldn't be able to tell if files are bad. You could always opt to redownload them from CDN again, since they are very small files. But if they aren't needed to install, then no need since clrmamepro will check that your files are good anyway.

The dat does pretty clearly state CDN, implying that these are in CDN format which they indeed are. Looking at that nusPy code again, it's the downloader that adds the .app extension (as well as renaming the tmd and cert)
So I would say the dat is functioning as intended. But I do wish that it was more clearly labeled. Especially for those that download huge No-Intro packs of these (not just trying to use the dat with files they already got elsewhere), and then have no idea how to install them because the filenames are all weird. There don't seem to be any explanations (at least easily available that I could find) on how to get those weirdly named files to install either. Mine might be the most in-depth post about it.

Edit2: If installing works after renaming the files and making sure the tik is present. It would be fairly trivial for me to write a small tool to batch rename all the .app files and also generate the .tiks so that you run it on your (CDN named) collection once and you can put them straight into WUPInstaller. Could also make it generate the .h3 files though I'd have to look at them a bit more closely to figure out how to do that. It would be a useful tool for people who download No-Intro packs anyway, so it's potentially a useful tool for a lot of people.
So don't feel like you need to hand rename everything. Just try it with one game and see if it's able to install after that.
because they're not the same format. that's why we have scripts for these things. you can't take a rom and just install it on to the wii u. it's impossible. it has to be converted first. plus, clrmame pro typically zips this stuff, which is also a format wup installer will not recognize.
They're just .app files without the extension, the extension is not in the file name on CDN but installers need it to be there.
They're not zipped because they're encrypted, so there's no point in compressing them.
You're making a lot of incorrect assumptions in this case. But I understand where you're coming from because what you describe is how it has typically worked on other consoles (you had to pack them into WAD on the Wii and NSP on the Switch), it just doesn't apply to the Wii U. The files on CDN are the same files WUPInstaller uses, minus the tik.
 
Last edited by The Real Jdbye,
  • Like
Reactions: Aglaophotis

godreborn

Welcome to the Machine
Member
Joined
Oct 10, 2009
Messages
38,471
Trophies
3
XP
29,138
Country
United States
I'm pretty sure on NUS they're just named with strings of hexadecimal letters and numbers and NUS downloader tools rename them to work with installers. And the dat probably just has them with their original NUS filenames. No clue why the naming is different on the servers compared to what installers expect though. It was the same on the Wii - the files on NUS were separate .app, .title, .tmd files and the NUS downloaders had to pack those into WADs in order to be usable with installers. I guess it's because you're not meant to be installing those files that way and when installed through the eShop they use a different installation method calling different install functions that install from NUS directly and work with that sort of file naming. Whereas installing from files was intended for dev usage and the files generated by dev tools probably already have the correct .app/.h3 naming used for installation that way.

I'd guess there has to be some "simple" way to figure out how the files need to be renamed for them to be installable the same way NUS downloaders originally figure it out. The TMD contains all the needed info (4 byte/8 hexadecimal character "content ID", 2-byte index, 2-byte type), using that info the files should be renamed from content ID -> index.type. I think that's how the NUS downloaders do it. But not sure about what type values correspond to what file extensions.

MVG makes significant mistakes in every video about homebrew that anybody with some knowledge about the scene could spot. But for his target audience, his videos serve their purpose well. They're not targeted at you or me, they're targeted at the general audience that doesn't care about the fine details, and probably wouldn't bother to watch a video that went more in depth, and if his videos can get those people interested in homebrew/hacking then that's a good thing.
Apparently, he's even admitted to making mistakes in videos on purpose to encourage discussion in the comments. Sounds like a cop-out excuse to me.

Edit: Had a look at this code https://github.com/ajd4096/nuspy/blob/master/nuspy/nuspy.py#L276
It appears the numbered files without a file extension from CDN are simply .app files. So you can add the .app extension to them and they should be correct without further renaming.
tmd needs to be renamed to title.tmd
cert needs to be renamed to title.cert
title.tik should be present. It's not in the dat, because it's not available on CDN. You normally get it by purchasing a game. They may have been deleted when you ran clrmamepro since they're not in the dat. But for eShop only titles (which is all this dat contains AFAIK) we just generate a fake ticket which you can do using a tool like https://github.com/dojafoja/Kii-U-Generator so nothing special is needed to regenerate the .tiks.
The .h3 files are not in the No-Intro database. Unsure why they were not included, as they are available on NUS with the .h3 extension. It's possible running clrmamepro would've deleted these files, since they're not in the dat. Or perhaps it just ignores them? You'll have to tell me.
They contain SHA-1 hashes to verify the data. I'm not sure if they're absolutely needed for WUPInstaller to install the files. The WUPInstaller code doesn't check for them but the internal system functions for installation could. If they've been deleted, you could try installing without them after renaming everything else.
I couldn't find a tool to generate them. Doing that would negate any benefits having those files would have since if the files are bad, the SHA-1 hashes would also be generated based on the bad data, so the installer wouldn't be able to tell if files are bad. You could always opt to redownload them from CDN again, since they are very small files. But if they aren't needed to install, then no need since clrmamepro will check that your files are good anyway.

The dat does pretty clearly state CDN, implying that these are in CDN format which they indeed are. Looking at that nusPy code again, it's the downloader that adds the .app extension (as well as renaming the tmd and cert)
So I would say the dat is functioning as intended. But I do wish that it was more clearly labeled. Especially for those that download huge No-Intro packs of these (not just trying to use the dat with files they already got elsewhere), and then have no idea how to install them because the filenames are all weird. There don't seem to be any explanations (at least easily available that I could find) on how to get those weirdly named files to install either. Mine might be the most in-depth post about it.

Edit2: If installing works after renaming the files and making sure the tik is present. It would be fairly trivial for me to write a small tool to batch rename all the .app files and also generate the .tiks so that you run it on your (CDN named) collection once and you can put them straight into WUPInstaller. Could also make it generate the .h3 files though I'd have to look at them a bit more closely to figure out how to do that. It would be a useful tool for people who download No-Intro packs anyway, so it's potentially a useful tool for a lot of people.
So don't feel like you need to hand rename everything. Just try it with one game and see if it's able to install after that.

They're just .app files without the extension, the extension is not in the file name on CDN but installers need it to be there.
They're not zipped because they're encrypted, so there's no point in compressing them.
You're making a lot of incorrect assumptions in this case. But I understand where you're coming from because what you describe is how it has typically worked on other consoles (you had to pack them into WAD on the Wii and NSP on the Switch), it just doesn't apply to the Wii U. The files on CDN are the same files WUPInstaller uses, minus the tik.
no, no, no, I thought he was talking about installing a normal snes rom like a .smc.
 

Norris

Well-Known Member
Member
Joined
Oct 13, 2021
Messages
614
Trophies
0
Age
16
XP
1,261
Country
United States
so just to be clear have the entire nus servers been archived all dlc all updates all games in every region?
 

Norris

Well-Known Member
Member
Joined
Oct 13, 2021
Messages
614
Trophies
0
Age
16
XP
1,261
Country
United States

The Real Jdbye

*is birb*
Member
Joined
Mar 17, 2010
Messages
23,338
Trophies
4
Location
Space
XP
13,916
Country
Norway
my freinds and i are in the legacy console speedrunning community look into it we have backups of all the updates for those games except i think switch and some ps3 stuff we probaly most likely have it ill ask them about it is that something that has been missing for a while?
Since 2018 it looks like. Or theoretically, it's been missing forever, but it was only discovered in 2018.
 

Norris

Well-Known Member
Member
Joined
Oct 13, 2021
Messages
614
Trophies
0
Age
16
XP
1,261
Country
United States
Since 2018 it looks like. Or theoretically, it's been missing forever, but it was only discovered in 2018.
whats the patch called im pretty sure we have it is it something thats like really obscure and was only around for a couple of days our team basically archives all updates we can
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
  • No one is chatting at the moment.
    Sonic Angel Knight @ Sonic Angel Knight: DAYTONAAAAAAAA!!!!!!!!!!