Hacking [WIP] Wii U game info database

[Quantumcat]

Update 11th December:
Now nearly all games, updates, DLC and demos are in the database. If you know one that's missing please let me know and I'll add it manually. Also any suggestions welcome. You can also search (since there's so many now). And you can click on the name of a game to see the individual files and their sizes that belong to it.

Screenshot:

If you have an application or homebrew that needs to know details about games, like individual files, sizes of the files or of the whole game etc, I can provide a php page for you to send a POST request to which will return this data for you! Just ask, I like helping people!




..... original post:

Since I had trouble finding a list of game and update file sizes from NUS on the web, I thought I'd create my own. A very bare-bones display is at http://quantumc.at/gamelist.php (specific suggestions for improvements on looks or useability welcome). There isn't much there right now (it is only stuff I downloaded for testing, and my internet is slow so I tried to stick to updates I thought would be small) but there will be more in the future, and I hope one day it will have everything!

My little downloader program sends data about the game or update you downloaded to my website, which then adds it to the database. It also uses info from the database to display the size in the list. The progress bars currently are based on the output from NUSGrabber (thanks @Moquedami for the suggestion), but later it will be based on the file sizes from the database (and can have an accurate total % complete). I compiled a new copy which I have uploaded to quantumc.at if anyone wants to try it (repository at https://github.com/quantumcat1/Wii-U-Downloader).

What I am hoping is that the authors of the other downloader programs (that can do more than mine and are more popular) might consider also sending data about what their users have downloaded to my website, so the database can grow much more quickly. They might also like to use the webservice I made for requesting game sizes from the database to also display sizes in their programs. If their programs are in Java I can provide the code for sending the http POST requests but if not they'll have to work it out themselves :-p

If you think this is a good idea could you please tag a dev :-) All I can think of right now is @cearp for FunKiiU.

 

[pietempgba]

i wish that title key site had file sizes

 

[bennyman123abc]

I can give you exact file sizes to a few games, updates, and dlcs if you need me to. I'm not going to re-download them because my internet speed is 4 mbps down on average so it takes FOREVER to download anything these days. I remember when 4 mbps was f*cking fast too lol

 

[Quantumcat]

I can give you exact file sizes to a few games, updates, and dlcs if you need me to. I'm not going to re-download them because my internet speed is 4 mbps down on average so it takes FOREVER to download anything these days. I remember when 4 mbps was f*cking fast too lol

I'll make a mini program tonight for you that will send through the data on what you have already (saves typing in all the .app and .h3 names!). I was going to do this anyway as my recycle bin is full of stuff I previously downloaded while testing.

Thanks!!

 

[bennyman123abc]

I'll make a mini program tonight for you that will send through the data on what you have already (saves typing in all the .app and .h3 names!). I was going to do this anyway as my recycle bin is full of stuff I previously downloaded while testing.

Thanks!!

Could you possibly just have it to where I can enter the file size manually? For one of the games that I have (Mario Kart 8 USA), I have a DDD dump which is accurate to a range of 10 MB typically (Which shouldn't be that big of a deal. Is it?). But also having the option for the .app and .h3 names is great too because I have a few other games in that format. Thanks for letting me help out with this project btw!

 

[Irastris]

Sorting alphabetically on the bare-bones display page would be nice.

 

[Moquedami]

Ey, this is a neat update!
I dont know about the download size database though. Is it really necessary? I mean, i think other programs get the download size directly from Nintendo servers when loading the game list, but i´m not sure that´s how it works.
If you are doing it this way just to experiment, that´s cool too. But I think it would take a community effort that would have cause more benefit in the beginning. By this time everybody already downloaded whatever they were interested in, and it´s not really probable that they would download again for the sake of it.

 

[bennyman123abc]

Ey, this is a neat update!
I dont know about the download size database though. Is it really necessary? I mean, i think other programs get the download size directly from Nintendo servers when loading the game list, but i´m not sure that´s how it works.
If you are doing it this way just to experiment, that´s cool too. But I think it would take a community effort that would have cause more benefit in the beginning. By this time everybody already downloaded whatever they were interested in, and it´s not really probable that they would download again for the sake of it.

Well if you haven't noticed at this point, the community is starting to grow for this project.

 

[Moquedami]

yeah, and i'm glad. But that's not the point i'm asking for.

 

[bennyman123abc]

yeah, and i'm glad. But that's not the point i'm asking for.

What are you asking then? This is just a project to quickly and easily get a game's file size before you buy it and realize you have to wait 2 weeks for it to download.

 

[Moquedami]

This is the part i´m asking: "i think other programs get the download size directly from Nintendo servers when loading the game list, but i´m not sure that´s how it works."
If this is the case, why would we need acommunity effort just to get the data thats its already there?
And again, if that´s for the sake of experimating, I´m super cool with that. I just don´t get the practical use

 

[dojafoja]

@Quantumcat I have a full sqlite database of every title size on nus. The only ones missing are those which I could not download the tmd with FunKiiU. I am including it in th next release of my GUI for FunKiiU. You want a copy of it?

--------------------- MERGED ---------------------------

Here it is http://www.mediafire.com/file/807f7scw0v51gcy/data
This database contains the title id, file size in bytes, and a human readable size for every single title on NUS except those which the tmd couldn't be downloaded.
I also have a script to update this database against the title id's on that key site.
Its up do date as of yesterday morning but new titles get added all the time.

If you don't know how to query an SQL database, I can help. I use Python because it's super easy. I could technically write this to a file line by line if its easier

if the title size in the database is 0 that means that I couldn't download the tmd for that title

Forgot to mention, the db table is called "titles" the title id field is called "title_id" and there are two fields with size info.
"size" field has the human readable size and "raw_size" field has the actual title size in bytes

I'd love to help you out so if you can use the db as is, great! If it's easier to parse them into your program by other means ie: json,csv, etc.let me know and I will send you that output

 

[Quantumcat]

Sorting alphabetically on the bare-bones display page would be nice.

Done (and can sort by size)

 

[Irastris]

Done (and can sort by size)

Sweet, thanks.

 

[bennyman123abc]

Done (and can sort by size)

Did you ever finish the other program that allows the file sizes to be entered without downloading the title?

 

[Quantumcat]

Did you ever finish the other program that allows the file sizes to be entered without downloading the title?

I'm working on it :-) What is your folder structure? I am currently assuming game name/titleid/app and h3 files but if different let me know. It is working but there seems to be a bug in my update.php because it created some rows with blank titleids :-(

 

[Quantumcat]

Have a look at what's in the database now. Hilarious :-) I will clean it up when I get home.

 

[bennyman123abc]

Have a look at what's in the database now. Hilarious :-) I will clean it up when I get home.

You should've screenshot the image and posted it here

Oh lord I'm a fucking idiot

 

[Quantumcat]

You should've screenshot the image and posted it here

Oh lord I'm a fucking idiot

Haha yeah I should have :-) All cleaned up now.

Having my program send data to the database when users download stuff might become redundant. @dojafoja has a way of fetching file sizes in his Funkii-UI program. I just need to learn Python to fully understand what it's doing, then I can steal the logic bwahaha

 

[bennyman123abc]

Haha yeah I should have :-) All cleaned up now.

Having my program send data to the database when users download stuff might become redundant. @dojafoja has a way of fetching file sizes in his Funkii-UI program. I just need to learn Python to fully understand what it's doing, then I can steal the logic bwahaha

I know Python. Send me a link to this program and I will check it out