Homebrew WIP NSP Organizer and mass downloader

blawar

Developer
OP
Developer
Joined
Nov 21, 2016
Messages
1,708
Trophies
1
Age
40
XP
4,311
Country
United States
Nut
This is a program that automatically downloads all games from the CDN, and organizes them on the file system.

It will also download any titles you do not have a key for (for archiving), by enabling sansTitleKey in nut.json. These titles are saved with the .nsx file extension, and can be unlocked at a later time when a title key is found.

If you only wish to rename / organize files, and not download anything, edit nut.json and set all downloading options to false.

Organization
It saves the titles in the following format by default:

base games: titles/{name}[{id}][v{version}].nsp

DLC: titles/DLC/{baseName}/{name}[{id}][v{version}].nsp

Updates: titles/updates/{baseName}/{name}[{id}][v{version}].nsp

Demos: titles/demos/{name}[{id}][v{version}].nsp

Demo Updates: titles/demos/updates/{baseName}/{name}[{id}][v{version}].nsp

Titlelist
This program will load any titlekeys files named *.titlekeys.txt including titlekeys.txt

Whitelist
Place title id's that you want to download in whitelist.txt, separated with a newline.

If you want to download all games, leave the file empty.

Blacklist
Place title id's that you do not want to download in blacklist.txt, separated with a newline.

Installation
Place all of these files in your already configured CDNSP directory if you want it to download from the CDN.

If all you want is to organize your existing NSP's, just place all of these files in a dir that contains your NSP files (or sub directory) with the titleid in the filename in brackets.

run nut.py

https://github.com/blawar/nut
 
Last edited by blawar,

masterspike52

Well-Known Member
Member
Joined
Jan 25, 2017
Messages
192
Trophies
0
Age
28
XP
506
Country
United States
@blawar can you show me via a video of sorts that this will work?

well i rewrote some of it and fixed the errors but it still does the same thing cdnsp does and asks to check the cert
 
Last edited by masterspike52,

DeuX

Well-Known Member
Member
Joined
Jun 14, 2018
Messages
175
Trophies
0
Age
40
XP
605
Country
Switzerland
Would it be possible to organize the titles already downloaded? I already have most of the titles in a external hard drive so I would need to organize them before starting pulling the missing ones from CDN
 

masterspike52

Well-Known Member
Member
Joined
Jan 25, 2017
Messages
192
Trophies
0
Age
28
XP
506
Country
United States
Would it be possible to organize the titles already downloaded? I already have most of the titles in a external hard drive so I would need to organize them before starting pulling the missing ones from CDN
from what it looks like, you just blacklist the ones you own by id and then leave the whitelist blank to download the rest
 

DeuX

Well-Known Member
Member
Joined
Jun 14, 2018
Messages
175
Trophies
0
Age
40
XP
605
Country
Switzerland
from what it looks like, you just blacklist the ones you own by id and then leave the whitelist blank to download the rest

Still would need to check one by one what titles I have and add them to the blacklist. It would take a ton of time considering I have near 800 NSP files lol
 

Poketrekker

Active Member
Newcomer
Joined
Mar 3, 2016
Messages
28
Trophies
0
Age
35
XP
285
Country
Awesome work! Like @DeuX I have over 1000 NSP's already and it would be wonderful if we could get it working with existing .NSP's as I may have some dupes from different CDNSP versions :)
 

blawar

Developer
OP
Developer
Joined
Nov 21, 2016
Messages
1,708
Trophies
1
Age
40
XP
4,311
Country
United States
Would it be possible to organize the titles already downloaded? I already have most of the titles in a external hard drive so I would need to organize them before starting pulling the missing ones from CDN

just, it scans the current directory (and sub directories for existing NSP's and will organize them too. the filename just needs to have the titleid somewhere in brackets.

--------------------- MERGED ---------------------------

from what it looks like, you just blacklist the ones you own by id and then leave the whitelist blank to download the rest

no, it will pick up existing NSP's as well, as long as they have the title id in the filename in brackets like Mario Kart 8 Deluxe[0100152000022000].nsp

--------------------- MERGED ---------------------------

Still would need to check one by one what titles I have and add them to the blacklist. It would take a ton of time considering I have near 800 NSP files lol

You do not @DeuX , just make sure the NSP's are in the same directory, or a sub directory under the script and it will automatically pick them up as long as they have the title id in brackets in the filename like Mario Kart 8 Deluxe[0100152000022000].nsp
 

DeuX

Well-Known Member
Member
Joined
Jun 14, 2018
Messages
175
Trophies
0
Age
40
XP
605
Country
Switzerland
just, it scans the current directory (and sub directories for existing NSP's and will organize them too. the filename just needs to have the titleid somewhere in brackets.

--------------------- MERGED ---------------------------



no, it will pick up existing NSP's as well, as long as they have the title id in the filename in brackets like Mario Kart 8 Deluxe[0100152000022000].nsp

--------------------- MERGED ---------------------------



You do not @DeuX , just make sure the NSP's are in the same directory, or a sub directory under the script and it will automatically pick them up as long as they have the title id in brackets in the filename like Mario Kart 8 Deluxe[0100152000022000].nsp

That’s neat! So if I’m storing my games in an external harddrive I should execute the script from that drive?
 

blawar

Developer
OP
Developer
Joined
Nov 21, 2016
Messages
1,708
Trophies
1
Age
40
XP
4,311
Country
United States
That’s neat! So if I’m storing my games in an external harddrive I should execute the script from that drive?

Correct. But remember it needs to be in a valid CDNSP directory with a valid titlekeys.txt. It uses the titlekeys.txt to rename / move / organize. Any title id's found that arent in titlekeys.txt get ignored.

If you do not want it to download anything, just organize, put some garbage in whitelist.txt and save it. Then it will effectively reorganize only. I'm going to clean it up a lot. But like a lot of you, I had 1000's of NSP's tha twere not consistently named and management was a nightmare :(

--------------------- MERGED ---------------------------

How much space does that take up?

1.21 TB if you download everything, but i have a few DLC and updates in there.
 
  • Like
Reactions: Don Jon

DeuX

Well-Known Member
Member
Joined
Jun 14, 2018
Messages
175
Trophies
0
Age
40
XP
605
Country
Switzerland
Correct. But remember it needs to be in a valid CDNSP directory with a valid titlekeys.txt. It uses the titlekeys.txt to rename / move / organize. Any title id's found that arent in titlekeys.txt get ignored.

If you do not want it to download anything, just organize, put some garbage in whitelist.txt and save it. Then it will effectively reorganize only. I'm going to clean it up a lot. But like a lot of you, I had 1000's of NSP's tha twere not consistently named and management was a nightmare :(

--------------------- MERGED ---------------------------



1.21 TB, but i have a few DLC and updates in there.

I'm using CDNSP GUI by BOB, I'll check the regular CDNSP
 

blawar

Developer
OP
Developer
Joined
Nov 21, 2016
Messages
1,708
Trophies
1
Age
40
XP
4,311
Country
United States
I'm using CDNSP GUI by BOB, I'll check the regular CDNSP

I use CDNSP GUI by bob too, it works. The bob gui I use, also has the regular CDNSP.py script it in. Just place these files in that directory and it will work.

You will have to add one line of code to the top of the CDNSP.py script because the people who wrote that script have a bug where they import one library (tqdm) in their main routine, however I am using the script as a library / module so that module never gets imported and errors out when they try to call it. Unforuntately, there is no github repo for me to submit the bug report.
 
Last edited by blawar,

DeuX

Well-Known Member
Member
Joined
Jun 14, 2018
Messages
175
Trophies
0
Age
40
XP
605
Country
Switzerland
I use CDNSP GUI by bob too, it works. The bob gui I use, also has the regular CDNSP.py script it in. Just place these files in that directory and it will work.

You will have to add one line of code to the top of the CDNSP.py script because the people who wrote that script have a bug where they import one library (tqdm) in their main routine, however I am using the script as a library / module so that module never gets imported and errors out when they try to call it. Unforuntately, there is no github repo for me to submit the bug report.

Can I ask the mods why the CDNSP.py script cant be posted on gbatemp? This would be a lot easier if I could just include that file, there is nothing copyrighted in it.

Didn't work for me. I put your script inside the CDNSP GUI, added "from tqdm import tqdm" to the top of CDNSP.py and executed your script. It opened a window fast, created a cache folder and closed. Didn't have much time to tinker with it yet, so I guess I did something wrong
 

blawar

Developer
OP
Developer
Joined
Nov 21, 2016
Messages
1,708
Trophies
1
Age
40
XP
4,311
Country
United States
Didn't work for me. I put your script inside the CDNSP GUI, added "from tqdm import tqdm" to the top of CDNSP.py and executed your script. It opened a window fast, created a cache folder and closed. Didn't have much time to tinker with it yet, so I guess I did something wrong

I just updated the repo, do not copy the titlekeys.txt from the repo if you are using CDNSP as it has all of the keys zero'd out. I also added the CDNSP.py file since I do not believe that file on its own violates any rules. If the mod take this down, i'll remove it.

I added the ability to organize your existing NSP's only without CDNSP. If that is what you want, extract all of these files on their own (except CDNSP.py). Make sure the NSP files are in or under the directory you extract the files.

https://github.com/blawar/nut
 
  • Like
Reactions: morrison22 and DeuX

DeuX

Well-Known Member
Member
Joined
Jun 14, 2018
Messages
175
Trophies
0
Age
40
XP
605
Country
Switzerland
I just updated the repo, do not copy the titlekeys.txt from the repo if you are using CDNSP as it has all of the keys zero'd out. I also added the CDNSP.py file since I do not believe that file on its own violates any rules. If the mod take this down, i'll remove it.

I added the ability to organize your existing NSP's only without CDNSP. If that is what you want, extract all of these files on their own (except CDNSP.py). Make sure the NSP files are in or under the directory you extract the files.

https://github.com/blawar/nut

Worked Perfectly now! They are organized and looking good :D
Now to see if I can get it to work with CDNSP GUI
Thank you!
 

blawar

Developer
OP
Developer
Joined
Nov 21, 2016
Messages
1,708
Trophies
1
Age
40
XP
4,311
Country
United States
Worked Perfectly now! They are organized and looking good :D
Now to see if I can get it to work with CDNSP GUI
Thank you!

You're welcome, I am glad you got it working!

It will work with the GUI, that is what I use.

I am going to add a configuration file / options to, among other things, specify different formats for how it structures the files. I think some people would want all of the base files in the same directory, and then squirrel away updates and DLC in sub directories perhaps?
 
  • Like
Reactions: morrison22

DeuX

Well-Known Member
Member
Joined
Jun 14, 2018
Messages
175
Trophies
0
Age
40
XP
605
Country
Switzerland
You're welcome, I am glad you got it working!

It will work with the GUI, that is what I use.

I am going to add a configuration file / options to, among other things, specify different formats for how it structures the files. I think some people would want all of the base files in the same directory, and then squirrel away updates and DLC in sub directories perhaps?

Yeah, it's a good idea, not everyone likes's to keep files the same way as the neighbor.
My CDNSP GUI has no CDNSP.py, so I'm a bit unsure how to do it. Maybe you can make it work with CDNSP GUI py file instead? Since I believe more people use the GUI version by BOB than the regular one
 

blawar

Developer
OP
Developer
Joined
Nov 21, 2016
Messages
1,708
Trophies
1
Age
40
XP
4,311
Country
United States
Yeah, it's a good idea, not everyone likes's to keep files the same way as the neighbor.
My CDNSP GUI has no CDNSP.py, so I'm a bit unsure how to do it. Maybe you can make it work with CDNSP GUI py file instead? Since I believe more people use the GUI version by BOB than the regular one

Look at the repo, I added a CDNSP.py, just extract it to your CDNSP GUI folder. And it probably would work with your CDNSP GUI if you renamed it to CDNSY.py

https://github.com/blawar/nut
 

morrison22

Well-Known Member
Member
Joined
Nov 26, 2005
Messages
618
Trophies
0
XP
1,898
Country
United States
This looks very promising! Thank you. I will be testing it later.

The readme states the following:

It saves the games in the following dir structure: titles/{name}/{name}[{title_id}].nsp
DLC are saved to: titles/{name}/DLC/{name}[{title_id}].nsp
Updates are saved to: titles/{name}/updates/{name}[{title_id}].nsp


If I do not want folders created for each NSP and instead, want all NSPs in one folder with the naming scheme shown below, what would need to be done to make this happen?

Game Name [BASE] [TitleID] [v].nsp
Game Name [UPD] [TitleID] [v].nsp
Game Name [DLC] [TitleID] [v].nsp
Game Name [DEMO] [BASE] [TitleID] [v].nsp
Game Name [DEMO] [UPD] [TitleID] [v].nsp


Also #1, How does the renaming handle invalid characters such as ":" for example? Can it be specified to replace ":" with "-"?
Also #2, Once a title is blacklisted, can it still be set to download updates only for that title when available? (Updates share the same TitleID as the Base Title, so I'm not sure how this would work.)

This is an excellent project that has come to life and is very much needed! Thank you for doing this! A+
 
Last edited by morrison22,

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
    The Real Jdbye @ The Real Jdbye: i always pee in the water