Homebrew WIP nxdumptool - Nintendo Switch Dump Tool

justinbug2 · Apr 12, 2021

Johnmariozelda99 said:
Well I'll be. That works. I wonder why 1.1.12 works on Atmosphere 0.19.0 but 1.1.13 doesn't. Definitely odd. Now if I could find a save manager that works on AMS 0.19.0 Id be set

JKSV works fine

Sayshi · May 15, 2021

I'm a new user here, but I have a question.

When dumping a game through NXdumptool (so I can play it without needing to insert the gamecard all the time), is it necessary (or a danger) to keep the certificate?
I plan to install through Awoo Installer.

I understand that sharing the game with the certificate is dangerous, but for personal use, do I need it to access cloud saves/online play/download updates from Nintendo servers etc? If I attempt to do the above without the certificate, will it get me banned?

Likewise, is there a risk of doing anything online without a certificate? (access cloud saves/online play/download updates).

Any help would be much appreciated!

Draxzelex · May 15, 2021

Sayshi said:
I'm a new user here, but I have a question.

When dumping a game through NXdumptool (so I can play it without needing to insert the gamecard all the time), is it necessary (or a danger) to keep the certificate?
I plan to install through Awoo Installer.

I understand that sharing the game with the certificate is dangerous, but for personal use, do I need it to access cloud saves/online play/download updates from Nintendo servers etc? If I attempt to do the above without the certificate, will it get me banned?

Likewise, is there a risk of doing anything online without a certificate? (access cloud saves/online play/download updates).

Any help would be much appreciated!

Certificates for cartridge dumps are actually useless if you install them instead of mounting them. The reason being is that certificates are meant to be used with raw .XCI files which are supposed to be mounted. When you install an .XCI file, you are actually converting it to an .NSP file then installing it. .NSP files do not use certificates but instead they use tickets which are not interchangeable with one another. So if you install an .XCI file with the certificate, it will be lost during the installation.

Sayshi · May 15, 2021

Draxzelex said:
Certificates for cartridge dumps are actually useless if you install them instead of mounting them. The reason being is that certificates are meant to be used with raw .XCI files which are supposed to be mounted. When you install an .XCI file, you are actually converting it to an .NSP file then installing it. .NSP files do not use certificates but instead they use tickets which are not interchangeable with one another. So if you install an .XCI file with the certificate, it will be lost during the installation.

I see... I thought the certificate was used to verify legitimate copies. So I thought it would be a good idea to keep it.
So I assume mounting is safer than installing then as the certificate will be applied to it?

So if I installed an .NSP file, that means I can't go online at all for anything then since Nintendo would detect it I assume.

Draxzelex · May 15, 2021

Sayshi said:
I see... I thought the certificate was used to verify legitimate copies. So I thought it would be a good idea to keep it.
So I assume mounting is safer than installing then as the certificate will be applied to it?

So if I installed an .NSP file, that means I can't go online at all for anything then since Nintendo would detect it I assume.

Mounting may be theoretically safer but not only is it only possible with one CFW that is on the verge of becoming irrelevant but people have already been banned just for mounting .XCI files. Also going online with a hacked console is never a good idea regardless if you installed an .NSP file or not.

Sayshi · May 15, 2021

Draxzelex said:
Mounting may be theoretically safer but not only is it only possible with one CFW that is on the verge of becoming irrelevant but people have already been banned just for mounting .XCI files. Also going online with a hacked console is never a good idea regardless if you installed an .NSP file or not.

Alright, thank you very much for the help.
I'll hold off then and just stick with playing my games via game card. Thanks!

pcwizard7 · May 16, 2021

Draxzelex said:
Certificates for cartridge dumps are actually useless if you install them instead of mounting them. The reason being is that certificates are meant to be used with raw .XCI files which are supposed to be mounted. When you install an .XCI file, you are actually converting it to an .NSP file then installing it. .NSP files do not use certificates but instead they use tickets which are not interchangeable with one another. So if you install an .XCI file with the certificate, it will be lost during the installation.

XCI Explorer allows you to remove or insert them into a xci which you could mount them

HinaNaru Cutie · May 23, 2021

DarkMatterCore said:
This is known as a title takeover or application override. It gives homebrew applications access to a ~3.5 GiB memory pool - and yes, even though some people may disagree, it's always the better practice when it comes to homebrew launching on the Switch.

Running homebrew apps under applet mode only provides them with access to a ~300-something MiB memory pool. Depending on what you want to do, it may not be desirable.

excuse me but i have been trying to dump the spongebob cook off game that i digitally bought, installed on my vanilla version of my switch not the homebrew one - and i was wondering if it was possible to use this tool to dump the game's files? i have been trying to find it in the emmuc option but it's not there at all

... i even used lockpick to dump the sysnand but it still won't show up in the list of games to dump..really lost on how to do this T_T...any help would be nice...

edit; i found out what i was doing wrong lol. so i am good

Draxzelex · Jun 6, 2021

pcwizard7 said:
XCI Explorer allows you to remove or insert them into a xci which you could mount them

But that is only doable on SX OS which less and less people are running nowadays due to TX not pushing an update to support the higher firmware versions in 6 months so mounting is also becoming less relevant.

DarkMatterCore · Jun 14, 2021

Finally began to work on Borealis integration.

Functionally speaking, the rewrite is almost complete at this moment - only the UI is missing.

DarkMatterCore · Jun 28, 2021

Still working on the UI. I'll slowly post updates as development progresses.

Connection to a USB host is going to be one of the upcoming big features - the label at the top right corner displays the USB speed (1.0, 2.0, 3.0).

This picture is slightly outdated, btw. Scrollbars were added today.

duckbill007 · Jun 28, 2021

@DarkMatterCore what speed of dumping do you achieve via USB?
DBI dumps NSP through MTP without any soft/drivers on PC side ~130-150 MB/sec on USB3 from NAND.
I wonder, if there is a possibility to speed up if using own protocol (add thiis to dbibackend).

pcwizard7 · Jun 28, 2021

@DarkMatterCore where will batch mode be? Any changes there?

DarkMatterCore · Jun 28, 2021

duckbill007 said:
@DarkMatterCore what speed of dumping do you achieve via USB?
DBI dumps NSP through MTP without any soft/drivers on PC side ~130-150 MB/sec on USB3 from NAND.
I wonder, if there is a possibility to speed up if using own protocol (add thiis to dbibackend).

Are you sure you're properly measuring the speeds on the USB host end? 130 - 150 MB/s sounds way too good for MTP, which usually adds overhead on its own even on Android phones. Unless you really are referring to megabytes (MB) and not mebibytes (MiB).

People are currently reaching ~60 MiB/s (60 mebibytes/second) under USB3 for raw gamecard dumps, which in my experience should be faster than dumping eMMC/SD card NCAs.

Personally, I'm using tqdm to calculate transfer speeds with Python. Speed calculation algorithms are a bit tricky, so I'd suggest to try another one and measure speeds from the USB host itself.

If you're interested, full documentation for my custom USB ABI is available here.

pcwizard7 said:
@DarkMatterCore where will batch mode be? Any changes there?

It's still planned, but it will be one of the last features to be reimplemented. It will most likely be mapped to a button press, and a hint for it will be displayed at the bottom.

duckbill007 · Jun 28, 2021

DarkMatterCore said:
which usually adds overhead on its own even on Android phones

What overhead are you talking about? Maybe for small files it does matter, but for big ones there are no overhead: there is short request packet from PC side with command GET_OBJECT, small packet with responce on switch side and after that raw stream of file data switch->PC without any headers/waiting/etc until all file data is sent.

Speed in DBI calculated as ((total sent bytes) >>20)/(total passed seconds)

Raw read from NAND is about 150-170 MB/sec (via ncmContentStorageReadContentIdFile)
Read + hash calculating is about 120-130 MB/sec

I found out that read/write speed (nand and usb) highly depends on buffer size. Right now DBI uses 128K chunks for USB operations and 4MB for NAND read/write.

Also, I do not use stdio wrappers, only direct FS calls.

DarkMatterCore · Jun 29, 2021

duckbill007 said:
What overhead are you talking about? Maybe for small files it does matter, but for big ones there are no overhead: there is short request packet from PC side with command GET_OBJECT, small packet with responce on switch side and after that raw stream of file data switch->PC without any headers/waiting/etc until all file data is sent.

Speed in DBI calculated as ((total sent bytes) >>20)/(total passed seconds)

Raw read from NAND is about 150-170 MB/sec (via ncmContentStorageReadContentIdFile)
Read + hash calculating is about 120-130 MB/sec

I found out that read/write speed (nand and usb) highly depends on buffer size. Right now DBI uses 128K chunks for USB operations and 4MB for NAND read/write.

It adds overhead because MTP depends on additional OS drivers/components to work, which aren't needed when you're using a custom USB protocol - in other words, data comes straight from the USB driver. The MTP implementation found within Windows isn't exactly speedy, for example, and it can easily freeze for some seconds before starting a data transfer stage (in my experience).

Excessive overhead with small files isn't also too much of an issue in this case. People have dumped the entire RomFS from Animal Crossing: New Horizons (60K+ small files) in under 8 minutes using USB2, and this was months ago with a Java-based client. I have yet to see how it performs using the Python client with USB3.

Furthermore, I'd say the speed calculation algorithm you're using probably isn't accurate enough. It seems there are some other variables you're not handling, like the number of bytes transferred during the last second, which is why I suggested using a PC tool or library to measure USB speeds there. tqdm's speed calculation is pretty robust, and it's not hard to use.

I'll take a look at the chunk sizes you're using - truth be told, right now I'm using 8 MiB chunks for everything. I have not conducted any extensive benchmarks since I sold my console some months ago.

FS calls are irrelevant for nxdumptool, since it will only send data to the USB host (none of the dumpeable data is retrieved using FS file handles).

duckbill007 · Jun 29, 2021

I have no experience with Windows implementation of MTP. I only using Linux with gvfs over libmtp.
I do not get what is wrong with dividing total sent data by total spent time?
Just verified measurement via "time gvfs-copy "mtp://[usb:001:104]/4: Installed games/Disgaea 4 Complete+ [B+U131072].nsp" ."
It gives the same result. Of course it is average speed for the whole file.

8Mib chunks is not a good idea - on tests I got worst performance on this size.

I do agree that on small files MTP sucks - depends on file size it can greatly increase transferred data.

Of course DBI's data transfer is multithreaded: reading from system and sending to USB are done in separate threads. Before this speed was almost twice lower.

DarkMatterCore · Jun 30, 2021

Video from the latest commit showing off tickering text and scrollbars: https://cdn.discordapp.com/attachme...0-15E684BAAD5E714176706B74D5E247A9-ffmpeg.mp4

duckbill007 said:
I have no experience with Windows implementation of MTP. I only using Linux with gvfs over libmtp.
I do not get what is wrong with dividing total sent data by total spent time?
Just verified measurement via "time gvfs-copy "mtp://[usb:001:104]/4: Installed games/Disgaea 4 Complete+ [B+U131072].nsp" ."
It gives the same result. Of course it is average speed for the whole file.

8Mib chunks is not a good idea - on tests I got worst performance on this size.

I do agree that on small files MTP sucks - depends on file size it can greatly increase transferred data.

Of course DBI's data transfer is multithreaded: reading from system and sending to USB are done in separate threads. Before this speed was almost twice lower.

I'll change the chunk sizes I'm currently using and check if any of my testers reports back any meaningful improvements, then. I'm already using multiple threads as well.

Anonymous42456 · Jul 15, 2021

Why is it when I extract music (.wavs) from the DQ classic games, they don't play after opening them in a media player?

DarkMatterCore · Jul 16, 2021

Anonymous456 said:
Why is it when I extract music (.wavs) from the DQ classic games, they don't play after opening them in a media player?

Are you sure they're typical RIFF-encoded WAVs? File extensions mean nothing at all.

Homebrew WIP nxdumptool - Nintendo Switch Dump Tool

Well-Known Member

Active Member

Well-Known Member

Active Member

Well-Known Member

Active Member

Well-Known Member

Well-Known Member

Well-Known Member

Finding my light.

Finding my light.

Well-Known Member

Well-Known Member

Finding my light.

Well-Known Member

Finding my light.

Well-Known Member

Finding my light.

Well-Known Member

Finding my light.

Similar threads

Popular threads in this forum