Homebrew RELEASE JKSV (save manager) for Switch

[cucholix]

ok no worries i can wait for it to be fixed in the meantime i will use checkpoint its just i like the jksv for the bcat function some games require it like tetris 99

You no longer need bcat with latest Tetris 99 update, just inject the main save to unlock all event themes. https://gbatemp.net/threads/request...and-the-splatoon-theme-unlocked.545476/page-4

 

[froggy55]

You no longer need bcat with latest Tetris 99 update, just inject the main save to unlock all event themes. https://gbatemp.net/threads/request...and-the-splatoon-theme-unlocked.545476/page-4

dude this is awesome ive been wondering for a while if there was a save that unlocked all themes im missing a bunch on my stock save but this would be good for my emmumc thank you so much i will try to inject with checkpoint

 

[JK_]

So, here's how things stand as of today:

• Save backup and restore seem to be finally working fine for the rewrite code. It has passed both the Minecraft and New Pokemon Snap tests. That's a pretty good sign that it's gonna work for almost anything.
• Side note: The main issue was actually caused by the library I wrote. JKSV's rewrite uses a small wrapper I wrote to completely bypass libnx's fs_dev file for reading and writing to the SD card. I made a small mistake that caused some issues with minizip writing ZIP files and a small issue with the path wrapper class.
• Save backups now have a meta data file that contains more data than they actually need to fully backup and restore saves in the rewrite. The master branch used to just sort of guess, you could say. This can be used a little down the line to just import saves you don't have installed yet.
• I've implemented the information panel and corrected the play time calculations to finally fix that issue from when Nintendo changed it to nanoseconds.
• Boost mode is now enabled at startup instead of just using a hacked up version of Freebird's code to accomplish the same thing.
• SVI file format has been updated and changed. Rewrite will not be backwards compatible with original ones. It wasn't exactly the most used feature anyway.
• Right now I'm sorting through some stuff so that when you create and delete saves, the changes are reflected in the title selection view immediately. This is getting kind of tricky, but it's getting there. Trying not to cause data races and crashes.
• Google Drive code is basically done. It just needs to be integrated. Still trying to figure out a good way to handle it.

Also had some revelations lately. Finally coming to terms with the real reason behind my extreme perfectionism and feeling like I'm never really good at anything even when I am. Basically, rewrite code is usable if anyone is will to build it. It's not up to speed with the master branch yet. For those with issues with it taking so long though, you should be fine now.

 

[Commercial-Magician]

So, here's how things stand as of today:

• Save backup and restore seem to be finally working fine for the rewrite code. It has passed both the Minecraft and New Pokemon Snap tests. That's a pretty good sign that it's gonna work for almost anything.
• Side note: The main issue was actually caused by the library I wrote. JKSV's rewrite uses a small wrapper I wrote to completely bypass libnx's fs_dev file for reading and writing to the SD card. I made a small mistake that caused some issues with minizip writing ZIP files and a small issue with the path wrapper class.
• Save backups now have a meta data file that contains more data than they actually need to fully backup and restore saves in the rewrite. The master branch used to just sort of guess, you could say. This can be used a little down the line to just import saves you don't have installed yet.
• I've implemented the information panel and corrected the play time calculations to finally fix that issue from when Nintendo changed it to nanoseconds.
• Boost mode is now enabled at startup instead of just using a hacked up version of Freebird's code to accomplish the same thing.
• SVI file format has been updated and changed. Rewrite will not be backwards compatible with original ones. It wasn't exactly the most used feature anyway.
• Right now I'm sorting through some stuff so that when you create and delete saves, the changes are reflected in the title selection view immediately. This is getting kind of tricky, but it's getting there. Trying not to cause data races and crashes.
• Google Drive code is basically done. It just needs to be integrated. Still trying to figure out a good way to handle it.

Also had some revelations lately. Finally coming to terms with the real reason behind my extreme perfectionism and feeling like I'm never really good at anything even when I am. Basically, rewrite code is usable if anyone is will to build it. It's not up to speed with the master branch yet. For those with issues with it taking so long though, you should be fine now.

Hi what a coincidence! I was just playing around with the SVI export feature and tried to export a save file thats on my emuNand and create a new save file on my sysNand using that SVI I created. For some reason at least with the latest release version, when doing those steps, the save does import but when you check the save on Data Management, it shows the save file but the icon has a ? mark and app like for example Pokemon Home can't see the save. Is this something that I've done wrong or will this work better on the rewrite your working on for JKSV? Also thanks for your hard work!

 

[Deleted member 413168]

So, here's how things stand as of today:

• Save backup and restore seem to be finally working fine for the rewrite code. It has passed both the Minecraft and New Pokemon Snap tests. That's a pretty good sign that it's gonna work for almost anything.
• Side note: The main issue was actually caused by the library I wrote. JKSV's rewrite uses a small wrapper I wrote to completely bypass libnx's fs_dev file for reading and writing to the SD card. I made a small mistake that caused some issues with minizip writing ZIP files and a small issue with the path wrapper class.
• Save backups now have a meta data file that contains more data than they actually need to fully backup and restore saves in the rewrite. The master branch used to just sort of guess, you could say. This can be used a little down the line to just import saves you don't have installed yet.
• I've implemented the information panel and corrected the play time calculations to finally fix that issue from when Nintendo changed it to nanoseconds.
• Boost mode is now enabled at startup instead of just using a hacked up version of Freebird's code to accomplish the same thing.
• SVI file format has been updated and changed. Rewrite will not be backwards compatible with original ones. It wasn't exactly the most used feature anyway.
• Right now I'm sorting through some stuff so that when you create and delete saves, the changes are reflected in the title selection view immediately. This is getting kind of tricky, but it's getting there. Trying not to cause data races and crashes.
• Google Drive code is basically done. It just needs to be integrated. Still trying to figure out a good way to handle it.

Also had some revelations lately. Finally coming to terms with the real reason behind my extreme perfectionism and feeling like I'm never really good at anything even when I am. Basically, rewrite code is usable if anyone is will to build it. It's not up to speed with the master branch yet. For those with issues with it taking so long though, you should be fine now.

What is the format that you went with for meta data? I also include meta data in backups that sphaira creates, i'd like to keep compatibility between jksv and sphaira.

 

[JK_]

@TotalJustice Here's what I went with. If you'd rather do it some other way, just let me know. You said something about Skyrim a while back and that's what I did shortly after. If there's something I missed or you wanna change something, let me know.

Edit: Shit. I did forget something. The save index. I'll get to that ASAP.

 

[Blythe93]

Probably a silly question but restoring backups created on an older version will work on the latest version of the rewrite? I'm still using the latest version before the rewrite so I thought I might as well ask before I do something bad. ^^"

 

[Deleted member 413168]

@TotalJustice Here's what I went with. If you'd rather do it some other way, just let me know. You said something about Skyrim a while back and that's what I did shortly after. If there's something I missed or you wanna change something, let me know.

Edit: Shit. I did forget something. The save index. I'll get to that ASAP.

Thanks! it's not too far off with what i went with here https://github.com/ITotalJustice/sp...sphaira/source/ui/menus/save_menu.cpp#L40-L60.

I didn't really need to store all the data in that struct, the main 2 things were the journal size and data size. But the extra data may be useful for a save viewer which i thought i may implement later on, so i figured that i might as well dump everything.

Would you rather stick with your struct? I can replace mine with what you use, and keep mine around for backwards compatibility with old saves created with sphaira.

 

[JK_]

@Blythe93 Yes, it will work. It will just fail to find the meta file and continue like normal for the most part. I wasn't aware of games like Skyrim back then. I played Skyrim a long time ago and had no urge to replay it on Switch when it came out. It's not like Chrono Trigger where I bust out the SNES every year and delicately put my ancient cartridge in it to make sure no harms comes to it.

@TotalJustice Let me check some stuff out quick. I'll get back to ya later today. After looking at it closer, I think this is actually better, even if it has some extra stuff you don't really need:

I just took a look at the structs you need to pass to create save data. I actually remember soft bricking my Switch by creating BCAT without the correct owner ID when I first added it. It needs to be the same ID as the BCAT module IIRC. Unfortunately, I don't have any BCAT data on my Switch at the moment to see. Obviously, 99% of the time that application and owner ID are the same. That should more than cover it and it only weighs in at 65 bytes. I don't see the point in saving the other stuff, like the account ID etc, because I'm not locking saves to a specific account. I'm always open to hear what you have to say though. I don't even see the one struct used in libnx at all. It's just defined. Never used. Sometimes I see shit in there that really makes me wonder. Anyways, I'll check in now and then today, or whenever.

 

[Deleted member 413168]

@Blythe93 Yes, it will work. It will just fail to find the meta file and continue like normal for the most part. I wasn't aware of games like Skyrim back then. I played Skyrim a long time ago and had no urge to replay it on Switch when it came out. It's not like Chrono Trigger where I bust out the SNES every year and delicately put my ancient cartridge in it to make sure no harms comes to it.

@TotalJustice Let me check some stuff out quick. I'll get back to ya later today. After looking at it closer, I think this is actually better, even if it has some extra stuff you don't really need:
View attachment 511305
I just took a look at the structs you need to pass to create save data. I actually remember soft bricking my Switch by creating BCAT without the correct owner ID when I first added it. It needs to be the same ID as the BCAT module IIRC. Unfortunately, I don't have any BCAT data on my Switch at the moment to see. Obviously, 99% of the time that application and owner ID are the same. That should more than cover it and it only weighs in at 65 bytes. I don't see the point in saving the other stuff, like the account ID etc, because I'm not locking saves to a specific account. I'm always open to hear what you have to say though. I don't even see the one struct used in libnx at all. It's just defined. Never used. Sometimes I see shit in there that really makes me wonder. Anyways, I'll check in now and then today, or whenever.

I am basically saving everything thats found in `FsSaveDataInfo` and `FsSaveDataExtraData` aside from anything that's padding or unused. Yeah most of it is not needed, but like i said, it was for potentially having a save info viewer. Things like the account ID + UID can be used to show the account that it's tied to. I figured that it's better to save *all* the meta data, can't go wrong then

I think the only missing thing from my meta dump compared to yours if the journal/save size max. I assume that's from the nacp? Are you storing that for system saves? Or have you seen those values change in an application update?

 

[JK_]

I think the only missing thing from my meta dump compared to yours if the journal/save size max. I assume that's from the nacp? Are you storing that for system saves? Or have you seen those values change in an application update?

I just saved them because I remember I was debugging and recording them way back and noticed it. Yeah, they're in the NACP. Here's Minecraft and then New Pokemon Snap, for example. Keep in mind, I'm assuming JKSV's fetching functions are working correctly here:

Keep in mind, the New Snap one seems a little off, but that could be my own mistake. I also extended them both to 3GB to make sure that was working correctly. I just sort of made the assumption that if there was a upper bounds to the journal size, that's what was used? I'm not sure if libnx has a way to pull that and digging through the containers to find out doesn't exactly sound like fun if I had to be honest.

One thing I would like to note: I do think saving the file as .nx_save_meta.bin is a good idea. That way they'll both be compatible with each other. Right now I have it as .jksv_save_meta.bin. I'll change it asap.

 

[Deleted member 413168]

I just saved them because I remember I was debugging and recording them way back and noticed it. Yeah, they're in the NACP. Here's Minecraft and then New Pokemon Snap, for example. Keep in mind, I'm assuming JKSV's fetching functions are working correctly here:
View attachment 511379 View attachment 511380
Keep in mind, the New Snap one seems a little off, but that could be my own mistake. I also extended them both to 3GB to make sure that was working correctly. I just sort of made the assumption that if there was a upper bounds to the journal size, that's what was used? I'm not sure if libnx has a way to pull that and digging through the containers to find out doesn't exactly sound like fun if I had to be honest.

One thing I would like to note: I do think saving the file as .nx_save_meta.bin is a good idea. That way they'll both be compatible with each other. Right now I have it as .jksv_save_meta.bin. I'll change it asap.

I think the journal / save size max is for the first time a save is created by that title, like when you first launch it, qlaunch creates a save file for it. You can test this behavior by creating a forwarder and enabling saving inside the nacp.

Basically its just the default value. Games can override them (if needed) when extending the save size, such as skyrim and minecraft.

For libnx, the only save structs are in fs.h and begin with `FsSaveData` if you wanted to look around. There is the function which allows you to set the extra data (like, the whole struct). I assume that exists for receiving save files from another console and/or cloud. It also keeps the timestamp in tact, which is cool. I've been debating using this when restoring saves. The idea is to extends the fs as normal, restore the save as normal, then read the extra data for the existing save to get the account id / uid and then overwrite the values saved in .nx_save_meta.bin and restore that extra data. Only benefit of doing it this way is keeping the timestamp the same, which may be a nice feature for those using cloud saves as well.


Also, when restoring a save, i extract everything from the zip to the save fs (after creating a backup). Do you first nuke the save fs and then restore each file? Because it's just occurred to me that it's possible for stale files that exist in save fs and not in the backup to get left inside. ie, when restoring a old save over a new one in botw will have all the album artwork left over from the newer save (in sphaira, well i think it does anyway, only just thought of this when typing the above )

 

[JK_]

Do you first nuke the save fs and then restore each file? Because it's just occurred to me that it's possible for stale files that exist in save fs and not in the backup to get left inside.

Yes. That's how I always did it. Even on 3DS. My process was always wipe the root, commit, and restore and commit as needed.

For libnx, the only save structs are in fs.h and begin with `FsSaveData` if you wanted to look around. There is the function which allows you to set the extra data (like, the whole struct). I assume that exists for receiving save files from another console and/or cloud. It also keeps the timestamp in tact, which is cool. I've been debating using this when restoring saves. The idea is to extends the fs as normal, restore the save as normal, then read the extra data for the existing save to get the account id / uid and then overwrite the values saved in .nx_save_meta.bin and restore that extra data. Only benefit of doing it this way is keeping the timestamp the same, which may be a nice feature for those using cloud saves as well.

Earlier, I looked at fs.h, highlighted the struct, and ctrl + f and only found the struct definition with it. I didn't notice the functions until just now. Why have the struct defined but have the functions take a void pointer? Yeah, sure it gets the job done, but... nevermind. We cannot question our devkitpro and switchbrew overlords. I forgot about that.

 

[Deleted member 413168]

Yes. That's how I always did it. Even on 3DS. My process was always wipe the root, commit, and restore and commit as needed.


Earlier, I looked at fs.h, highlighted the struct, and ctrl + f and only found the struct definition with it. I didn't notice the functions until just now. Why have the struct defined but have the functions take a void pointer? Yeah, sure it gets the job done, but... nevermind. We cannot question our devkitpro and switchbrew overlords. I forgot about that.

Ah okay, thank you. I'll fix that bug in sphaira. Yes i find it strange / a little inconstant in some libnx functions to accept a void* + a size of the data, even though we know what the struct is. My guess is that the function was implemented first and then the structs were added after, but the functions were not updated. The same can be said for a few of the fs gamecard functions, they accept a void* but we know what the structs are :/

 

[Commercial-Magician]

Hi what a coincidence! I was just playing around with the SVI export feature and tried to export a save file thats on my emuNand and create a new save file on my sysNand using that SVI I created. For some reason at least with the latest release version, when doing those steps, the save does import but when you check the save on Data Management, it shows the save file but the icon has a ? mark and app like for example Pokemon Home can't see the save. Is this something that I've done wrong or will this work better on the rewrite your working on for JKSV? Also thanks for your hard work!

Am I the only one with this problem? Appreciate any feedback

 

[JK_]

Am I the only one with this problem? Appreciate any feedback

Are you trying to import a Pokemon save and Home doesn't see it? Am I getting this right? The ? icon in data management is because the system doesn't actually have the game installed. JKSV exports the icon with SVI and uses that to fill it in when inside it. I know it worked to unlock those Pokemon in the games themselves. If you want, I can throw up a build of the rewrite code to see if that works better later. I'm kind of hesitant to until we get this save meta situation handled though. Any save exported with it might become invalid later, kind of?

 

[Commercial-Magician]

Are you trying to import a Pokemon save and Home doesn't see it? Am I getting this right? The ? icon in data management is because the system doesn't actually have the game installed. JKSV exports the icon with SVI and uses that to fill it in when inside it. I know it worked to unlock those Pokemon in the games themselves. If you want, I can throw up a build of the rewrite code to see if that works better later. I'm kind of hesitant to until we get this save meta situation handled though. Any save exported with it might become invalid later, kind of?

Yeah thats exactly the test im trying to do. I also build the lastest version in your git and now when I use the export SVI function and i relaunch JKSV the app crashes. Guessing its to do with the rewrites your working on currently.

 

[JK_]

Yeah thats exactly the test im trying to do. I also build the lastest version in your git and now when I use the export SVI function and i relaunch JKSV the app crashes. Guessing its to do with the rewrites your working on currently.

It's possible the rewrite is trying to read the files from the master branch and that's what's causing the crash. The current code on git doesn't have anyway to differentiate the two. That's what I was saying earlier about them not being compatible.

@TotalJustice
After going through both structs, this is what I ended with:

It comes in at 85 bytes. The rest of the structs are just useless padding. Let me know when you get a chance. The magic doesn't need to be JKSV either. NXSM? Whatever works.

Edit: I also wouldn't be bothered dropping the system save ID from it either. I don't like touching those in any kind of significant way. Deleting or creating them.

Edit 2: I just caught that I set the save data index to uint8_t instead of uint16_t. Fixed that.

 

[Commercial-Magician]

It's possible the rewrite is trying to read the files from the master branch and that's what's causing the crash. The current code on git doesn't have anyway to differentiate the two. That's what I was saying earlier about them not being compatible.

@TotalJustice
After going through both structs, this is what I ended with:
View attachment 511415
It comes in at 85 bytes. The rest of the structs are just useless padding. Let me know when you get a chance. The magic doesn't need to be JKSV either. NXSM? Whatever works.

Edit: I also wouldn't be bothered dropping the system save ID from it either. I don't like touching those in any kind of significant way. Deleting or creating them.

Ah gotchu! Well Im in no rush so I'll wait for the rewrite to finish! But just want to know, does that feature suppose to work on the latest version or is my problem common? So just to be clear, emuNand has a game installed were sysNad doesn't. So when I export SVI of a game thats not on my sysNand and then try to import that SVI on the sysNand, the ? icon appears. Is this not the intended feature? Does the save file have to already been created for me to import any save, or is it possible to create a save file of a game thats not in the sysNand if i have it already on the emuNand?

 

[YeBen]

Hi!
I want to backup saves in emunand and restore them in sysnand.
How safe is it to do this via JKSV? Does anyone have any experience? I don't want to be banned.