Hacking Question XCI Scrubber?

[ChiefChronoX]

I've searched but couldn't find anything regarding this, does someone know if there is a "XCI Scrubber" (like the good old WiiScrubber) available or in the works?

 

[MHDestination]

I've searched but couldn't find anything regarding this, does someone know if there is a "XCI Scrubber" (like the good old WiiScrubber) available or in the works?

Wondered the same... There's much unused space on the cartridges.

But we won't see some kind of "scrubber" before the actual release of an backuploader.

We can't be 100% sure that scrubbed images will run. There might be some safety measures
that check for free space/garbage data/total cartridge size ....

 

[MHDestination]

Same question
https://gbatemp.net/threads/trim-switch-backup.504742/

 

[SimonMKWii]

Literally impossible unless the nca contents are decrypted, then repacked into the xci.
You cannot compress encrypted contents.

 

[MHDestination]

Literally impossible unless the nca contents are decrypted, then repacked into the xci.
You cannot compress encrypted contents.

Where exactly is the unused space?

I thought xci files are just raw images of the cartridges. And those contain multiple nca files.
If that was true, shouldn't the free space consist of unused sectors next to the nca files and not actually inside them?

 

[Arck]

This is how you get the used space of a XCI:

Read value at 0x118, multiply this value by 0x200, and add 0x200.
You get the used space, then beyond this value you have unused data filled with FFFFFFF...

Here you go, now someone with a simple programming knowledge anyone can make a simple XCI trimmer

 

[MHDestination]

This is how you get the used space of a XCI:

Read value at 0x118, multiply this value by 0x200, and add 0x200.
You get the used space, then beyond this value you have unused data filled with FFFFFFF...

Here you go, now someone with a simple programming knowledge can make a simple XCI trimmer

What if they use some kind of padding?
The free space might be somewhere inbetween.

 

[Arck]

What if they use some kind of padding?
The free space might be somewhere inbetween.

that's the padding, remove it and you'll have your XCI trimmed, now idea if it'll work with Xecuter XCI loader

 

[MHDestination]

that's the padding, remove it and you'll have your XCI trimmed, now idea if it'll work with Xecuter XCI loader

But that way you'll remove free space at the end of the image.
By padding single files, there might be unused sectors somewhere between the files / or at the beginning.

Compressing the whole xci might be the better solution. That won't change the contents at all.
But might slightly impact performance (additional CPU and memory usage)

 

[Arck]

But that way you'll remove free space at the end of the image.
By padding single files, there might be unused sectors somewhere between the files / or at the beginning.

no, it's just full of FFFFFFF, nothing else.

Read switchbrew if you want more info.

 

[MHDestination]

no, it's just full of FFFFFFF, nothing else.

Read switchbrew if you want more info.

Okay, i've read the page. You're right the xci-files always end with FFFFF. That might be cut of.

But in addition to that, there might still be unused space inbetween. As a result of the partition and file offsets.

 

[Maximilious]

For now I have my xci files unpacked and ready to xfer to SD card. For long term storage I'm just keeping them packed in the original rar files.

Trimming would be nice for the SD card of course, hopefully someone makes one once the SX or CFW is out and they can test to see if it works.

 

[MHDestination]

I wrote a small tool to cut off the unused sectors at the end of xci files. Thanks to the idea of @Arck

Download:
https://github.com/Destiny1984/XCI-Cutter/releases

Before cutting, the tool checks if the sectors are really unused (filled with FF).
It refuses to cut, if it finds any data after the gamefiles.

I don't know if those xci-files would run in a backuploader.
But the process is reversable. The tool can be used to restore those sectors.


Hadn't much time to put this together.
It might crash now and then

EDIT / Small update:
Don't write to source-file in uncut-mode.
Copy to a destination-file instead (as it does in cut-mode)

 

[lostalice]

thanks

 

[ChiefChronoX]

I wrote a small tool to cut off the unused sectors at the end of xci files. Thanks to the idea of @Arck

Download:
https://drive.google.com/open?id=1e32CGB3pzFXvua7b-oeiau0syW18lgJn

Before cutting, the tool checks if the sectors are really unused (filled with FF).
It refuses to cut, if it finds any data after the gamefiles.

I don't know if those xci-files would run in a backuploader.
But the process is reversable. The tool can be used to restore those sectors.


Hadn't much time to put this together.
It might crash now and then

Danke! You might want to create a new thread for this.

edit:
I did a quick test with Cartoon Network Battle Crashers (because it's a small game when it's rarred).

Rarred: 411MB
Cut-XCI: 526MB

I'm still able to unpack the Cut-XCI with hactool just fine.

It seems there's still some progress to be made since it "missed out" on 115MB of unused space, but great first attempt

 

[MHDestination]

Danke! You might want to create a new thread for this.

edit:
I did a quick test with Cartoon Network Battle Crashers (because it's a small game when it's rarred).

Rarred: 411MB
Cut-XCI: 526MB

I'm still able to unpack the Cut-XCI with hactool just fine.

It seems there's still some progress to be made since it "missed out" on 115MB of unused space, but great first attempt

Thanks!

Yes, compression would still be better.
This app just cuts space at the end of the xci files. There might still be free sectors somewhere inbetween.

But there's a chance that these cut files would run in a backuploader.

 

[ChiefChronoX]

Thanks!

Yes, compression would still be better.
This app just cuts space at the end of the xci files. There might still be free sectors somewhere inbetween.

But there's a chance that these cut files would run in a backuploader.

Maybe these source files from WiiScrubber might help you to see how they did it back then?
https://github.com/jjgod/wiiscrubber-ng

 

[thequickbrownfox]

I wrote a small tool to cut off the unused sectors at the end of xci files. Thanks to the idea of @Arck

Download:
https://drive.google.com/open?id=1O6lbkio-I2PNPgJWQdc5_od7UiV14GlM

Before cutting, the tool checks if the sectors are really unused (filled with FF).
It refuses to cut, if it finds any data after the gamefiles.

I don't know if those xci-files would run in a backuploader.
But the process is reversable. The tool can be used to restore those sectors.


Hadn't much time to put this together.
It might crash now and then

EDIT / Small update:
Don't write to source-file in uncut-mode.
Copy to a destination-file instead (as it does in cut-mode

Man, you are incredible! Thank you!
Now it's up to TX to make their backup loader working with trimmed files.

 

[linuxares]

Man, you are incredible! Thank you!
Now it's up to TX to make their backup loader working with trimmed files.

It might work right out of the box.

 

[andijames]

That's really nice good work. Have we got a sticky thread for useful tools or do we need one whacking up?