Hacking ndstrim 1.0

recover

Well-Known Member
OP
Member
Joined
Aug 6, 2007
Messages
161
Trophies
1
Website
Visit site
XP
252
Country
Update:
Azimuth have done some very good work on a GUI (ruby) and archive (7z/RAR) support.
biggrin.gif

If you want the latest stuff, I suggest you check out Azimuth's ndstrim thread!
Big hug to Azimuth for his improvements
smile.gif


History:
I want to share with you all my own nds rom trimmer.
I got my R4 a while back and needed a trimmer, I tried a few, but most of them were slow and didn't run on linux.
Anyway, after I had done some research I found out that there were two ways rom trimmers worked, let us call them 1st generation and 2nd generation.
1st generation just trimmed the roms from the end of the file until it found something else than the hex data 00 or FF, this often caused problems with wifi and download play.
2nd generation read the size of the actual rom which were written down at 4 bytes in the nds header, then they trimmed the rom to that size+136 bytes (for wifi data).

I started to make my own trimmer and I coded it to be fast, simple and reliable.
I finally came up with a really good trimmer which I'm now deciding to release as open source.
My trimmer is a 2nd generation trimmer, so it should be able to handle all the commercial roms without problems with wifi or download play.

Download:
In the file ndstrim-1.0.tar.bz2 (you can open it with winrar/7-zip) you'll find the source, binaries for windows and linux as well as bat/sh-files to make it easier to trim many roms at once.
If you are planning to use ndstrim on windows, you only need the files "ndstrim.exe" and "trim.bat".
If you are planning to use ndstrim on linux, you only need the files "ndstrim" and "trim.sh" (make sure they are both executable - chmod +x ndstrim trim.sh).
Feel free to host this on any site you want, it's open source after all!

How to make it all work:
To trim roms, put them in the same directory as the files you unpacked and run trim.bat (or trim.sh if you are on linux), a window will popup and it will go through all the files ending in .nds in the same directory, trimming each one (and also telling you how much space you "saved" by trimming the roms).
The trimmed roms will appear in a subdirectory called "trimmed" with their original filename.
Windows users: Do not drag roms on ndstrim.exe, it won't work!

I compiled everything on a 32-bit computer (Windows XP + MinGW and Ubuntu Linux 7.04+gcc)
Even though it was compiled on a 32-bit computer it should be able to run fine on 64-bit computer (but I have not tested this).

Here's a screenshot of how it could look like (I did this on linux so the window might not look the same for you):
ndstrim-1.0.png

Running ndstrim without specifying output will show you how much space you will save when trimming the file (without the file being trimmed).

Mac users:
Update: n45800 was so kind to compile ndstrim 1.0 for Mac (Universal binary). You can check his post or download it. Thanks n45800!
I have not tested the program on a Mac since I don't own one (nor have any experience with them), so there are no Mac binaries available from me.
Even so, ndstrim should work perfectly on Macs, it should only need to be compiled to work (I can't help you with this).
Let me know how it works on your Mac.

Source code:
Here are the source for those who are interested, the code should be very easy to understand even to novice programmers (ndstrim.c):
CODE/* ndstrim by [email protected]
Trims NDS roms fast and reliable.
ROM size is available in four bytes at 0x80-0x83.
Wifi data is stored on 136 bytes after ROM data.
Filesize is checked to be at least 0x200 bytes to make sure it contains a DS cartridge header.
Filesize is then checked to be at least the rom size+wifi to avoid errors.
Source code licensed under GNU GPL version 2 or later.

Sources:
http://nocash.emubase.de/gbatek.htm
http://forums.ds-xtreme.com/showthread.php?t=1964
http://gbatemp.net/index.php?showtopic=44022
*/

#include
#include

//#define DEBUG
#define BUFFER 1000000 //1 MB buffer size

//The rom size is located in four bytes at 0x80-0x83
struct uint32 {
unsigned data:32; //4*8
};

int main(int argc, char *argv[]) {
if (argc < 2) {
fprintf(stderr,"%s: Too few arguments.\n",argv[0]);
fprintf(stderr,"%s: Usage: %s [output]\n",argv[0],argv[0]);
exit(1);
}

//Debug?
char debug=0;
#ifdef DEBUG
debug=1;
#endif

//Open input
#ifdef DEBUG
printf("%s: Opening input file '%s'.\n",argv[0],argv[1]);
#endif
FILE *input;
if ((input=fopen(argv[1],"rb")) == NULL) {
fprintf(stderr,"%s: fopen() failed in file %s, line %d.\n",argv[0],__FILE__,__LINE__);
exit(1);
}

//Get input filesize
fseek(input,0,SEEK_END);
unsigned int filesize=ftell(input);
#ifdef DEBUG
printf("%s: Filesize: %d bytes.\n",argv[0],filesize);
#endif

//Check if file is big enough to contain a DS cartridge header
if (filesize = 3) {
//Open output
#ifdef DEBUG
printf("%s: Opening output file '%s'.\n",argv[0],argv[2]);
#endif
FILE *output;
if ((output=fopen(argv[2],"wb")) == NULL) {
fprintf(stderr,"%s: fopen() failed in file %s, line %d.\n",argv[0],__FILE__,__LINE__);
exit(1);
}

//Reset input pos
rewind(input);

//Start copying
#ifdef DEBUG
printf("%s: Copying data.\n",argv[0]);
#endif
char buffer[BUFFER];
unsigned int fpos=0;
unsigned int tocopy=BUFFER;
while (fpos < newsize) {
if (fpos+BUFFER > newsize) {
tocopy=newsize-fpos;
}
fread(&buffer,tocopy,1,input);
fwrite(&buffer,tocopy,1,output);
fpos+=tocopy;
}

//Done
printf("%s: Trimmed '%s' to %d bytes (saved %.2f MB).\n",argv[0],argv[1],newsize,(filesize-newsize)/(float)1000000);

//Close output
#ifdef DEBUG
printf("%s: Closing output.\n",argv[0]);
#endif
if (fclose(output) == EOF) {
fprintf(stderr,"%s: fclose() failed in file %s, line %d.\n",argv[0],__FILE__,__LINE__);
exit(1);
}
}

//Close input
#ifdef DEBUG
printf("%s: Closing input.\n",argv[0]);
#endif
if (fclose(input) == EOF) {
fprintf(stderr,"%s: fclose() failed in file %s, line %d.\n",argv[0],__FILE__,__LINE__);
exit(1);
}

return 0;
}


Help me help you:
This is a very simple trimmer and whilst it is very fast compared to other trimmers I have tried, of course I want to make it better...
I'm not very skilled in coding GUI so I don't think I'll be doing that anytime soon.
I tried to optimize read/write as much as I could, but I'm sure it's possible to make it even faster, if anyone have any experience with this, I'll gladly listen (currently: the program uses a 1 MB buffer which means it reads and writes in 1 MB chunks).

Any suggestions are welcome!
 

party

Member
Newcomer
Joined
Oct 17, 2006
Messages
19
Trophies
0
XP
50
Country
Gambia, The
Nice one!
smile.gif


I just had a very quick look at it. Seems to work (Win XP SP2). Trimmed ~17.38MB off hotel dusk usa (haven't tested the resulting rom).

A few things that came to my mind:

-The "not" shouldn't be there, right?: "My trimmer is a 2nd generation trimmer, so it should not be able to handle all the commercial roms without problems with wifi or download play.

-Have a /? -? /help or -help parameter instead of an error message.

-Make the filename the only required parameter (and others optional): "ndstrim file.nds" would output file_trimmed.nds to the directory where file.nds is stored.
I'd love that because then I could put ndstrim in my "send to" folder, and trim roms with rightclick -> send to -> ndstrim. AFAIK that send to thingy only works if the target program accepts the filename as the first and only parameter. (hope this all makes sense)

-Whatever you change in future versions, please make sure that the bare executable is fully usable without the batch/shellscript. IMHO all the functions that are now outsourced to the batch/shellscript should go into the main executable anyways. But I guess you did it this way to have only one sourcecode for linux and windows and let the script take care of all the os-specific stuff, right?


Thanks for sharing!
 

recover

Well-Known Member
OP
Member
Joined
Aug 6, 2007
Messages
161
Trophies
1
Website
Visit site
XP
252
Country
A few things that came to my mind:

-The "not" shouldn't be there, right?: "My trimmer is a 2nd generation trimmer, so it should not be able to handle all the commercial roms without problems with wifi or download play.
Whoops, that might have been the biggest typo I've ever made in my whole life, fixed it.
tongue.gif


-Have a /? -? /help or -help parameter instead of an error message.
Perhaps, but I might just add the usage message on every error message instead...

-Make the filename the only required parameter (and others optional): "ndstrim file.nds" would output file_trimmed.nds to the directory where file.nds is stored.
I'd love that because then I could put ndstrim in my "send to" folder, and trim roms with rightclick -> send to -> ndstrim. AFAIK that send to thingy only works if the target program accepts the filename as the first and only parameter. (hope this all makes sense)
Yea, that should be possible.
I'm still not very good at command line processing, but I guess it would be possible to make a -i argument to make it "investigate" the rom file and post the info it does now and make outputting a trimmed rom with a prefix the default

-Whatever you change in future versions, please make sure that the bare executable is fully usable without the batch/shellscript. IMHO all the functions that are now outsourced to the batch/shellscript should go into the main executable anyways. But I guess you did it this way to have only one sourcecode for linux and windows and let the script take care of all the os-specific stuff, right?
Yes, I agree, it would be nice to be able to do these things without the shellscripts, but this is a first release.
I want to make it possible to specify multiple infiles to the executable (e.g. by dragging), but I find it hard to come up with a nice way to decide if files should be outputted with a prefix or in a separate directory (since I guess there are a lot of people who want both).

Thanks for sharing!
No problems!
smile.gif



I would be happy to add a simple GUI in dotnet, if you think it would be helpful.
Yes, that might be helpful, feel free to help and post suggestions
smile.gif


Thanks for your responses!
biggrin.gif
 

scandal_uk

Not Really There
Member
Joined
Oct 3, 2005
Messages
322
Trophies
0
Location
UK
XP
580
Country
United Kingdom
Well when I get the time I will create a proper one for you.

For the meantime, here is a quick & dirty proof-of-concept version (which I haven't tested as I don't have any ROMs on the PC I'm working on);

http://scandal.uk.googlepages.com/Trim-concept.rar

To open it, double-click Trim.exe in the BIN folder (no other files are required), or you can drag & drop many files onto the program for batch trimming - none of the ouputted trimmed roms will overwrite existing files (it does rename them though).

I will improve the look and add a list to batch convert roms once I've tested it (later today perhaps).

I have included ALL of the source for anyone's use and built it in .NET 1.0 so that it's compatable with all versions of the .NET Framework.

The code is heavily-commented VB so that even non-programmers can see what it's doing and how it's doing it.

Hope that helps (and it's only 16KB unzipped)!
 

recover

Well-Known Member
OP
Member
Joined
Aug 6, 2007
Messages
161
Trophies
1
Website
Visit site
XP
252
Country
Ok, I'll look into it later today (I'm in linux right now).
I'm very grateful that you want to help me, even more that you are releasing the source as well
biggrin.gif
 

scandal_uk

Not Really There
Member
Joined
Oct 3, 2005
Messages
322
Trophies
0
Location
UK
XP
580
Country
United Kingdom
I'm happy to help, and if you look closely you'll notice that the source is all yours - I just changed it to use .NET Framework objects (for file reading etc).

It's through contributions such as yours that the scene continues to be good.

I needed this one as I use G6 Lite, but my new G6 doesn't use a manager so I wanted a clean, quick trimmer.

It certainly seems to work with the 5-6 ROMs I just tried! At 16k it runs quick too.
 

recover

Well-Known Member
OP
Member
Joined
Aug 6, 2007
Messages
161
Trophies
1
Website
Visit site
XP
252
Country
I've looked into the program and the source and everything looks nice
smile.gif

I've also become convinced that automatically appending a suffix to the output file is the way to go, therefore enabling batch-processing by e.g. dragging nds roms on the exe, it's also a very good way to see if you have already trimmed the rom.
I'll probably code a bit over the weekend and apply these changes and release ndstrim 1.1.
biggrin.gif

There are still some issues to work out like writing the trimmed rom in a separate directory so you can easier copy the roms after they have been trimmed, perhaps it would be a good idea to setup two basic directories such as "untrimmed" and "trimmed" which ndstrim automagically go after when it's run?
Any thoughts?
smile.gif
 

scandal_uk

Not Really There
Member
Joined
Oct 3, 2005
Messages
322
Trophies
0
Location
UK
XP
580
Country
United Kingdom
Well I quickly added some methods for you to easily trim a few ROMs. It's a little bit more of a GUI now.
smile.gif


I even added a little scissors icon...
biggrin.gif


The source can be compiled with Visual Studio 2002 or greater (any .NET version, basically).

It's a little bigger now but I think it runs OK...


http://scandal.uk.googlepages.com/NDSTrim-GUI.rar

Edit:
Yes, I think it would be best if you could choose an output directory - an input one might be little unnecessary though. In this version of the GUI you can queue up multiple roms.

It would be a simple matter to enable opening Zip files for roms, however, that would be a good addition too.
 

recover

Well-Known Member
OP
Member
Joined
Aug 6, 2007
Messages
161
Trophies
1
Website
Visit site
XP
252
Country

madlobster

Active Member
Newcomer
Joined
Oct 21, 2006
Messages
37
Trophies
0
Website
Visit site
XP
69
Country
United States
Some ROMS don't trim right. 0040, 0132, 0192, 0318, and 0357 in particular.
Code:
D:\test>"\nds\To trim\ndstrim.exe" "0040 - Hanjuku Eiyuu DS - Egg Monster Hero (JP).nds"
\nds\To trim\ndstrim.exe: ROM size: 0 bytes.
\nds\To trim\ndstrim.exe: ROM size + wifi: 136 bytes.
\nds\To trim\ndstrim.exe: Can save: 33554296 bytes.

It appears that bytes 0x80-0x83 are 00 00 00 00.
blink.gif
 

flai

Androgynous Apparently :)
Member
Joined
Sep 30, 2006
Messages
850
Trophies
0
Website
Visit site
XP
72
Country
It looks very nice but I think you should perhaps ask a coder around here to help with a GUI, it would make it more accessible to people not in the know, great piece of software though!
 

scandal_uk

Not Really There
Member
Joined
Oct 3, 2005
Messages
322
Trophies
0
Location
UK
XP
580
Country
United Kingdom
Some ROMS don't trim right. 0040, 0132, 0192, 0318, and 0357 in particular.
Code:
D:\test>"\nds\To trim\ndstrim.exe" "0040 - Hanjuku Eiyuu DS - Egg Monster Hero (JP).nds"
\nds\To trim\ndstrim.exe: ROM size: 0 bytes.
\nds\To trim\ndstrim.exe: ROM size + wifi: 136 bytes.
\nds\To trim\ndstrim.exe: Can save: 33554296 bytes.

It appears that bytes 0x80-0x83 are 00 00 00 00.Â
blink.gif



Yes, this can be a problem with some games, it needs looking into so I'll see what I can find out. I also have a feeling that the added 136 bytes should be 138 bytes....

It looks very nice but I think you should perhaps ask a coder around here to help with a GUI, it would make it more accessible to people not in the know, great piece of software though!

I tried !! :'(

I know it's just .Net but that's to make the code more accessible for people who want to re-use it.
 

recover

Well-Known Member
OP
Member
Joined
Aug 6, 2007
Messages
161
Trophies
1
Website
Visit site
XP
252
Country
Some ROMS don't trim right. 0040, 0132, 0192, 0318, and 0357 in particular.
Code:
D:\test>"\nds\To trim\ndstrim.exe" "0040 - Hanjuku Eiyuu DS - Egg Monster Hero (JP).nds"
\nds\To trim\ndstrim.exe: ROM size: 0 bytes.
\nds\To trim\ndstrim.exe: ROM size + wifi: 136 bytes.
\nds\To trim\ndstrim.exe: Can save: 33554296 bytes.

It appears that bytes 0x80-0x83 are 00 00 00 00.Â
blink.gif


That's odd, I'll take a deeper look into it, thanks for the report
smile.gif



Here is a pre-compiled mac version (CLI, Universal Binary).
I'll see if I can whip up a quick Cocoa GUI later today.
And another small bug report. You should probably have a test for if the file actually exists before you try doing anything, otherwise it bombs with the error "./ndstrim: fopen() failed in file ndstrim.c, line 45.".
Thanks, the mac binaries should be useful to some people.

And with that file check
tongue.gif

You see, there are no standard way to check if files exist, so either it fails when trying to open the file (that probably means that the file doesn't exist, or maybe the user doesn't have the permissions needed).
But perhaps I should write something else in that error message, like "(file doesn't exist?)" or something. =)
 

BrianTokyo

NDSTokyoTrim Dev.
Member
Joined
Jun 28, 2007
Messages
658
Trophies
1
Website
www.eden.fm
XP
340
Country
Example:

Using ROM: 0938 - Theme Park (US)(M6).nds

Your trimmer left it at 29,040,336 bytes

----------------------------------------------
It should be:

0938 - Theme Park (US)(M6).nds: 4514232
Processing Finished.

Trimming took 0.05 seconds.

Size: 29,040,200 bytes
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
  • No one is chatting at the moment.
    Sonic Angel Knight @ Sonic Angel Knight: DAYTONAAAAAAAA!!!!!!!!!!