ndstrim 1.0

Discussion in 'NDS - Flashcarts and Accessories' started by recover, Sep 2, 2007.

Sep 2, 2007

ndstrim 1.0 by recover at 7:36 PM (10,974 Views / 0 Likes) 57 replies

  1. recover
    OP

    Member recover GBAtemp Regular

    Joined:
    Aug 6, 2007
    Messages:
    161
    Country:
    Sweden
    Update:
    Azimuth have done some very good work on a GUI (ruby) and archive (7z/RAR) support. [​IMG]
    If you want the latest stuff, I suggest you check out Azimuth's ndstrim thread!
    Big hug to Azimuth for his improvements [​IMG]

    History:
    I want to share with you all my own nds rom trimmer.
    I got my R4 a while back and needed a trimmer, I tried a few, but most of them were slow and didn't run on linux.
    Anyway, after I had done some research I found out that there were two ways rom trimmers worked, let us call them 1st generation and 2nd generation.
    1st generation just trimmed the roms from the end of the file until it found something else than the hex data 00 or FF, this often caused problems with wifi and download play.
    2nd generation read the size of the actual rom which were written down at 4 bytes in the nds header, then they trimmed the rom to that size+136 bytes (for wifi data).

    I started to make my own trimmer and I coded it to be fast, simple and reliable.
    I finally came up with a really good trimmer which I'm now deciding to release as open source.
    My trimmer is a 2nd generation trimmer, so it should be able to handle all the commercial roms without problems with wifi or download play.

    Download:
    In the file ndstrim-1.0.tar.bz2 (you can open it with winrar/7-zip) you'll find the source, binaries for windows and linux as well as bat/sh-files to make it easier to trim many roms at once.
    If you are planning to use ndstrim on windows, you only need the files "ndstrim.exe" and "trim.bat".
    If you are planning to use ndstrim on linux, you only need the files "ndstrim" and "trim.sh" (make sure they are both executable - chmod +x ndstrim trim.sh).
    Feel free to host this on any site you want, it's open source after all!

    How to make it all work:
    To trim roms, put them in the same directory as the files you unpacked and run trim.bat (or trim.sh if you are on linux), a window will popup and it will go through all the files ending in .nds in the same directory, trimming each one (and also telling you how much space you "saved" by trimming the roms).
    The trimmed roms will appear in a subdirectory called "trimmed" with their original filename.
    Windows users: Do not drag roms on ndstrim.exe, it won't work!

    I compiled everything on a 32-bit computer (Windows XP + MinGW and Ubuntu Linux 7.04+gcc)
    Even though it was compiled on a 32-bit computer it should be able to run fine on 64-bit computer (but I have not tested this).

    Here's a screenshot of how it could look like (I did this on linux so the window might not look the same for you):
    [​IMG]
    Running ndstrim without specifying output will show you how much space you will save when trimming the file (without the file being trimmed).

    Mac users:
    Update: n45800 was so kind to compile ndstrim 1.0 for Mac (Universal binary). You can check his post or download it. Thanks n45800!
    I have not tested the program on a Mac since I don't own one (nor have any experience with them), so there are no Mac binaries available from me.
    Even so, ndstrim should work perfectly on Macs, it should only need to be compiled to work (I can't help you with this).
    Let me know how it works on your Mac.

    Source code:
    Here are the source for those who are interested, the code should be very easy to understand even to novice programmers (ndstrim.c):
    CODE/* ndstrim by recover89@gmail.com
    Trims NDS roms fast and reliable.
    ROM size is available in four bytes at 0x80-0x83.
    Wifi data is stored on 136 bytes after ROM data.
    Filesize is checked to be at least 0x200 bytes to make sure it contains a DS cartridge header.
    Filesize is then checked to be at least the rom size+wifi to avoid errors.
    Source code licensed under GNU GPL version 2 or later.

    Sources:
    http://nocash.emubase.de/gbatek.htm
    http://forums.ds-xtreme.com/showthread.php?t=1964
    http://gbatemp.net/index.php?showtopic=44022
    */

    #include
    #include

    //#define DEBUG
    #define BUFFER 1000000 //1 MB buffer size

    //The rom size is located in four bytes at 0x80-0x83
    struct uint32 {
    unsigned data:32; //4*8
    };

    int main(int argc, char *argv[]) {
    if (argc < 2) {
    fprintf(stderr,"%s: Too few arguments.\n",argv[0]);
    fprintf(stderr,"%s: Usage: %s [output]\n",argv[0],argv[0]);
    exit(1);
    }

    //Debug?
    char debug=0;
    #ifdef DEBUG
    debug=1;
    #endif

    //Open input
    #ifdef DEBUG
    printf("%s: Opening input file '%s'.\n",argv[0],argv[1]);
    #endif
    FILE *input;
    if ((input=fopen(argv[1],"rb")) == NULL) {
    fprintf(stderr,"%s: fopen() failed in file %s, line %d.\n",argv[0],__FILE__,__LINE__);
    exit(1);
    }

    //Get input filesize
    fseek(input,0,SEEK_END);
    unsigned int filesize=ftell(input);
    #ifdef DEBUG
    printf("%s: Filesize: %d bytes.\n",argv[0],filesize);
    #endif

    //Check if file is big enough to contain a DS cartridge header
    if (filesize = 3) {
    //Open output
    #ifdef DEBUG
    printf("%s: Opening output file '%s'.\n",argv[0],argv[2]);
    #endif
    FILE *output;
    if ((output=fopen(argv[2],"wb")) == NULL) {
    fprintf(stderr,"%s: fopen() failed in file %s, line %d.\n",argv[0],__FILE__,__LINE__);
    exit(1);
    }

    //Reset input pos
    rewind(input);

    //Start copying
    #ifdef DEBUG
    printf("%s: Copying data.\n",argv[0]);
    #endif
    char buffer[BUFFER];
    unsigned int fpos=0;
    unsigned int tocopy=BUFFER;
    while (fpos < newsize) {
    if (fpos+BUFFER > newsize) {
    tocopy=newsize-fpos;
    }
    fread(&buffer,tocopy,1,input);
    fwrite(&buffer,tocopy,1,output);
    fpos+=tocopy;
    }

    //Done
    printf("%s: Trimmed '%s' to %d bytes (saved %.2f MB).\n",argv[0],argv[1],newsize,(filesize-newsize)/(float)1000000);

    //Close output
    #ifdef DEBUG
    printf("%s: Closing output.\n",argv[0]);
    #endif
    if (fclose(output) == EOF) {
    fprintf(stderr,"%s: fclose() failed in file %s, line %d.\n",argv[0],__FILE__,__LINE__);
    exit(1);
    }
    }

    //Close input
    #ifdef DEBUG
    printf("%s: Closing input.\n",argv[0]);
    #endif
    if (fclose(input) == EOF) {
    fprintf(stderr,"%s: fclose() failed in file %s, line %d.\n",argv[0],__FILE__,__LINE__);
    exit(1);
    }

    return 0;
    }


    Help me help you:
    This is a very simple trimmer and whilst it is very fast compared to other trimmers I have tried, of course I want to make it better...
    I'm not very skilled in coding GUI so I don't think I'll be doing that anytime soon.
    I tried to optimize read/write as much as I could, but I'm sure it's possible to make it even faster, if anyone have any experience with this, I'll gladly listen (currently: the program uses a 1 MB buffer which means it reads and writes in 1 MB chunks).

    Any suggestions are welcome!
     


  2. party

    Newcomer party Member

    Joined:
    Oct 17, 2006
    Messages:
    19
    Country:
    Germany
    Nice one! [​IMG]

    I just had a very quick look at it. Seems to work (Win XP SP2). Trimmed ~17.38MB off hotel dusk usa (haven't tested the resulting rom).

    A few things that came to my mind:

    -The "not" shouldn't be there, right?: "My trimmer is a 2nd generation trimmer, so it should not be able to handle all the commercial roms without problems with wifi or download play.

    -Have a /? -? /help or -help parameter instead of an error message.

    -Make the filename the only required parameter (and others optional): "ndstrim file.nds" would output file_trimmed.nds to the directory where file.nds is stored.
    I'd love that because then I could put ndstrim in my "send to" folder, and trim roms with rightclick -> send to -> ndstrim. AFAIK that send to thingy only works if the target program accepts the filename as the first and only parameter. (hope this all makes sense)

    -Whatever you change in future versions, please make sure that the bare executable is fully usable without the batch/shellscript. IMHO all the functions that are now outsourced to the batch/shellscript should go into the main executable anyways. But I guess you did it this way to have only one sourcecode for linux and windows and let the script take care of all the os-specific stuff, right?


    Thanks for sharing!
     
  3. scandal_uk

    Member scandal_uk GBAtemp Regular

    Joined:
    Oct 3, 2005
    Messages:
    165
    Location:
    UK
    Country:
    United Kingdom
    I would be happy to add a simple GUI in dotnet, if you think it would be helpful.
     
  4. recover
    OP

    Member recover GBAtemp Regular

    Joined:
    Aug 6, 2007
    Messages:
    161
    Country:
    Sweden
    Whoops, that might have been the biggest typo I've ever made in my whole life, fixed it. [​IMG]

    Perhaps, but I might just add the usage message on every error message instead...

    Yea, that should be possible.
    I'm still not very good at command line processing, but I guess it would be possible to make a -i argument to make it "investigate" the rom file and post the info it does now and make outputting a trimmed rom with a prefix the default

    Yes, I agree, it would be nice to be able to do these things without the shellscripts, but this is a first release.
    I want to make it possible to specify multiple infiles to the executable (e.g. by dragging), but I find it hard to come up with a nice way to decide if files should be outputted with a prefix or in a separate directory (since I guess there are a lot of people who want both).

    No problems! [​IMG]


    Yes, that might be helpful, feel free to help and post suggestions [​IMG]

    Thanks for your responses! [​IMG]
     
  5. scandal_uk

    Member scandal_uk GBAtemp Regular

    Joined:
    Oct 3, 2005
    Messages:
    165
    Location:
    UK
    Country:
    United Kingdom
    Well when I get the time I will create a proper one for you.

    For the meantime, here is a quick & dirty proof-of-concept version (which I haven't tested as I don't have any ROMs on the PC I'm working on);

    http://scandal.uk.googlepages.com/Trim-concept.rar

    To open it, double-click Trim.exe in the BIN folder (no other files are required), or you can drag & drop many files onto the program for batch trimming - none of the ouputted trimmed roms will overwrite existing files (it does rename them though).

    I will improve the look and add a list to batch convert roms once I've tested it (later today perhaps).

    I have included ALL of the source for anyone's use and built it in .NET 1.0 so that it's compatable with all versions of the .NET Framework.

    The code is heavily-commented VB so that even non-programmers can see what it's doing and how it's doing it.

    Hope that helps (and it's only 16KB unzipped)!
     
  6. recover
    OP

    Member recover GBAtemp Regular

    Joined:
    Aug 6, 2007
    Messages:
    161
    Country:
    Sweden
    Ok, I'll look into it later today (I'm in linux right now).
    I'm very grateful that you want to help me, even more that you are releasing the source as well [​IMG]
     
  7. Maikel Steneker

    Member Maikel Steneker M3 Fanboy

    Joined:
    May 16, 2007
    Messages:
    3,396
    Country:
    Netherlands
    Nice! I'm temporary using Windows XP atm, and it looked like I had to use NDSTokyoTrim with Wine when I'm going back to Ubuntu...
     
  8. scandal_uk

    Member scandal_uk GBAtemp Regular

    Joined:
    Oct 3, 2005
    Messages:
    165
    Location:
    UK
    Country:
    United Kingdom
    I'm happy to help, and if you look closely you'll notice that the source is all yours - I just changed it to use .NET Framework objects (for file reading etc).

    It's through contributions such as yours that the scene continues to be good.

    I needed this one as I use G6 Lite, but my new G6 doesn't use a manager so I wanted a clean, quick trimmer.

    It certainly seems to work with the 5-6 ROMs I just tried! At 16k it runs quick too.
     
  9. recover
    OP

    Member recover GBAtemp Regular

    Joined:
    Aug 6, 2007
    Messages:
    161
    Country:
    Sweden
    I've looked into the program and the source and everything looks nice [​IMG]
    I've also become convinced that automatically appending a suffix to the output file is the way to go, therefore enabling batch-processing by e.g. dragging nds roms on the exe, it's also a very good way to see if you have already trimmed the rom.
    I'll probably code a bit over the weekend and apply these changes and release ndstrim 1.1. [​IMG]
    There are still some issues to work out like writing the trimmed rom in a separate directory so you can easier copy the roms after they have been trimmed, perhaps it would be a good idea to setup two basic directories such as "untrimmed" and "trimmed" which ndstrim automagically go after when it's run?
    Any thoughts? [​IMG]
     
  10. scandal_uk

    Member scandal_uk GBAtemp Regular

    Joined:
    Oct 3, 2005
    Messages:
    165
    Location:
    UK
    Country:
    United Kingdom
    Well I quickly added some methods for you to easily trim a few ROMs. It's a little bit more of a GUI now. [​IMG]

    I even added a little scissors icon... [​IMG]

    The source can be compiled with Visual Studio 2002 or greater (any .NET version, basically).

    It's a little bigger now but I think it runs OK...


    http://scandal.uk.googlepages.com/NDSTrim-GUI.rar

    Edit:
    Yes, I think it would be best if you could choose an output directory - an input one might be little unnecessary though. In this version of the GUI you can queue up multiple roms.

    It would be a simple matter to enable opening Zip files for roms, however, that would be a good addition too.
     
  11. JacobReaper

    Member JacobReaper I Love You....

    Joined:
    Jun 5, 2007
    Messages:
    1,677
    Location:
    Spoomo
    Country:
    United States
    i really like this trimmer.. this one and the NDStokyotrim
     
  12. recover
    OP

    Member recover GBAtemp Regular

    Joined:
    Aug 6, 2007
    Messages:
    161
    Country:
    Sweden
    Glad you like it [​IMG]
     
  13. madlobster

    Newcomer madlobster Member

    Joined:
    Oct 21, 2006
    Messages:
    37
    Country:
    United States
    Some ROMS don't trim right. 0040, 0132, 0192, 0318, and 0357 in particular.
    Code:
    D:\test>"\nds\To trim\ndstrim.exe" "0040 - Hanjuku Eiyuu DS - Egg Monster Hero (JP).nds"
    \nds\To trim\ndstrim.exe: ROM size: 0 bytes.
    \nds\To trim\ndstrim.exe: ROM size + wifi: 136 bytes.
    \nds\To trim\ndstrim.exe: Can save: 33554296 bytes.
    It appears that bytes 0x80-0x83 are 00 00 00 00. [​IMG]
     
  14. n45800

    Newcomer n45800 Member

    Joined:
    May 28, 2007
    Messages:
    25
    Country:
    United States
    Here is a pre-compiled mac version (CLI, Universal Binary).
    I'll see if I can whip up a quick Cocoa GUI later today.
    And another small bug report. You should probably have a test for if the file actually exists before you try doing anything, otherwise it bombs with the error "./ndstrim: fopen() failed in file ndstrim.c, line 45.".
     
  15. flai

    Member flai Androgynous Apparently :)

    Joined:
    Sep 30, 2006
    Messages:
    850
    Country:
    United Kingdom
    It looks very nice but I think you should perhaps ask a coder around here to help with a GUI, it would make it more accessible to people not in the know, great piece of software though!
     
  16. scandal_uk

    Member scandal_uk GBAtemp Regular

    Joined:
    Oct 3, 2005
    Messages:
    165
    Location:
    UK
    Country:
    United Kingdom
    I tried !! :'(

    I know it's just .Net but that's to make the code more accessible for people who want to re-use it.
     
  17. madlobster

    Newcomer madlobster Member

    Joined:
    Oct 21, 2006
    Messages:
    37
    Country:
    United States
  18. recover
    OP

    Member recover GBAtemp Regular

    Joined:
    Aug 6, 2007
    Messages:
    161
    Country:
    Sweden
    Thanks, the mac binaries should be useful to some people.

    And with that file check [​IMG]
    You see, there are no standard way to check if files exist, so either it fails when trying to open the file (that probably means that the file doesn't exist, or maybe the user doesn't have the permissions needed).
    But perhaps I should write something else in that error message, like "(file doesn't exist?)" or something. =)
     
  19. BrianTokyo

    Member BrianTokyo NDSTokyoTrim Dev.

    Joined:
    Jun 28, 2007
    Messages:
    649
    Country:
    Japan
    Hi, nice little app [​IMG] (altho .net sucks [​IMG]) Why do you always add the wifi block though? Shouldn't it detect whether it's needed or not?
     
  20. BrianTokyo

    Member BrianTokyo NDSTokyoTrim Dev.

    Joined:
    Jun 28, 2007
    Messages:
    649
    Country:
    Japan
    Example:

    Using ROM: 0938 - Theme Park (US)(M6).nds

    Your trimmer left it at 29,040,336 bytes

    ----------------------------------------------
    It should be:

    0938 - Theme Park (US)(M6).nds: 4514232
    Processing Finished.

    Trimming took 0.05 seconds.

    Size: 29,040,200 bytes
     

Share This Page