Unofficial DeSmuME build: Unused memory finder tool

Discussion in 'NDS - ROM Hacking and Translations' started by Nagato, Jun 11, 2013.

  1. Nagato
    OP

    Nagato GBAtemp Advanced Fan

    Member
    533
    513
    Jul 15, 2011
    United States
    I have a lot of free time for the next month and a half and have been itching to do something programming and romhacking related, so I decided to try adding a feature to DeSmuME that would help with finding free memory for rom hacks. It's probably solving a problem that most people don't need a solution for, but I learned a few things in the process so it was worth the effort in my opinion. I am posting this in case there is anyone else who could benefit from it.

    The only usage I can think of for this is for rom hackers who are looking for places in memory to write their code, redirect buffers, or load new files into memory without accidentally overwriting something important. Sometimes it can be a pain to find a good unused buffer with enough space for what you need, so I made this hoping to speed up the process a little bit.

    Functions
    [​IMG]

    Region: The region dropdown in the top left lets you select which region (arm7 or arm9) of memory to search. It's the same setting that you would set on the memory viewer menu.
    Size: The size is the minimum amount of free space you searching for. Anything equal or above that number will be shown in the results window.
    Search Range: The search range is optional. You can narrow down the memory ranges you want to search. The default values are from 0x00000000 until 0x10000000.
    Set: You must press the set button after entering a range or else it won't save the range.


    Results window: The address column is the real address in memory. You can plug the address in with the correct region into the memory viewer window and you should be able to see the free space. The size column gives you the size in hex and decimal.

    Export: The export button exports the contents of the result window into a text file.
    Here is an example of the output:
    Warning: Spoilers inside!
    Clear: Clears the results window.

    Save: Saves the entire search buffer so it can be loaded back into memory later. In the saved data, bit 1 signifies that the action took place in the arm9 region, and bit 2 signifies the action took place in the arm7 region. (Warning: Output file is large.)
    Load: Loads data back into the search buffer. Only the first 0x10000000 bytes will be read.
    Reset: Zero out the entire search buffer. Note: Searching the memory will automatically mark any bytes that are non-null in memory, so it will only be completely blank until you search again.

    Close: Closes window (but does not clear search buffer).

    The search buffer is cleared every time a game is loaded or reloaded. Save states do not have any effect on the search buffer which is why I added the save and load features. The longer you play the game, the more accurate the results should be.

    How It Works
    Most memory read and write functions (except for some internal system stuff) have been modified to mark a bit in the search buffer if a specific location has been written to. Sometimes data that looks blank is actually used somewhere in-game, read commands are also marked. The memory locations returned in the results window will have been completely untouched (to the extent that my modifications know). As is still the chance that the results are not entirely accurate once you get into far off regions, but it should be good enough for the areas in memory that most hackers will be interested in.

    Other Notes
    This build is compiled with the gdbstub enabled. You can start the gdbserver by passing the parameters "--arm9gdb=(port)" or "--arm7gdb=(port)" in the command prompt. From there you can attach something like IDA Pro or arm-eabi-gdb to the specified port.

    The EXE was compiled with VS2010 using the latest SVN code as of June 9th, 2013 (rev 4672).

    I have not had a chance to do extensive testing so there may be bugs. Because this is an unofficial build, do not report any bugs found with this build to the DeSmuME team unless you have tested an can prove that the bug is not a result of my changes.

    Download (+ source patch): http://www.mediafire.com/download/siavuq6jmt870ls/DeSmuMe-custom-memaccessview-r3.rar

    Update (6/11/2013):
    - Automatically mark overlay regions as used memory
    - Added overlay information output to console
    Update (6/12/2013):
    - Fixed small bug in ARM7 mode.
     
    Rydian, Fishaman P, GHANMI and 2 others like this.
  2. FAST6191

    FAST6191 Techromancer

    pip Reporter
    23,364
    9,167
    Nov 21, 2005
    I might have to have a look at this as it will likely be nicer than fiddling around with deadbeef padding*, far less aggro than doing it by hand and less clunky than just overwriting wifi error codes (nice long ASCII messages, sometimes in several languages all end to end and in the binary from boot).

    *though it looks like the main issue I have with that of games sometimes flood filling/"initialising" memory might still trouble it.

    Possible future addon - a decode of the overlay table and a quick breakdown of the ranges. I would not hardcode it out as some overlays are just that rare and the extension might only been needed for a little while.
     
  3. Nagato
    OP

    Nagato GBAtemp Advanced Fan

    Member
    533
    513
    Jul 15, 2011
    United States
    I think you will still have that problem here because anything the game does to the memory will be logged. On the other hand, you have some options to undo it to an extent (resetting the entire memory or loading a modified buffer, or some combination or whatever).

    Can you elaborate on what you mean by a decode of the overlay table? Just a list of all of the overlays?
     
  4. FAST6191

    FAST6191 Techromancer

    pip Reporter
    23,364
    9,167
    Nov 21, 2005
    Yeah the reset thing would help but some of them seem to go in for non null fills.

    The overlay table lists the location in memory the overlays are destined for and their size (compressed or otherwise) is equally easy to find. From here you could get the lowest overlay location in memory and also the highest/final location (it can go many ways* so you will have to add sizes to location and figure out the highest number to get the whole picture). Basically it will hopefully stop something the game would never use as it is earmarked for an overlay but being an overlay area it might only be used for a rarely called function and thus your heuristics might flag as a free area.

    *there can also be gaps but this is potentially less pressing -- the compilers are fairly smart about this.
     
  5. Nagato
    OP

    Nagato GBAtemp Advanced Fan

    Member
    533
    513
    Jul 15, 2011
    United States
    Alright, I just want to make sure that I'm understanding your suggestion properly. You're suggesting that I read the overlay tables and mark the potential overlay areas as used so that my tool doesn't think it's free space? If I'm understanding correctly, then yeah, that should be pretty easy to implement.

    Edit: I updated the download link in the first post according to how I interpreted your suggestion. I didn't feel like messing with the layout (I hate Win32 GUIs...) so I just wrote the overlay information to the console. It looks like this:
    Code:
    ----------------Begin Overlay Info----------------
    ARM9 Overlay Table (1 total):
    Overlay[0000]
            Overlay ID: 0000
            RAM Address: 0x021354e0
            RAM Size: 0x00001320
            BSS Size: 0x00000000
            Static Initialization Start Address: 0x021367e8
            Static Initialization End Address: 0x021367ec
            File ID: 0x00000000
            Reserved: 0x01000f58
    ARM7 Overlay Table (0 total):
    -----------------End Overlay Info-----------------
    It also marks the areas in memory as used.
     
  6. AsPika2219

    AsPika2219 Pikachu going beach!

    Member
    1,198
    220
    Jun 17, 2010
    Malaysia
    Cats City
    Nice tool! Better than RAM Search! Time for try this features! :lol:
     
  7. Blazer

    Blazer GBAtemp Regular

    Member
    281
    99
    Aug 10, 2008
    United States
    This actually looks like it could be pretty useful to me, great work Nagato!
     
  8. Fishaman P

    Fishaman P Speedrunner

    Member
    3,239
    505
    Jan 2, 2010
    United States
    Wisconsin
    So, it basically checks the game's code and highlights any areas of the memory that the ROM never references?

    If so, I can actually see a nice use for this!
     
  9. Nagato
    OP

    Nagato GBAtemp Advanced Fan

    Member
    533
    513
    Jul 15, 2011
    United States
    Thanks. :) I think if I had made this sooner then it might've actually been useful on one of the problems that we ran into on Fire Emblem. Oh well.

    Pretty much that, yes. It doesn't actually check the game's code, though. You play the game normally and it attempts to track memory access (read/write) positions and marks them as used. Anything left unmarked is assumed to be unused. I've had problems in the past with trying to find enough unused free space, so I decided to make this for future purposes.
     
  10. StorMyu

    StorMyu "I'm too old for this"

    Member
    897
    437
    Jan 2, 2010
    France
    I must say, this is pretty cool Nagato. ;)
     
  11. Blazer

    Blazer GBAtemp Regular

    Member
    281
    99
    Aug 10, 2008
    United States
    That's actually exactly what I was thinking... that and that if I had time, I'd use it to see if I couldn't try to fix the issue once again, though IIRC the issue was a bit complicated and it might not be worth the trouble at this point, now that most everyone who wanted to play the game has, haha.
     
  12. Gryphon93

    Gryphon93 GBAtemp Regular

    Member
    145
    14
    Nov 30, 2008
    Interesting tool, Nagato. Nice work! Have you asked someone on the DeSmuMeE Dev-team to add it in as a "permanent" feature? I think that would be nice. Creating a branch in the SVN repo and commiting it would do the trick to begin with, assuming they accept it. You can always ask on IRC.

    I may actually have use for this in a near future, so thanks again for making the tool!
     
  13. Nagato
    OP

    Nagato GBAtemp Advanced Fan

    Member
    533
    513
    Jul 15, 2011
    United States
    I'm not even sure if it's worth the effort at this point because as you said, everyone who wanted to play it has at this point. If you want to be a perfectionist then I say go for it, though.

    I'm not really interested in going through whatever process it would take to get it included in the official release. I'm not even sure if my implementation method would be up to their standards, either (it was my first time looking into the DeSmuME code so I just did what I could). And besides, it's a pretty niche feature that a lot of people won't have a need for most likely. Thanks for the suggestion, though. And good luck with whatever it is you want to use it for!
     
  14. snapman

    snapman Member

    Newcomer
    23
    6
    Aug 31, 2011
    United States
    So it locates the parts of memory that are never called... those lost, un-named chunks of game memory... the nameless game memory...
     
  15. Nagato
    OP

    Nagato GBAtemp Advanced Fan

    Member
    533
    513
    Jul 15, 2011
    United States
    Are you trying to imply something? Because it sounds like you're trying to imply something. :|
     
  16. Rydian

    Rydian Resident Furvertâ„¢

    Member
    27,883
    8,108
    Feb 4, 2010
    United States
    Cave Entrance, Watching Cyan Write Letters
    The Nameless Game is the name of a DS game involving hidden secretive stuff, and... stuff.
    I haven't played it, obviously...

    http://en.wikipedia.org/wiki/Nanashi_no_Game
     
  17. snapman

    snapman Member

    Newcomer
    23
    6
    Aug 31, 2011
    United States
    Interesting choice for the screenshot. ^_^
     
  18. snapman

    snapman Member

    Newcomer
    23
    6
    Aug 31, 2011
    United States
    Seriously though, am I the only person who noticed what game is in that screenshot? Nagato was the lead hacker on the Nanashi no Game English translation patch, and that's a screenshot of the opening scene of that game's sequel, Nanashi no Game: Me (The Nameless Game: The EYE) with English text!

    Check it:


    In short: eye see what you did there ^_^