Discussion SNES Online - Mod Discussion

18Phoenix Sep 7, 2019.

  1. 18Phoenix
    OP

    18Phoenix GBAtemp Advanced Fan

    Member
    6
    Nov 21, 2005
    Germany
    ... and will never appear here.
     
  2. reminon

    reminon GBAtemp Fan

    Member
    5
    Feb 7, 2016
    United States
    Am I correct to assume that 1D92C is the memory address and 0x1E1B4 is the physical binary address?

    I get that 1E1B4 is the first address you get if you search for 29000094 in hex, but how would you have found that address had we not been lucky enough for it to be the first? 29000094 has multiple entries in the main executable according to hxd. Of course I may be mistaken.

    I guess my main question is: how does one find the relative binary address after finding the appropriate memory address in IDA pro?
     
  3. Falo

    Falo GBAtemp Advanced Fan

    Member
    9
    Jul 22, 2012
    Germany
    It's the elf header.
    A Switch executable almost always starts at 0x0, so memory addresses are identical to the decompressed binary address, the only thing that comes before that is the elf header.
    In Ida pro you can see the binary address, example:

    Code:
    .text:000000000001D92C 29 00 00 94                 BL              RsaCheckHash
    .text:000000000001D930 E8 03 1F 32                 MOV             W8, #2
    .text:000000000001D934 E8 CB 01 B9                 STR             W8, [SP,#0x1340+var_1178]
    .text:000000000001D938 C0 01 00 36                 TBZ             W0, #0, loc_1D970
    
    if you click on the line with RsaCheckHash, then ida pro will tell you in the lower left corner this: "0001E0B4 | 000000000001D92C: sub_1D590+39C"
    0001E0B4 = binary address,
    000000000001D92C = memory address,
    sub_1D590+39C = function name + offset

    Note: i named the function like that and i did use my own nx2elf.exe to convert the files, it generates better info/data than the public one, places the imagebase at 0x0 instead of 0x0000007100000000 and works with 32 bit nso's like Grandia 1, but it works the same way.
     
    18Phoenix likes this.
  4. reminon

    reminon GBAtemp Fan

    Member
    5
    Feb 7, 2016
    United States
    Awesome! Thank you so much. I'm new to this, and find it intriguing. Edit: Which loader are you using?
     
    Last edited by reminon, Sep 15, 2019
  5. Falo

    Falo GBAtemp Advanced Fan

    Member
    9
    Jul 22, 2012
    Germany
    If you convert nso to elf, then you don't need a loader.
    Elf is a generic format with hundreds of supported executable formats, Switch is ARM32 / ARM64, so no special loader is required.
     
  6. reminon

    reminon GBAtemp Fan

    Member
    5
    Feb 7, 2016
    United States
    I get that now. I was loading the nso in ida and the elf in hxd. When I could just load the elf in both. My bad.
     
  7. DarkAkuma

    DarkAkuma Advanced Member

    Newcomer
    3
    Sep 20, 2008
    United States
    I'm sorry. I've been very busy lately, and as I don't own a switch this has been kind of low priority for me when I've had free time.

    Previously I got as far as looking at a few footers, and just assuming the rest of the footer besides the Preset ID was remains of Header #2's from previous formats. That wasent adding up. Some data is obviously gone, like PCM/Var data sizes. Other data like FPS, ROM size, volume and ROM Map override don't seem to be clearly represented. That was as far as I got as I realized I needed to dump all the footers and compare. I didn't have the time. Thankfully it seems Falo did, so...

    Well. As I said, I don't own a switch. So this is only my best guess right now without being able to test.

    Code:
    47 02 00 00 // Bytes used to mark this region as valid, formatted footer data. Like <footer> in HTML terms.
    BB AA       // Preset ID in reverse order.
    XX YY       // The confusing parts. Ill explain my theory below.
    43 61 6E 31 // Bytes used to mark this region as valid, formatted footer data. Like </footer> in HTML terms.
    The bytes between the Preset ID and "Can1" seem similar between games, but can have different orders. I think this is because they are used as a sort of dictionary type of look up table.

    For example, "70 02" I believe might be the new representation of "Max Input". The code wants to know how many controllers it should check for, sees something like "#define MAX_INPUT 0x70", searches the footer for 0x70, then checks the byte after it and sees "0x02". So 2 players. (To support this idea, look at Super Puyo Puyo 2 and Super Family Tennis. The only 4 player capable games in the roster. They are the only ones with "70 04". The rest either have "70 01" or "70 02")

    This assumption is only based off a quick look though. It seems to match up as far as I can gather from memories or what I believe of the player counts in these games.

    Building on that idea, that would mean the rest are probably similar.

    "65 0C" probably does indeed mean "Special chip", or "#define SPECIAL_CHIP 0x65" (0x0C meaning SuperFX).

    That just leaves "0A 00 00 00", "0C 00 00 00", "0E 00 00 00", "1E 00 00 00", 12 00 00 00". These could be 4 byte groups, or 2 2byte groups. If I had to guess they are likely the carry over of what I dubbed "Unknown 3"/"Unknown 4" in the SNESClassic/WiiU Header2. There, those values were different on occasion, but I couldn't find noticeable differences when adjusting the value. If that's the same value, either is probably fine.

    If the dictionary idea holds up, what would need to be figured out to construct proper footers would be.

    Code:
    3C 05
    4A 06
    50 04
    55 02
    63 01
    6A 38
    74 XX - 06, 07
    76 XX - 28, 46, 78, 82, 91, A5
    I know this isn't much atm, but its a starting point. I've only looked over this stuff for a few minutes today now. So I'm sure we can refine it more.

    For testing purposes, I'd currently conclude that a basic minimum Switch SFROM footer would be as such:

    Code:
    47 02 00 00 BB AA 74 06 70 02 0A 00 00 00
    Just change "BB AA" to the reversed Preset ID from my list. That should be good enough to test with for now.

    As soon as I can get this format figured out, I'll update SFROM Tool in some way to support Switch .sfroms.
     
    Last edited by DarkAkuma, Sep 15, 2019
  8. Falo

    Falo GBAtemp Advanced Fan

    Member
    9
    Jul 22, 2012
    Germany
    Your theory is correct, i did analyze the "ReadRom" / "ReadFooter" function a bit more now.
    The data from PresetId to FooterSize is looped like this:

    Code:
    int64* ptrData = ptrRom + szRom;
    do{
        byte type = ptrData[pFooter];
     
        if(type & 0x20)
        {
            byte value = ptrData[pFooter + 1];
            pFooter += 2;
        }
        else
        {
            int len = ptrData[pFooter + 1] | ptrData[pFooter + 2] << 8 | ptrData[pFooter + 3] << 16
            pFooter += 4;
            //then read n-bytes of data in the same way as len
        }
    }while(ptrData + pFooter < ptrFooterSize)
    example:
    Code:
    70 02 -> type 0x70, len = 0x1, value: 0x2
    50 04 00 00 1E 00 00 00 -> type 0x50, len = 0x4 value: 0x1E
    
    Super Mario World™ 2: Yoshi's Island™:
    47 02 00 00 3D 12 = PresetId
    65 0C = Super FX
    74 06 = maybe DSP type ?
    70 01 = ???
    55 02 00 00 4A 06 = maybe Clock Speed (1610)
    12 00 00 00 = FooterSize
    43 61 6E 31 = "Can1"
    the emulator only checks these types:
    multi byte: 0x41, 0x44, 0x47, 0x50, 0x53, 0x55
    2 byte: 0x61, 0x63, 0x65, 0x68, 0x6A, 0x6D, 0x70, 0x72, 0x74, 0x76
     
    Last edited by Falo, Sep 16, 2019
    18Phoenix, hippy dave and wicksand420 like this.
  9. DarkAkuma

    DarkAkuma Advanced Member

    Newcomer
    3
    Sep 20, 2008
    United States
    Interesting.

    Not sure how I missed the coloration of the 4 bytes to being footer size, but that does add up. 0x0000000A, 0x0000000C, 0x0000000E and 0x00000012 from known games.

    Knowing that, that would leave Joe & Mac 2's "1E 00 00 00" as not included.

    I don't think "74 06" is DSP type. It's "74 06" used with every game from both regions, except for Super Tennis with which it is "74 07". Not a DSP game, and the 2 that are DSP games are 06. Honestly, this one has been puzzling me because of all games, Super Tennis does not stand out in any way. Its an incredible average game emulation wise. My best theory right now is that it "might" be related to network play. Like a value for packet size/frequency. That might make sense as Super Tennis seems like a game where timing may matter more. Why Super Family Tennis wouldn't be like this too, is that's Japanese. Japanese games are meant to be played only in the island of Japan and as such ping/latency time difference would be more standard, where as a western game would have people potentially playing it from many different countries.

    I am 99% certain that "70 XX" is related to MaxInput, or PlayerCount. I made this theory before even realizing that Super Puyo Puyo 2 and Super Family Tennis both have the value of 04 following 70. While they only support 2 players on the switch from what I understand, they are technically 4 player games.

    Outside of "76 XX", I don't have much for theories other than possible performance hacks. Like enabling/disabling layers, frame skips, etc.

    76 XX I feel might be a carry over of the Volume byte from WiiU/3DS/SNESC. But I don't know why they would. That's covered in the titlesdb.

    EDIT:

    Yea. I was including "3C 05" and "4A 06" as 2 byte pairs, but as they always follow what seems like a 4 byte value, that means they may be a part of that as a 6 byte value.

    That would clear up some that you say are not checked from my notes. Though you don't list 0x1E, and that clearly seems like a 4 byte value. Omission, or just not referenced?
     
    Last edited by DarkAkuma, Sep 16, 2019
    hippy dave likes this.
  10. Falo

    Falo GBAtemp Advanced Fan

    Member
    9
    Jul 22, 2012
    Germany
    To be honest, i don't know much about the internal workings of snes games.
    So yeah, it's not DSP. And it could be player count

    This Footer must be some special settings that they didn't want inside the executable or database.
    Each PresetId contains a lot of extra settings inside the executable, some are used for the online statistics, so i don't really get why a footer exists in the first place.

    Also, they did add Header Support, most likely for internal debug roms only, but the code is there,

    If a rom starts with:
    Offset 0x00 = 0x00
    Offset 0x04 = 0x01
    Offset 0x24 = 0x57
    then it reads the rom info (0x47 / 0x70 / 0x76 / 0x65 and 0x44) directly from a fixed size header struct and the footer is ignored.

    It's in my example...

    Code:
    Joe & Mac 2: Lost in the Tropics
    47 02 00 00 62 12 = PresetId (0x1262)
    74 06
    70 02
    50 04 00 00 1E 00 00 00 = type: 0x50, len: 0x4, value: 0x1E
    12 00 00 00 = FooterSize
    43 61 6E 31 = "Can1"
     
  11. DarkAkuma

    DarkAkuma Advanced Member

    Newcomer
    3
    Sep 20, 2008
    United States
    I'm not much of a asm guy my self. I've tried tracing around assembly routines and making sense of them. That's how I predicted Stunt Race FX's ID as working on SNESC. But I have done a bit of ROM Hacking, and seem to be decent at reverse engineering data to some degree.

    That matches up with either WiiU/SNESC .sfrom, or if you mean that it reads the ROM info from the same fixed size header, then that would sound more like 3DS. "Offset 0x24 = 0x57" makes sense for WiiU/SNESC, as that's the first letter of the Product ID code. The "W" in WUP-JXXY. If it was 3DS, it would be 4B (K in KTR-UXXY). 3DS used its own custom iteration of the WiiU header setup. It combined Header1 and 2 into a single 96 byte header before the ROM data.


    Oh. I skimmed over that part. Hmm.

    I'm not sure i get it. If "70 02 -> type 0x70, len = 0x1, value: 0x2" is true, than wouldn't "0x50, len: 0x4" mean "value: 0x1E000004"? Unless with this the length isnt assumed, but instead provided by the data itself. So the 2nd-4th bytes are the length.

    Anyway, knowing either way... No clue what the data could be. I'm not familiar with the game its used for though. best guess is that since its newly emulated, its just some custom hack enabler/adjustment.
     
  12. Falo

    Falo GBAtemp Advanced Fan

    Member
    9
    Jul 22, 2012
    Germany
    if((type & 0x20) > 0) then the length is fixed at 1, otherwise the length is written as a 3 byte value.

    I did compare all values with the japanese version and there is only 1 difference, Super Family Tennis:

    USA: 0x70 = 2, 0x74 = 7
    JPN: 0x68 = 3, 0x70 = 4, 0x74 = 6

    all other games match, Super Puyo Puyo 2 even matches the PresetId.

    0x55 must be the Super FX Clock Speed
    0x63 = snes header location memory bank, default = 0x0 (0x7FC0), Mario Kart uses 0x1 (0xFFC0)

    The leftover code matches the 96 Byte 3DS Header, but checks for WiiU data.
     
  13. DarkAkuma

    DarkAkuma Advanced Member

    Newcomer
    3
    Sep 20, 2008
    United States
    For a minute I was confused as to what you were talking about. But I think you are mistaking that "Super Family Tennis" is the japanese version of "Super Tennis". 2 different games! That's why they have different footers, and substantially different Preset IDs. In most cases, the Preset ID's between the English and japanese versions are only different by 1-2. Super Puyo Puyo 2 is just the same japanese ROM, regardless of which region it was released in on the switch. So it uses the same Preset ID.

    Actually, I was about to just post and ask if anyone would be interested in testing a theory about 0x63. I was thinking that it might be for DSP revision. The basis for that when Pilotwings does not use that setting is that I suspected that Pilotwings was using DSP-1A to avoid a bug, while Super Mario Kart is using DSP-1B. "63 01" presumably meaning DSP-1B. Are you certain about that pointing to the internal SNES ROM Header?

    FYI, the test id ask is simply to see if the demo bugs out and the plane crashes like with this. If the plane does crash, I'd be wrong and its using 1B. If not, it might be 1A, but then id want to know if inserting "63 01" and bumping the footer size value by 0x02 would then show that plane crash bug.

    Oh. And technically, SFX clock speed has not been found to exist with previous versions of canoe. Closest thing was the --boostfx command line param. But that was not officially used with any of the 3 prior existing SFX games. So they all use the same default value. But if 0x55 was clock speed, Starfox and Yoshis would be using different values now.

    More likely, it might be related to "Super FX GSU-1" for Starfox/Stunt Race FX, and "Super FX GSU-2" for Yoshi's island. Technically different chips, just Nintendo did not distinguish between the on the SNESC. They might now though, which would be nice. Might mean some DSP-2, DSP-3, DSP-4 support...
     
  14. RadMcFist

    RadMcFist Member

    Newcomer
    1
    Sep 16, 2019
    United States
    Not only does the plane not crash but it goes on to land beautifully. I can run any test you want, just tell me what to do (just keep in mind that I'm not an expert by any description of the word). In any case, thanks to you and Falo for trying to make some sense of this all!

    BTW, since I haven't seen anyone else mention it here, regardless of what enhancement chip a game uses, any footer I've tried works without even changing the Game ID. For example, Kirby Superstar, Super Mario RPG, Mega Man X 2 & X3 all work great with the Mario Kart footer (without any modifications) when they all use different chips than Mario Kart. In the case of the first two games, they use the same chip as Kirby Dreamland 3 which is included in the official games, but the Mega Man games use a different chip, one only they use, actually.

    On the other hand, Street Fighter Alpha 2 shows some pretty extreme graphic glitches on what I assume are the logo screens and then the software crashes completely (the only crash I've experienced, even the couple other games that don't load for me only show a black screen with no crash). Haven't even tried Star Ocean but it'll probably be the same.

    I couldn't get the official Star Fox 2 ROM from the SNES mini to work here but I only tried with the Yoshi's Island and Stunt Race footers. I've seen some people say the game works for them but are they using this version or the older, more common, incomplete beta that was available for years? As a curiosity, the Trials of Mana ROM extracted from Mana Collection also doesn't work, but that's probably because it was expanded for the translation.

    Street Fighter Alpha 2 doesn't use MSU1, no commercial game uses it since it's a "fictional" chip created by byuu. Trials of Mana doesn't use any chip at all, like all of Square's games for SNES.

    Check your ROM, both Pocky & Rocky games work great.

    The only thing most of these games have in common is that they were published by ENIX but most of them are by different developers so they don't really share anything. Because of that it makes sense that none of the Quintet games (Terranigma, ActRaiser, Gaia, Robotrek, Soul Blazer) work correctly but EVO, which is by Almanic, and Ogre Battle, by Quest (and only published by ENIX outside of Japan), do work correctly. I haven't tested the rest but I expect a lot of them do work so don't give up!

    We should probably try to get a list of games that work and don't work from those of us making our own mods to help determine when the game actually doesn't work or when it's just a bad rom or a simple mistake. Maybe once the footer and other stuff is understood a bit better. From my experience, the vast majority of games work. I've even thrown some very bad, sketchy ass roms I had lying around and they worked too. Even the ones that I was sure wouldn't work, like Mega Man X2 & X3 do work (didn't expect the Cx4 since these games are unlikely to be included officially). It's a shame about the Quintet games, but I'm sure they'll get sorted out eventually and it's even probable that both Act Raiser games are released officially eventually since they came out on Virtual Console.

    Are they corrupted (as in, graphical glitches) or do they look kinda out of focus? Cause that game uses the SNES hi-res mode for the menus, like Secret of Mana's initial menus or Rudra does for all text. It would be interesting to see if Super Family Tennis can be used to fix those two games. What is its footer?

    Someone please post something so I don't have to keep editing this massive post!
     
    Last edited by RadMcFist, Sep 16, 2019
    hippy dave and 18Phoenix like this.
  15. DarkAkuma

    DarkAkuma Advanced Member

    Newcomer
    3
    Sep 20, 2008
    United States
    Thanks. I guess that means it's being emulated using DSP-1A. Which is what I'd expect. This is the test of Pilotwings I'd like to see, using this footer.

    Code:
    47 02 00 00 BA 10 74 06 63 01 70 01 0C 00 00 00 43 61 6E 31
    You have to manually edit that in using a hex editor. If necessary, I can probably make an ips patch to do this as unlike the SNESC .sfrom format, the Switch .sfrom format lends itself better to using ips patches as normal.

    That is is strange. It suggests some more internal code changes to canoe that allow it to more automatically use a a chip as needed. On the SNESC/WiiU/3DS the Preset ID was what canoe used to know a game needed to emulate a chip.

    I did see some evidence on the the SNESC of chip emulation slightly working with other games Preset ID.

    SFA2 graphics are like that because canoe does not properly emulate the SDD1 chip. Instead its coded just to toss aside any calls to the SDD1 chip and continue on. Since its 99% graphics data thats compressed using the SDD1, it just displays static, or garbage data instead.

    I wanted to learn a little more about the footer format before I started trying to get SFA2 and my Star Ocean patch working. One step at a time.

    I'm not sure why Starfox 2 wouldn't work, unless as you say people are using the only prototype ROM, or maybe for that the proper Preset ID (0x1245, or "45 12" in reverse) is required.

    Yes, canoe doesn't like expanded HiROM games. It doesnt map or check the expanded regions at all. It can boot ExHiROM games, but as soon as it encounters data in those regions it crashes.


    There's no reason Megaman X2 and X3 shouldn't work. The SNES emulation code is derived from the coded used to make the SNESClassic emulator. That was derived from the WiiU VC SNES emulator code. That was likely derived from the original Wii's VC SNES emulator code. There no reason to remove support for it if you are not using it.

    What will be interesting is to find out if some new games work...
     
    18Phoenix likes this.
  16. RadMcFist

    RadMcFist Member

    Newcomer
    1
    Sep 16, 2019
    United States
    OK! I'll test it in a minute. At the very least, I can use a hex editor XD

    Edit:
    It works! No idea what that means, but it works. Ha ha.



    I'll try out a few different footers and with the correct Preset ID, although the few times I've tried using the real ID the games have not loaded at all. Maybe this version of canoe only recognizes the Preset IDs of the games officially released so far?



    Thanks for the explanation! It's great to have more people who actually know what they are doing taking a look at this instead of most of us just throwing stuff around in the dark to see what sticks XD Anything you wanna try out, I'm glad to help out.



    I'll test later it with a bunch of different footers and the real Preset ID and see what happens.

    Oof, that's a shame but I expected as much. There's a lot of translations I'd like to use with this that expand the roms. That must be why the translations for games like Fire Emblem Tracia 776 and Dragon Quest 3 appear to work at first but just give garbled text after a while. Do you think that this means that even the expanded roms that do load will eventually crap out? I'd rather know now than start playing Famicom Tantei Club 2 only to run into a brick wall later.



    That makes total sense and it's exactly what I thought when I saw X2 & X3 working, so I went and tried Alpha 2 that was also released on Virtual Console. But you already know how that went.
     
    Last edited by RadMcFist, Sep 16, 2019
    18Phoenix likes this.
  17. DarkAkuma

    DarkAkuma Advanced Member

    Newcomer
    3
    Sep 20, 2008
    United States
    While its good that it still works, thats not the purpose of the test. What id want to know is if the plane crashes during the demo like in that gif from the tweet i linked above. That footer is testing if a value is setting emulation to use DSP-1B instead of 1A.

    Well. I was making some IPS patches for my tests. Seems that you don't need the pilotwings patch, but you can try this SFA2 patch. It's a complete shot in the dark. I suspect it wont work and will either require adjustment, or a completely different approach.

    Sluffy did make a DQ3 patch to get it working. But such patches are not easy to make.

    Crimson Echoes is a great expanded ROM to see for yourself. As I understand that hack boots, but crashes very early. Otherwise, I wouldn't mess much with expanded ROMs right now.
     
    Last edited by DarkAkuma, Sep 16, 2019
    18Phoenix likes this.
  18. RadMcFist

    RadMcFist Member

    Newcomer
    1
    Sep 16, 2019
    United States
    Ah, I was sure that I had mentioned that it was another perfect landing for the plane. Sorry about that. I guess it didn't work after all.


    Cool! I'll give the patch a go later.

    Edit: The game didn't even start after the patch. The resulting rom didn't have a footer (was the patch supposed to add one? I patched it correctly!). Tried it like that, got nothing, tried with a footer that should work, nothing again.


    Understood! I'll check it out and keep my expectations low.
     
    Last edited by RadMcFist, Sep 16, 2019
    18Phoenix likes this.
  19. DarkAkuma

    DarkAkuma Advanced Member

    Newcomer
    3
    Sep 20, 2008
    United States
    NP. Guess that theory is wrong. Good to know at least that so I can move on.


    I may have another SFA2 test later. Though it cant be done in simple IPS form. Well see if it comes to that.

    For now I'd like to get the first test working. I suspect that one of the unused indexes Falo listed above might play a role in SDD1 graphics support, by providing a pointer/offset to where the decrypted tile data is located. As is right now I'm just assuming it will look for and find it immediately after the footer.
     
    18Phoenix likes this.
  20. 18Phoenix
    OP

    18Phoenix GBAtemp Advanced Fan

    Member
    6
    Nov 21, 2005
    Germany
    StarFox 2 runs fine. Used the SNESmini rom, no prototype.
     
Quick Reply
Draft saved Draft deleted
Loading...