1. NoobletCheese

    NoobletCheese GBAtemp Regular
    Member

    Joined:
    Aug 12, 2018
    Messages:
    165
    Country:
    United States
    Xenoblade Chronicles seems to have a unique case of the blurs -- its view frustum looks filtered, but the Wii sysmenu overlays aren't. This is unlike all other games I've observed so far, where the vfilter gets applied to everything. As it doesn't respond to patching vfilter or sample_pattern, I would speculate the game is perhaps using its own custom AA sample_pattern and applying it with GXSetCopyFilter.

    We must figure out how to patch GXSetCopyFilter
    .
     
    Draxikor and contra1980 like this.
  2. Draxikor

    Draxikor GBAtemp Regular
    Member

    Joined:
    Apr 24, 2019
    Messages:
    259
    Country:
    Mexico
    Which program you use to extract the 01.app from a wad?
     
  3. MaeseJesus

    MaeseJesus GBAtemp Advanced Fan
    Member

    Joined:
    Apr 3, 2013
    Messages:
    680
    Country:
    The program is named wwcxtool.exe, I got it long time ago somewhere here in gbatemp.

    It is a command line tool, so you need to know how to use it. On the folder you have the tool and 00000001.app I type cmd on the browser on the top and it will open it.

    The command goes like this:

    wwcxtool /u 00000001.app NameOfUncompressedfile.app

    Then, to compress it back again after modified:

    wwcxtool /c1 NameOfUncompressedfile.app 00000001.app

    And that's how I do it. Not every Wiiware game has a compressed 01.app, if it's over 2.5mb or larger the chances of being compressed are slim, and you can edit those from the get-go.
     
    NoobletCheese and Draxikor like this.
  4. contra1980

    contra1980 Newbie
    Newcomer

    Joined:
    Apr 10, 2021
    Messages:
    3
    Country:
    Russia
    The Legend of Zelda - Twilight Princess EUR - Matching code 08080A0C0A0808! Everything works, there is no filter like on NTSC. But the code 04080C100C0804 was not found!
     
  5. MaeseJesus

    MaeseJesus GBAtemp Advanced Fan
    Member

    Joined:
    Apr 3, 2013
    Messages:
    680
    Country:
    Not every game uses the same "code", but which each one uses is sort of meaningless because searching for the strings before is simpler and much more thourough.

    Most use 08 08 0A 0C 0A 08 08 though. It is weird that the other one wasn't on there, every game I modified myself did use it, yet not behind the 06 06 06 06 etc string, but the other one posted here.

    About GXSetCopyFilter and compiling Wii stuff, I won't be able to help, I cannot use devkitpro because I don't have a 64 bit computer.

    I don't want to bother other people with this, but maybe blackb0x could make a quick work of it if it's compiling just what you mentioned to check it on a hex editor.

    In any case all of this is a big breakthrough already.
     
    Draxikor likes this.
  6. NoobletCheese

    NoobletCheese GBAtemp Regular
    Member

    Joined:
    Aug 12, 2018
    Messages:
    165
    Country:
    United States
    Just to clarify, Devkitpro is the third party open-source SDK. As far as I can tell, Wii games are compiled with RVL SDK -- Nintendo's official closed-source SDK. To discover how GXSetCopyFilter appears as machine code in the hex data, I think we need to build some .dols with RVL SDK. It seems the final version of RVL SDK is 3.3.2, and it's against forum rules to post a link to it.

    My plan for today is to write an autohotkey script to search through DK's main.dol for some custom byte patterns that I think look like a vfilter int array, which if found would hopefully appear adjacent to a GXSetCopyFilter call.
     
    Last edited by NoobletCheese, Apr 12, 2021
    MaeseJesus likes this.
  7. NoobletCheese

    NoobletCheese GBAtemp Regular
    Member

    Joined:
    Aug 12, 2018
    Messages:
    165
    Country:
    United States
    Haven't discovered GXSetCopyFilter yet, but found the filters for DKC Returns and Sonic Colors:
    DK: 04 04 10 10 10 04 04
    Sonic: 04 04 0F 12 0F 04 04
     
    MaeseJesus and Draxikor like this.
  8. MaeseJesus

    MaeseJesus GBAtemp Advanced Fan
    Member

    Joined:
    Apr 3, 2013
    Messages:
    680
    Country:
    Do you appreciate any difference by modifying those values? I haven't got the chance to try it yet.
     
  9. NoobletCheese

    NoobletCheese GBAtemp Regular
    Member

    Joined:
    Aug 12, 2018
    Messages:
    165
    Country:
    United States
    Yes, and that is how I'm determining if I found the right values.

    My method is to search main.dol for a string of 7 bytes with the following pattern:

    Last 3 bytes = reversed copy of first 3 bytes
    and bytes 1,2,3,5,6,7 are not all equal
    and bytes 1,2,6,7 are not zeros
    and bytes 1,3,5,7 are not equal
    and bytes 2,4,6 are not equal
    and no bytes are > 0x40
    and bytes sum to 0x40
    or bytes are preceded by known AA sample patterns (060606.. or 030209...)

    This gives me a list of candidates with offsets so I can see if they're near other known filter patterns, which seems to increase the chance of it being a vfilter and not just some random data.

    I'll post the script once its finished.
     
    Last edited by NoobletCheese, Apr 18, 2021
    Draxikor and MaeseJesus like this.
  10. MaeseJesus

    MaeseJesus GBAtemp Advanced Fan
    Member

    Joined:
    Apr 3, 2013
    Messages:
    680
    Country:
    Thanks again. I looked around on the Main.dol and the only 04 04 10 10 10 04 04 is found at offset 553D28 on the European version, and it's the only string like that on the entire file, and does not have 060606 030209 before it but "03 00 00 00 08 00 00 00 05 00 00 00 0A 00 00 00 07 00 00".

    Is that true for the NTSC version?
     
    Draxikor likes this.
  11. NoobletCheese

    NoobletCheese GBAtemp Regular
    Member

    Joined:
    Aug 12, 2018
    Messages:
    165
    Country:
    United States
    I've got the NTSC version and it's at offset 554890, but still preceded by that same string.
    If you look another 0x11 bytes further you should see a 00001516150000.
     
    MaeseJesus likes this.
  12. Razor83

    OP Razor83 GBAtemp Fan
    Member

    Joined:
    Dec 23, 2009
    Messages:
    370
    Country:
    I just checked and managed to find the GXNtsc480ProgSoft function listed in the official GC and Wii SDK's, take a look at the RenderMode.html file found in the following locations:-

    DolphinSDK1.0/man/gfx/gx/Framebuffer/RenderMode.html
    RVL_SDK/man/en_US/gx/Framebuffer/RenderMode.html

    Out of interest would the GXSetCopyFilter.html file found in the same SDK folder be of any help?

    Your doing great work here NoobletCheese, i'm really glad somebody is taking an interest in this issue :)
     
    Last edited by Razor83, Apr 14, 2021
  13. NoobletCheese

    NoobletCheese GBAtemp Regular
    Member

    Joined:
    Aug 12, 2018
    Messages:
    165
    Country:
    United States
    Those are definitions of GXRenderModeObj's, which is what we're currently patching in the hex editor.

    That is the page I was quoting in post 31 and 34. If we can find the hex values corresponding to its arguments, then we should be able to reliably set vfilter and sample_pattern for any game, since I believe all games set them via that function, as according to the SDK, VIConfigure (the main function which sets the GXRenderModeObj) does not set its vfilter and sample_pattern elements.



    Looking at the sample_pattern 03020906030A03020906030A09020306090A09020306090A, I notice it also contains symmetry and delimiters which may be useful for finding more of them with the right search pattern.

    Code:
    0302090603  0A  0302090603  0A  0902030609  0A  0902030609  0A
    AAAAAAAAAA  DD  AAAAAAAAAA  DD  BBBBBBBBBB  DD  BBBBBBBBBB  DD
    
    edit: also according to SDK, the sample pattern values must all be less than 0x0C, and the y-coordinates must be in ascending order for each quadrant -- that should help a lot.

    edit2: also the SDK says the sum of all 7 bytes for the vfilter should be 0x40, which will help rule out false positives.
     
    Last edited by NoobletCheese, Apr 15, 2021
    Draxikor and Razor83 like this.
  14. Razor83

    OP Razor83 GBAtemp Fan
    Member

    Joined:
    Dec 23, 2009
    Messages:
    370
    Country:
    NoobletCheese likes this.
  15. NoobletCheese

    NoobletCheese GBAtemp Regular
    Member

    Joined:
    Aug 12, 2018
    Messages:
    165
    Country:
    United States
    After patching all found 24 byte patterns in Xenoblade according to a search rule specified by the SDK for AA sample_pattern where:

    - all bytes <= 11
    - y-coordinates are in ascending order

    I observe no change in image clarity.

    Additionally the following image shows what AA should look like when enabled:

    [​IMG]

    As Xenoblade doesn't appear to have such antialiased edges, I suspect Xenoblade is not using AA.

    Since it is not using vfilter either, I suspect it's simply the game's depth of field effect causing the blur, as it seems to get worse the further away from the camera objects are.

    Or it could be some other bespoke pixel shader the developer is using.
     
    Last edited by NoobletCheese, Apr 15, 2021
    Draxikor and kingjinxy2 like this.
  16. NoobletCheese

    NoobletCheese GBAtemp Regular
    Member

    Joined:
    Aug 12, 2018
    Messages:
    165
    Country:
    United States

    I am so glad you mentioned Wii Backup Fusion as I saw it uses cygwin1.dll to run wit.exe silently at the command line, which will now allow me to make a GUI tool to automate the vfilter patching. i.e taking an iso/wbfs as an input file, and output a patched [GameID].dol onto SD/USB for use by USB Loader.

    Wit.exe is Wiimms ISO tool which can extract single files from wbfs/iso files, such as main.dol. It even works for GameCube games! Sidenote: I am confident to patch the deflicker out of Starfox Adventures, as I saw its main.dol contains the same vfilter used by Mario Kart Wii.

    To extract main.dol:
    Code:
    wit extract --source "c:\my games\game.wbfs" --dest "c:\destination folder" --files +/sys/main.dol --overwrite
    

    How I envisage it will all work


    A database of game ID's and their known filters will be kept in the cloud, eg. Pastebin (I have already set this up).

    We currently have the following entries:

    DonkeyKongCountryReturns,NTSC-U,SF8E01,04041010100404
    MetroidOtherM,NTSC-U,R3OE01,08080A0C0A0808
    MarioKartWii,NTSC-U,RMCE01,07070C0C0C0707
    MarioKartWii,NTSC-U,RMCE01,05050F0E0F0505
    SonicColours,NTSC-U,SNCE8P,04040F120F0404
    SuperMarioGalaxy,NTSC-U,RMGE01,20002000000000
    SuperMarioGalaxy2,NTSC-U,SB4E01,20002000000000
    Ntsc480Prog,,,00001516150000
    Ntsc480ProgSoft,,,08080A0C0A0808
    Ntsc480Aa,,,04080C100C0804

    If you come across a game which doesn't use any of the above filters, my tool should be able to find it by its search patterns. Currently it is capable of finding vfilters for Donkey Kong, Sonic Colours, Mario Kart and Mario Galaxy just from looking at byte patterns. In addition to this it will also match filters from the pastebin.

    Every time someone finds a unique vfilter which isn't in the pastebin, they should add it to the pastebin. Over time we should have enough vfilters that the majority of games should be easy to patch.


    USB Loader GX integration

    This appears to be achievable by USB Loader scraping the game's filter from the cloud and patching it with its Search_and_patch_Video_To function.

    If a game isn't in the cloud, it is still likely to work by patching instances of commonly used ones that are in the cloud. The probability of this working will also increase over time as there will be enough known vfilters that one of them is likely to match the one used by any game.


    Timeline

    I am aiming to have something ready in the next 10 days or so. I need to spend a lot of time making sure my patching function uses all the correct offsets and doesn't corrupt any data in main.dol. This will probably take a couple of days to iron out any bugs, then I can get started on the GUI. I am writing it in Autohotkey which makes it a little bit difficult as its internal handling of integers can be a bit unpredictable at times.
     
    Last edited by NoobletCheese, Apr 17, 2021
  17. MaeseJesus

    MaeseJesus GBAtemp Advanced Fan
    Member

    Joined:
    Apr 3, 2013
    Messages:
    680
    Country:
    I have not found values different from the ones you've written up there between the games I have, and they're several dozens, although of course, they were mostly PAL versions. This fortunately means that all the information found can be used for any game independently from what region they are.

    04080C100C0804 is by far the most common, being in every game I've checked so far. 08080A0C0A0808 is the second most common, which makes me think both of these are the default options upon developing for the console. The others seem to be mostly used by Nintendo-related games for the most part. A handful of games use 07 07 0C 0C 0C 07 07, and none use the Galaxies' string at all.

    Your tool sounds wonderful, I've been doing this with normal hex searches myself.
    Do you need a list of each game's entries, or just of all the different entries themselves?

    Edit: Small detail but, even for games only working in 480i, you can remove the filters, which can be bad for you. The picture will be clearer, yes, but because it is interlaced, but you'll see the constant interlacing (two frames of the picture interchanging fast) which can harm your eyes a lot and very quickly.

    I think this is the reason of why 480i games like Mega Man 9 and 10 use such aggressive filters. Not having them in combination with huge pixels results in a very jumpy picture that tortures your eyes.

    So anyone reading, don't do that. I need to try if you can still patch 480p through USB Loader GX (only works for retail games) while also having the Main.dol modified to remove the filters. I have Rune Factory Frontier and Sega Bass Fishing which are two of those 480i only games, so I'll get to that this weekend.
     
    Last edited by MaeseJesus, Apr 17, 2021
  18. NoobletCheese

    NoobletCheese GBAtemp Regular
    Member

    Joined:
    Aug 12, 2018
    Messages:
    165
    Country:
    United States
    Ideally the cloud data should store each game's vfilter plus its offset.

    If we don't include the offset, then USB Loader integration may not be possible as USB Loader won't necessarily know if the string is a real vfilter or some other random data.

    eg. Mario Galaxy's 20002000000000 is too generic to simply "find and replace all".

    Even "find and replace all" instances that follow a known sample_pattern like 060606... or 030209... won't work because there are games like Donkey Kong and Sonic Colours whose vfilter isn't preceded by a sample_pattern (or any other identifiable part of GXRModeObj).

    So USB Loader could only patch those games if it knows the offset.

    However this creates another problem: in my tool, the user will typically just "patch all known vfilters" because it is a convenient one-click solution. But this fails to identify which offset is the one the game uses, and so doesn't help USB Loader integration.

    In the end perhaps we don't need USB Loader integration, because the pool of total unique vfilters is probably not that large so using the tool will be a one-click solution most of the time anyway.

    Since there is no harm to having the offsets in the cloud, I will implement that anyway, and hopefully one user per game will be caring enough to do a few trial and errors to find which offset the game uses, and upload that data to cloud. This will be partially assisted by displaying the adjoining GXRModeObj members when user clicks on each vfilter in the list of found vfilters after searching the dol.
     
    Last edited by NoobletCheese, Apr 18, 2021
    Draxikor and XFlak like this.
  19. XFlak

    XFlak Wiitired but still kicking
    Member

    Joined:
    Sep 12, 2009
    Messages:
    11,311
    Country:
    Cyprus
    I was a out to reply to this, but then I saw this next post
    ModMii does the same thing, but actually you can just use the portable zip for wit, it should come with all the files needed. Cygwin files don't need to be in the system 32 folder, they can just be saved to the same directory as wit.exe and it will get picked up from there, so no need to run any kind of complicated installation

    Edit: btw for hex editing with ModMii I use hexalter.exe, in case it's helpful

    Edit2: and yes, knowing the offsets is vital
     
    Last edited by XFlak, Apr 18, 2021
    NoobletCheese likes this.
  20. MaeseJesus

    MaeseJesus GBAtemp Advanced Fan
    Member

    Joined:
    Apr 3, 2013
    Messages:
    680
    Country:
    I wrote down the offsets and VFilters used by some of the games I have personally edited.

    Code:
    Battalion Wars 2
    RWBP01    PAL
        VFilter:                Offsets
        08 08 0A 0C 0A 08 08    4D71AA
                                4D7222
                                4D725E
                                4D729A
                                4D72D6
                                4D7312
                                4D734E
                                4E9772
                             
    Final Fantasy Fables: Chocobo Dungeon
    R7FPGD    PAL
        VFilter:                Offsets
        08 08 0A 0C 0A 08 08    4A0512
                                4A058A
                                4A0602
                                4A063E
                             
    Donkey Kong Country Returns
    SF8P01    PAL
        VFilter:                Offsets
        04 04 10 10 10 04 04    553D28    > No string before it.
        08 08 0A 0C 0A 08 08    57E782
                                57E7FA
                                57E836
                                57E872
                                58DA8E
        04 08 0C 10 0C 08 04    58DACA
    
    Donkey Kong: Jungle Beat NPC
    R49P01
        VFilter:                Offsets
        20 00 20 00 00 00 00    3A39EA
                                3A3A26
                                3A3A62
                                3A3A9E
                                3A3ADA
                                3A3B16
                                3A3B52
                                3A3B8E
                                3A3BCA
                                3A3C06
        08 08 0A 0C 0A 08 08    477006
                                477CE2
                                477D5A
                                477D96
                                477DD2
        04 08 0C 10 0C 08 04    477042
     
    Super Mario Galaxy 1
    RMGP01    PAL
        VFilter:                Offsets
        20 00 20 00 00 00 00    53641A
                                536456
                                536492
                                5364CE
                                53650A
                                536546
                                536582
                                5365BE
                                5365FA
                                536636
        08 08 0A 0C 0A 08 08    5E4BB6
                                5E5822
                                5E589A
                                5E58D6
                                5E5912
        04 08 0C 10 0C 08 04    5E4BF2
     
    Super Mario Galaxy 2
    SB4P01    PAL
        VFilter:                Offsets
        20 00 20 00 00 00 00    6490F2
                                64912E
                                64916A
                                6491A6
                                6491E2
                                64921E
                                64925A
                                649296
                                6492D2
                                64930E
        08 08 0A 0C 0A 08 08    71C0E6
                                71CDAA
                                71CE22
                                71CE5E
                                71CE9A
        04 08 0C 10 0C 08 04    71C122
     
    Kirby's Adventure Wii
    SUKP01    PAL
        VFilter:                Offsets
        07 07 0C 0C 0C 07 07    7149EA
                                714AFA
                                714B82
                                714C92
                                714DA2
                                714E2A
        08 08 0A 0C 0A 08 08    73D7BE
                                73E532
                                73E56E
                                73E5AA
                                73E5E6
        04 08 0C 10 0C 08 04    714A2E
     
    Kirby's 20th Anniversary Collection
    S72E01    NTSC US
        VFilter:                Offsets
        07 07 0C 0C 0C 07 07    4106EA
                                4107FA
                                41083E
                                4108C6
                                4109D6
                                410A1A
        08 08 0A 0C 0A 08 08    42C01E
                                42CD92
                                42CDCE
                                42CE0A
                                42CE46
        04 08 0C 10 0C 08 10    41072E
     
    Mario Kart Wii
    RMCP01    PAL
        VFilter:                Offsets
        07 07 0C 0C 0C 07 07    2417EA
                                241862
                                24189E
                                241916
                                24198E
                                2419CA
                                254412
                                25448A
                                2544C6
                                25453E
                                2545B6
                                2545F2
        05 05 0F 0E 0F 05 05    241826
                                2418DA
                                241952
                                241A06
     
    Super Mario Sluggers
    RMBE01    NTSC US
        VFilter:                Offsets
        08 08 0A 0C 0A 08 08    6F4A76
                                6F8D12
                                6F8D8A
                                6F8DC6
                                6F8E02
                                6F8E3E
                                6F8EB6
        04 08 0C 10 0C 08 04    6F4AB2
     
    Monster Hunter Tri
    RMHP08    PAL
        VFilter:                Offsets
        08 08 0A 0C 0A 08 08    6173B2
                                6173EE
                                61742A
                                617466
                                6174A2
                                628016
        04 08 0C 10 0C 08 04    628052
     
    New Super Mario Bros. Wii
    SMNP01    PAL
        VFilter:                Offsets
        07 07 0C 0C 0C 07 07    2F9122
                                2F919A
                                2F91D6
                                2F924E
                                2F92C6
                                2F9302
        08 08 0A 0C 0A 08 08    33C736
                                33D51A
                                33D592
                                33D5CE
                                33D60A
        04 08 0C 10 0C 08 04    33C772
                     
    
    Metroid Prime Trilogy
    R3MP01    PAL
        VFilter:                Offsets
        08 08 0C 0A 0C 08 08    558AAA
                                558B22
                                558B5E
                                558B9A
                                5897DE
        04 08 0C 10 0C 08 04    56881A
     
    Red Steel 2
    RD2X41    Region Free? Bought in Spain.
        VFilter:                Offsets
        08 08 0A 0C 0A 08 08    4291D6
                                47D50E
                                48C49A
                                48C4D6
                                48C512
                                48C54E
                                48C58A
                                48C5C6
        04 08 0C 10 0C 08 04    47D54A
     
    Rune Factory Frontier
    RUFP99    PAL
        08 08 0A 0C 0A 08 08    3423DE
                                347C32
                                347C6E
                                347CAA
                                347CE6
                                347D22
                                347D5E
        04 08 0C 10 0C 08 04    34241A
     
    Mario Sports Mix
    RMKP01
        VFilter:                Offsets
        08 08 0A 0C 0A 08 08    49C5AE
                                4A1312
                                4A134E
                                4A138A
                                4A13C6
                                4A1402
                                4A143E
        04 08 0C 10 0C 08 04    49C5EA
    

    I'll be adding more as I have the time. Most of these are for PAL games, but as I said before, while the offsets wont be as useful, the listed filters will be equal for NTSC games.

    I also tried to load an edited Main.Dol while also forcing 480p to 480i only games and all seems to work together on Rune Factory Frontier, I hope the other games behave in the same way.

    Disaster: Day of Crisis is another game that has a stray filter without the "06 06 06" string before it, but it is not new. I'll add that one at another time.
     
    Last edited by MaeseJesus, Apr 18, 2021
Draft saved Draft deleted
Loading...

Hide similar threads Similar threads with keywords - (De)Flicker, Possible, Disable