Hacking Rominator for NDS

[IBNobody]

Well....

It took 30-40 seconds to recognize my 418 file collection. (10GB Compressed, 93GB Uncompressed according to Rominator) I'll have to see if a refresh/addition takes this long.

Rominator saw all my zips, but it didn't iconize them. My guess is that it's something in Rominator, as all my zips contain 1 file. All my files are good, according to AdvanSCENE.

I'm batch-identifying them now, but that process is very slow. How does that identification process work? Are you doing a partial decompress or are you doing the whole thing?

Don't worry about zip compression yet... But you do need to get zip decompression working. After all, the best part of your program is the flashcard management. Sure, I can have "Mary Kate & Ashley's Sweet 16" in a zip... but what if I really wanted to play it on my R4?

Misc Questions:

Did you implement the menu bar change thing that would allow the saves/tools/options windows to toggle state?

How will you indicate trainers on Zips?

EDIT:

Wow... That... Took... A long time! But now I have all my icons sucked in. Closing and opening the program is fast. Icons are retained. They are also retained after a refresh (thank god).

There's something screwy going on with Pac'n Roll (US). The ROM's CRC is 0xA9A1D018. This matches the DB in Rominator. However, I cannot identify the ROM or suck the icon in. The containing zip is okay. This is the only item that's failing. All other icons sucked just fine.

The ROM validator doesn't work for zips. It tells you the zip CRC and not the actual ROM's CRC.

EDIT 2: Add Zendoku and Dynasty Warriors to the undetected / no-icon list... These aren't like Pac'n Roll because the Rominator DB won't even recognize them by showing boxart. (Pac'N Roll has no icon, but has a DB entry.)

 

[NemesisHunter]

Checking out on the update as well

.

 

[Typhoe]

Crashes when executing the beta 102807 linux version with this error.

Runtime Error 4: Failed Assertion
Press OK to Continue
Press Cancel to Quit.

Please report what caused this error along with the information below.
../Common/loaderX86.cpp:268
Failure Condition: lib->mLibraryHandle
Missing shared library /usr/lib/libc.so

Then the same window will appear only difference of...
../Common/loaderX86.cpp:282
Failure Condition: functionEntry
Could not resolve function 'time' in /usr/lib/libc.so

../Common/loaderX86.cpp:282
Failure Condition: functionEntry
Could not resolve function 'localtime_r' in /usr/lib/lib.so

Tried it on two boxes. Gentoo ~x86 arch and Ubuntu Gutsy Gibbon

Hi,

got exactly the same messages on Gutsy 64bit.

Here is the log I get in the terminal from where I launched rominator:
./rominatorbeta_102807

(rominatorbeta_102807:6941): Gtk-WARNING **: Error loading theme icon 'gtk-cancel' for stock: Impossible de charger le module de chargement d'images : /usr/lib32/gtk-2.0/2.10.0/loaders/svg_loader.so : /usr/lib32/gtk-2.0/2.10.0/loaders/svg_loader.so: Ne peut ouvrir le fichier d'objet partagé: Aucun fichier ou répertoire de ce type

(rominatorbeta_102807:6941): Gtk-WARNING **: Error loading theme icon 'gtk-cancel' for stock: Impossible de charger le module de chargement d'images : /usr/lib32/gtk-2.0/2.10.0/loaders/svg_loader.so : /usr/lib32/gtk-2.0/2.10.0/loaders/svg_loader.so: Ne peut ouvrir le fichier d'objet partagé: Aucun fichier ou répertoire de ce type

(rominatorbeta_102807:6941): Gtk-WARNING **: /usr/lib/gtk-2.0/2.10.0/immodules/im-cedilla.so: wrong ELF class: ELFCLASS64

(rominatorbeta_102807:6941): Gtk-WARNING **: Loading IM context type 'cedilla' failed
Erreur de segmentation (core dumped)

Regards,
Typhoe

 

[hankchill]

Well....

It took 30-40 seconds to recognize my 418 file collection. (10GB Compressed, 93GB Uncompressed according to Rominator) I'll have to see if a refresh/addition takes this long.

Yeah, I found the initial caching period to take a bit, but after that, it was smooth sailing from then on correct? Next relaunch should be snappy.

Rominator saw all my zips, but it didn't iconize them. My guess is that it's something in Rominator, as all my zips contain 1 file. All my files are good, according to AdvanSCENE.

It can't iconize them because their icons need to be stored in the cache. Each NDS game has it's icon data stored at different locations, so an instant recognition of icons would not work. An identification process would have to take place.

I'm batch-identifying them now, but that process is very slow. How does that identification process work? Are you doing a partial decompress or are you doing the whole thing?

As mentioned above, because the icon location is in a funny location for all games, I have to do an entire decompress. Maybe after it's completely stable, I'll add in some smart methods like seeking directly to the game for decompression and only decompressing to the last byte of the icon

Don't worry about zip compression yet... But you do need to get zip decompression working. After all, the best part of your program is the flashcard management. Sure, I can have "Mary Kate & Ashley's Sweet 16" in a zip... but what if I really wanted to play it on my R4?

I guess you didn't find out on your own, but if you drag a Zip to your flash card, it will be decompressed and flashed

So enjoy your Mark Kate & Ashley's Sweet 16

Misc Questions:

Did you implement the menu bar change thing that would allow the saves/tools/options windows to toggle state?

Nope, forgot about it

I'll fix that.

How will you indicate trainers on Zips?

Currently, I can't. What I will do is probably extend the details window to include information pertaining to the file (such as format, actual bytes, etc)

EDIT:

Wow... That... Took... A long time! But now I have all my icons sucked in. Closing and opening the program is fast. Icons are retained. They are also retained after a refresh (thank god).

Told ya it would be snappy

Yes I made it "smart" so if you did a refresh, it would only add in the icons whose data did not exist and just append it to my super happy 2D Array

. Heck, you could go and delete the iconCache.rsd file, and it will just go and recache everything that's missing... unless they're zips

There's something screwy going on with Pac'n Roll (US). The ROM's CRC is 0xA9A1D018. This matches the DB in Rominator. However, I cannot identify the ROM or suck the icon in. The containing zip is okay. This is the only item that's failing. All other icons sucked just fine.

That's weird... I'd have to see the zip to rectify this issue...

The ROM validator doesn't work for zips. It tells you the zip CRC and not the actual ROM's CRC.

Nope, the Rom Validator doesn't work for zips. Nothing in the Toolbox works with Zips - Shoulda mentioned that too

EDIT 2: Add Zendoku and Dynasty Warriors to the undetected / no-icon list... These aren't like Pac'n Roll because the Rominator DB won't even recognize them by showing boxart. (Pac'N Roll has no icon, but has a DB entry.)

I'll try with these games a compression and then put it in Rominator -- See what happens.

Phew, that's a lot for 5:00 AM!

As for you Linux guys, looks like something's not right here... wonder if there's something we're missing. Does the Windows release work properly under WiNE?

--Henry

 

[IBNobody]

EDIT:

The error message I got when trying to copy my zips to my "flashcart" was:

Error #001,
unauthorized device has been detected.

No... Seriously...

Error!

There was an error while copying the rom to flash. It is possible that your rom or Flash Drive was ejected or the Rom does not exist anymore

Now, maybe this was being caused by something else. My "flashcart" was actually a temp drive on my HD.

One other error: My settings initially pointed to a dir with some unzipped Roms. When I changed the game directory in the settings to my zipped directory and did a refresh, the old roms remained in my list. Shouldn't a dir change and a refresh purge the library list?

 

[IBNobody]

Whoops! I see that I edited most of my info out of my post. Here's the jist of what I said:

1. Transferring Zipped ROMs to flashhcarts didn't work. It gave me the error mentioned above.

2. Your entries for Dynasty and Zendoku were the old versions. The scene nuked these and released propers for these. Thus, the corrected versions weren't recognized.

3. I have no clue why Pac'n Roll fails. It is recognized and brings up boxart, but it can't find the icon.

4. How deep in a file have you found icons? If you can just restrict your search to the first 1MB, maybe the scan-in would be speedier.

 

[hankchill]

One other error: My settings initially pointed to a dir with some unzipped Roms. When I changed the game directory in the settings to my zipped directory and did a refresh, the old roms remained in my list. Shouldn't a dir change and a refresh purge the library list?

No, this will not purge your library. Your library is only a cache of games that are added, so technically you can drag games from all over your hard drive to the library into Rominator and it will retain the information unless removed.

Error #001,
unauthorized device has been detected.

Yeah, I got the same error when I dragged Super Mario Galaxy onto Rominator

1. Transferring Zipped ROMs to flashhcarts didn't work. It gave me the error mentioned above.

I'll have to take a peekaroni.

2. Your entries for Dynasty and Zendoku were the old versions. The scene nuked these and released propers for these. Thus, the corrected versions weren't recognized.

I'll update the database for tonight's update

3. I have no clue why Pac'n Roll fails. It is recognized and brings up boxart, but it can't find the icon.

I tried with Pac'n Roll... Works fine

4. How deep in a file have you found icons? If you can just restrict your search to the first 1MB, maybe the scan-in would be speedier.

I wrote a program that views the game icons and shows its location in the ROM... they can get pretty deep, so I'll examine about 100 and see how deep they go. Indeed it would speed up the process to do a partial decompress... you know what I can probably do eh -- if I was *really* good, I would have it partially decompress the header, get the Icon Location from the header, and then set the max byte size to the location of the found header & continue decompression, and stop at that point+ 512 bytes, so the entire icon can be retrieved. That would take a while but blow my mind away -- I'd have to sit down and code that one good, but it can be done!

 

[IBNobody]

It's not a problem with Pac'n Roll. It appears to be a caching issue.

1. I removed PNR from my games list, re-added and re-identified it. Result: No icon.

2. I decompressed PNR and recompressed it using 7Zip's ZIP and DEFLATE settings to create a clean zip with no comments or extra files. (There were comments in the original.) I removed PNR and re-added/ID'd it. Result: No icon.

3. I took the uncompressed PNR and added it. Result: Icon!

4. I removed the uncompressed PNR and added the compressed PNR. Result: Icon!

Apparently, I was able to get the icon into the cache with #3 and use it in #4. My guess was that the cached icon was corrupted.

---

Let's revisit something you talked about before.

Your beta only displays icons for ROMs in the database. Why make us zippers spend 20 minutes pulling the icons out of our ROMs when you're only going to display ones in the database? Why recreate the icon cache when the end user can't add anything to the cache?

It makes the most sense to take a pre-built icon database (i.e. that SQLite icon cache file) and bundle it with your full database.

 

[hankchill]

It's not a problem with Pac'n Roll. It appears to be a caching issue.

1. I removed PNR from my games list, re-added and re-identified it. Result: No icon.

2. I decompressed PNR and recompressed it using 7Zip's ZIP and DEFLATE settings to create a clean zip with no comments or extra files. (There were comments in the original.) I removed PNR and re-added/ID'd it. Result: No icon.

3. I took the uncompressed PNR and added it. Result: Icon!

4. I removed the uncompressed PNR and added the compressed PNR. Result: Icon!

Apparently, I was able to get the icon into the cache with #3 and use it in #4. My guess was that the cached icon was corrupted.

---

Let's revisit something you talked about before.

Your beta only displays icons for ROMs in the database. Why make us zippers spend 20 minutes pulling the icons out of our ROMs when you're only going to display ones in the database? Why recreate the icon cache when the end user can't add anything to the cache?

It makes the most sense to take a pre-built icon database (i.e. that SQLite icon cache file) and bundle it with your full database.

Ah, but the end user can add to the cache

Each game added that has a readable icon is added to the cache, so no matter how many times that game is added/readded, it will always display the cached icon. (this is another reason why rZip rocks

)

I'm against doing the pre-built icon database, because up to games 1570 would already be a 10MB database, which I don't want to host and would only get bigger with each database update. Whatever the people want to have in their database, in reality, they'll have to scan themselves. Heck, at least they only have to do it once.

That Pac'N Roll though... such strange behaviour

 

[IBNobody]

Ah, but the end user can add to the cache

Each game added that has a readable icon is added to the cache, so no matter how many times that game is added/readded, it will always display the cached icon. (this is another reason why rZip rocks

)

I'm against doing the pre-built icon database, because up to games 1570 would already be a 10MB database, which I don't want to host and would only get bigger with each database update. Whatever the people want to have in their database, in reality, they'll have to scan themselves. Heck, at least they only have to do it once.

That Pac'N Roll though... such strange behaviour

You're right about the hosting... There may be ways around it, depending if you want to leech icons from somewhere.

But if you are going with the caching, why can't you let us pull in icons from games that aren't in your database? (I tried - you can't.) This includes ROMs and zipped ROMs. If you are going to stick with a caching system, take it to its logical conclusion and allow for caching of all ROM icons, even those not in the DB.

You've put together a "smart" program to determine the icon location. Does Rominator not know where the icons are located on a game-to-game basis?

 

[hankchill]

Ah, but the end user can add to the cache

Each game added that has a readable icon is added to the cache, so no matter how many times that game is added/readded, it will always display the cached icon. (this is another reason why rZip rocks

)

I'm against doing the pre-built icon database, because up to games 1570 would already be a 10MB database, which I don't want to host and would only get bigger with each database update. Whatever the people want to have in their database, in reality, they'll have to scan themselves. Heck, at least they only have to do it once.

That Pac'N Roll though... such strange behaviour

You're right about the hosting... There may be ways around it, depending if you want to leech icons from somewhere.

But if you are going with the caching, why can't you let us pull in icons from games that aren't in your database? (I tried - you can't.) This includes ROMs and zipped ROMs. If you are going to stick with a caching system, take it to its logical conclusion and allow for caching of all ROM icons, even those not in the DB.

You've put together a "smart" program to determine the icon location. Does Rominator not know where the icons are located on a game-to-game basis?

Rominator only knows the icon location when it opens the Rom. In the header, there is a 4 byte value which tells the icon location, which is why the Zip would need to be unzipped anyways so it can go to that location and extract the icon. Honestly there's not much of another way around it. Besides, Rominator also has the ability to edit a game's internal data (header and Icon), so caching won't even work then. The location of the icon is 100% different for each game, so no matter what you're going to have to do a little processing. I thought you said a long time ago you don't mind a little pre-processing

 

[IBNobody]

Rominator only knows the icon location when it opens the Rom. In the header, there is a 4 byte value which tells the icon location, which is why the Zip would need to be unzipped anyways so it can go to that location and extract the icon. Honestly there's not much of another way around it. Besides, Rominator also has the ability to edit a game's internal data (header and Icon), so caching won't even work then. The location of the icon is 100% different for each game, so no matter what you're going to have to do a little processing. I thought you said a long time ago you don't mind a little pre-processing

I figured as much. So... Unzip enough to get the icon location, then unzip 512 bytes past the start of the icon. (Like you said before.)

I don't mind pre-processing. I was just trying to lessen it in case I have to rebuild the cache.

 

[cheatah]

Just did a quick run

The beta windows version seems to run under wine. During the initial setup it crashes out but after that it seems to be fairly stable although the cursor disappears when it is over the rom area and you can't drag any roms into the program rendering it not very usable.

wine-0.9.47

 

[zloty]

So does this prog support 7zip yet?

I'd love to swap from OfflineList to this but without 7zip support it's useless to me ATM...

 

[IBNobody]

So does this prog support 7zip yet?

I'd love to swap from OfflineList to this but without 7zip support it's useless to me ATM...

No, and it probably won't, unless someone is willing to write a decompression routine in RealBasic.

He added zip support because there was a RealBasic module for zip decompression.

If you're using one of the less popular (7z) or proprietary (rar) formats, you're SOL.

 

[zloty]

Actually, hankchill mentioned that he was working on adding 7zip support

calling 7z less popular is misleading; morelike, less-known...

 

[hankchill]

Actually, hankchill mentioned that he was working on adding 7zip support

calling 7z less popular is misleading; morelike, less-known...

I never said I was adding it. I said I would look into it, but no guarantees.

No, there will not be 7z support. Sorry.

I would call 7z less popular. everyone knows about it, but honestly I wouldn't want it as my compression method choice, because it isn't as supported as, say, standard Zip.

 

[zloty]

but it
compresses smaller
software runs faster
and
it's open source

it will take over zip eventually, it's like IE and firefox. You should try and accomodate for it!

 

[hankchill]

but it
compresses smaller
software runs faster
and
it's open source

it will take over zip eventually, it's like IE and firefox. You should try and accomodate for it!

Well, maybe when it takes over Zip then I will reconsider, but as it is, Zip seems to be the compression method of choice.

 

[IBNobody]

but it
compresses smaller
software runs faster
and
it's open source

it will take over zip eventually, it's like IE and firefox. You should try and accomodate for it!

If you are willing to convert the algorithms for 7z to RealBasic or create an RB plug-in that works on MacOS, XP, Vista and Linux, Hank would jump at the chance to use them.