Homebrew RetroArch - A new multi-system emulator

[Jacobeian]

Off course, i was not talking about huge ROM file or CD image file that cannot be stored in RAM entirely at load time, that was not my point at all and is irrelevant to discussion (you are basically saying that loading a full 10MB ROM takes more RAM than only loading chunks of 2MB from time to time, i think everyone agree with that lol, it´s obvious). I am speaking of nes, snes or genesis files, that other emulators have no problem fully decompressing into RAM. After that, the file is closed and does not exist anymore for the emulator, there is no file system and no uncompressed file into RAM, only raw data from the uncompressed file copied where it should be, which the emulator decodes as it emulates the console.

Flash drive and hdd are similar, anything that use a file system and drive based hardware is similar, it IS less efficient than DIRECT memory access because the data transfer rate and CPU processing required is nowhere the same (where do you think the file system is stored ? it needs to be loaded from the drive as well then processed by the file API while a direct RAM access only takes memory access time, which is generally completely transparent on modern consoles)

It's not rocket science, really.

 

[the_randomizer]

Here's hoping a new version will be released soon!

 

[Toad King]

Here's hoping a new version will be released soon!

Not to dash your hopes, but at the moment most of the development work is going into the Android port, and a new release will probably be on hold until that is in a usable state. We may have an intermediate release, but only if Android development takes too long.

 

[Rydian]

Off course, i was not talking about huge ROM file or CD image file that cannot be stored in RAM entirely at load time, that was not my point at all and is irrelevant to discussion (you are basically saying that loading a full 10MB ROM takes more RAM than only loading chunks of 2MB from time to time, i think everyone agree with that lol, it´s obvious).

I was responding to what you posted. You said "always" and "period", which is why I corrected you. Since there's common exceptions, it is not a "period" statement and is not "always" true.

I am speaking of nes, snes or genesis files, that other emulators have no problem fully decompressing into RAM.

That's because those emulators have a lot more free RAM. This is why I brought up PC emulators working with huge disc images, to give an example of how other emulators handle things depending on the resources available and the data size being worked with.

Flash drive and hdd are similar, anything that use a file system and drive based hardware is similar
[...]
(where do you think the file system is stored ? it needs to be loaded from the drive as well

I take it the concept of a RAMdrive is one you're unfamiliar with?

How do you think Linux LiveCDs work? They load the entire "harddrive" into RAM, filesystem and all... hell, they often have multiple partitions. One is the discs's data, then they have a storage area (for saving downloaded files and installing programs), a section of RAM dedicated to use as actual RAM, etc...

And before you say something like "well that's linux", that's just an extreme example. RAMdrives are used elsewhere as well, even for applications within Windows.

 

[the_randomizer]

Not to dash your hopes, but at the moment most of the development work is going into the Android port, and a new release will probably be on hold until that is in a usable state. We may have an intermediate release, but only if Android development takes too long.

Well, that's fine, no biggie I can wait. I was mostly referring to how the Snes9x Next core may have a memory leak (as in gradual performance loss over time). Games that run full speed upon immediately launching the core tend to slow down over the period of an hour (sorry, just want to emphasize the issue) or so, this is especially noticeable in the audio department. I'm not sure if more optimizations need to be made or what, but I have full confidence that this issue can be completely neutralized. Just thought I'd let you guys know about it.

 

[Jacobeian]

That's because those emulators have a lot more free RAM. This is why I brought up PC emulators working with huge disc images, to give an example of how other emulators handle things depending on the resources available and the data size being worked with.

Not really, standalone emulators have GUI taking up huge memory space.
From studying the code, I think it's all because of the design of Retroarch which would force the need of the ROM buffer to be duplicated to be copied into proper core memory: indeed they took out the zip decompressing code out of the original cores when it existed (for memory saving I think) so they cannot pass the zipped file directly to the emulator code (so that it decompress it to proper memory locations like standalone cores do) but instead must first unzip the file externally and then pass it to the cores.

In regard to your initial statement that made me react, when an emulator unzips a file(s), there is no temporary unzipped file(s) in RAM or wherever, it unzips the CONTENT of the file(s), which is plain ROM data, into memory buffer(s) , which is then accessed through direct memory access (NOT file access) by the emulator core. So nothing got "deleted", it's just a buffer, it's cleared or overwritten when you load another game. Cores in Retroarch do the same, the difference is that Retroarch unzips the file itself then write the content of the file into another file on HDD or SD so that it can then be loaded by the emulator core in memory. At the end, memory usage is exactly the same (mind the extra memory used by standalone emulators extra features)

It's all due to the multi-core design. I'm not saying it's bad, I'm only saying that it has his benefits and disadvantages .

I take it the concept of a RAMdrive is one you're unfamiliar with?

How do you think Linux LiveCDs work? They load the entire "harddrive" into RAM, filesystem and all... hell, they often have multiple partitions. One is the discs's data, then they have a storage area (for saving downloaded files and installing programs), a section of RAM dedicated to use as actual RAM, etc...

And before you say something like "well that's linux", that's just an extreme example. RAMdrives are used elsewhere as well, even for applications within Windows.

I don't think you understood what i was saying or maybe we are simply not on the same page
l am quite familiar with these concepts but it has NOTHING to do with the initial discussion and is completely unrelated to Wii programming which I was referring to in the first place ^^
If if it's for the sake of saying something or showing off technical knowledge, I am afraid I will leave the discussion since It's not much interest to me...
For the last time, I was just saying that anything using a FILESYSTEM on a device (be it on a HDD or even in RAM if you really want) will ALWAYS be slower than a DIRECT MEMORY access to RAM. Is that really so hard to acknowledge ?

With a file system, you need a library to manage the file (opening, seeking, reading, closing) while a direct memory access is generally done through a SINGLE CPU LOAD instruction.
I can't make it simpler, I'm sorry

 

[quepaso]

The android port looks like its making very good progress via the twitter pics so far.

 

[Rydian]

Not really, standalone emulators have GUI taking up huge memory space.

This one opted for no GUI because it needs the RAM for other things. This isn't like a single-system emulator like NES/Genesis ones where the range of data to be loaded is known and within a small range, they have to take into account the larger things that can happen (which is why I made the comparison to different system emulators on the PC and how they handle loading data).

In regard to your initial statement that made me react, when an emulator unzips a file(s), there is no temporary unzipped file(s) in RAM or wherever, it unzips the CONTENT of the file(s), which is plain ROM data, into memory buffer(s) , which is then accessed through direct memory access (NOT file access) by the emulator core.

The difference between a "file" and a "stream of data" is the existence of metadata within the filesystem, but when loading a file into RAM you still need something pointing out the start and end of the block of data, the raw data in any sort of RAM needs additional info somewhere marking what and where it is.

So nothing got "deleted", it's just a buffer, it's cleared or overwritten when you load another game.

Yes, the data in RAM is marked as free to overwrite when it's not needed anymore.

Just like what happens when you delete a file from FAT/NTFS and other filesystems on a harddrive. The file's data is NOT overwritten, the info marking out the file is removed (and the filesystem metadata includes things like the filename, which is why recovered files can lack this depending on the deletion/recovery methods and filesystem in use).

It's all due to the multi-core design. I'm not saying it's bad, I'm only saying that it has his benefits and disadvantages.

I'm just saying the idea that other emulators don't create a temporary copy of the unzipped ROM is false, but I agree RA should delete the temporary file when it's not needed anymore.

For the last time, I was just saying that anything using a FILESYSTEM on a device (be it on a HDD or even in RAM if you really want) will ALWAYS be slower than a DIRECT MEMORY access to RAM. Is that really so hard to acknowledge ?

You brought that up multiple posts into the conversation... The original conversation was whether other emulators make a temporary copy of the file or not when unzipping it.

 

[Toad King]

Can we not have another dozen pages about ZIP loading? Trust me, the other devs and I know the pros and cons of the methods we use and we have plans for getting the few games we can't run yet running.

 

[papermanzero]

A principle question.
Due to libretro the api is separating the UI and the core.
So it must be possible to exchange the genesis GX core code (which is based on 1.7.0) with the current release code (1.7.1) in order to update the core easily.

So the development of the cores is independent of the development of retroarch itself.


Another question:
Can the UI and the ergonomic software flow be changed?
Means, if the core is running (e.g. snes9x_next) and I am pressing the home button on the wiimote, classic controller etc. the settings menu shall be opened instead of the rom selection menu.

 

[Toad King]

A principle question.
Due to libretro the api is separating the UI and the core.
So it must be possible to exchange the genesis GX core code (which is based on 1.7.0) with the current release code (1.7.1) in order to update the core easily.

So the development of the cores is independent of the development of retroarch itself.


Another question:
Can the UI and the ergonomic software flow be changed?
Means, if the core is running (e.g. snes9x_next) and I am pressing the home button on the wiimote, classic controller etc. the settings menu shall be opened instead of the rom selection menu.

1. Yes, but recent changes in RetroArch changed the pixel format, so you'll get a corrupted image with the 0.9.7.1 codebase unless you compile the core with "FRONTEND_SUPPORTS_RGB565=0". You could update RetroArch Wii and use that instead, but please note that the menu overlay currently has a bug because of the format switch, and has a bad flicker effect. It's still usable though.

2. That's something I'll keep in mind.

 

[papermanzero]

Great to hear. =)

1) Okay. So I will wait for the next update. The reason behind the question is to use retroarch as main emulator for wiiflow.
Means retroarch will be used to emulate most systems in order to provide the same UI and the same functionality. So retroarch will be used to harmonize the usage of wiiflow and the emulators.

2) Alright, it's only a proposal. Most emulators like mednafen have only one menu for loading roms and emulator settings (e.g. entering roms loading menu via settings entry). That's why I am asking. It would be more comfortable (and also a slight solution) to have one menu and to reach all features via this menu. At the moment the loading menu (for roms) and the emulator settings are separated.

 

[hicandturkey]

Is there a quicker way to exit RetroArch instead of having to open up the settings and locate the option to Exit RetroArch. The reason I ask is because I'm using Wiiflow as the "frontend" for my roms and it takes a lot longer to quit RetroArch and get back to WiiFlow than say FCEUGX (where I just hit the HOME button and selected "Return to Loader"). I'm even fine with a special button combination that will instantly kill it, if that's possible.

 

[Toad King]

Is there a quicker way to exit RetroArch instead of having to open up the settings and locate the option to Exit RetroArch. The reason I ask is because I'm using Wiiflow as the "frontend" for my roms and it takes a lot longer to quit RetroArch and get back to WiiFlow than say FCEUGX (where I just hit the HOME button and selected "Return to Loader"). I'm even fine with a special button combination that will instantly kill it, if that's possible.

There is one already, the combo for this is in the README in the first post.

 

[hicandturkey]

There is one already, the combo for this is in the README in the first post.

Gotcha. Somehow I didn't realize I could expand the entire Read Me. Thanks for pointing out the obvious.

 

[seam]

hey guys, im testing out some of my cps1 games, and i noticed that Carrier Airwing freezes during the first level... can anyone else verify this issue? my rom is cawing.zip(1,440,062)

 

[nakata6790]

hey guys, im testing out some of my cps1 games, and i noticed that Carrier Airwing freezes during the first level... can anyone else verify this issue? my rom is cawing.zip(1,440,062)

No it doesn't necessarily freeze. I had this problem too, here's what i did:
Midway through the first level, there is a formation of 3 enemy planes that come from the rear (left of screen)
that have a corrupt sprite, and 3 after that. If you are quick and manage to shoot them all down before they make a turn back, you can keep playing.
Same thing with some enemy types in level 3 and 4, however it doesn't freeze there.
I managed to finish the game fine, otherwise. One tip: have a savestate after each level.

 

[seam]

No it doesn't necessarily freeze. I had this problem too, here's what i did:
Midway through the first level, there is a formation of 3 enemy planes that come from the rear (left of screen)
that have a corrupt sprite, and 3 after that. If you are quick and manage to shoot them all down before they make a turn back, you can keep playing.
Same thing with some enemy types in level 3 and 4, however it doesn't freeze there.
I managed to finish the game fine, otherwise. One tip: have a savestate after each level.

ah, i see what you mean about the corrupt sprites. theyre like big black blotches that fly by, ha. after u kill them the level continues just fine, thanks for the tip!

 

[seam]

so after testing all my cps1 roms, the only other game i have a problem with is 1941(1941.zip) ive downloaded every different dump i could find, they all come up with the "corrupt or missing file" error, or whatever that message is. any idea on that one nikata? or anyone else thats gotten it to work...

edit: well i eventually just combined files from a few different versions of the rom and finally got it working.

 

[nakata6790]

Glad you managed to run it, dude.

Btw, for anyone interested, here's a list/pic of ~400 (selected after tested by me) fba games that run from playable to perfect on retroarch wii so far, maybe it helps other people to know what games to look out for: