Hacking VCFE Wip...

[RadioShadow]

^ So we can finally inject any size NES rom? Sweet!

Freezing problem? Is that a typo? I thought we were talking about a corrupted image problem.

Man you really don't know how to follow the scientific method do you?

We should be working with super mario bros with the title chagned to "Super Mario Bros. 12345678" as that is the one I can't get to fix. Also changing the padding from "00" to "FF" is out of no where. There are too many variables in this test to prove anything but I'll see what I can do.

Yeah, that was a typo. The padding 'FF' was so you could easily see where the padding data started. Doesn't matter what value you use.

But the VCbrlyt9.0 is working fine. The banner images doesn't become corrupt and can be compressed (so the LZ77 compress is working fine).


EDIT:

Odd, the latest version of U8 Tool isn't packing the icon.bin files correctly (no LZ77). *gets pictures*

^ That black dot is a dead thunderfly (thunderbug), not from the image. >_<

I haven't edited any of the files inside the icon.bin file.

 

[HowardC]

Did you open the icon.bin and pack it back up, or just leave as-is?

 

[RadioShadow]

Unpacked the icon.bin, then packed it with no LZ77.

 

[HowardC]

Well it'd have to be the lack of lz77 that's doing it because I just decompressed a few icon.bins and then repacked them (without lz77) and the decompressed ones match the newly packed ones exactly. Either that or it's actually the IMET packing that's screwed up and not the packing of the icon.bin itself.

Hmm...

Actually I can't replicate the error.... they turn out fine for me. Specific examples?

 

[RadioShadow]

Well I'll try unpacking some other icon.bin files and see if the same affect happens. I may have just done something wrong. ^^'


Also thanks to this post, I think I found the height and width of the Logo Images in the SNES banner.brlyt file.

Search for the hex values '4490000043000000'. The values apparently are stored in big endian.

Now the 'my_BackSNES_a.tpl' has a width of 256 pixels and a height of 128 pixels.

The first four hex bytes are the width while the other four are the width so:

'44900000' is 256 pixels for the Width.
'43000000' is 128 pixels for the Height.

If those are the correctly values, then it means we can customise the values! Just I have no idea how big endian works.

I'll test that out tomorrow.

 

[HowardC]

Whoops... nevermind... big -endian is standard order.

Those don't add up to 256 or 128 though. As I've said in previous posts though, I don't think the pixel size has anything to do with the actual texture size.

 

[marioxb]

WAIT! HowardC! Don't release your save icon creator just yet! I knew it! I knew it! I knew Japan wouldn't use those cheesy English text save icon console "logos" for Neo Geo and N64!

I just purchased Custom Robo V2 for Japanese N64 and it uses a different icon than the US, and slightly different than your alternate.

I will purchase a Japanese Neo Geo game in a few minutes as well to get it's icon and I suggest you wait so you can have all the official icons in your creator. I GUARANTEE that they won't use some cheesy English text that says NEOGEO like US and PAL do, my N64 test has proven it! Didn't dump my filesystem just yet, but here's the save file (copy to SD- private/wii/title/NAPJ to view on your wii via SD card. The logo is similar to your N64 alternate, but without a border.

data.bin save file for Japanese Custom Robo V2 N64

I will have the Japanese N64 as well as Neo Geo icons for you in this post very shortly.

 

[HowardC]

That's great man, but why did you send me a data.bin?? The n64 save icon is stored in a regular tpl.

 

[marioxb]

That was just to look at. And it was quick to copy to SD and upload right away. Because, like I said, at the time, I haven't created a wad or dumped my filesystem yet (in the process right now). I just got Metal Slug 2 Neo Geo and it's different than US as well.

EDIT! Filesystem dump/ wad creation complete!

Here are the OFFICIAL JAPANESE save icons for N64 and Neo Geo. Different than USA, possibly the same as your alternates. At least this way we have the real ones. You can probably replace your alternate images with these ones.

Here are the tpls:

save_banner.tpl (Japanese N64):
http://www.mediafire.com/?tkiaumihuzg

banner.bin (Japanese Neo Geo):
http://www.mediafire.com/?w4yjmo2k1lm

 

[Forsaekn]

@HowardC: Why is your U8Tool considerably slower than other U8 packer / unpackers? Im curious because it is a huge difference.

Also I just had the same issue when compressing an archive that has the image structure to it, the picture comes out all distorted and "shredded" horizontally. Its very odd, standard u8 setting, no LZ77 compression. file sizes of the TPLs were 554,560 bytes, and the archive was rather large, maybe it bugs out when it archives too large of a file... hmm, anyways, I would love for you to fix this issue.

Cheers,
Fors

 

[HowardC]

@HowardC: Why is your U8Tool considerably slower than other U8 packer / unpackers? Im curious because it is a huge difference.

Also I just had the same issue when compressing an archive that has the image structure to it, the picture comes out all distorted and "shredded" horizontally. Its very odd, standard u8 setting, no LZ77 compression. file sizes of the TPLs were 554,560 bytes, and the archive was rather large, maybe it bugs out when it archives too large of a file... hmm, anyways, I would love for you to fix this issue.

Cheers,
Fors

It isn't slower except in certain cases. The short version of the answer is mine is slower because it actually works.

Try upacking a neogeo html archive if you don't believe me.

"the picture" doesn't tell me anything. Icon.bin, Banner.bin, a specific part of the images inside them?

 

[HowardC]

That was just to look at. And it was quick to copy to SD and upload right away. Because, like I said, at the time, I haven't created a wad or dumped my filesystem yet (in the process right now). I just got Metal Slug 2 Neo Geo and it's different than US as well.

EDIT! Filesystem dump/ wad creation complete!

Here are the OFFICIAL JAPANESE save icons for N64 and Neo Geo. Different than USA, possibly the same as your alternates. At least this way we have the real ones. You can probably replace your alternate images with these ones.

Here are the tpls:

save_banner.tpl (Japanese N64):
http://www.mediafire.com/?tkiaumihuzg

banner.bin (Japanese Neo Geo):
http://www.mediafire.com/?w4yjmo2k1lm

Yeah they pretty much look exactly like my alternatives. Regardless I need to add official slots for them as they are confirmed different. It's strange that the english versions don't use these seeing as how they look better and the text is still in english.

 

[marioxb]

That was just to look at. And it was quick to copy to SD and upload right away. Because, like I said, at the time, I haven't created a wad or dumped my filesystem yet (in the process right now). I just got Metal Slug 2 Neo Geo and it's different than US as well.

EDIT! Filesystem dump/ wad creation complete!

Here are the OFFICIAL JAPANESE save icons for N64 and Neo Geo. Different than USA, possibly the same as your alternates. At least this way we have the real ones. You can probably replace your alternate images with these ones.

Here are the tpls:

save_banner.tpl (Japanese N64):
http://www.mediafire.com/?tkiaumihuzg

banner.bin (Japanese Neo Geo):
http://www.mediafire.com/?w4yjmo2k1lm

Yeah they pretty much look exactly like my alternatives. Regardless I need to add official slots for them as they are confirmed different. It's strange that the english versions don't use these seeing as how they look better and the text is still in english.

I know. It makes sense for NES and SNES, since the logos/system names are totally different in Japan, but they could have come up with something better for us, and stuck with the Japanese for N64 and NG.

I just thought of something. What about alternate icons (as well as banner images) for Mark III (original Japanese name for Master System), Famicom Disk System, TurboGrafx-16 CD-Rom, and PC Engine CD-Rom? And if they ever do decide to come out with them, SuperGrafx (supposedly under PC Engine) and Game Gear (as Master System)? I don't know, probably too much trouble..

 

[HowardC]

Well it's fairly easy for me to add new slots, so we can always go back and do that, but for now official ones are good enough. Once we get vcfe up and running and it's actually working, I intend to look at the various open-source emulators out there and make custom channel versions for us to use. No drastic changes to them mind you, just an option to have them auto-boot a rom that is contained in the same folder as them and auto-exit if a rom is loaded via that method. At that point it'd make sense to expand functionality assuming by then we figure out how to make real save files. I've decided to merge the icons you made with mine to make a better set of alternate icons btw. I'm mix and matching the best ones. Also the three alternates for snes will be solved via a pal switch and a folder for pal icons. Seemed the best solution to me. Don't expect a release for a while though as I'm taking my time and atm I'm knee-deep in the rom application.

Btw has anyone ever experimented in making the emulator itself a seperate app that is called by the loader? From what I understand 6.app actually contains a loader that launches 1.app. If one were to make a custom 6.app it could launch a emulator stored on the sd card or a seperate folder and thus we could save a lot of space. Just a thought.



Btw disregard forsaken's comments for the moment.... he's contacted me via pm and what he is working on are system menu archives, so at least for now, it doesn't concern us.

 

[HowardC]

Just a quick update.... I injected a goodtools-cleaned "Ice Climber (U) [!].zip" into my iceclimber wad and the resulting 1.app is IDENTICAL to the original! This is great news as it means the rom copying code, the unzipping code and the nes injecting code are all working! I haven't put this on my wii yet, but seeing as how it's identical to the original I really don't see the point. I've yet to test the neogeo injection on my wii either but all things point to it being successful as well.
I need to get tgcd working and then the rest of the games are a simple copy job for the most part.

Just a quick question before I retire for the night....

Do you guys think I should rename the roms when I copy them over? There are some cases where this isn't possible (like snes, in which the save name used depends upon the rom name and neogeo, where the rom name is always "game.bin") but in the case of sega games and the tg16, it should be possible to change the name of the rom. Do you think I should or use the original rom name? See this hasn't been tested and changing rom names could have unforseen side-effects so it might mean a longer debugging process on our part, but it would look much nicer.

 

[HowardC]

Good morning! I have news to report on the tg16-cd front.

I had high hopes for the bincuesplit utility included with the wiiengine (pcengine emulator for the wii) that claims to turn a bin/cue image into a hcd set, which is what the wii's vc emulator uses. Turns out, at least at first glance, it only half works. The program cuts the iso up into data and audio tracks, converts the audio tracks into ogg, and makes a hcd file with all the size info inside. Unfortunately, the vc tg16 emulator doesn't use the data tracks as-is. they are converted to something and/or trimmed, but I haven't figured out what yet. I know this because the data track from a monster lair cd is around 12-15 megs, but the monster lair data track in the vc wad is only 2 megs. Either there is some trimming going on, or some sort of compression. I thoguht it might be bzlib compression, because that is often used on regular pce roms, but I've since ruled that out. 12 gold stars and a bonus point to anyone that can figure out if the files are compressed and if so what compression they are using.

Anyway, one of the things that makes it hard is the fact that I don't have a proper image to work with. I was working with monster lair, but the disc image I had was in mds format, which doesn't work wiht bincuesplit. I since downloaded a random japanese image, ran it through the converter and installed it on my wii. Other than a loud buzzing sound, the image didn't do anything. Anyway, I got ahold of a gradius 2 iso in bin/cue format, which should work much better.

It appears that there are some variations of the tg 16 cd formats as well as numerous settings for the emulator.

The gradius II ini file (found in the 5.app) looks like this:
--------------------------
NAME=KMCD2003
ROM=KMCD2003.hcd
BACKUPRAM=0
MULTITAP=1
HDS=0
RASTER=0
POPULUS=0
SPRLINE=0
PAD5=0
NOFPA=1
IRQMODE=1
CD_VOLUME=1.0
ADPCM_VOLUME=0.8
PSG_VOLUME=0.3
;PATCH=1
------------------------

Most of it doesn't matter, but irqmode is probably pretty important.

monster lair looks like this:
-------------------
NAME=MONLAIR
ROM=/monlair.hcd
BACKUPRAM=1
MULTITAP=1
HDS=0
RASTER=0
POPULUS=0
SPRLINE=0
NOFPA=1
---------------------

Notice that the irq setting is missing? I do know that gradius II is apparently in "super cdrom 2" format while monsterlair is in what I think is regular old "pce cd" format. Any pc engine, turbografx experts out there that can explain the various incarnations to me should do so as I'm getting confused. Anyway, my gradius II image needs looking at so later.

*EDIT*

I made a hcd set from my gradius II iso and compared it to the official hcd set in the gradius II wad. Again, the data tracks on the unofficial one are much larger. I thought by odd chance that the data tracks might work anyway, but unfortuantely I get the same loud buzzing sound and nothing else. The interesting thing is the hcd files (similar to a cue sheet on an iso) for the official wad and the one generated by bincuesplit on the unofficial one are identical? the hcd has track offset and size info, so shouldn't they have different values as they are different sizes? This makes me think more and more that the data tracks are compressed.

 

[RadioShadow]

Do you guys think I should rename the roms when I copy them over? There are some cases where this isn't possible (like snes, in which the save name used depends upon the rom name and neogeo, where the rom name is always "game.bin") but in the case of sega games and the tg16, it should be possible to change the name of the rom. Do you think I should or use the original rom name? See this hasn't been tested and changing rom names could have unforseen side-effects so it might mean a longer debugging process on our part, but it would look much nicer.

Just stick with the original names. Otherwise you'll just be making unnecessary work for yourself. Its not like the Wii Emulator says "Loading rom". Its not going to make a difference what the rom is called.

 

[HowardC]

Sweet! Looks like somebody else figured out the compression algorythm. It's a variant of good old zlib.

Read about it here.....Playing with the "Official" PCE Emulator

The guy was even nice enough to make a little exe that does the funky compression, so that saves me the trouble. I may go ahead and throw this data towards the guy who did bincuesplit though so it can be added to the tool. Of course it's open source, so I could do it myself.

Of all places to find useful data... the maxconsole forums? Who'd have figured!

*EDIT*

Tried the tools, they are junk... but I have confirmed from looking at one of the official data track files that the format proposed in that thread is indeed correct, the tools are just coded badly.

Here's the explaination in plain english:

The data tracks are zlib compressed, but not as a whole, rather they are compressed in chunks of 262144 bytes. The beginning of the file is a header, the first 4 bytes tell how many chunks there are and then for each chunk there is an 8 byte entry, the first 4 being the offset to the chunk of data and the next 4 being the compressed size of the chunk.

So to turn a bin file made from bincuesplit into a bin file that the wii can use, I simply need to break the bin into chunks of 262144, compress each chunk with zlib, reassemble them into a single file, and then build the header! I'm pretty sure I already have an app somewhere that uses zlib, so this shouldn't take long, but I'm in no hurry to do it. The good news is, tgcd injection on the wii IS possible, thanks to the fact that hudson soft used the same file format on a psp title as they do for the wii vc.

 

[HowardC]

Ok.. some more to report on the tg cd front.

I built a decompressor to compare the vc bin files with the ones that bincuesplit make. Well they are similar, but not identical. First off, the bincuesplit file is around 7.62 megs while the decompressed vc file is around 6.75 megs. Secondly the bincuesplit file has this wierd 16 byte header at the top of it. Then right after sector 1 (2048 bytes) there's 19 lines of garbage on the bincuesplit file, which throws off the whole offset. I'm not sure if it's necessary to remove this or not as I haven't built the compressor to test a wad, but we'll see.

In other words, we are close, but not quite there yet.


**DOH!**

Disregard what I just said.... the decrypted files are virtually identical, it's just the decrypted files are a iso, NOT a bin and that is what threw me off. After I converted to iso the only difference was a bit of padding at the end of the decrypted file due to me not knowing the exact size of the last chunk. I'll get to work on the compressor, which should be just as easy.

I can also take what I've learned, combine it with the tools others have made and increase compatability with various image formats, making it quite simple to make a hcd set from any type of image.

You see tgcd images are stored one of three ways:

1. iso/cue/wav Where an original iso is split into it's various tracks.
2. bin/cue Where the iso is saved in bin format (with a cue sheet of course)
3. Misc Where it is indeed a disc image, but it's saved in mds, nrg, or some other crazy format.

And we want to convert it to

2. bin/ogg/hcd Where the tracks are split, the isos are compressed and the wav files are converted to ogg, with a special hcd cue sheet

BinCueSplit only works on bin/cue sets but is the only tool that'll output a hcd file and convert the wavs to oggs automatically.
Another tool, turborip, will convert virtually any kind of misc disc image, as well as bins and isos to iso/cue/wav format, but it doesn't make a hcd file and doesn't convert the wavs to oggs. The tool in the article I linked to just randomly crashes and doesn't seem to work well. (Understandable seeing as the guy made a windows app in vm on a linux box.) What we want is a tool that'll take any sensible image file and iso/cue/wav set and convert it to bin/ogg/hcd.

So I propose that I do the following:

1. Check to see if the inputted set is a image or in iso/cue/wav format.
2. If it isn't use turborip to convert it.
3. Write my own app that converts a cue sheet into a hcd file.
4. Find (or make) a command line tool that converts wav to ogg.
5. Run my compressor on the iso portions of the new set.
6. ????
7. Profit!

Any suggestions would be welcome in this regard. I'd much rather have a singular tool, but converting various image formats into isos is a bit beyond the scope of what I'm willing to do and all of the hcd converters are simply too picky in regards to what format they work on. Couple that with the fact that there is a growing trend in the tg16 community to release rips in iso/cue/wav format, which bincuesplit can NOT handle and I don't know of a simplier way to do it.

p.s. Some other things I learned about the tg16 emulator thanks to the psp community:

1. You can replace the bios quite easily... it's stored by itself in the syscard3p.pce file.
2. The emulator SHOULD have support for the "arcade" card and the games that ran on it... to activate it you put "arcade, 1" at the end of your hcd file.

 

[HowardC]

Success!

To my knowledge I just ran the first injected tg16-cd game this morning! My compressor isn't quite as efficent as the one nintendo uses (my files end up a few thousand bytes bigger) but it works! I used bincuesplit on my gradius II iso and then ran the data file through my compressor. I then replaced the data file in my official gradius II vc wad with the one I made and it booted up fine! There is still work to be done as the tg-cd conversion process is quite tedious to the average user, but tg-cd injection IS possible and everything seems to work just great as the emu has great compatability (not to mention it's region free, as I injected a japanese iso into a pal wad and ran it on a ntsc machine!)

Once I get my hcd generator made and fine a suitable ogg converter, I'll ditch bincuesplit in favor of the more flexible turborip. I also need to find some isos that use the "arcade" board and test those.

So to recap:

NeoGeo injection code is done.
Nes injection code is done.
tg16-cd injection code is almost done.
All other systems use roms as-is and thus no conversion is necessary.

So the rom maker/copier should be done by this weekend.
All that's left in my toolset after that is the ticket manager, which is 75% done.

So if we are lucky vcfe itself will get a release before thanksgiving and will DEFINATELY get a release by the end of the year!