Hacking VCFE Wip...

[HowardC]

That's good news! How did you fix it?

The buffering technique seems to be working well, but I want to go ahead and release one that just adds a few bytes first. We want to keep it as small as possible. Knowing a working solution though, I can add in support for a 12kb buffer in a flash if we need that much, which I have a suspicion that we don't.

After trick or treat (which is tomorrow here) I'll have time to work on this and get a release out. I've successfully added unicode support to everything but the save icon injector (cause it's wierd as hell on some consoles) and fixed a bunch of bugs (not big ones, but a bug is a bug) along the way. I must say though, guitar hero world tour has me a bit distracted. Of course anything that involves a virtual "nuge" riding a buffalo to the stage is distracting.

My Wii or the 'banner.brlyt' file?

Don't forget all of the Save Names use Unicode except for the SNES which uses Shift JIS.

The only difference with Shift JIS is ASCII characters only uses one bytes while the Japanese characters use two bytes.

Your wii.

No, unfortunately, there is more wierdness than that. The sega style consoles can be unicode, shift jis, or BOTH! I'll probably just force output to unicode and be done with it as the wii apparenlty can read any of them, unicode is used 90% of the time anyway and the end user won't notice a difference. N64 is particularly wierd. I'll have to double-check, but I think the bytes are reversed for the unicode characters (first byte is last in each character). Also the end character differs between the latin based files and the asian ones, or rather the japanese entry doesn't seem to have an end character at all!

 

[HowardC]

Here is a complete set of English-text banner templates for the four Japanese systems that vary from the US/PAL systems available on Virtual Console. They are now formatted just like the US releases in regards to format of the Players: and Released: text, with everything else being in English as well. The systems use their Japanese names, in English. Famicom has two versions of the Icon.bin. The original, and one that now has the Family Computer logo in English. See the included readme for more details on that.

Genesis= Mega Drive (also the name in Europe, but with different logos, the Japanese logos are included here)

Nintendo Entertainment System= Family Computer (Famicom for short, with alternate colors than the NES banners)

Super Nintendo Entertainment System= Super Famicom (with alternate colors than the SNES banners)

TurboGrafx-16= PC Engine (with alternate colors than the TG16 banners)

Yes, that's right kids, the NES, SNES and TG16 systems in Japan use alternate color schemes for their banners than the US and PAL regions.

Also included are some new alternate logos for the Icon Generator. (Just replace the alt save icons folder.) I changed Howard's alternate NES banner for something I think looks a little better (was there ever an official logo of the initials NES by the way?). I made English save icons for the Famicom and Super Famicom ones that were only in Japanese, and I put in alternate banners for the PAL SNES, which used the Japanese four color circles logo, as opposed to the changed USA four circles logo. Since there is already an alternate SNES set of logos, I didn't know where else to put the PAL ones, so for now they appear as an alternate for PC Engine.

HowardC, feel free to use these in your tools, and or post them somewhere. Feedback is welcome.

Banner Templates and Alt Icon images


Famicom:

Super Famicom:

PC Engine:

Japanese Mega Drive:

Alternate Icon Generator images:

I really appreciate the effort mario, but a lot of this is already done. Yes the japaneese regions do indeed have alternate color schemes... this is why they are INCLUDED with the brlyt editor. Text templates are not necessary either, the whole point of the brlyt editor is to get away from usining templates if we can possibly help it. With unicode support we'll be able to properly edit the japanese banners as well. I haven't announced it because it most likey won't work initially, but I intend to add a "region change" option to the brlyt editor. Although I might have to poke around in the brlan files a bit, we should be able to change all instances of "JPN" to "ENG" in the brlyt and have the text (besides the title) actually show up on an english console! After that point, you can either leave the original japanese text or choose to re-save each entry and have it show up in english.

To answer your question, no there was never an official nntendo logo with just "NES" in it, but at 48x48 pixels wide (actually even less due to the blue border) you can actually read "nes" while you can't read "nintendo entertainment system". This is the same reason I used the famicom logo rather than the famicom text logo in the alternate sets btw. Also I dunno if you are using an old version of the icon generator or I just forgot to upload the newer ones, but the alternate save icons have been revised somewhat and the nes logo looks much better.

The icon.bins should be very useful, but unless I'm missing something you only made one alternate (the famicom) right?

We'll post this stuff on the main page, but you need to clean up your archive first. Seperate the icon.bins into one archive and the alternate save icon images into another as they aren't related. Also please remove the original/official ones as they waste space. I'd do it myself, but I'm busy juggling the 4 or 5 apps that need revised.

So keep up the good work, just in the future make sure it is work that hasn't already been done.

 

[marioxb]

OK, thanks. Will do. Yeah, I only made one alternate icon.bin (Famicom), since it's the only one that's not in English originally. As far as the save icons, the one I have from you only has alternates for SNES (US), NES and Neo Geo and N64.

 

[HowardC]

I'm understanding the n64 save format a little more and I thought I'd share my findings....

First off each savecomments_XX.bin file has a header exactly 64 bytes long, followed by the text, followed by a footer.
The header is essentially the same regardless of the game or language except:

Bytes 29 and 30 contain the two letter language code for the entry. (jp, en, es... ect)

Bytes 48 and 46 contain the length of the text in hex (the length is repeated twice, much like in brlyt files).

Bytes 59 & 60 and 55 & 56 contain the total length of the file minus 4. (again, the same value repated twice)

The text itself is indeed in "reversed unicode" meaning every other byte has to be swapped to read the value properly.
As for the footer, it is a seemly random combination of "00" and "BB" (the hex value for the fast-forward symbol).
Hopefully this footer isn't very important and I can just use the existing one.

So if my first attempt at n64 save text injection didn't work, that is why, as I didn't change the header values.

The good news is figuring out this header solves one of the major save injector problems and we are much closer to a release. I intend to release all the apps at one time this go around since for most of them the only major change is unicode support.

 

[HowardC]

As you guys figuered out the byrlt and brlan files, maybe one of you could give me a hint on how to loop a sound.bin.
As I understood, it can be done with a sound_start.bin (intro) and a sound_loop.bin (loop^^)
Problem is, I don't know how to make a corresponding byrlt or brlan file...

Any Hints?

No man, unless this is something I've never heard of it doesn't work that way. Considering the structure of an 0.app depends upon there being a file called sound.bin, I don't see how that'd work. Right now any user-created sounds are in wav format with a simple header applied and saved as sound.bin. To make a looping sound.bin the sound file (and there's only one, not two seperate files) is stored in BNS format, a format we currently don't have tools for. I meantion my findings on the bns format earlier in this thread.

 

[RadioShadow]

I'm understanding the n64 save format a little more and I thought I'd share my findings....

First off each savecomments_XX.bin file has a header exactly 64 bytes long, followed by the text, followed by a footer.
The header is essentially the same regardless of the game or language except:

Bytes 29 and 30 contain the two letter language code for the entry. (jp, en, es... ect)

Bytes 48 and 46 contain the length of the text in hex (the length is repeated twice, much like in brlyt files).

Bytes 59 & 60 and 55 & 56 contain the total length of the file minus 4. (again, the same value repated twice)

The text itself is indeed in "reversed unicode" meaning every other byte has to be swapped to read the value properly.
As for the footer, it is a seemly random combination of "00" and "BB" (the hex value for the fast-forward symbol).
Hopefully this footer isn't very important and I can just use the existing one.

So if my first attempt at n64 save text injection didn't work, that is why, as I didn't change the header values.

The good news is figuring out this header solves one of the major save injector problems and we are much closer to a release. I intend to release all the apps at one time this go around since for most of them the only major change is unicode support.

That explains a lot. I guess they can use both lines?

And if there are no tools to make BNS sound files, how did they 'Homebrew Channel' sound.bin to loop?

 

[HowardC]

I dunno if it can use two lines or not, that'd be something to test.

Well the thing about team tweezer is, for all the praise they get (and to some degree they have earned it) they are real jerks about releasing certain stuff. They already have brlyt and brlan files figured out (except for the text, which we figured out) and made an app that lets you preview the files, but they never released it. The bns files are partially documented at wiibrew and my guess is they figured it out and made a file (probably manually), but again, never posted the full findings. Either that or they edited their brlan files to use two sounds, but again, the process is totally undocumented and that isn't how "real" vc wads do it.


Btw, I have successfully converted all the save text functions to use unicode except for snes. As you said it uses shift JIS, and that poses a real problem. See the weakness of using forms 2.0 to add unicode support is I have to first convert the text to unicode manually by taking two bytes at a time and using that value to make a unicode character in the textbox. Normally this works regardless of language because english unicode characters and english ascii characters are the same. With shift JIS, I have to know the language of origin coming in because the values of two english ascii characters, when combined, will also make a japanese character. Other than doing some real hackish things which wouldn't work 100% of the time (like checking to see if the first byte is a 00) I don't know how to detect the language.

I can use the rom's key name of course, but that would mean japanese wads could only save japanese text and vice-versa. I'd really like a solution that allows us to use any language freely when we create save files.

 

[RadioShadow]

That does cause a bit of a problem. It's a shame we can't force SNES Wads to use Unicode. Couldn't you just toggle between JIS and UNICODE?

The way Shift Jis works is if the first byte is between 81 - FA, this tells the Wii if it is a Japanese character and what ever the next hex byte is, is the Japanese letter to display. Since ASCII doesn't use any bytes between 81 - FA and the byte is not one of those, this tells the Wii just to use 1 hex byte and what ever that hex byte is, it displays that ASCII character.



Anyway, I might have a solution for the 'compression' issue.

Source 1

icon.bin and banner.bin are identical in format. icon.bin contains the layout, animation, and bitmap data for the channel thumbnail icon (what you see in the System Menu overview, and what you see in the Channel Management screen), while banner.bin contains the same data for the large full-screen banner. The files begin with an IMD5 header, which wraps an LZ77 compressed block of data. LZ77 compression is the same as used in the GBA and Nintendo DS BIOS routines. The compressed data is nothing but another U8 archive, with the following structure:


Source 2

Find yourself the source of a DS emulator. Most of them simulate bios/firmware functions and iirc, LZ77 decoding is one of them.


If we can find the source code for a DS Emulator, we could use that as the LZ77 compressor. This site explains a bit about it:

Source 3
QUOTE

SWI 11h (GBA/NDS7/NDS9) - LZ77UnCompWram
SWI 12h (GBA/NDS7/NDS9) - LZ77UnCompVram (NDS: with Callback)
Expands LZ77-compressed data. The Wram function is faster, and writes in units of 8bits. For the Vram function the destination must be halfword aligned, data is written in units of 16bits.
If the size of the compressed data is not a multiple of 4, please adjust it as much as possible by padding with 0. Align the source address to a 4-Byte boundary.

r0 Source address, pointing to data as such:
Data header (32bit)
Bit 0-3 Reserved
Bit 4-7 Compressed type (must be 1 for LZ77)
Bit 8-31 Size of decompressed data
Repeat below. Each Flag Byte followed by eight Blocks.
Flag data (8bit)
Bit 0-7 Type Flags for next 8 Blocks, MSB first
Block Type 0 - Uncompressed - Copy 1 Byte from Source to Dest
Bit 0-7 One data byte to be copied to dest
Block Type 1 - Compressed - Copy N+3 Bytes from Dest-Disp-1 to Dest
Bit 0-3 Disp MSBs
Bit 4-7 Number of bytes to copy (minus 3)
Bit 8-15 Disp LSBs
r1 Destination address
r2 Callback parameter (NDS SWI 12h only, see Callback notes below)
r3 Callback structure (NDS SWI 12h only, see Callback notes below)

 

[HowardC]

Actually we probably CAN force unicode for the snes. That true type font in the 4.app is the actual font used to render text and it contains the character sets of all the languages in it, not just english. The entire english character set is put in the "FF" offset as well as the usual place (00), for numbers and "engrish". I see no reason why unicode text in the english entries wouldn't work. The trick is in the reading of the text though.

If what you say about shift jis is true though, I can probably write a function. I perfectly willing to take your word for it, but a bit of documented re-assurance would be nice as well

As for the lz77 compression.... if we were to go as far as rip apart a ds emulator, that'd be one heck of a chore. As much as I'd like to fix the c64 issues, that will probably be something to work on dead last, after even the vcfe is released.

*Edit*

81-FA? That couldn't be right. Games like Mario 64 use FF14 and FF16 for the "64" part in their japanese entries. Like I said, the entrie ascii set is stored in the ff offset as well.


*Edit2*

That isn't right. I did a quick print out of all the ascii characters beyond 129 (81 in hex) and while they aren't english characters, they are french, spanish, crylic, ect.... So the first character couldn't be used to determine if it's an ascii value or not.

 

[RadioShadow]

UNICODE and SHIFT JIS uses different hex entries to display the necessary characters.

So '??' would be display in hex as:

UNICODE = FF16FF14
SHIFT JIS = 82558253


Now displaying '64' in ASCII:

UNICODE = 00360034
SHIFT JIS = 3634


And the English characters in the 'FF' set (UNICODE) are the fixed width LATIN characters. The ASCII characters have there own single bytes using the hex values being 00 - FF.

Check these sites for more details:

UNICODE
SHIFT JIS
ASCII

That isn't right. I did a quick print out of all the ascii characters beyond 129 (81 in hex) and while they aren't english characters, they are french, spanish, crylic, ect.... So the first character couldn't be used to determine if it's an ascii value or not.

That's a good point. *does more research on SHIFT JIS*

EDIT:

Right, below is a image of the single byte characters that can be used (yellow and blue areas):

Source

QUOTEThe single-byte characters 0x00 to 0x7F match the ASCII encoding, except for a yen sign at 0x5C and an overline at 0x7E in place of the ASCII character set's backslash and tilde respectively. The single-byte characters from 0xA1 to 0xDF map to the half-width katakana characters found in JIS X 0201.

The 'Extended ASCII Codes' don't seem to get used. That explains why UNICODE is better to use.

Now if the hex byte is anywhere between '81 - 9F' or 'E0 - FC', this means a two byte character is to be used and the second byte determines which Japanase character to use.

 

[HowardC]

hmm.... that's a real pain in the butt then.....

I'm gonna have to read in some japanese snes titles and see what the results are. Shift JIS isn't natively supported by.... well ANYTHING, so if I have to do some big conversion table to get things to display properly then I might just leave asian character out in this one instance.

 

[HowardC]

Good News!

I've managed to cobble together a function that properly reads shift_jis data!

Mind you I have to first convert the string to bytes and they convert it a few more times before finally outputting readable text, but it seems to work.
Now writing it back will be the tough part. I'll keep you posted.

*EDIT*

Managed to write a function to convert ascii back in shift-jis. The function is even crazier than the read function though. Hopefully it'll be compliant enough to work.

 

[HowardC]

I added the before-mentioned regon force option to the brlyt editor. I don't have time to install wads atm (NOW I'm tkaing things DOWN from halloween.

) so when the release comes it'll be up to you guys to test it.

Here's how it works: When a region force value other than "NONE" is selected, the region force function first checks to see if that region is already in the brlyt. If not then it checks to see what regions ARE in the brlyt and assuming the brlyt isn't region free (like mega-drive japan) it takes the first region present and replaces all it's entries (T_YEAR, T_PLAY, ect...) with the region you wish to force it to along with the group for that region found at the bottom of the file. The only exception is the "T_VCTitle" entry. Why? Well after studying the animation brlan files I've determined why the title shows up regardless of region and that I do NOT have to edit these files so long as the original vctitle entry is in place.

The reason for this is more of a bug in vc animation files than a feature. You know where when you first click on a vc game the title bounces up? Well that is done via the animation files and instead of calling the proper region, they try to call ALL regions instead of doing it the intelligent way that the year and player entries do (the wii automatically will set the region). Because of this, the game title will always display because in the Japanese animation files, only a JPN region is called for vctitle and apparently the wii's error handling allows this title even though it's the wrong region, because no other regions are present. The year and player entires don't have any code in the animation files and thus they don't get the special error handling and don't show up. If I were to change the T_VCTitle entry in the banner.brylt then the japanese brlyts wouldn't have the proper t_vctitle entires in the animation files and thus a title would NEVER show up. Leaving it in tact should assure that it is always visible.

Of course this is just theory, again I haven't had time to test, but if all else fails I can edit the animation files as well, it'd just be a pain in the butt.

To the end user that is lost at this point, what this means is that japanese vc games can be set to have the year and number of players show up even on non japanese wiis and vice versa! To our other foreign friends, this means if you install a vc title that wasn't available in your countries shop channel and it uses the newer style brlyts that only have region entries for countries that get a release, you can make the year and player entries show up as well!


I have some stuff to button up on the u8tool, but we are getting close to a new release of EVERY tool I've made so far. It should happen sometime this week and while the main new feature is unicode support, bugs were found and fixed in almost every app, so it is reccomened you download the new tools when they arrive even if you aren't interested in the new feature.

 

[RadioShadow]

Looking forward to the tool updates. That region free feature will be useful.

Would you also be able to add a function that allows a user to say change the Japanese Year Text To English Text?

 

[HowardC]

Well, it does this. If you merely change the region and don't alter the entries then everything remains in japanese, BUT if you start to edit the text and save it the editor will determine that the file is now in english and saves the year and player data with english text.


*edit*

Maybe I need to explain that better. See the region force isn't done until last when save is clicked, so any changes will remain in the original language until you save it. If you then edit the file again, when the file is first loaded up it'll only find the new langauge you forced it to and thus when you set the text entries they will be converted to that language.

With that being said, you may still wish to edit the text. See the upper "FF" numerals are used for japanese entries and while they are perfectly readable english numbers they look a little funky compared to the REAL english numbers. I'm not going to do a big conversion function, so those numbers will remain the same unless you actually edit them.

 

[RadioShadow]

Cool!

 

[HowardC]

I'm packaging up the tools as we speak. All but the brlyt editor are ready for release. Unfotunately, rapidshare has changed it's policy regarding uploads, which means I need to find a new site to upload files to. Because of this I won't upload anything tonight. If anyone has any suggestions let me know.

On an unrelated note, I noticed by chance that the latest version of the wiiEngine (tg16 pce emulator) includes a utility that allows you to split a iso into hcd format, which is the format the VC emulator uses. Assuming this works even half-way, I should be able to use it to make a tg16 cd injector as it's open-source and I can change any tiny bits that need altered.

Corsario is working on documenting neogeo for me as we speak and we've already determined the format for msx, mastersystem and c64, so in theory the injector, (which I work on next now that we've finally tackled the corrupted banner issues) will be the first to support ALL vc consoles!

I'm looking forward to this next app because once it is done we will be ready to start on vcfe itself, which comparatively should be quite simple to build!

 

[DoctorWhosThat]

HowardC, i'd like to say nice work with these injection tools. Regarding the VCFE, will it be an all-in-one program which consists of all the tools that you have created? As I think this will make things a lot easier if it is.

All I can say for now is, keep up the good work!

 

[RadioShadow]

I'm packaging up the tools as we speak. All but the brlyt editor are ready for release. Unfotunately, rapidshare has changed it's policy regarding uploads, which means I need to find a new site to upload files to. Because of this I won't upload anything tonight. If anyone has any suggestions let me know.

- megaupload
- filefactory
- GBAtemp itself

 

[marioxb]

Mediafire...

Speaking of which, HowardC, here are those Alternate Save Icons I posted before, this time by themselves. Alternate FC and SFC with English text, alt SNES-PAL, alt NES:

Alternate Icon Generator images:

http://www.mediafire.com/?y2zw5m1yqmd

And then here is an alternate icon.bin file for Famicom that says Family Computer in English, instead of the normal Japanese:

http://www.mediafire.com/?yyqvzzd5ydz