Introducing Recom, a comprehensive and superior compression format.

Qriist

Member
OP
Newcomer
Joined
Oct 24, 2010
Messages
22
Trophies
0
XP
177
Country
United States
Lastest download: 2012-01-15
http://www.multiupload.com/O565IAN0I7

This is a toolset I have been working on for quite some time. The basic premise is very simple and adheres to two points:
A: The files must be compressed to the smallest possible file size in the most reasonable amount of time.
B: The file integrity must be protected at all costs!!!

I am a semi-amateur programmer and have written these tools in AutoHotKey_L 64-bit. I have tested over and over using many different scenarios. Alas, I can't predict everything. This release contains the first module for Recom that I will be releasing, Revolution. Revolution processes Gamecube and Wii games.

Main features:
-Automatic! You place your game images and archives in the indicated directory and Recom will process it in turn. Split RAR's and binary splits are no problem!
-Safe! At no point will your original image be deleted until all stages of compression and other forms of processing complete successfully.
-Efficient! I have spent a large amount of time messing with these tools to find the fastest, best, reproducible settings imaginable.
-Naming! If the option is enabled and a ReDump dat is available, the program will rename your output game based on the dat file at hand, with semi-auto updating of the dat files. (Once ReDump restores the dat files to the website I can make it entirely automatic..) This allows for standardized naming conventions and easy updating of already processed files. (Renaming is much faster.)
-Unified! For images with garbage data, this data is moved to an external file. Users can opt to download only the scrubbed archive (or the scrubbed archive+junk data) and the unRecom extractor will detect and use the proper technique. This means that "casual" users can easily grab the tiny scrubbed archive while the "collectors" acquire the entire thing, and both seed the same torrent!
-Economic! If everyone uses this method, even those with slow connections or limited space will be able to download more games in less time.
-Futureproof! If an amazing new compression format or technique pops up, I'll be able to implement it and the end user will be able to run all previously Recom'd games through it.
While the script looks quite primitive and unrefined, I do intend to add a better visual interface if the community likes what they see. (That is, if I get good reception on the METHODS at hand.)

This software is still BETA, though I have processed over a hundred gamecube and wii images. I can vouch that all characters A-Z, 0-9, parentheses, and the ampersand (&) are properly picked up while renaming. If other special characters are found in the redump dats that use special codes, I don't know them. The program will sorta fritz in this case... Just kill the plugin (ex, "Revolution.exe") and inform me of the game and system. Your original game will NOT be touched, mmmkay?

User feedback prompted me to make an intrinsically dynamic system. You need to run the Recom Initialization program before first run. Your "bit-ness" and RAM capacity will be detected and Recom will automatically compile the best possible settings for you! The meta files will allow your instance of Recom to "communicate" with previous instances.

Good luck! I hope you guys start using this format from here on out!

Here is the latest Recom:

Here is the latest Readme:
Recom by Qriist

"Recom" is a toolset that aims for the maximum amount of compression on supported game types, with a secondary focus on making damn sure your data is intact. Many tools do not check file integrity after compressing or when moving files from one drive to another, a serious flaw that should be addressed! This is my brainchild, as a result. ALL data-intensive operations are verified against the desired and actual outcome and redone until the computer gets it right.

For the most effective results you will need at least 12gb of RAM. Sorry, that's just the way it is, blame 7-zip. To that end, however, I have created a solution that allows ALL XP and higher users the ability to automatically detect and set your BEST LZMA value. This value is then used in future Recom processes to determine if you can (possibly) further compress this particular file. Once again, for best results, you need a fairly strong computer, though CPU is much less of a concern. This may seem hefty, keep in mind that your storage considerations will decrease, sometimes dramatically.

Programming for the Recom toolset (includes Recom.exe and all Recom plugins) is done in AutoHotKey by a self-taught programmer, though I cannot speak for other used tools.

Once extracted, inside the Recom folder, you will see a minimum of 4 folders, Recom.exe, Recom.ahk, Recom.ini, and this readme. Here is a quick rundown of the functions:
-Recom Initilization.exe: This program compiles all other plugins with the correct detected settings. You must run it once and wait for the success message.
-Recom.ahk: This is the launcher program, with .ahk being the source code. This isn't very advanced right now, little more than a generic loop for the plugins, but I do plan to add a few things to it.
-Recom.ini: Basic config file. Very easy to understand.
-(Folder) RecomPlugins: This is the folder that Recom will run through, launching each one of these programs in turn. If you are perhaps dabbling in what makes Recom tick, you will like to know that the source for each plugin is included; additionally, if you launch Recom.ahk (Requires AutoHotKey installed!) then all plugins will run as source, the AHK (NOT the .exe) version of each plugin. Good for debugging changes. Each plugin can be run independently.
-(Folder) RecomTemp: This folder is where all the magic actually happens. If you kill Recom intentionally or unintentionally you may safely delete anything in this folder. Recom will start from scratch anyways.
-(Folder) RecomHold: While you can specify your own output directory in the Recom.ini, all completed files will default to (Recom)\RecomHold\(system)\(Relative dir). This means that relative folder structure is kept 100% intact while moving your heavily compressed files safely away from the scope of the Recom process, with the sole exception of Alias.
-(Folder) RecomTools: This folder contains all the tools used by any plugin of Recom. All tools are freeware or shareware, so there is no need to crack anything. While it is true that most shareware programs are crippleware, my implementation of such tools will NEVER fail due to such limitations. I make sure that each tool is not crippled in what it needs to do.



[Getting Started]
Extract the latest Recom into a folder of your choosing and you'll see all of the items described above, as well as some folders that reflect default settings of the currently released plugins. If you want automatic verification and renaming, grab a copy of the latest redump zip, currently located here: ( http://www.mediafire.com/?uzyx5lr5rns99 ), and run the "Recom Initialization.exe" to analyze your system and compile all of your plugins with the right settings. Put some games in the appropriate folders and run Recom.exe. You will get a notice about the dats, which can take a fair amount of time to go away, usually no more than ten minutes. Then, your games will be processed according to all of your settings, most of which should be good while at their defaults.



[Supported formats]
===The following input archives are currently supported.
Binary splits (.001)
7-zip (.7z)
RAR
Split RARs (x.r00 AND x.part1.rar)
...Many more to come!


===Supported Wii/GC specific inputs.
.wia
.ciso
.wbi
.wdf
.gcm


===Supported (mainly) PSP inputs
.jso
.dax
.cso



[Currently implented/released plugins]


===Revolution
Revolution addresses Gamecube and Wii games. Because these are so very similar in structure, the fabulous Wit is capable of handling both types. There is a lot that goes into processing GC/Wii games and the most strenuous requirement is harddrive space. To process any one game you will need about 5x the size of the ISO itself free on your Recom drive. Not too big of a deal with GameCube games, but certainly a big requirement with Dual-Layer Wii games. Fortunately, there will be an initial free space check to make sure all is well before the real meat of the operation starts. (NOT YET IMPLEMENTED, SORRY!) As for the process itself, this plugin will start with an original ISO and copy it to WIA/NONE format using Wit. The WIA will then be copied out to an uncompressed scrubbed ISO. At this point, a quick test will be made to make sure that the new ISO is properly created. A delta patch will be made between the newly made scrubbed ISO and the original ISO. This delta patch will be applied to the scrubbed ISO and this NEW ISO will be binary compared against the very first ISO, the original. Assuming all is well the two temp ISOs will be nixed and 7z will do its magic on the WIA. Once the 7z is produced and verified, the 7z and the xdelta will be kept and moved to the output folder.

===Alias
Alias is a plugin that will keep your games named in compliance with your ReDump dats. The plugin reads the folders and dats as specified in the Recom Ini and runs through all of the meta files in your output directory to find games that have a hash match but not a title match; Alias will move such games and all related files to that game's sponsor plugin folder and then run the plugin in question. This is the ONLY plugin that will ever read your finished games and even then it doesn't modify them, it just calls the right plugin. Safety of your data is paramount!



[Ini Options Explained]
[Recom]
MoveFinishedFilesTo= Specify a valid directory and you can output the relative game structure to this new root folder. Otherwise, RecomHold will be utilized. Note that %system% can be used as a folder variable in the path; this will pick up the "root" folder of your input games and place that "root" folder at %system%.
Example: Z:Video Games\%system%\Games <- This will become:
Z:Video Games\Gamecube\Games <- or whatever "root" folder your input games reside in.
DebugMode= if the word "active" is specified, 7zip will use copy mode, greatly reducing the time required to analyze problems in the input files.
WindowMode= the words min,max,hide will cause the console window for each program to do just that. If no word is specified the console window will pop up as normal.
Torrent Creation Mode= This option lets you create or not create torrents of your finished games with the following keywords: all, ReDump, none. These options allow you to always create torrents, create torrents only if your game matches a ReDump dat entry, or never create torrents.
Create Open or Private Torrent= A second option for filtering torrent creation criteria. Available options are private,open,both,none. For open torrents, all trackers in the [Open Torrent Trackers] section (along with the trackon.org trackers, if enabled) will be consolidated into one non-private torrent with many trackers. For Private torrents, one temporary torrent with a throw-away tracker and the private flag set is made; each of your priavte trackers under the [Private Torrent Trackers] section are cycled into a new, exported torrent with that tracker. For both types of torrents, you can have any number of keys with any key name to facilitate any arbitrary number of trackers, just make sure your announce URLs are correct. If you don't want to make any torrents at all, set this to none.
Use Trackon For Open Torrents= Scans trackon.org for available torrents with an "Excellent!" rating to add to your open torrent. Since it uses the internet this feature is disabled by default. Set it to true if you want it.


[Folders] & [Dats]
Specify the exact name of the plugin for these entries, and also the exact folders and dats to look in. The folders you reference are folders that are present inside Recom root directory. In the case of the dats, do NOT include the timestamp. As an example, if you have "Nintendo - Wii (20111222 06-15-24).dat", only include "Nintendo - Wii" so that future updated dats will be automatically selected.


[Advanced]
Symbol Substitution= These are symbols that will be replaced with spaces in the ini file, so required due to the nature of some of the commands in the ReDump routines. If you encounter a name error (big message box with a load of programming jibberish,) just attempt to find the ReDump name and and that symbol to this list. Run Recom again and all should be fine. REPORT THESE TO ME, PLEASE! I can include such symbols in the next release.
Active Plugins= List the name of the plugins you want ran. If you don't want one to run automatically, ever, just remove it from this list.

For those of you familiar with Wiimm's wit, this builds on his WIA format to handle Gamecube and Wii games. Kudo's to him, that format is nasty awesome.

The following are some sample results from my UNTOUCHED, INTACT ISOs, compressed with my format. In red following each game, it shows what the scrubbed ISO + winrar looks like, size wise.
The Legend of Zelda - Skyward Sword (Korea) 3gb (ISO+RAR=3.83gb)
Rayman Origins (USA): 1.44gb (ISO+RAR=3.44GB)
Super Smash Bros. Brawl (USA) (v1.01) 5.11gb (ISO+RAR=6.73GB)
Geist (Europe) 340mb (ISO+RAR=647mb)
Metroid Prime Trilogy: 4.41gb (ISO+RAR=7.43gb)
 

Qriist

Member
OP
Newcomer
Joined
Oct 24, 2010
Messages
22
Trophies
0
XP
177
Country
United States
RAM is getting ungodly cheap nowadays. You can easily get 16gb (4x4) for 150 bucks, and even cheaper. Still, it isn't a requirement to run this program, as YOUR best setting is detected and written into the code.
 

xist

ΚΑΤΑ ΤΟΝ ΔΑΙΜΟΝΑ ΕΑΥΤΟΥ
Member
Joined
Jul 14, 2008
Messages
5,859
Trophies
0
XP
984
Country
Maybe i'm going mental but the link points to the wrong files...i think you meant this one -> http://www.multiupload.com/RNY4W8BIA8

Also,

Hey Qriist...
 

Qriist

Member
OP
Newcomer
Joined
Oct 24, 2010
Messages
22
Trophies
0
XP
177
Country
United States
Derp. Didn't include the download link, how stupid. It is there now. -.- The link in the spoiler is indeed intended as pointing to the redump archives.

Hey Xist!!!!! How are you? It has been quite some time, eh?
 

xist

ΚΑΤΑ ΤΟΝ ΔΑΙΜΟΝΑ ΕΑΥΤΟΥ
Member
Joined
Jul 14, 2008
Messages
5,859
Trophies
0
XP
984
Country
Hey Xist!!!!! How are you? It has been quite some time, eh?

Been a while indeed....and my fortune is about as good as CEX's was.....mostly bad....:(

Anyway, it's nice to see that you're still working on these neat little apps. For some reason i never really made it over to PSX-Scene....
 
D

Deleted_171835

Guest
I was actually reading about this on *snip* yesterday. It's pretty crazy that you managed to compress Metroid Prime Trilogy to 4.4GB.

Although I wasn't able to get this to work with my Prime Trilogy .wia file.
 

Ace

GBATemp's Patrick Bateman
Member
Joined
Apr 8, 2009
Messages
1,034
Trophies
0
Age
29
Location
Manhattan
Website
goo.gl
XP
538
Country
Well, I'd actually love to see this work on the DS, since I carry lots of games on my DSTWO. It seems utterly unbelieveably good!
 
  • Like
Reactions: 1 person

Presto99

You slammin, bro?
Member
Joined
Feb 24, 2010
Messages
915
Trophies
1
Age
28
Location
Skyworld
Website
https
XP
320
Country
United States
This seems awesome...excuse my n00bness, but this is just for archiving, right? Obviously none of the USB loaders support loading a Wii game like this, right?
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
    Psionic Roshambo @ Psionic Roshambo: https://youtu.be/hYkEl-3x92A?si=JSDoEc2QCDOrtimz