ZIP file Order?

Discussion in 'Computer Games and General Discussion' started by raing3, Nov 9, 2009.

Nov 9, 2009

ZIP file Order? by raing3 at 12:35 PM (703 Views / 0 Likes) 2 replies

  1. raing3
    OP

    Member raing3 CNET_Richard

    Joined:
    Apr 2, 2009
    Messages:
    529
    Location:
    Australia
    Country:
    Australia
    I was hoping someone knew a Windows based application which would allow me to sort what order files were added to a ZIP archive in. For example WinRAR has this option but only for Solid RAR archives and uses a file called RarFiles.lst which has a different filename/filemask on each line for the order to add the files in.

    I desperately need some assistance to speed up the extraction of files from a ZIP archive which contains almost 1,000 tiny files (and increasing) so that I can sort the most important files to the start of the ZIP archive and have the rest of the files added afterwards.

    ^
    [​IMG] I hope this makes sense to others.
     
  2. FAST6191

    Reporter FAST6191 Techromancer

    pip
    Joined:
    Nov 21, 2005
    Messages:
    21,746
    Country:
    United Kingdom
    I might be able to dig an application that can pull this off (I have none to hand) but can I instead suggest you rethink your method:

    If it is zip then it is straight up file compression on a file by file basis whereas stuff like 7zip tends to use data hence it working so well for big collections of roms (note that using 7zip in this manner will probably make extraction even longer).
    If you must zip them then perhaps a lower setting is in order, as someone who has done compression algorithms from the ground up in general computing most of the compression gains are achieved with the simplest of methods or weakest of settings (case in point; ever noticed how a trimmed DS rom and a plain zip compressed rom are about the same size or if you go in for the PSP as your signature implies how a basic level of CSO is not that far off a level 9 in terms of percentage gained).

    I will flip it again; you say 1000 odd files, I am guessing some are of the "potential to be used" rather than "every day", how about shifting them to a secondary archive.
    Similarly if you have 1000 files in a folder you have probably messed up somewhere or could stand to sort them better. The time to do this is now (or ideally a few months back but as my time machine is reserved for more important things like looking at dinosaurs that is not going to happen) before it becomes truly entrenched.

    The next is what sorts of files are they; if they are all documents or all images or all something else we might have different options including no compression for formats with compression already inbuilt (most image formats, music formats, video formats and a good chunk of document formats).

    Rather than straight up compression how about diff files or something like it, for instance xdelta is built for this (although we usually use it in these circles for distributing rom hacks).

    Next there are all manner of compressed file systems ranging from compressed mountable ram drives to actual set in magnetism tracks. By similar token how about a new partition of the drive you are using or even a new drive for the files in question.
     
  3. raing3
    OP

    Member raing3 CNET_Richard

    Joined:
    Apr 2, 2009
    Messages:
    529
    Location:
    Australia
    Country:
    Australia
    Hmm... thanks for the great explanation and amount of obviously time you clearly took in replying but I guess I should have explained more about my problem.

    Firstly the decompression is occuring on the PSP and I can't really change from the ZIP compression format to another format because I'm not sure how to implement decompression of other formats as well as the ZIP compression format is fairly lightweight compared to other formats (from what I understand).

    The ZIP archive I have consists of the following files and directories:
    Code:
    Changelog.txt
    Credits and Help.txt
    GameList.txt
    cheat.db
    CMF\*.cmf (400 files)
    NitePR\*.txt (385 files)
    and that is roughly the order I wish them to be added to the archive in because the decompression routine in my program looks for the first instance of a file, decompresses it and then stops searching through the ZIP file. At the moment the amount of files doesn't cause too much of an issue but as the amount of files continue to grow it will eventually take longer and longer to find the required file in the archive.

    I've considered double packing the CMF and NitePR directories so seeking through the archive to find the required will be faster however that will break compatibility with previous versions of the PSP decompression software I made for the archive (which I have done too much already) as well as make the archives look more cluttered and require decompression of an extra file to extract the CMF/NitePR files when needed.

    At the moment I'm just using WinRAR to add all the files to a ZIP archive which adds them in a seemingly random order (doesn't seem to be based on timestamp or filename).

    If you do manage to be able to think of an application which would allow me to choose how the files are added to the ZIP archive that would be great.
     

Share This Page