[giantpune]

its not defragmenting anything. using --analyze it is only reading. thats why the numbers are the same before and after.

theres also the perl script here http://forums.gentoo.org/viewtopic-p-31114...t-.html#3111409. it doesnt print out as much info as youd probably like, but you can always add more print() in there for more details

heres a quick bit that prints the filename and the number of fragments..

Warning! Spoiler inside. 

CODE

#!/usr/bin/perl -w

#this script search for frag on a fs
use strict;

#number of files
my $files = 0;
#number of fragment
my $fragments = 0;
#number of fragmented files
my $fragfiles = 0;

#search fs for all file
open (FILES, "find " . $ARGV[0] . " -xdev -type f |");

while (defined (my $file = <FILES>)) {
        #quote some chars in filename
        $file =~ s/!/\\!/g;
        $file =~ s/#/\\#/g;
        $file =~ s/&/\\&/g;
        $file =~ s/>/\\>/g;
        $file =~ s/</\\</g;
        $file =~ s/\$/\\\$/g;
        $file =~ s/\(/\\\(/g;
        $file =~ s/\)/\\\)/g;
        $file =~ s/\|/\\\|/g;
        $file =~ s/'/\\'/g;
        $file =~ s/ /\\ /g;
    print ( $file );
        #nb of fragment for the file
        open (FRAG, "filefrag $file |");
        my $res = <FRAG>;
        if ($res =~ m/.*:\s+(\d+) extents? found/) {
                my $fragment = $1;
                $fragments+=$fragment;
                if ($fragment > 1) {
                        $fragfiles++;
                }
        print ( $fragment . "\n" );
                $files++;

        } else {
                print ("$res : not understand for $file.\n");
        }
        close (FRAG);
}
close (FILES);

print ( $fragfiles / $files * 100 . "% non contiguous files, " . $fragments / $files . " average fragments.\n");

and this version only prints the info if the file has more than 1 fragment

Warning! Spoiler inside. 

CODE

#!/usr/bin/perl -w

#this script search for frag on a fs
use strict;

#number of files
my $files = 0;
#number of fragment
my $fragments = 0;
#number of fragmented files
my $fragfiles = 0;

#search fs for all file
open (FILES, "find " . $ARGV[0] . " -xdev -type f -print0 |");

$/ = "";

while (defined (my $file = <FILES>)) {
        open (FRAG, "-|", "filefrag", $file);
        my $res = <FRAG>;
        if ($res =~ m/.*:\s+(\d+) extents? found/) {
                my $fragment = $1;
                $fragments += $fragment;
                if ($fragment > 1) {
            print ( $file . "\t" . $fragment ."\n" );
                        $fragfiles++;
                }
                $files++;
        } else {
                print ("$res : not understand for $file.\n");
        }
        close (FRAG);
}
close (FILES);

print ( $fragfiles / $files * 100 . "% non contiguous files, " . $fragments / $files . " average fragments.\n");

Edited by giantpune, 27 November 2010 - 10:09 PM.

[Wiimm]

maybe pyfragtools is what you are looking for?

http://ubuntuforums.org/showthread.php?t=169551

This site has an intersting note:

By default, the defragger counts files with less than .5 frags/MB as unfragmented.

Hey, defrag freaks, do you read this:
This means <200 frags per GB counts like unfragmented!!!!!

I postetd so many times that a disc of 1 GB with <20 fragments is like unfragmented. And the tools makers say *200*.

And now I have a question to myself: Why do I spent my time in things that are not important?

CoD I'm coming, where are my victims?

[giantpune]

this is the output from the perl script. it seems to find the exact same files as the one that allows 200 fragments/GB

CODE

$ sudo '/home/jenna/Desktop/perlFragChecker.pl' '/media/WiiFat500/wbfs'
/media/WiiFat500/wbfs/Rock Band Green Day Rock_[R36E69].wbfs    16
/media/WiiFat500/wbfs/RFPP01.wbfs    2
/media/WiiFat500/wbfs/Guitar Hero 5_[SXEE52].wbf1    48
/media/WiiFat500/wbfs/Guitar Hero - Warriors of Rock_[SXIP52].wbfs    7732
/media/WiiFat500/wbfs/Guitar Hero 5_[SXEE52].wbfs    7685
/media/WiiFat500/wbfs/Pikmin 2_[R92P01].wbfs    4176
/media/WiiFat500/wbfs/New Super Mario Bros. Wii_[SMNE01].wbfs    4175
/media/WiiFat500/wbfs/James Bond - Golden Eye_[SJBE52].wbfs    2
/media/WiiFat500/wbfs/Guitar Hero Aerosmith Custom - Mini Concerts Edition_[CGVEMC].wbfs    4608
/media/WiiFat500/wbfs/Guitar Hero Aerosmith Custom - Mini Concerts Edition 2_[CGVEM2].wbfs    4608
/media/WiiFat500/wbfs/.____deleteme.wbfs.olOsd1.tmp    3
10.5769230769231% non contiguous files, 318.730769230769 average fragments.

does this mean that those 200 frags are really negligible?

Edited by giantpune, 27 November 2010 - 10:21 PM.

[oggzee]

I haven't read in detail the last couple of pages, but I guess you're having problem with preallocation on linux?
In wbfs_file, preallocation is #ifdefed only for windows, because that's the only os that needs such measures to prevent fragmentation, and on windows preallocating has no performance impact, only positive effects.

[Wiimm]

Here is my decision, discuss it if you like:

1. The test options --sparse --defrag and --pa-limit=size will be removed.
2. I implement a new option --prealloc with an optional parameter.
3. --prealloc=0 disables preallocation. This is the default for NON-CYGWIN.
4. --prealloc=1 enables an smart preallocation ignoring unused blocks.
5. --prealloc=2 or simple --prealloc preallocates the complete CISO or WBFS file. This is the default for CYGWIN because --prealloc=1 doesn't work like thought if the sparse holes are small.
6. Modes '1' and '2' are only different for WBFS and CISO files. ISO files will always use '1', and WDF and WIA files always '2' if nit disabled.
7. Later I will implement the preallocation for non image files (e.g. extracted files).

[oggzee]

--prealloc=2 or simple --prealloc preallocates the complete CISO or WBFS file. This is the default for CYGWIN because --prealloc=1 doesn't work like thought if the sparse holes are small.

sounds good to me.

[Wiimm]

Ok, I have committed v1.23,beta4 (nice digit order) inti the beta branch.

New option: --prealloc

CODE

--prealloc [=mode]
       This option enables or disables the disc space preallocation. If
       enabled the tools try to allocate disc space for the new files before
       writing the data. This reduces the fragmentation but also disables the
       sparse effect for prealocated areas.
         The optional parameter decides the preallocation modus: OFF (or 0),
       SMART (or 1), ALL (or 2). If no parameter is set, ALL is used.
         Mode 'OFF' disables the preallocation. This is the default for all
       non Cygwin releases because preallocation has only advantages on
       Windows systems.
         Mode 'SMART' looks into the source disc to find out the writing
       areas. SMART is only avalable for ISO, CISO and WBFS file types. For
       other file types ALL is used instead.
         Mode 'ALL' preallocate the whole destination file. This is the
       default for Cygwin. Because of the large holes in plain ISO images, the
       SMART mode is used for ISOs instead.

[W hat]

I've used WIA enough to report in. After testing 12 games, I get compression ratios of 50-60% almost every time with 7-zip+LZMA1 (Ultra). Some games do still only compress to 80-90%, but that's still great. I love WIA.

[Wiimm]

WIT: Wiimms ISO Tools v1.23a.beta5 - 2010-12-03
Beta test version, for EXPERIMENTAL use only.

WIT (Wiimms ISO Tools) is a set of command line tools to manipulate Wii ISO images and WBFS containers. The two main tools are called wit (Wiimms ISO Tool) and wwt (Wiimms WBFS Tool, the WBFS manager).

For productive versions see the main WIT thread or visit http://wit.wiimm.de/.

Download this EXPERIMENTAL release:

• wit-v1.23a.beta5-r2068-i386.tar.gz : Linux/i386 version
• wit-v1.23a.beta5-r2068-x86_64.tar.gz : Linux/x86_64 version
• wit-v1.23a.beta5-r2068-mac.tar.gz : Mac OS universal binaries (ppc, i386 and x86-64)
• wit-v1.23a.beta5-r2068-cygwin.zip : Cygwin (Windows) version
• Other releases are available from http://wit.wiimm.de/download.html.

Highlights of this release:

• Preallocation to avoid fragmentation
  I have implemented disc space preallocation. Under Windows it can avoid fragmentation. Therfor it is only enabled for Cygwin by default. It can be disabled with --prealloc=0 or enabled with --prealloc. See http://wit.wiimm.de/opt/prealloc for more details.
  .
  
• ID wildcards for some wwt commands
  The wwt commands EXTRACT + REMOVE + TOUCH + VERIFY wil support now ID wildcards to select different discs. See http://wit.wiimm.de/info/disc-id.html for more details.
  The needed preallocation function is not avaialable for Mac. So Mac will ignore --prealloc.
  .
  
• wwt EXTRACT rewritten
  "wwt EXTRACT" is now able to patch while extracting and to create WIA files like "wit COPY". It support also the options --fst --pmode --psel --raw --files and --sneek. http://wit.wiimm.de/wwt/cmd-extract.html for more details.
  .

Change log:

CODE

wit v1.23a.beta5 r2068 - 2010-12-03

- Preallocation of disc space implemented. It is only enabled for Cygwin by
   default. It can be disabled with --prealloc=0 or enabled with --prealloc.
   The needed preallocation function is not avaialable for mac.
- wwt EXTRACT + REMOVE + TOUCH + VERIFY:
   These commands accept now ID6 pattern like '...P' or '+1'. Each '.' is a
   wildcard that matches any single character. A '+' matches any number of
   characters. For details see http://wit.wiimm.de/info/disc-id.html
- wwt EXTRACT is now able to patch while extracting and to create WIA files.
   It support also the options --fst --pmode --psel --raw --files and --sneek.


Known bugs:

- The GameCube support is in development and EXPERIMENTAL!
- wit COPY --raw: Copying an image in raw mode to a WBFS file is buggy.
- There are new problems while reading from non seekable sources (e.g. pipes).
   The reason is the complete new and more intelligent wiidisc lib which makes
   more lookups ==> The best is to use version 1.01c for pipes.
- No WIA/bzip2 support for MAC/x86_64. (bug in the Mac bzip2 library)


Missing docu:

- wit commands FILES + EXTRACT => see built in help
- Tool wdf => see built in help

The source is published under the license of GPL2. Checkout the sources directly from the SVN repository or browse the source. These URLs are different from the product version. Visit http://wit.wiimm.de/ for more information.

[Wiimm]

WIT: Wiimms ISO Tools v1.26a.beta1 - 2011-01-11
Beta test version, for EXPERIMENTAL use only.

WIT (Wiimms ISO Tools) is a set of command line tools to manipulate Wii ISO images and WBFS containers. The two main tools are called wit (Wiimms ISO Tool) and wwt (Wiimms WBFS Tool, the WBFS manager).

For productive versions see the main WIT thread or visit http://wit.wiimm.de/.

Download this EXPERIMENTAL release:

• wit-v1.26a.beta1-r2156-i386.tar.gz : Linux/i386 version
• wit-v1.26a.beta1-r2156-x86_64.tar.gz : Linux/x86_64 version
• wit-v1.26a.beta1-r2156-mac.tar.gz : Mac OS universal binaries (ppc, i386 and x86-64)
• wit-v1.26a.beta1-r2156-cygwin.zip : Cygwin (Windows) version
• Other releases are available from http://wit.wiimm.de/download.html.

Highlights of this release:

• I have made some changes in the make system to support local libraries with own makefiles (libbz2.a), option -static and cross compiling (Linux/i386 on Linux/x86_64). Hope that all work as usual.
  Please test it!
  .
  
  .

Change log:

CODE

wit v1.26a.beta1 r2156 - 2011-01-11

- I have made some changes in the make system to support local libraries with
   own makefiles (libbz2.a), option -static and cross compiling (Linux/i386 on
   Linux/x86_64). Hope that all work as usual.
    - BZIP2 support is now included as source and should work for all systems.
      This enables WIA/bzip2 support for Mac/x86_64.
    - The distributions of Linux/i386 and Linux/X86_64 are linked static to
      to avoid GLIBC version issues (e.g. fallocate() support).
    - On Linux/x86_64: Enter "M32=1 make ..." to create i386 binaries.
      I386 libraries and include files are needed.
- "wit COPY/CONVERT/EXTRACT/DIFF" support option --sections.


Known bugs:

- The GameCube support is in development and EXPERIMENTAL!
    - GameCube conversion to WIA fails.
- There are new problems while reading from non seekable sources (e.g. pipes).
   The reason is the complete new and more intelligent wiidisc lib which makes
   more lookups ==> The best is to use version 1.01c for pipes.

The source is published under the license of GPL2. Checkout the sources directly from the SVN repository or browse the source. These URLs are different from the product version. Visit http://wit.wiimm.de/ for more information.

[giantpune]

any chance you could change the makefile to support the jobserver? in prior revisions, "make -j4" works as expected. with whatever changes were made to add the bzib library, it causes make to spit out some crap about the jobserver.

Warning! Spoiler inside. 

CODE

$ make -j6
***  create templates.sed    
***     asm ssl-asm.o        
*** ----------  ENTER ./src/libbz2/  ----------                  
***  create install.sh        
***  create load-titles.sh    
make[1]: warning: jobserver unavailable: using -j1.  Add `+' to parent make rule.
*****  object blocksort.o      
***  create version.h        
***  object gen-ui.o          
***  object LzFind.o          
***  object Lzma2Dec.o        
***  object Lzma2Enc.o        
***  object LzmaDec.o        
***  object LzmaEnc.o        
***  object cert.o            
***  object dclib-utf8.o      
***  object debug.o          
***  object file-formats.o    
***  object iso-interface.o  
***  object lib-bzip2.o      
*****  object huffman.o        
***  object lib-ciso.o        
*****  object crctable.o        
*****  object randtable.o      
***  object lib-file.o        
*****  object compress.o        
***  object lib-lzma.o        
***  object lib-sf.o          
***  object lib-std.o        
src/lib-std.c: In function ‘SetupLib’:
src/lib-std.c:472: warning: ignoring return value of ‘getcwd’, declared with attribute warn_unused_result
***  object lib-wdf.o        
***  object lib-wia.o        
*****  object decompress.o      
***  object libwbfs.o        
***  object match-pattern.o  
***  object patch.o          
***  object rijndael.o        
***  object sha1_one.o        
***  object sha1dgst.o        
***  object titles.o          
***  object tools.o          
*****  object bzlib.o          
***  object ui.o              
***  object wbfs-interface.o  
src/wbfs-interface.c: In function ‘ScanPartitions’:
src/wbfs-interface.c:217: warning: ignoring return value of ‘fgets’, declared with attribute warn_unused_result
***  object wiidisc.o        
***  object wit-mix.o        
*****      ar libbz2.a          
***  object wtest.o          
*****  ranlib libbz2.a          
*** ----------  LEAVE ./src/libbz2/  ----------                  
***  helper gen-ui            
*** +object wdf.o            
*** +object wit.o            
***    tool wtest            
*** +object wwt.o            
***    tool wdf              
***    link wdf -> wdf-cat    
***    link wdf -> wdf-dump  
***    tool wit               wit-mix.o
***    tool wwt              
HINT: try 'make help'

in your makefile, you can change this section

CODE

$(LIB_FILES):
    @printf "$(LOGFORMAT)" "----------  ENTER "./src/$(patsubst %.a,%,$@)/"  ----------" "" ""
    @XFLAGS="$(XFLAGS)" $(MAKE) --no-print-directory -C "src/$(patsubst %.a,%,$@)"
    @printf "$(LOGFORMAT)" "----------  LEAVE "./src/$(patsubst %.a,%,$@)/"  ----------" "" ""
    @cp -p "src/$(patsubst %.a,%,$@)/$@" .

change "make" to "$(MAKE)" and the jobserver stuff is working again.

other than this, it seems to work good

Edited by giantpune, 11 January 2011 - 08:00 PM.

[Wiimm]

@pune: ok

[giantpune]

oops... spoke too soon. it build fine on windows and linux x64. but it fails to build on OSx with this

Warning! Spoiler inside. 

CODE

l$ make -j2
***  create templates.sed    
***     asm ssl-asm.o        
*** ----------  ENTER ./src/libbz2/  ----------                  
*****  object blocksort.o      
***  create install.sh        
*****  object huffman.o        
*****  object crctable.o        
*****  object randtable.o      
*****  object compress.o        
*****  object decompress.o      
*****  object bzlib.o          
***  create load-titles.sh    
***  create version.h        
***  object gen-ui.o          
*****      ar libbz2.a          
*****  ranlib libbz2.a          
*** ----------  LEAVE ./src/libbz2/  ----------                  
***  object LzFind.o          
***  object Lzma2Dec.o        
***  object Lzma2Enc.o        
***  object LzmaDec.o        
***  object LzmaEnc.o        
***  object cert.o            
***  object dclib-utf8.o      
***  object debug.o          
***  object file-formats.o    
***  object iso-interface.o  
***  object lib-bzip2.o      
***  object lib-ciso.o        
***  object lib-file.o        
***  object lib-lzma.o        
***  object lib-sf.o          
***  object lib-std.o        
***  object lib-wdf.o        
***  object lib-wia.o        
***  object libwbfs.o        
***  object match-pattern.o  
***  object patch.o          
***  object rijndael.o        
***  object sha1_one.o        
***  object sha1dgst.o        
***  object titles.o          
***  object tools.o          
***  object ui.o              
***  object wbfs-interface.o  
***  object wiidisc.o        
***  object wit-mix.o        
***  object wtest.o          
***  helper gen-ui            
Undefined symbols:
  "___strcat_chk", referenced from:
      _bzopen_or_bzdopen in libbz2.a(bzlib.o)
      _bzopen_or_bzdopen in libbz2.a(bzlib.o)
  "___stack_chk_guard", referenced from:
      ___stack_chk_guard$non_lazy_ptr in libbz2.a(bzlib.o)
      ___stack_chk_guard$non_lazy_ptr in libbz2.a(compress.o)
      ___stack_chk_guard$non_lazy_ptr in libbz2.a(blocksort.o)
     (maybe you meant: ___stack_chk_guard$non_lazy_ptr)
  "___stack_chk_fail", referenced from:
      _bzopen_or_bzdopen in libbz2.a(bzlib.o)
      _BZ2_compressBlock in libbz2.a(compress.o)
      _BZ2_blockSort in libbz2.a(blocksort.o)
  "_fopen$UNIX2003", referenced from:
      _bzopen_or_bzdopen in libbz2.a(bzlib.o)
  "_fdopen$UNIX2003", referenced from:
      _bzopen_or_bzdopen in libbz2.a(bzlib.o)
ld: symbol(s) not found
collect2: ld returned 1 exit status
make: *** [gen-ui] Error 1
make: *** Waiting for unfinished jobs....

[Wiimm]

It runs here without probs on "Darwin Kernel Version 9.8.0: i386"

[giantpune]

mine says
Mac OS X 10.6.4 (10f569)
kernel version 10.0.0
if you have any ideas about what i may have done to break it, ill try them. but all i did is checkout the source and type make.