Hacking wwt+wit: Wiimms WBFS+ISO Tools

[Wiimm]

@sylver78
I have uploaded your Mac OS universal version and changed the announcements.
Thank you.

 

[sylver78]

You're welcome !
To do it I had to generate one version for each architecture and then I used the command "lipo" to join the 3 binaries of each tool into an universal binary ...
I'm wondering what would be the best option to make it automatically from the makefile (a new make option like "make macosdistribution" for example ?).

 

[drh]

ok, I have my mass storage device connected to an OSX Server machine up in my loft.

I wanted to check if the speed of transfers locally on that machine.

Its running OSX Server 10.5.8.

When I try to use the same tools (that work fine on my OSX 10.6.3), I am getting the following error:

dyld: unknown required load command 0x80000022
Trace/BPT Trap

Anyone know if I can fix this maybe with some different compile switches?

 

[sylver78]

ok, I have my mass storage device connected to an OSX Server machine up in my loft.

I wanted to check if the speed of transfers locally on that machine.

Its running OSX Server 10.5.8.

When I try to use the same tools (that work fine on my OSX 10.6.3), I am getting the following error:

dyld: unknown required load command 0x80000022
Trace/BPT Trap

Anyone know if I can fix this maybe with some different compile switches?
try using these options (untested, found on the internet) :
For gcc :

-mmacosx-version-min=10.4

for the linker :
CODE-syslibroot /Developer/SDKs/MacOSX10.4u.sdk/
-macosx_version_min 10.4
-F/Developer/SDKs/MacOSX10.4u.sdk/System/Library/Frameworks

Replace 10.4 with 10.5 if you want the binary to working only from 10.5.

I missed this too so my universal binary version may only work on 10.6.x !

 

[Wiimm]

@sylver78

you muste create 2 files.

Makefile.local (don't ignore the uppercase 'M')

# remove trailing '#' to uncomment

# defines (should not needed)
#DEFINES += -DMACCRO=VALUE

# compiler flags for all 3 mac variants
#CFLAGS += -my_flag

# ld flags
#LDFLAGS += my_flags

you can confirm this settings with: make flags

Makefile.user

#----------------------------------------------------------------
# make manual: http://www.gnu.org/software/make/manual/make.html
#----------------------------------------------------------------

# indent your commands with tabs
# prefix with @ : don't show command
# prefix with - : ignore errors
# see Makefile for examples
# you can also call Make recurse

#!-----------------------------------------------------------------------------

# build you own rule:
.PHONY : mac
mac: mac-ppc mac-i386 mac-x86_64

ÂÂÂÂ# commands that built universal binaries
ÂÂÂÂ@echo "built universal"
ÂÂÂÂ
#!-----------------------------------------------------------------------------
ÂÂ 
.PHONY : mac-ppc
mac-ppc: 
ÂÂÂÂ@XFLAGS="-aaa" make clean all
ÂÂÂÂ@mkdir -p bin/ppc
ÂÂÂÂ@mv $(TOOLS) bin/ppc


.PHONY : mac-i386
mac-i386: 
ÂÂÂÂ@XFLAGS="-bbb" make clean all
ÂÂÂÂ@mkdir -p bin/i386
ÂÂÂÂ@mv $(TOOLS) bin/i386


.PHONY : mac-x86_64
mac-x86_64: 
ÂÂÂÂ@XFLAGS="-ccc" make clean all
ÂÂÂÂ@mkdir -p bin/x86_64
ÂÂÂÂ@mv $(TOOLS) bin/x86_64

XFLAGS="-aaa -bbb -ccc" : define compiler flags for the sub call of make

 

[drh]

try using these options (untested, found on the internet) :
For gcc :

-mmacosx-version-min=10.4

for the linker :

-syslibroot /Developer/SDKs/MacOSX10.4u.sdk/ 
-macosx_version_min 10.4 
-F/Developer/SDKs/MacOSX10.4u.sdk/System/Library/Frameworks

Replace 10.4 with 10.5 if you want the binary to working only from 10.5.

I missed this too so my universal binary version may only work on 10.6.x !

How do I implement these, at the moment, I just use "make" with no options.

 

[Wiimm]

edit Makefile.local (new file)

CFLAGS += -mmacosx-version-min=10.4
LDFLAGS += -syslibroot /Developer/SDKs/MacOSX10.4u.sdk/
LDFLAGS += -macosx_version_min 10.4
LDFLAGS += -F/Developer/SDKs/MacOSX10.4u.sdk/System/Library/Frameworks

 

[sylver78]

I managed to build a 10.5 compatible binary using these settings :

CFLAGSÂÂÂÂ+= -mmacosx-version-min=10.5 -isysroot /Developer/SDKs/MacOSX10.5.sdk
LDFLAGSÂÂÂÂ+= -mmacosx-version-min=10.5 -isysroot /Developer/SDKs/MacOSX10.5.sdk

 

[drh]

yea, that worked, thanks guy's :-)

 

[sylver78]

I managed to build a 10.4 and above compatible set of binaries ...
here are my custom files (thanks Wiimms

):
Makefile.local :

#defines for 10.4 and above
CFLAGSÂÂÂÂ+= -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk
LDFLAGSÂÂÂÂ+= -mmacosx-version-min=10.4 -isysroot /Developer/SDKs/MacOSX10.4u.sdk
CC="gcc-4.0"

#defines for 10.5 and above
#CFLAGSÂÂÂÂ+= -mmacosx-version-min=10.5 -isysroot /Developer/SDKs/MacOSX10.5.sdk
#LDFLAGSÂÂÂÂ+= -mmacosx-version-min=10.5 -isysroot /Developer/SDKs/MacOSX10.5.sdk

Makefile.user :

#----------------------------------------------------------------
# make manual: http://www.gnu.org/software/make/manual/make.html
#----------------------------------------------------------------

# indent your commands with tabs
# prefix with @ : don't show command
# prefix with - : ignore errors
# see Makefile for examples
# you can also call Make recurse

#!-----------------------------------------------------------------------------

# build you own rule:
.PHONY : mac-distrib
mac-distrib: mac doc $(INSTALL_SCRIPTS) gen-mac-distrib

.PHONY : mac
mac: mac-ppc mac-i386 mac-x86_64

# commands that built universal binaries
ÂÂÂÂ@echo "building universal binaries"
ÂÂÂÂ@for i in $(TOOLS); do \
ÂÂÂÂÂÂÂÂlipo -create bin/ppc/$${i} bin/i386/$${i} bin/x86_64/$${i} -output bin/$${i}; \
ÂÂÂÂÂÂÂÂlipo -create bin/ppc/$${i} bin/i386/$${i} bin/x86_64/$${i} -output $${i}; \
ÂÂÂÂdone;
ÂÂÂÂ@rm -rf bin/ppc bin/i386 bin/x86_64;
ÂÂÂÂ@echo "done"

#!-----------------------------------------------------------------------------
ÂÂ 
.PHONY : mac-ppc
mac-ppc:
ÂÂÂÂ@echo "building PowerPC"
ÂÂÂÂ@XFLAGS="-arch ppc -mcpu=powerpc" make clean all
ÂÂÂÂ@mkdir -p bin/ppc
ÂÂÂÂ@mv $(TOOLS) bin/ppc

.PHONY : mac-i386
mac-i386: 
ÂÂÂÂ@echo "building i386"
ÂÂÂÂ@XFLAGS="-arch i386 -mfpmath=sse -march=prescott" make clean all
ÂÂÂÂ@mkdir -p bin/i386
ÂÂÂÂ@mv $(TOOLS) bin/i386


.PHONY : mac-x86_64
mac-x86_64: 
ÂÂÂÂ@echo "building X86-64"
#ÂÂÂÂ@XFLAGS="-arch x86_64 -mfpmath=sse -march=core2" make clean all
ÂÂÂÂ@XFLAGS="-arch x86_64 -mfpmath=sse" make clean all
ÂÂÂÂ@mkdir -p bin/x86_64
ÂÂÂÂ@mv $(TOOLS) bin/x86_64

.PHONY : gen-mac-distrib
gen-mac-distrib:

ÂÂÂÂ@echo "***ÂÂcreate $(DISTRIB_PATH)"

ÂÂÂÂ@rm -rf $(DISTRIB_PATH)
ÂÂÂÂ@mkdir -p $(DISTRIB_PATH)/bin $(DISTRIB_PATH)/scripts $(DISTRIB_PATH)/lib $(DISTRIB_PATH)/doc

ÂÂÂÂ@cp -p $(DISTRIB_FILES) $(DISTRIB_PATH)
ÂÂÂÂ@cp -p $(TOOLS) $(DISTRIB_PATH)/bin
ÂÂÂÂ@cp -p lib/*.txt $(DISTRIB_PATH)/lib
ÂÂÂÂ@cp -p $(DOC_FILES) $(DISTRIB_PATH)/doc
ÂÂÂÂ@cp -p $(SCRIPTS)/*.{sh,txt} $(DISTRIB_PATH)/scripts

ÂÂÂÂ@chmod -R 755 $(DISTRIB_PATH)
ÂÂÂÂ@chmod a+x $(DISTRIB_PATH)/*.sh $(DISTRIB_PATH)/scripts/*.sh $(DISTRIB_PATH)/bin*/*
ÂÂÂÂ@chmod -R a+X $(DISTRIB_PATH)

ÂÂÂÂ@tar -czf $(DISTRIB_PATH).tar.gz $(DISTRIB_PATH)

Then you can do a "make mac" or "make mac-distrib".
Note that the 10.4 SDK is not installed by default with xcode, you'll have to select it when installing xcode to be able to build 10.4 compatible binaries ...

Here is the new version of the distribution :
http://www.multiupload.com/U78LIP0C7V (same version as previous one but compatible with MacOS 10.4 to 10.6)

 

[drh]

Good job Sylver,

I checked the Developer directory, I realised I had 10.5 by default and not 10.4, so skipped the 10.4 stuff as I'm never likely to need that now anyway but its nice to have a complete distrib available now.

The instructions for 10.5 worked sweet, so I've left my OSX Server dumping a load of files onto another drive.

 

[exobuzz]

Should wwt allow me to transfer a game directly from a backup dvd to a wbfs partition? I get an error when trying. Looks as though it's trying to seek in the stdin which won't work of course.

# dd if=/dev/dvd | wwt ADD -p /dev/sdm2 -
*****ÂÂwwt: Wiimms WBFS Tool v0.40a r790 x86_64 - Dirk Clemens - 2010-03-31ÂÂ*****
WBFSv1 #1 opened: /dev/sdm2
- ADD 1/1 [R8PE01] ISO:- (stdin)

************************************************************************
*****ÂÂIt seems, that the caching area for the game is too small!ÂÂ*****
*****ÂÂPlease report this to the author.ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ *****
*****ÂÂTechnical data: ID=R8PE01, OFF=40000, SIZE=7c0000ÂÂÂÂÂÂÂÂÂÂ *****
************************************************************************

!! wwt: ERROR #19 [READ FILE FAILED] in ReadSF() @ lib-sf.c#433
!!ÂÂÂÂÂÂSeek failed [F=4,200000]: - (stdin)
!!ÂÂÂÂÂÂ-> Illegal seek
!! wwt: ERROR #22 [WBFS ERROR] in wbfs_add_disc_param() @ libwbfs/libwbfs.c#1414
!!ÂÂÂÂÂÂerror reading disc
!! wwt: ERROR #22 [WBFS ERROR] in exec_add() @ wwt.c#2239
!!ÂÂÂÂÂÂError while creating disc [R8PE01] @/dev/sdm2

 

[Wiimm]

@exobuzz

Reading from pipe needs a cache because reading already done data.
I know that the cache must be about 256 MiB for some discs with update partitions.

Please test the following 2 variants ...

# Direct access of the dvd
wwt ADD -p /dev/sdm2 /dev/dvd

# copy only game (data) partition from pipe
dd if=/dev/dvd | wwt ADD -p /dev/sdm2 - --psel=game

... and posts the results.

 

[exobuzz]

Thanks. Wills try. Never though to pass the /dev/dvd directly to it! Was this mentioned in the documentation and I missed it, or it is a hidden feature? :-)

btw what happens if --raw is chosen? i did try this, and it sat there for ages (the hd accessing but never seemed to finish), until i got bored and did ctrl+c. That was using dd and piping again.

and I forgot to say thanks for a great tool. Nice to have some powerful commandline tool, as I do all the copying remotely with the drive connected to my headless fileserver.

 

[exobuzz]

The direct access to the dvd worked great. I didn't try the copy only game partition from pipe though.

 

[MaK11-12]

Great app!
I was just wondering what version of file system does this app work with?
Becuase it says that i have 6 errors becuase of the old libwbfs v0 .
Do i need to format the drive again? Or can i convert it or something?

MaK
ps. sorry for the overload of questions...

 

[exobuzz]

One thing I noticed using this, was when I added an item to the wbfs partition, and then viewed the partition in it in WBFS Backup Manager http://sites.google.com/site/completesg/ba...-backup-manager on windows the titles showed up as

"name ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "

as though the string termination was different/incompatible/broken. Any info on this? (I'm not saying necessarily your program is at fault, but wondered what the difference is) ?

 

[Skeeve]

I have an Request for enhancement, Wiim.

wit copy --wbfs *.iso --dest /Volumes/HOMEBREW/wbfs/%X/

will copy (example) to

WBFS:/Volumes/WIIDISK/wbfs/Toy Story Mania! [R5IP4Q].wbfs/R5IP4Q.wbfs

whereas I'd like to have it as

WBFS:/Volumes/WIIDISK/wbfs/Toy Story Mania! [R5IP4Q]/R5IP4Q.wbfs

Could you change it so that either if %X is used as a directory name no extension is added or add an option to suppress the extension?

BTW: Which %X-s are allowed?

Update: As wit can't create directories (can it?) I've created a workaround script "iso2wbfs":

#!/bin/sh
d=/Volumes/WIIDISK/wbfs
while [ -n "$1" ]; do
ÂÂÂÂtarget=`/usr/local/bin/wit copy --wbfs "$1" --dest $d/%X/ --test | perl -ne 'print $1 if m#WBFS:(.*)\.wbfs/.*\.wbfs\s*$#'`
ÂÂÂÂmkdir -p "$target"
ÂÂÂÂ/usr/local/bin/wit copy --wbfs "$1" --dest "$target"
ÂÂÂÂshift
done

 

[Wiimm]

Great app!
I was just wondering what version of file system does this app work with?
Becuase it says that i have 6 errors becuase of the old libwbfs v0 .
Do i need to format the drive again? Or can i convert it or something?
I have found some errors in libwbfs. I made some mods and just call it v1. One of these errors is a initial wrong free blocks table: The last blocks are marked as used, but they are not.

One thing I noticed using this, was when I added an item to the wbfs partition, and then viewed the partition in it in WBFS Backup Manager http://sites.google.com/site/completesg/ba...-backup-manager on windows the titles showed up as

"name ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "

as though the string termination was different/incompatible/broken. Any info on this? (I'm not saying necessarily your program is at fault, but wondered what the difference is) ?
My tools show always the title of the title db (taken from WiiTDB). You can disable this with -T0. On the other haned "wit dump -l iso_file" will show title and internal name of the ISO together.

I have an Request for enhancement, Wiim.

wit copy --wbfs *.iso --dest /Volumes/HOMEBREW/wbfs/%X/

will copy (example) to

WBFS:/Volumes/WIIDISK/wbfs/Toy Story Mania! [R5IP4Q].wbfs/R5IP4Q.wbfs

whereas I'd like to have it as

WBFS:/Volumes/WIIDISK/wbfs/Toy Story Mania! [R5IP4Q]/R5IP4Q.wbfs

Could you change it so that either if %X is used as a directory name no extension is added or add an option to suppress the extension?
'%X' is a short cut for '%T [%I].%E'. Just use '%T [%I]' instead. The next version will have a '%Y'.


QUOTE(Skeeve @ Apr 11 2010, 05:04 PM) BTW: Which %X-s are allowed?

See docu file wit.txt and search for command "COPY".



QUOTE(Skeeve @ Apr 11 2010, 05:04 PM)

Update: As wit can't create directories (can it?) I've created a workaround script "iso2wbfs":
...

If using --DEST (or -D) instead of --dest (or -d) then the missing directories are created automatically. (this is valid for all tools)

 

[Skeeve]

BTW: Which %X-s are allowed?
See docu file wit.txt and search for command "COPY".



QUOTE(Skeeve @ Apr 11 2010, 05:04 PM) Update: As wit can't create directories (can it?) I've created a workaround script "iso2wbfs":
...

If using --DEST (or -D) instead of --dest (or -d) then the missing directories are created automatically. (this is valid for all tools)

Wiimm, I'm deeply impressed! Your tool becomes better each time I use it

BTW. You're german (too)?