Hacking Configurable USB Loader

[George Dawes]

wbfs_file write sparse file whcih are smaller.

It does? AFAIK wbfs_file writes wbfs files and scrubbed .isos.

Sparse files are a function of the File System.

 

[oggzee]

wbfs_file write sparse file whcih are smaller.

It does? AFAIK wbfs_file writes wbfs files and scrubbed .isos.
Sparse files are a function of the File System.

It does.

 

[DIXES]

What's meaning of ntfs getf -37 error ? Thanks !!!
I use v50b, launching some .iso files. (Use WBM convert some .wbfs files to .iso files)

That means the file is fragmented in more than 5000 places which is the current limit. It looks like the current method of creating sparse files makes the files highly fragmented, at least it seems that is the case with wbfs_file, but probably with WBM too.

Can you try using explorer on windows:
- copy that file to a different hdd for example C:
- remove the file on USB
- copy the file back from C: to USB
This will loose the sparse properties but i think it will not be fragmented anymore.

To solve the issue I think that we would need to adjust the sparse file creation...

Thanks for your replying.
I try using Defraggler to defrag that HDD, then sparse isos can work fine.
It seems not a good solution, but it's a only way currently maybe.

 

[oggzee]

Thanks for your replying.
I try using Defraggler to defrag that HDD, then sparse isos can work fine.
It seems not a good solution, but it's a only way currently maybe.

Good to hear. (I hope I understood correctly that after de-fragmenting the drive the sparse files that previously gave an error now work properly?)

As an aside note it looks like the sparse file creation will need to be revisited. Currently it sets the sparse flag and writes the blocks where they are needed. Maybe what it should do is switch between sparse and non-sparse mode, when creating holes or writing data, using also SetEndOfFile to indicate the size of sparse / data block. I'm not sure however how can that be done and if I have time to look into it...

 

[fig2k4]

I didn't think that sparse files would cause any more fragmentation than writing a normal file, but obviously it does. I'll have a quick check as well to see if writing to sparse files can be optimised.

 

[oggzee]

I didn't think that sparse files would cause any more fragmentation than writing a normal file, but obviously it does. I'll have a quick check as well to see if writing to sparse files can be optimised.

Yeah i didn't think so either, but comparing the results with the windows defrag tool confirms this (the report does give you a list of the most fragmented files), if I just copy a file using explorer it doesn't get fragmented, while if that file is created using wbfs_file it is unfortunately fragmented .

 

[fig2k4]

Yeah i didn't think so either, but comparing the results with the windows defrag tool confirms this (the report does give you a list of the most fragmented files), if I just copy a file using explorer it doesn't get fragmented, while if that file is created using wbfs_file it is unfortunately fragmented .

I was going to suggest using SetFileValidData() to preallocate the real size of the data. But reading here, I found that it won't work with sparse files.
http://msdn.microsoft.com/en-us/library/aa...28VS.85%29.aspx

BTW, this little tool looks quite handy. It lists the fragments for a file in the properties dialog and allows single files to be defragmented from the explorer context menu.
http://www.mutexed.com/code/FragExt/#defrag-manager

 

[hackotedelaplaqu]

Please answer my previous post.
Can I burn directly sparse files iso like normal ones ?

 

[fig2k4]

Yes. As far as sparse unaware apps are concerned, it's just a regular ISO file.

 

[hackotedelaplaqu]

Thanks fig2k4.
By the way, your manager is nice and fast

 

[Burton]

If I convert my fat32 drive to Ntfs what version of configurable usb loader I should use?

the 50 normal or the 50-Fat?

 

[Sacohen9665]

If I convert my fat32 drive to Ntfs what version of configurable usb loader I should use?

the 50 normal or the 50-Fat?

I believe Dr. Clipper answered that a couple of pages back and said the Fat would be better optimized for it because it still needs to run on cIOS 222.

 

[Retal]

If configurable USB loader supports NTFS then why isn't this mentioned in the first post? It seems like quite a breakthrough feature to just omit to mention.

 

[cman1783]

NTFS support is working really well. I was using a 160GB Maxtor formatted as WBFS. I have a spare 1.5TB Seagate that I formatted into 2 partitions (1st partition is 200GB for homebrew and the like and the rest is NTFS). It's working flawlessly with all emulators and USB CFG.

 

[GolfDude]

its not mentioned b/c ntfs support is only in the loader, the HBC doesnt support it yet, and youd still need a fat partition for your other hoembrew.. and until ntfs catches on with otehr homebrewers, youll still have to deal with FAT

 

[HemiSync]

NTFS support is working really well. I was using a 160GB Maxtor formatted as WBFS. I have a spare 1.5TB Seagate that I formatted into 2 partitions (1st partition is 200GB for homebrew and the like and the rest is NTFS). It's working flawlessly with all emulators and USB CFG.

How do you have it working with emulators & which ones? I didn't think the HBC utilized ntfs yet.

 

[bnm81002]

I have a MAJOR PROBLEM, I can't access the loader through my USB drive, I don't see any of my games displayed, I don't see my created theme, I updated to v50b2 and it was working fine till later on that evening, then nothing, I can't even access the Configurable Loader through the HBC either, same problems, it's not the drive because the Loader GX works fine, how do I go about reinstalling the Configurable Loader without losing any of my games? I am using the WBFS partition as well, thanks

 

[oggzee]

cfg v50 (release)

Cfg_USB_Loader_50.zip
cfg50.dol
cfg50-fat.dol
(or online update)

Changes:

cfg v50 (release)

* Optimizations for highly fragmented files (either fat or ntfs)

cfg v50b2 (beta2)

* Fixed crash when using flat /wbfs file layout without subdirectories
* Fixed crashes when starting HBC forwarder discs
And added safety checks of memory regions when loading disc
* Raised number of fragments limit to 20000
* Properly identify dual-layer iso
* Removed obsolete ehcmodules for IOS 222 revisions 2 and 3
External ehcmodules for these versions are still supported
* Use the new ehcmodule with fat/ntfs support also for wbfs
(but still uses wbfs mode for wbfs partition)
Can be overriden using an external ehcmodule4
External module for fat/wbfs has been renamed
from ehcmodule_fat.elf to ehcmodule_frag.elf
* In case the new fragments method fails for any reason for FAT
it will fallback to the old method
* Other cleanups

cfg v50b (beta)

* .iso files on NTFS support
The file name layout is the same as for .wbfs files:
/wbfs/gameid.iso or /wbfs/gameid_title/gameid.iso
* Fixes and cleanups for NTFS support (fixed ntfs getf -1 error)
* option: partition=ntfs1 accepted

About NTFS support:

FAT support in ehcmodule has been rewritten with a new
generic wii disc emulation system that is:
- light weight / zero overhead
- filesystem independent
- fileformat independent
It works by supplying the ehcmodule with a list of sector fragments
that specify the location of data using direct sector addressing.
To see the list of fragments one can use debug=1 and they will
be printed out before starting the game.
The number of fragments if limited to 5000, that number is also
the max theoretical number of fragments on a wbfs partition (actually
4600, for a dual layer disc with a 2mb wbfs block size). In normal
conditions the number of fragments should be a lot lower most commonly
just a single big block. Fragments are used to describe both physical
address on hdd and virtual adress on wii disc so if a .wbfs file is used
the list will be composed of 3 fragments - disc header, update partition
and game partition.
libntfs however doesn't seem stable enough for write access at the moment,
so the ntfs partition is mounted read-only meaning install and remove can't
be done from inside the loader for now.

Credits: WiiPower for libntfs modification which returns the list of fragments.

cfg v50a (alpha)

* Fix for PeppaPig (from NeoGamma by WiiPower)
* Fixes and cleanups for NTFS

cfg v50x (experimental)

* Rewritten FAT support in ehcmodule with a generic system
* NTFS support
* Improved gui speed with large number of games
(most noticable in grid, flow and flow-z gui styles)
* Print on the intro screen if the ios is reloaded a second time
(in case the setting from config.txt is different from default)
* The -fat version 'simple' option does not change 'hide_hddinfo'
* Changed WBFS ERROR: read error while installing a game to a WARNING.
Note: the read error check has been introduced in v47, all previous
versions including the original loader 1.5 and all other loaders
silently ignore it.
* Changed default value of install_partition=only_game
To avoid errors caused by modchips when trying to copy the update partition.
* Minor cosmetic changes to cover download when trying different urls.

 

[Retal]

its not mentioned b/c ntfs support is only in the loader, the HBC doesnt support it yet, and youd still need a fat partition for your other hoembrew.. and until ntfs catches on with otehr homebrewers, youll still have to deal with FAT

...but I don't need NTFS support in anything but the loader.

 

[cman1783]

NTFS support is working really well. I was using a 160GB Maxtor formatted as WBFS. I have a spare 1.5TB Seagate that I formatted into 2 partitions (1st partition is 200GB for homebrew and the like and the rest is NTFS). It's working flawlessly with all emulators and USB CFG.

How do you have it working with emulators & which ones? I didn't think the HBC utilized ntfs yet.

Sorry, 200GB is FAT32.