Hacking 4 TB Seagate / DIOS MIOS issues

[Elliander]

I would bet it would be more feasible to just have a 512 sector version and a separate 4k sector version. I kinda think the first failed attempt was just that, though, a hard coded 4k sector size. Can't remember exactly, though.

I agree. Actually, right as you replied I was in the middle of editing my post to suggest essentially the same thing:

​

​

Maybe there could be a separate DIOS MIOS install. The current version can detect a 4k drive and produce a message, so maybe have USB Loader include both a 512 and 4k version together, and have USB Loader handle the detection. It could have a toggle in settings to switch between 512 and 4k mode. That would in turn allow people to snip out the extra code for auto-detection and still maintain the ability for people of different hard drive types to use it.​

​

As you said, have a separate version for 512 and 4k. Then just let USB Loader handle which version to use.

 

[Rydian]

Even if by some chance there are still some to find, the fact remains that my 1 TB I purchased before they shifted to 4k works fine. So why would I downgrade to 500 GB when I need more space (not less) and where there isn't even a likelihood of it working?

You seemed tired of trying multiple drives so I suggested something that would give you the biggest chance of success. If you still want larger and want to keep trying drives until you find one that works, nobody's stopping you.

That said, age and capacity do not share the same line, especially when you get into 3.5" externals versus 2.5" externals.

 

[Elliander]

I thought PS3 had a HDD size limit too. maybe it's only for internal storage?

Just an update on that: I finished transferring my PS1 games from the internal hdd to the 4 TB external. I had no problems loading PS1 games. I also had no problems loading PS3 games from the external hdd. I haven't tested PS2 games yet - waiting for Swap Magic to arrive so I can use that for a hardware driven PS2 loading mode - but I have no reason to believe that they wouldn't work. (not going to bother with PS2 classics because I value hardware mode. I currently play with the physics discs because software mode is too annoying.) So anyway, it looks like the PS3 is completely unaffected by the 4k sector size, and there are no problems with having a single 4 TB partition in FAT32. (well, technically 3.72ish TB when you count lost space.)

Since I can't find another hard drive option I will likely either wait until DIOS MIOS is updated to support 4k drives, or until it becomes possible to use a powered USB hub with the Wii for multiple external hdd. (Would that even be possible?)

 

[Elliander]

You seemed tired of trying multiple drives so I suggested something that would give you the biggest chance of success. If you still want larger and want to keep trying drives until you find one that works, nobody's stopping you.

That said, age and capacity do not share the same line, especially when you get into 3.5" externals versus 2.5" externals.

That is a good point. I could always try looking for internals and use it with a hard drive enclosure.Also, I could try the refurbished market. (though I usually have a high failure rate with refurbished drives.) and of course I could try fixing a broken hdd (It was 2 TB, not 3 TB as I thought) which I know has a 512 byte sector size. That one only failed because I plugged in and unplugged repeatedly and had a major failure to data, but the partitions were recoverable so it might work if I don't do that again. (that's also why I am unwilling to use multiple different drives to swap with the Wii)

Still, the fact that it will get more and more difficult to find one that will work at all actually makes me more hopeful that someone will eventually get DIOS MIOS to work with newer drives. If that wasn't the case there would be no incentive. I have no problem waiting a year or more if that's what it would take.

 

[Rydian]

The 2.5" 320GB drive I ordered off Amazon a few weeks back works with it just fine. It doesn't have to be out of stock or collecting dust, just you're not going to find any 4TB drives built on Windows 98 standards. XD

 

[Elliander]

The 2.5" 320GB drive I ordered off Amazon a few weeks back works with it just fine. It doesn't have to be out of stock or collecting dust, just you're not going to find any 4TB drives built on Windows 98 standards. XD

Also no 2 TB. I still might try a 1.5 TB in a few weeks, but anything smaller than 1 TB would be a downgrade for me. You probably just got lucky and had one arrive that was from an older stock pile.

 

[Rydian]

My 2TB internal is 4K, but that's for my PC so it never became an issue.

The 320GB I ordered online from Amazon? Of course it's older, they're not being made anymore, but that doesn't mean they're not being sold retail. This is the type of thing I keep pushing you to look for. Lower capacity and older drives doesn't mean refurb or something.

 

[Elliander]

I did some research and I learned about a 4 TB external hard drive that was released by Seagate before the shift to 4k sector drives. The "Seagate FreeAgent GoFlex 4 TB" with part number 9ZQAD5-500 is supposed to use 512 byte physical sectors, and uses an adapter to logically interpret it as 4k sector size for newer operating systems. This review appeared to confirm that so I went and found one "used - like new" on ebay and am going to test it out. If it doesn't work I can use it with my PC so it won't be any problem. I will update this thread with whatever I find out.

 

[PsyBlade]

how do you plan to partition it?

 

[Elliander]

how do you plan to partition it?

Good question. Well, assuming it really is a 512 byte drive I can use it with the attachment to be interpreted as 4k or without it to interpret it as 512 bytes. Since there is a 2 TB ceiling on 512-bytes I will likely be forced to go with multiple partitions, but I read that this ceiling has more to do with limitations of the MBR. I will TRY to go with one partition, but I really doubt that it will work in 512 byte mode so I will probably go with 2 partitions of 2 TB each.

A question about that though: The DIOS MIOS partition needs to be primary, but what about the Wii games for USB Loader GX and EmuNAND content in either USB Loader GX and/or Sneek? And could I safely put Wii games on both partitions?

Also, what if I made both partitions primary?

 

[Cyan]

DIOS MIOS : Primary and MBR, first listed partition of the partition table only
Wii games : Primary, logical, GPT, any partition number
EmuNAND : it depends on the cIOS you are using:

• rev17: 1st FAT32 partition of a 512 bytes/sector HDD, NandEmuPath must be on root, Full EmuNAND only
• rev18: adds Partial EmuNAND mode
• rev21: adds EmuNAND paths (can be in sub-folders)
• rev21 d2x beta: adds partition selection. officially supported in d2x v3
• d2x v4: adds 4096 bytes/sector support

You can make up to 4 primary partitions.
The MBR (partition table) can hold 4 partitions informations (4x [partition type + partition starting sector])
These partitions can be either primary or extended.
An extended partition is only an additional partition table located at one of the four designated primary "partition starting sector" instead of a partition itself, which can hold another number of (logical) partitions. It can also be a GPT instead of Extended, in that case the GPT table is located right after the MBR instead of the partition starting sector.
GPT is required with drives > 2TB, as MBR/LBA can only address sector number with 32 bit. (Edit: Or maybe it only requires that the partition starting sector is bellow 2TB?)

So, if you format the drive above the 2TB limit, the drive might switch to GPT, and it will not work with DIOS MIOS either.
You'll have to see the possible choices in your Partition manager.
You could end up with a 4TB with only 2TB used, but at least you found a working drive

Edit2:
I don't know if you can have 0xEE (GPT) as second MBR's partition table.
Is it possible to set hybrid partition table? the first MBR's partition array to a Primary, and the second to GPT?
Of course, you'll have to put the first primary partition not on LBA1 but at LBA34 (after GPT table), but I'm not sure other softwares will find the GPT.

 

[Elliander]

DIOS MIOS : Primary and MBR, first listed partition of the partition table only
Wii games : Primary, logical, GPT, any partition number
EmuNAND : it depends on the cIOS you are using:

• rev17: 1st FAT32 partition of a 512 bytes/sector HDD, NandEmuPath must be on root, Full EmuNAND only
• rev18: adds Partial EmuNAND mode
• rev21: adds EmuNAND paths (can be in sub-folders)
• rev21 d2x beta: adds partition selection. officially supported in d2x v3
• d2x v4: adds 4096 bytes/sector support

Thanks! That really helps! What about with the EmuNAND used by Sneek and Uneek? What I will probably do with EmuNAND is have USB Loader handle all the Wii Ware games that fully work with it, and any game that doesn't work with USB Loader along with any DLC game I will have Uneek handle those games. That should take care of the problem I was having with games taking forever to load, but in the case of Uneek I still can't figure out how to remove Priiloader from the EmuNAND which was also causing slowdown in Uneek. In any case, I need to figure out which partition I would put the EmuNAND into for Uneek.

So, if you format the drive above the 2TB limit, the drive might switch to GPT, and it will not work with DIOS MIOS either.
You'll have to see the possible choices in your Partition manager.
You could end up with a 4TB with only 2TB used, but at least you found a working drive

That makes sense, but then again, as long as the partition that DIOS MIOS is working with is less than 2 TB it should be fine, right? So in theory if I am able to create a 3 TB FAT32 and a 1 TB FAT32, the 1 TB partition would work with DIOS MIOS, and even if the 3 TB partition switches to GPT it should work too, right?

What about the limitations of WBFS? Can a WBFS partition be larger than 2 TB on a 512 byte partition? Are there any advantages to using a WBFS partition over FAT32? I know it can only be used to load the Wii games, but are there any advantages?

I am a bit more eager to get a larger drive working than I was yesterday, because my 1 TB drive had a few problems last night. When loading Bomberman (Gamecube) it played OK, but on one of the levels the drive hanged and there was a black screen. After that I went to play a Wii game, Smash Bros Melee, and it didn't load at all. Just a black screen. I am checked the partition for errors all night (Elapsed Time: 11:37:39), and none of the sectors report as bad, and all the data appears to be intact, but overall that was a bad sign. Defragmenting that drive now.

I don't know if you can have 0xEE (GPT) as second MBR's partition table.
Is it possible to set mixed partition ? the first MBR's partition table to Primary, and the second to GPT?
Of course, you'll have to put the first primary partition not on LBA1 but at LBA34 (after GPT table), but I'm not sure other softwares will find the GPT.

That's a good question. I have some pretty good partition managers so I will certainly find out.

 

[Rydian]

https://sites.google.com/site/completesg/how-to-use/sneek/removing-preloader

WBFS has no advantages over FAT32 from what I read, people say it shouldn't even be used anymore.

 

[Cyan]

Thanks! That really helps! What about with the EmuNAND used by Sneek and Uneek?

I don't remember for neek.

That makes sense, but then again, as long as the partition that DIOS MIOS is working with is less than 2 TB it should be fine, right? So in theory if I am able to create a 3 TB FAT32 and a 1 TB FAT32, the 1 TB partition would work with DIOS MIOS, and even if the 3 TB partition switches to GPT it should work too, right?

I don't understand what you want to do. And I think you didn't understand, because that sentence doesn't make sense.
First, you can't make a 3TB FAT32 (FAT32 is limited to 2TB, as it's a 32bit, same problem than with MBR partitions).
DIOS MIOS is not "working as long as it's less than 2TB", because if it's bigger than 2TB, then it's : Not FAT32, not MBR. So of course it wouldn't work if it's >2TB. But it will not work only based on the partition size being <2TB.
If you create a 3TB + 1TB, then you can't boot DM on the "1TB" because it's the second partition. you need "1TB + 3TB" (in that order).
If you create a 1TB and a 3TB, the partition table will use GPT because you have a partition >2TB, so DM will not work at all, even with the 1TB partition because that partition will be defined in the GPT, not the MBR.

That's why I wondered in my "edit2" above if hybrid mode was possible.

What about the limitations of WBFS? Can a WBFS partition be larger than 2 TB on a 512 byte partition? Are there any advantages to using a WBFS partition over FAT32? I know it can only be used to load the Wii games, but are there any advantages?

I think WBFS can be larger. I don't remember the partition size limit.
Though, you are limited by the number of games the partition can hold. By default it can hold (byte/Sector)-12 : 500 games on a 512 byte/sector HDD, 4084 games on a 4kiB/sector HDD.
But with d2x, you can "fake" the sector size of a 512B HDD to use 4KiB WBFS header to hold 4084 games. the 500 games limit is not a problem now.

There's no advantage to use WBFS over FAT32, except maybe "I never had any problem with dual layer games" (Metroid Other M, Xenoblade, etc., none of these ISO blacked screen or froze with WBFS, while FAT32 users had problems).

 

[PsyBlade]

That's why I wondered in my "edit2" above if hybrid mode was possible.

I don't think that you can (or should want to) use GPT as a replacement for the second stage MBR format
there is a hybride mode but it works differently

first a little explanation for elliander
MBR and GPT are both partitioning tables
they record (outside of the patritions) which partitions exist on a drive
(like the "contents" page in a book record which chapter is where)
a partition isn't MBR or GPT a drive is

a drive can even contain both at the same time (thats the hybrid mode)
you can create a MBR table that holds only some partitions
(but only up to 3 instead of 4 primary since one is used to mark the drive as GPT)
and an GPT one that contains all (important since GPT aware tools ignore the MBR)
as long as both tables agree about the exact(!) starting point and size of the partitions this works fine

both the primary and logical partitions from the MBR can be included as primary in GTP
(GPT only supports primary, since there can be 128+ of them instead of 4 there was no need for indirection/logical partitions)

 

[Cyan]

I didn't thought about having a partition listed in both MBR and GPT at the same time!
It actually make sense.

 

[Elliander]

you can't make a 3TB FAT32 (FAT32 is limited to 2TB, as it's a 32bit, same problem than with MBR partitions).
DIOS MIOS is not "working as long as it's less than 2TB", because if it's bigger than 2TB, then it's : Not FAT32, not MBR. So of course it wouldn't work if it's >2TB. But it will not work only based on the partition size being <2TB.

I know that's the way it's supposed to be, but there are two cases where that isn't true. One is that, at least for the 4k drive (which I know we are not talking about here) I have no problems formatting the entire drive to FAT32.

The second is a weird program I came across which is required for using Swap Magic with the PS3 for PS2 backup loading. It appears to require that the drive have 512 byte sector size (because I couldn't get it to work with the drive that had 4k) and also the drive must first be formatted to NTFS through Windows XP. Then I have to use a program that creates a FAT32 partition on top of it which is viewed as FAT32 by the operating system, but in a partition manager it doesn't recognize the partition so it's not really FAT32. It's supposed to allow larger than 4 GB files and larger partition sizes while at the same time working with programs that need it to be FAT32. I have no idea what else it would work with, but I am curious to see if it would work with DIOS MIOS and/or Wii backup loading.

If you create a 3TB + 1TB, then you can't boot DM on the "1TB" because it's the second partition. you need "1TB + 3TB" (in that order).
If you create a 1TB and a 3TB, the partition table will use GPT because you have a partition >2TB, so DM will not work at all, even with the 1TB partition because that partition will be defined in the GPT, not the MBR.

So both partitions would use GPT?

That's why I wondered in my "edit2" above if hybrid mode was possible.


I think WBFS can be larger. I don't remember the partition size limit.
Though, you are limited by the number of games the partition can hold. By default it can hold (byte/Sector)-12 : 500 games on a 512 byte/sector HDD, 4084 games on a 4kiB/sector HDD.
But with d2x, you can "fake" the sector size of a 512B HDD to use 4KiB WBFS header to hold 4084 games. the 500 games limit is not a problem now.

There's no advantage to use WBFS over FAT32, except maybe "I never had any problem with dual layer games" (Metroid Other M, Xenoblade, etc., none of these ISO blacked screen or froze with WBFS, while FAT32 users had problems).

hmm. So with d2x would the "4KiB WBFS header" work with FAT32, or only with a WBFS partition?

Alternatively, in the event that I am not able to get this to work the way I want to, how does the read speed of a Gamecube disc compare with the read speed of a class 10 SDXC card? They are already up to 256 GB cards, and since I know that SDXC can work with the Wii it might just be simpler to use an SD card for DIOS MIOS, and devote an entire hard drive to one partition that everything else would use. And actually, even if I can get it to work, hard drives don't last forever and it will become more and more difficult to find 512 byte drives so I might someday have to do something like that anyway.

I don't think that you can (or should want to) use GPT as a replacement for the second stage MBR format
there is a hybride mode but it works differently

first a little explanation for elliander
MBR and GPT are both partitioning tables
they record (outside of the patritions) which partitions exist on a drive
(like the "contents" page in a book record which chapter is where)
a partition isn't MBR or GPT a drive is

a drive can even contain both at the same time (thats the hybrid mode)
you can create a MBR table that holds only some partitions
(but only up to 3 instead of 4 primary since one is used to mark the drive as GPT)
and an GPT one that contains all (important since GPT aware tools ignore the MBR)
as long as both tables agree about the exact(!) starting point and size of the partitions this works fine

both the primary and logical partitions from the MBR can be included as primary in GTP
(GPT only supports primary, since there can be 128+ of them instead of 4 there was no need for indirection/logical partitions)

So I would need to avoid using GPT on the hard drive that will use DIOS MIOS entirely? Or would hybrid mode solve the problem since DIOS MIOS would see a MBR?

 

[Rydian]

Elliander, are you sure you're not seeing "FAT" and assuming "FAT32" when it's really "vFAT" in some of these cases?

That said, if the sectors are 4K, then you can go above 2TB for FAT32. But you're trying to avoid 4K.

 

[PsyBlade]

I know that's the way it's supposed to be, but there are two cases where that isn't true. One is that, at least for the 4k drive (which I know we are not talking about here) I have no problems formatting the entire drive to FAT32.

Thats because the MBR limits are 32 bit sector addresses.
2^32*8Kib=32TiB but 2^32*512B = 2TiB

MBR in itself only requires partitions to start before that absolute address and to be less in size
it does not impose a limit on the absolute end address
but whatever reads the disk (usually the operating system, but DM in this case) might assume 32 bit addresses for everything
that would effectively limit all partitions to end before the limit too

The second is a weird program I came across which is required for using Swap Magic with the PS3 for PS2 backup loading. It appears to require that the drive have 512 byte sector size (because I couldn't get it to work with the drive that had 4k) and also the drive must first be formatted to NTFS through Windows XP. Then I have to use a program that creates a FAT32 partition on top of it which is viewed as FAT32 by the operating system, but in a partition manager it doesn't recognize the partition so it's not really FAT32. It's supposed to allow larger than 4 GB files and larger partition sizes while at the same time working with programs that need it to be FAT32. I have no idea what else it would work with, but I am curious to see if it would work with DIOS MIOS and/or Wii backup loading.

As far as I can tell thats only a standard fat32 formater.
It only forgot to change the filesystem type filed in the MBR.
Since most tools besides partition managers largely ignore it nowadays it still works.
If it bothers you it should be easy to fix.

So both partitions would use GPT?

the partitions can't use GPT
only the drive can and since it MBR can't record all partitions the drive is switched to GPT
(the default is to use either MBR or GPT, hybrid is rather uncommon)

hmm. So with d2x would the "4KiB WBFS header" work with FAT32, or only with a WBFS partition?

wbfs only, but ooth fat32 don't have the 512 game limit to begin with

So I would need to avoid using GPT on the hard drive that will use DIOS MIOS entirely? Or would hybrid mode solve the problem since DIOS MIOS would see a MBR?

the presence of GPT is not a problem for DM, only the absence of MBR is
that means you can either use either MBR only or GPT+MBR hybrid

 

[Elliander]

As far as I can tell thats only a standard fat32 formater.
It only forgot to change the filesystem type filed in the MBR.
Since most tools besides partition managers largely ignore it nowadays it still works.
If it bothers you it should be easy to fix.

hmm. Well now I am confused. Why would it be required for PS2 backup loading if it's an ordinary Fat32?

the presence of GPT is not a problem for DM, only the absence of MBR is
that means you can either use either MBR only or GPT+MBR hybrid

I see. What program would you suggest I use to create it as a Hybrid?

Elliander, are you sure you're not seeing "FAT" and assuming "FAT32" when it's really "vFAT" in some of these cases?

That said, if the sectors are 4K, then you can go above 2TB for FAT32. But you're trying to avoid 4K.

I'm sure. I used FAT32 GUI formatter with 32k cluster sizes each time, with the exception of with this tool which I still don't completely understand.