support for drives larger than 2TiB
Hello,
I've been playing with a SATA 2.5T drive and things don't look too bad. All four controllers I've tested worked fine and the driver and kernel worked just fine. Even BIOSes don't seem too bad. At least the two boards I tested (both about three years old) didn't have much problem recognizing upto 2TiB and could access and boot fine although I'm fairly sure there will be BIOSes which would behave erratically.
I also tested installing w/ openSUSE 11.3 and it worked fine. It automatically chose GPT and alignment and everything just worked (tm). I think the situation shouldn't be too different for any distro which uses up-to-date parted.
The only problem is that everything which is necessary for booting needs to be located below 2TiB limit. Please note that this is much stricter restriction than the 128GiB limit we had due to LBA28. That limit was caused by BIOSes using LBA28 and vendors could and did update and be done with it in many cases. However, 2TiB limit is inherent in the BIOS programming interface and currently the only way to overcome it is using a completely different BIOS interface (EFI, that is). Vendors are not likely to introduce EFI for already released products although they're much more likely to release updates so that BIOSes can access upto 2TiB if they don't work already. We'll be stuck with 2TiB limit on much more configurations for longer period of time.
So, distro installers need to try to locate everything needed for bootstrapping below 2TiB limit (ie. a dedicated boot partition below the limit). Drives > 2TiB aren't on the market yet but aren't too far away. Let's make sure things will be ready by the next distro release cycle.
Thanks.
Signature · 1 line
tejun
Message forwarded to [email protected], and also to the Debian BTS, package "debian-installer", bug #590169.
Thanks for the analysis and advice, Tejun.
Signature · 4 lines
"One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh
Quoted 4 lines · Tejun Heo
The only problem is that everything which is necessary for booting needs to be located below 2TiB limit. Please note that this is much stricter restriction than the 128GiB limit we had due to LBA28. That limit was caused by BIOSes using LBA28 and vendors could and did
Quoted 8 lines
update and be done with it in many cases. However, 2TiB limit is inherent in the BIOS programming interface and currently the only way to overcome it is using a completely different BIOS interface (EFI, that is). Vendors are not likely to introduce EFI for already released products although they're much more likely to release updates so that BIOSes can access upto 2TiB if they don't work already. We'll be stuck with 2TiB limit on much more configurations for longer period of time.
The only thing that would please me no end with newer replacements for BIOS is the ability to have 4k boot sectors. Imagine what we can do with 4k what we can't do with 512 bytes.
Signature · 3 lines
http://www.munted.org.uk One very high maintenance cat living here.
Quoted 3 lines · Alex Buell
The only thing that would please me no end with newer replacements for BIOS is the ability to have 4k boot sectors. Imagine what we can do with 4k what we can't do with 512 bytes.
Are you saying that 4K sectors have some special nice implications for the boot process, or that the boot process is the last major hangup to fully supporting a device with 4K sectors, which would give us an 8X boost in capacity on all the codepaths that work via sector humbers? I suspect you mean the latter, but it's early in the morning still.. ;)
As a side consideration - moving from 512 to 4K moves the associated limit from 2 TiB to 16 TiB. Given the current rate of device density increase, how much time will that buy us, and what do we do then?
Quoted 11 lines · On Sat, 2010-07-24 at 09:49 -0400, [email protected] wrote:
Quoted 3 lines · Alex Buell
The only thing that would please me no end with newer replacements for BIOS is the ability to have 4k boot sectors. Imagine what we can do with 4k what we can't do with 512 bytes.
Are you saying that 4K sectors have some special nice implications for the boot process, or that the boot process is the last major hangup to fully supporting a device with 4K sectors, which would give us an 8X boost in capacity on all the codepaths that work via sector humbers? I suspect you mean the latter, but it's early in the morning still.. ;)
It would be interesting to see how newer BIOSses cope with 4k boot sectors. I'm sure the latest ATAPI standards do allow 4k boot sectors, I just want to know how this will be implemented for devices with larger physical sectors than the more usual 512 byte sectors.
Quoted 3 lines
As a side consideration - moving from 512 to 4K moves the associated limit from 2 TiB to 16 TiB. Given the current rate of device density increase, how much time will that buy us, and what do we do then?
There are now 3TB devices out there. But noone can boot from 4k devices yet on existing PC systems.
I'm sure a market to provide 3rd party BIOSes able to do this will develop shortly. I know of one: coreboot.
Signature · 3 lines
http://www.munted.org.uk One very high maintenance cat living here.
[email protected] put forth on 7/24/2010 8:49 AM:
Quoted 3 lines
As a side consideration - moving from 512 to 4K moves the associated limit from 2 TiB to 16 TiB. Given the current rate of device density increase, how much time will that buy us, and what do we do then?
The crystal ball tells me that SSD adoption will kick this date a little farther into the future given that SSD capacity lags mechanical by a large margin. In the not too distance future, PCs/laptops/netbooks will all transition to shipping with SSD as their sole/main internal storage device, with USB3/eSATA thumb drives of 1TB and above permanently replacing external USB/eSATA mechanical drives.
Servers will probably transition at about the same time to shipping with solely SSD storage internally for boot/OS and maybe some other primary and a little secondary storage, with mechanical storage being optional, whether internal or external. Mechanical in the form of FC/iSCSI SAN and NAS arrays will still rule large data needs well after the aforementioned transitions above.
Signature · 1 line
Stan
Hello,
Quoted 5 lines · Stan Hoeppner
[email protected] put forth on 7/24/2010 8:49 AM:
Quoted 3 lines
As a side consideration - moving from 512 to 4K moves the associated limit from 2 TiB to 16 TiB. Given the current rate of device density increase, how much time will that buy us, and what do we do then?
I don't really see it happening on ATA hard drives. Maybe on USB / external ones. There simply are too many compatibility implications.
Quoted 4 lines
In the not too distance future, PCs/laptops/netbooks will all transition to shipping with SSD as their sole/main internal storage device, with USB3/eSATA thumb drives of 1TB and above permanently replacing external USB/eSATA mechanical drives.
After hearing the same story for the good part of the decade, I'm a bit skeptical about this too. The unit capacity price ratio hasn't really changed that much. Maybe requirements for storage capacity have reached / are approaching saturation point, so as long as the price keeps coming down, SSDs can be mainstream without closing the unit capacity price difference. At this point, hybrid seems much more realistic to me, but then again, maybe we really are reaching the saturation point. Who knows?
Thanks.
Signature · 1 line
tejun
Quoted 1 line · Tejun Heo
saturation point. Who knows?
I don't think any design decision should be made on the assumption that drives won't get larger.
There is always a market for bigger capacity drives. Today I use SSD for system drive, and then much larger capacity drives for long term storage. The term "drives are either new or full" still holds true in the 2TB era, it'd be the same if there were 8TB drives available.
If it was economically possible to store uncompressed video I'm sure we would, if it was anywhere economical. With 1080p60 that's a LOT of data.
If there is cheap storage available, people will stop erasing data and just let it pile up. I know I do, but I'm less cost sensitive than most.
Signature · 1 line
Mikael Abrahamsson email: [email protected]
Quoted 5 lines
However, 2TiB limit is
inherent in the BIOS programming interface and currently the only way to overcome it is using a completely different BIOS interface (EFI, that is).
Nope, look at the Int13 extensions, it already support 64-bit LBA.
Yuhong Bao
_________________________________________________________________
The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail.
http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4
Quoted 8 lines · Yuhong Bao
Quoted 5 lines
However, 2TiB limit is
inherent in the BIOS programming interface and currently the only way to overcome it is using a completely different BIOS interface (EFI, that is).Nope, look at the Int13 extensions, it already support 64-bit LBA.
Yuhong Bao
64-bit LBA? I assume you meant 48-bit LBA from ATA-7. It will address up to PBs I believe. (KB, MB, GB, TB, PB, ....)
But I've seen lots of "48-bit LBA" supporting controllers that hit limits at 500GB, 1TB, etc. and needed new firmware updates.
Secondly, if you look at the table on the right of
http://en.wikipedia.org/wiki/Master_boot_record#Disk_partitioning you see that the starting sector of a partition is defined with a 32-bit value.
ie. 2TB with 512 byte sectors.
The normal solution is to move to a GPT which requires EFI if you want to boot from it.
Greg
Quoted 9 lines · Greg Freemyer
Secondly, if you look at the table on the right of
http://en.wikipedia.org/wiki/Master_boot_record#Disk_partitioning you see that the starting sector of a partition is defined with a 32-bit value.ie. 2TB with 512 byte sectors.
The normal solution is to move to a GPT which requires EFI if you want to boot from it.
Can't one have a GPT with a guard boot sector which can read the real partition table?
I think Tejun must mean some big problem at the BIOS APIs required for the bootloader. Tejun, what exactly croaks at the 2TB boundary (with 512KiB sectors)?
Signature · 4 lines
"One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh
Hello,
Quoted 6 lines · Henrique de Moraes Holschuh
Quoted 5 lines · Greg Freemyer
Secondly, if you look at the table on the right of
http://en.wikipedia.org/wiki/Master_boot_record#Disk_partitioning you see that the starting sector of a partition is defined with a 32-bit value.
Quoted 7 lines
Quoted 4 lines
ie. 2TB with 512 byte sectors.
The normal solution is to move to a GPT which requires EFI if you want to boot from it.
Can't one have a GPT with a guard boot sector which can read the real partition table?
Yeah, parted already does that although there seem to be some bugs and they sometimes are inconsistent. In principle, BIOSes don't really care about the partition table (they look at it primarily to figure out geometry and stuff) and can boot as long as the initial boot sector is accessible.
Quoted 3 lines
I think Tejun must mean some big problem at the BIOS APIs required for the bootloader. Tejun, what exactly croaks at the 2TB boundary (with 512KiB sectors)?
The 2TiB limit is coming from BIOS software interrupt for disk access using 32bit for addressing. If there's 64bit extension to BIOS disk access, everything should work but still locating boot partition under 2TiB is pretty much required at this point, I think.
Thanks.
Signature · 1 line
tejun
Quoted 3 lines
64-bit LBA? I assume you meant 48-bit LBA from ATA-7. It will address up to PBs I believe. (KB, MB, GB, TB, PB, ....)
Yes, the sector number in Int13 extensions really is 64-bit. Look at the Ralf Brown Interrupt List, for example.
Yuhong Bao
_________________________________________________________________
The New Busy is not the old busy. Search, chat and e-mail from your inbox.
http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_3
(cc'ing hpa, hi!)
Hello,
Quoted 6 lines · Yuhong Bao
Quoted 4 lines
However, 2TiB limit is
inherent in the BIOS programming interface and currently the only way to overcome it is using a completely different BIOS interface (EFI, that is).Nope, look at the Int13 extensions, it already support 64-bit LBA.
Ah, yeah, that sounds much more logical than requiring transition to EFI. Does anyone know how wide spread the support for this extension is in BIOSes? Do the bootloaders support this?
Thanks.
Signature · 1 line
tejun
Quoted 9 lines · Tejun Heo
(cc'ing hpa, hi!)
Hello,Quoted 5 lines · Yuhong Bao
Quoted 4 lines
However, 2TiB limit is
inherent in the BIOS programming interface and currently the only way to overcome it is using a completely different BIOS interface (EFI, that is).
Quoted 8 lines
Quoted 1 line
Nope, look at the Int13 extensions, it already support 64-bit LBA.
Ah, yeah, that sounds much more logical than requiring transition to EFI. Does anyone know how wide spread the support for this extension is in BIOSes? Do the bootloaders support this?
Thanks.
Syslinux 4 supports it with full 64-bit capabilities. SeaBIOS (used in Qemu/KVM) supports it, and I know at least some hardware/firmware RAID solutions support it; I recently got access to a 3 TB SATA drive but due to NDA requirements I can't reveal the results of that testing.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Hello,
Quoted 4 lines · H. Peter Anvin
Syslinux 4 supports it with full 64-bit capabilities. SeaBIOS (used in Qemu/KVM) supports it, and I know at least some hardware/firmware RAID solutions support it; I recently got access to a 3 TB SATA drive but due to NDA requirements I can't reveal the results of that testing.
Hmmm... doesn't sound too bad then. Is there an easy to test whether 64bit LBA works? I have a 2.5TiB drive and can test the machines I have and probably publish the result too.
Thanks.
Signature · 1 line
tejun
Quoted 12 lines · Tejun Heo
Hello,
Quoted 4 lines · H. Peter Anvin
Syslinux 4 supports it with full 64-bit capabilities. SeaBIOS (used in Qemu/KVM) supports it, and I know at least some hardware/firmware RAID solutions support it; I recently got access to a 3 TB SATA drive but due to NDA requirements I can't reveal the results of that testing.
Hmmm... doesn't sound too bad then. Is there an easy to test whether 64bit LBA works? I have a 2.5TiB drive and can test the machines I have and probably publish the result too.
If you can set up the disk with GPT then just set up a bootable partition beyond the 2 TiB mark and install Syslinux 4 on it. (Note:
there is a bug in current versions of gdisk: attribute 2 is the legacy BIOS bootable attribute, but the author got the bit order wrong and so you have to set "attribute 61". parted doesn't support the legacy BIOS bootable attribute yet.)
If you don't want tomodify the contents of the disk then it can still be tested, but I don't have a ready-made test for you.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Hello,
Quoted 9 lines · H. Peter Anvin
If you can set up the disk with GPT then just set up a bootable partition beyond the 2 TiB mark and install Syslinux 4 on it. (Note:
there is a bug in current versions of gdisk: attribute 2 is the legacy BIOS bootable attribute, but the author got the bit order wrong and so you have to set "attribute 61". parted doesn't support the legacy BIOS bootable attribute yet.)If you don't want tomodify the contents of the disk then it can still be tested, but I don't have a ready-made test for you.
The disk doesn't contain any data. I'll try it with several different motherboards and report back.
Thanks.
Signature · 1 line
tejun
Quoted 6 lines · H. Peter Anvin
Syslinux 4 supports it with full 64-bit capabilities. SeaBIOS (used in
Qemu/KVM) supports it, and I know at least some hardware/firmware RAID solutions support it; I recently got access to a 3 TB SATA drive but due to NDA requirements I can't reveal the results of that testing.
Yes but is it possible to boot from a complete 4k sector with the new standards?
Signature · 3 lines
http://www.munted.org.uk One very high maintenance cat living here.
Quoted 10 lines · Alex Buell
Quoted 6 lines · H. Peter Anvin
Syslinux 4 supports it with full 64-bit capabilities. SeaBIOS (used in
Qemu/KVM) supports it, and I know at least some hardware/firmware RAID solutions support it; I recently got access to a 3 TB SATA drive but due to NDA requirements I can't reveal the results of that testing.Yes but is it possible to boot from a complete 4k sector with the new standards?
Yes, but currently there are no 4k *logical* sector products on the market (and $DEITY knows how many BIOSes would handle them correctly.) The BIOS interfaces should handle them fine, though.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 7 lines
I've been playing with a SATA 2.5T drive and things don't look too bad. All four controllers I've tested worked fine and the driver and kernel worked just fine. Even BIOSes don't seem too bad. At least the two boards I tested (both about three years old) didn't have much problem recognizing upto 2TiB and could access and boot fine although I'm fairly sure there will be BIOSes which would behave erratically.
In fact, Seagate is already selling 3TB external drives and CrunchGear was already able to get the internal 3TB drive inside out:
http://techcrunch.com/2010/07/01/hands-on-the-seagate-freeagent-goflex-desk-3tb-external-hard-drive/
Yuhong Bao
_________________________________________________________________
Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox.
http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1
Quoted 4 lines · Yuhong Bao
In fact, Seagate is already selling 3TB external drives and CrunchGear was already able to get the internal 3TB drive inside out:
http://techcrunch.com/2010/07/01/hands-on-the-seagate-freeagent-goflex-desk-3tb-external-hard-drive/
The key word there is _external_ --> by using USB3/2/FW interfaces, they neatly sidestep most BIOS issues. For the time being anyway.
Quoted 9 lines
Quoted 4 lines
In fact, Seagate is already selling 3TB external drives and CrunchGear was already able to get the internal 3TB drive inside out:
http://techcrunch.com/2010/07/01/hands-on-the-seagate-freeagent-goflex-desk-3tb-external-hard-drive/The key word there is _external_ --> by using USB3/2/FW interfaces, they neatly sidestep most BIOS issues. For the time being anyway.
A more detailed review from AnandTech:
http://www.anandtech.com/show/3858/the-worlds-first-3tb-hdd-seagate-goflex-desk-3tb-review