What to do about the 2TB limit on HDIO_GETGEO ?
Subsystems (4)
A message that is not in the archive (sent off-list or not yet received): [email protected]
(resending .. forgot to copy the lists originally)
We have a problem coming down the pipeline.
Practically all utilities that care about it, use ioctl(fd, HDIO_GETGEO) to determine the starting sector offset of a hard disk partition.
SCSI, libata, IDE, USB, Firewire.. you name it.
The return value uses "unsigned long",
which on a 32-bit system limits drive offsets to 2TB.
There will be single drives exceeding this limit within the next 12 months or less, and we already have RAID arrays that exceed 2TB.
So.. what's the replacement for HDIO_GETGEO on 32-bits ?
One candidate might seem to be the existing /sys/block/dev/partition/start which I expect is already 64-bit friendly.
But this requires about 150 lines of somewhat complex C code to access, using only the dev_t (from stat(2) on a file) as a starting point, or less if one relies upon the udev device name matching the sysfs device name.
Is it time now for HDIO_GETGEO64 to make an appearance? Similar to how the existing BLKGETSIZE64 is supplanting BLKGETSIZE ?
??
Quoted 2 lines · On Tue, 25 Mar 2008 00:02:10 -0400 Mark Lord <[email protected]> wrote:
Is it time now for
HDIO_GETGEO64to make an appearance? Similar to how the existing BLKGETSIZE64 is supplanting BLKGETSIZE ?
That sounds useful.
But you're the one who has investigated this - please make a recommendation?
Quoted 11 lines · Mark Lord
One candidate might seem to be the existing
/sys/block/dev/partition/startwhich I expect is already 64-bit friendly.But this requires about 150 lines of somewhat complex C code to access, using only the
dev_t(from stat(2) on a file) as a starting point, or less if one relies upon the udev device name matching the sysfs device name.Is it time now for
HDIO_GETGEO64to make an appearance? Similar to how the existing BLKGETSIZE64 is supplanting BLKGETSIZE ?
Probably a better thing to have would be a way to look up block devices in sysfs by device number.
-hpa
Quoted 9 lines · H. Peter Anvin
Quoted 8 lines · Mark Lord
One candidate might seem to be the existing
/sys/block/dev/partition/startwhich I expect is already 64-bit friendly.But this requires about 150 lines of somewhat complex C code to access, using only the
dev_t(from stat(2) on a file) as a starting point, or less if one relies upon the udev device name matching the sysfs device name.
Quoted 5 lines
Quoted 3 lines
Is it time now for
HDIO_GETGEO64to make an appearance? Similar to how the existing BLKGETSIZE64 is supplanting BLKGETSIZE ?Probably a better thing to have would be a way to look up block devices in sysfs by device number.
..
Yeah, that would be just as good, really. Maybe even better.
Mark Lord wrote (later on):
Quoted 9 lines
Instead, software has to search everything inside
/sys/block/looking for a "dev" file whose contents match, rather than just trying to access something like this:/sys/block/8:1/start
or
/sys/block/majors/8/minors/1/startOr any one of a number of similar ways to arrange it.
..
Greg ?
Quoted 8 lines · Mark Lord
Yeah, that would be just as good, really. Maybe even better.
Mark Lord wrote (later on):
Quoted 4 lines
Instead, software has to search everything inside
/sys/block/looking for a "dev" file whose contents match, rather than just trying to access something like this:
Quoted 7 lines
Quoted 5 lines
/sys/block/8:1/start
or
/sys/block/majors/8/minors/1/startOr any one of a number of similar ways to arrange it.
..
It shouldn't be under /sys/block... there are enough many things that scan /sys/block and assume any directory underneath it has the current format.
-hpa
Quoted 9 lines · H. Peter Anvin
Quoted 8 lines · Mark Lord
Yeah, that would be just as good, really. Maybe even better.
Mark Lord wrote (later on):
Quoted 4 lines
Instead, software has to search everything inside
/sys/block/looking for a "dev" file whose contents match, rather than just trying to access something like this:
Quoted 8 lines
Quoted 6 lines
Quoted 5 lines
/sys/block/8:1/start
or
/sys/block/majors/8/minors/1/startOr any one of a number of similar ways to arrange it.
..
It shouldn't be under
/sys/block... there are enough many things that scan/sys/blockand assume any directory underneath it has the current format.
..
So long as we only add things, and not remove them, then any software that scans /sys/block/ shouldn't care, really.
But yes, it could go elsewhere, too.
Perhaps a /sys/dev/ directory, populated with symbolic links (or hard links?) back to the /sys/block/ entries, something like this:
/sys/dev/block/8:0 -> ../../../block/sda
/sys/dev/block/8:1 -> ../../../block/sda/sda1
/sys/dev/block/8:2 -> ../../../block/sda/sda2
...
That's just a suggestion, really.
And what about character devices?
Perhaps Greg will chime in.
Quoted 32 lines · Mark Lord
Quoted 9 lines · H. Peter Anvin
Quoted 8 lines · Mark Lord
Yeah, that would be just as good, really. Maybe even better.
Mark Lord wrote (later on):
Quoted 4 lines
Instead, software has to search everything inside
/sys/block/looking for a "dev" file whose contents match, rather than just trying to access something like this:Quoted 8 lines
Quoted 6 lines
Quoted 5 lines
/sys/block/8:1/start
or
/sys/block/majors/8/minors/1/startOr any one of a number of similar ways to arrange it.
..
It shouldn't be under
/sys/block... there are enough many things that scan/sys/blockand assume any directory underneath it has the current format...
So long as we only add things, and not remove them, then any software that scans
/sys/block/shouldn't care, really.But yes, it could go elsewhere, too.
Perhaps a/sys/dev/directory, populated with symbolic links (or hard links?) back to the/sys/block/entries, something like this:
/sys/dev/block/8:0-> ../../../block/sda
/sys/dev/block/8:1-> ../../../block/sda/sda1
/sys/dev/block/8:2-> ../../../block/sda/sda2
...
Quoted 4 lines
That's just a suggestion, really.
And what about character devices?Perhaps Greg will chime in.
I've been waiting to see if sanity will take hold of anyone here.
Come on people, adding symlinks for device major:minor numbers in sysfs to save a few 10s of lines of userspace code? Can things get sillier?
You can add a single udev rule to probably build these in a tree in /dev if you really need such a thing...
And what's wrong with your new ioctl recomendation?
greg k-h
Quoted 40 lines · On Tue, 25 Mar 2008 12:25:15 -0700 Greg KH wrote:
Quoted 32 lines · Mark Lord
Quoted 9 lines · H. Peter Anvin
Quoted 8 lines · Mark Lord
Yeah, that would be just as good, really. Maybe even better.
Mark Lord wrote (later on):
Quoted 4 lines
Instead, software has to search everything inside
/sys/block/looking for a "dev" file whose contents match, rather than just trying to access something like this:Quoted 8 lines
Quoted 6 lines
Quoted 5 lines
/sys/block/8:1/start
or
/sys/block/majors/8/minors/1/startOr any one of a number of similar ways to arrange it.
..
It shouldn't be under
/sys/block... there are enough many things that scan/sys/blockand assume any directory underneath it has the current format...
So long as we only add things, and not remove them, then any software that scans
/sys/block/shouldn't care, really.But yes, it could go elsewhere, too.
Perhaps a/sys/dev/directory, populated with symbolic links (or hard links?) back to the/sys/block/entries, something like this:
/sys/dev/block/8:0-> ../../../block/sda
/sys/dev/block/8:1-> ../../../block/sda/sda1
/sys/dev/block/8:2-> ../../../block/sda/sda2
...Quoted 4 lines
That's just a suggestion, really.
And what about character devices?Perhaps Greg will chime in.
I've been waiting to see if sanity will take hold of anyone here.
Quoted 7 lines
Come on people, adding symlinks for device major:minor numbers in sysfs to save a few 10s of lines of userspace code? Can things get sillier?
You can add a single udev rule to probably build these in a tree in /dev if you really need such a thing...
And what's wrong with your new ioctl recomendation?
Ah, there's some sanity. :)
---
~Randy
Quoted 11 lines · Randy Dunlap
Quoted 8 lines
Come on people, adding symlinks for device major:minor numbers in sysfs to save a few 10s of lines of userspace code? Can things get sillier?
You can add a single udev rule to probably build these in a tree in /dev if you really need such a thing...
And what's wrong with your new ioctl recomendation?
Ah, there's some sanity. :)
It's not so much an issue of a few tens of lines of user space code, but rather the fact that something that should be O(1) is currently O(n).
-hpa
Quoted 5 lines · H. Peter Anvin
Quoted 4 lines · Randy Dunlap
Quoted 4 lines
Come on people, adding symlinks for device major:minor numbers in sysfs to save a few 10s of lines of userspace code? Can things get sillier?
Quoted 8 lines
Quoted 5 lines
Quoted 4 lines
You can add a single udev rule to probably build these in a tree in /dev if you really need such a thing...
And what's wrong with your new ioctl recomendation?
Ah, there's some sanity. :)
It's not so much an issue of a few tens of lines of user space code, but rather the fact that something that should be O(1) is currently O(n).
"should"? why? Is this some new requirement that everyone needs? I've _never_ seen anyone ask for the ability to find sysfs devices by major:minor number in O(1) time. Is this somehow a place where such optimization is warranted?
thanks,
greg k-h
Quoted 8 lines · Greg KH
Quoted 7 lines · H. Peter Anvin
Quoted 6 lines · Randy Dunlap
Quoted 6 lines
Come on people, adding symlinks for device major:minor numbers in sysfs to save a few 10s of lines of userspace code? Can things get sillier?
You can add a single udev rule to probably build these in a tree in /dev if you really need such a thing...
Quoted 8 lines
Quoted 3 lines
Quoted 2 lines
Quoted 1 line
And what's wrong with your new ioctl recomendation?
Ah, there's some sanity. :)
It's not so much an issue of a few tens of lines of user space code, but rather the fact that something that should be O(1) is currently O(n).
"should"? why? Is this some new requirement that everyone needs? I've _never_ seen anyone ask for the ability to find sysfs devices by major:minor number in O(1) time. Is this somehow a place where such optimization is warranted?
Well, when dealing with shell scripts a O(n) very easily becomes O(n^2).
For the stuff that I, personally, do, it's not a big deal, but people
with large number of disks have serious gripes with our boot times.
-hpa
Quoted 14 lines · H. Peter Anvin
Quoted 13 lines · Greg KH
Quoted 12 lines · H. Peter Anvin
Quoted 9 lines · Randy Dunlap
Quoted 8 lines
Come on people, adding symlinks for device major:minor numbers in sysfs to save a few 10s of lines of userspace code? Can things get sillier?
You can add a single udev rule to probably build these in a tree in /dev
if you really need such a thing...And what's wrong with your new ioctl recomendation?
Ah, there's some sanity. :)
It's not so much an issue of a few tens of lines of user space code, but rather the fact that something that should be O(1) is currently O(n).
Quoted 8 lines
Quoted 4 lines
"should"? why? Is this some new requirement that everyone needs? I've _never_ seen anyone ask for the ability to find sysfs devices by major:minor number in O(1) time. Is this somehow a place where such optimization is warranted?
Well, when dealing with shell scripts a O(n) very easily becomes O(n^2). For the stuff that I, personally, do, it's not a big deal, but people with large number of disks have serious gripes with our boot times.
How does this have anything to do with boot times? Do you really have a foolish shell script that iteratorates over every single disk in the sysfs tree for every disk? What does it do that for?
I thought we were talking about 2TB disks here, with a proposed new ioctl, not foolishness of boot scripts...
Quoted 3 lines
How does this have anything to do with boot times? Do you really have a foolish shell script that iteratorates over every single disk in the sysfs tree for every disk? What does it do that for?
Any time you want to get the sysfs information for a filesystem which is already mounted, that's what you're forced to do.
Quoted 2 lines
I thought we were talking about 2TB disks here, with a proposed new ioctl, not foolishness of boot scripts...
I pointed out that having a way to map device numbers to sysfs directories would have the same effect, *and* would be usable for other purposes. I'd rather see that than a new ioctl, and another, and another...
ioctl()s are also nasty since they're generally root-only (or rather, device-owner only). Since the information is already in sysfs, there is no benefit to this hiding. Otherwise one could consider a ioctl() "give me the sysfs name of this device."
-hpa
Quoted 7 lines · H. Peter Anvin
Quoted 3 lines
How does this have anything to do with boot times? Do you really have a foolish shell script that iteratorates over every single disk in the sysfs tree for every disk? What does it do that for?
Any time you want to get the sysfs information for a filesystem which is already mounted, that's what you're forced to do.
Quoted 6 lines
Quoted 2 lines
I thought we were talking about 2TB disks here, with a proposed new ioctl, not foolishness of boot scripts...
I pointed out that having a way to map device numbers to sysfs directories would have the same effect, *and* would be usable for other purposes. I'd rather see that than a new ioctl, and another, and another...
Again, a simple udev rule will give you that today if you really want it...
And I think 'udevinfo' can be used to retrieve this information as well.
thanks,
greg k-h
Quoted 3 lines · H. Peter Anvin
Well, when dealing with shell scripts a O(n) very easily becomes O(n^2).
For the stuff that I, personally, do, it's not a big deal, but people
with large number of disks have serious gripes with our boot times.
This should be a solved problem with scsi_mod.scan=async (or equivalent compile option). Are people still complaining about it, and if so, have they tried this option?
Signature · 4 lines
Intel are signing my paycheques ... these opinions are still mine "Bill, look, we understand that you're interested in selling us this operating system, but compare it to ours. We can't possibly take such a retrograde step."
Quoted 4 lines · Greg KH
Quoted 1 line · Mark Lord
Perhaps Greg will chime in.
I've been waiting to see if sanity will take hold of anyone here.
..
So have we. sysfs is a total nightmare to extract information from under program / script control. The idea presented in this thread, is to have it cross-index the contents with a method that actually makes it easy to access in many common scenarios, without requiring huge gobs of code in user space. Or in kernel space.
And it's not just a few 10s of lines of code currently, but rather about 80-100 lines just to find the correct device subdir, and *then* a few more 10s of lines of code to retrieve the value.
In a bulletproof fashion, that is. Sure it can be slightly smaller if niceties such as error checking/handling are omitted.
There's no guarantee that udev is present, and even if it were present, there's no guarantee that the names in /dev/ will match /sysfs/ pathnames, since udev is very configurable to do otherwise.
So lookups are by dev_t, which sysfs has no simple or even easy way of accomplishing. O(n) at a minimum.
If we make it easier to access, then more programs will use it rather than us having to expand our tricky binary ioctl interfaces.
Isn't that part of the idea of sysfs -- to limit the need for new ioctls ?
Cheers
Hello,
Quoted 17 lines · Mark Lord
So have we. sysfs is a total nightmare to extract information from under program / script control. The idea presented in this thread, is to have it cross-index the contents with a method that actually makes it easy to access in many common scenarios, without requiring huge gobs of code in user space. Or in kernel space.
And it's not just a few 10s of lines of code currently, but rather about 80-100 lines just to find the correct device subdir, and *then* a few more 10s of lines of code to retrieve the value.
In a bulletproof fashion, that is. Sure it can be slightly smaller if niceties such as error checking/handling are omitted.
There's no guarantee that udev is present, and even if it were present, there's no guarantee that the names in
/dev/will match /sysfs/ pathnames, since udev is very configurable to do otherwise.
Quoted 7 lines
So lookups are by
dev_t, which sysfs has no simple or even easy way of accomplishing. O(n) at a minimum.If we make it easier to access, then more programs will use it rather than us having to expand our tricky binary ioctl interfaces.
Isn't that part of the idea of sysfs -- to limit the need for new ioctls ?
The questions are...
1. Are we gonna push sysfs as the primary interface and not provide an alternative interface (ioctl here) which can provide equivalent information? There are people running their systems w/o sysfs but I think we're getting closer to this everyday.
2. Is udev an essential part of all systems? I'm not sure about this one. Lots of small machines run w/o udev and I think udev is a bit too high level to depend on for every system.
If both #1 and #2 are true, I agree with Mark that we need an easy to map from device number to matching sysfs nodes. Tools which are used early during boot and emergency sessions need this mapping and many of them are minimal C program w/o much dependency for a good reason. Requiring each of them to implement their own way to map device node to sysfs node is too awkward.
Probably something like /sys/class/block/MAJ:MIN or /sys/class/devnums/bMAJ:MIN?
Signature · 1 line
tejun
Quoted 12 lines · Tejun Heo
Hello,
Quoted 9 lines · Mark Lord
So have we. sysfs is a total nightmare to extract information from under program / script control. The idea presented in this thread, is to have it cross-index the contents with a method that actually makes it easy to access in many common scenarios, without requiring huge gobs of code in user space. Or in kernel space.
And it's not just a few 10s of lines of code currently, but rather about 80-100 lines just to find the correct device subdir, and *then* a few more 10s of lines of code to retrieve the value.
I think you are using either the wrong programming language, or your sysfs walking logic is quite convulted. Look at the udev and HAL code if you want to steal some compact, working sysfs code :)
Quoted 9 lines
Quoted 9 lines
In a bulletproof fashion, that is. Sure it can be slightly smaller if niceties such as error checking/handling are omitted.
There's no guarantee that udev is present, and even if it were present, there's no guarantee that the names in
/dev/will match /sysfs/ pathnames, since udev is very configurable to do otherwise.So lookups are by
dev_t, which sysfs has no simple or even easy way of accomplishing. O(n) at a minimum.
And again, is this a performance requiring operation?
Quoted 11 lines
Quoted 4 lines
If we make it easier to access, then more programs will use it rather than us having to expand our tricky binary ioctl interfaces.
Isn't that part of the idea of sysfs -- to limit the need for new ioctls ?
The questions are...
1. Are we gonna push sysfs as the primary interface and not provide an alternative interface (ioctl here) which can provide equivalent information? There are people running their systems w/o sysfs but I think we're getting closer to this everyday.
Exactly, originally you suggested a new ioctl, which would be trivial to add, and trivial to switch any program that was currently using an ioctl to get the disk size, to use it instead.
Since when is the major:minor view of devices the "standard" one that userspace uses? Last I looked, userspace uses symlinks and lots of other ways of directly accessing block devices in /dev/, and does not rely on major:minor.
And finally, I haven't seen a patch that implements this "shadow" tree, it would be interesting to see if it could even be done.
Quoted 3 lines
2. Is udev an essential part of all systems? I'm not sure about this one. Lots of small machines run w/o udev and I think udev is a bit too high level to depend on for every system.
My tiny little phone runs udev, I don't see why anyone wouldn't run it these days, except in very limited embedded applications with no dynamic devices. But if you are in that situation, you aren't querying the size of any random block device either :)
And heck, this phone is a very limited embedded application, with razor thin margins, if it can use udev, I'd be interested in hearing the justifications for anyone who says it is too large for their systems to use it.
Quoted 9 lines
If both #1 and #2 are true, I agree with Mark that we need an easy to map from device number to matching sysfs nodes. Tools which are used early during boot and emergency sessions need this mapping and many of them are minimal C program w/o much dependency for a good reason. Requiring each of them to implement their own way to map device node to sysfs node is too awkward.
Probably something like /sys/class/block/MAJ:MIN or /sys/class/devnums/bMAJ:MIN?
Why the preopcupation with major:minor? Just because you are able to grab it from an open file handle? Heck, why not just an ioctl to get the path within sysfs for the device currently open? :)
thanks,
greg k-h
Hello, Greg.
Quoted 6 lines · Greg KH
Quoted 4 lines
1. Are we gonna push sysfs as the primary interface and not provide an alternative interface (ioctl here) which can provide equivalent information? There are people running their systems w/o sysfs but I think we're getting closer to this everyday.
Exactly, originally you suggested a new ioctl,
Well, I like Mark but am not really him. :-)
Quoted 3 lines
which would be trivial to
add, and trivial to switch any program that was currently using an ioctl to get the disk size, to use it instead.
That should be the simplest solution for the problem at hand.
Quoted 4 lines
Since when is the major:minor view of devices the "standard" one that userspace uses? Last I looked, userspace uses symlinks and lots of other ways of directly accessing block devices in
/dev/, and does not rely on major:minor.
The fact that major:minor is the unique identifier of a device makes it a bit special compared to other names on filesystem.
Quoted 2 lines
And finally, I haven't seen a patch that implements this "shadow" tree, it would be interesting to see if it could even be done.
It's possible, all that's needed are symlinks. We do similar things all the time.
Quoted 9 lines
Quoted 3 lines
2. Is udev an essential part of all systems? I'm not sure about this one. Lots of small machines run w/o udev and I think udev is a bit too high level to depend on for every system.
My tiny little phone runs udev, I don't see why anyone wouldn't run it these days, except in very limited embedded applications with no dynamic devices. But if you are in that situation, you aren't querying the size of any random block device either :)
Quoted 4 lines
And heck, this phone is a very limited embedded application, with razor thin margins, if it can use udev, I'd be interested in hearing the justifications for anyone who says it is too large for their systems to use it.
I agree udev is affordable for most cases but it's still a major step to require it for every system. I would hate to hear that hdparm or fdisk doesn't work unless udev is online. These are tools which are used to recover systems.
Quoted 7 lines
Quoted 7 lines
If both #1 and #2 are true, I agree with Mark that we need an easy to map from device number to matching sysfs nodes. Tools which are used early during boot and emergency sessions need this mapping and many of them are minimal C program w/o much dependency for a good reason. Requiring each of them to implement their own way to map device node to sysfs node is too awkward.
Quoted 6 lines
Quoted 2 lines
Probably something like /sys/class/block/MAJ:MIN or /sys/class/devnums/bMAJ:MIN?
Why the preopcupation with major:minor? Just because you are able to grab it from an open file handle? Heck, why not just an ioctl to get the path within sysfs for the device currently open? :)
Because major:minor is the key attribute to devices?
Thanks.
Signature · 1 line
tejun
Quoted 9 lines · Tejun Heo
Quoted 5 lines
Since when is the major:minor view of devices the "standard" one that userspace uses? Last I looked, userspace uses symlinks and lots of other ways of directly accessing block devices in
/dev/, and does not rely on major:minor.The fact that major:minor is the unique identifier of a device makes it a bit special compared to other names on filesystem.
In particular, stat() and friends returns the device number, not a device name.
-hpa
Quoted 37 lines · Tejun Heo
Quoted 17 lines · Mark Lord
So have we. sysfs is a total nightmare to extract information from under program / script control. The idea presented in this thread, is to have it cross-index the contents with a method that actually makes it easy to access in many common scenarios, without requiring huge gobs of code in user space. Or in kernel space.
And it's not just a few 10s of lines of code currently, but rather about 80-100 lines just to find the correct device subdir, and *then* a few more 10s of lines of code to retrieve the value.
In a bulletproof fashion, that is. Sure it can be slightly smaller if niceties such as error checking/handling are omitted.
There's no guarantee that udev is present, and even if it were present, there's no guarantee that the names in
/dev/will match /sysfs/ pathnames, since udev is very configurable to do otherwise.Quoted 7 lines
So lookups are by
dev_t, which sysfs has no simple or even easy way of accomplishing. O(n) at a minimum.If we make it easier to access, then more programs will use it rather than us having to expand our tricky binary ioctl interfaces.
Isn't that part of the idea of sysfs -- to limit the need for new ioctls ?
The questions are...
1. Are we gonna push sysfs as the primary interface and not provide an alternative interface (ioctl here) which can provide equivalent information? There are people running their systems w/o sysfs but I think we're getting closer to this everyday.
2. Is udev an essential part of all systems? I'm not sure about this one. Lots of small machines run w/o udev and I think udev is a bit too high level to depend on for every system.
Quoted 8 lines
If both #1 and #2 are true, I agree with Mark that we need an easy to map from device number to matching sysfs nodes. Tools which are used early during boot and emergency sessions need this mapping and many of them are minimal C program w/o much dependency for a good reason. Requiring each of them to implement their own way to map device node to sysfs node is too awkward.
Probably something like /sys/class/block/MAJ:MIN
"Devices directories" are not supposed to contain duplicate entries. It would slow-down, or may even break things.
Quoted 1 line
or /sys/class/devnums/bMAJ:MIN?
These are no devices belonging to the class "devnums", so it may confuse things which crawl these directories to get "all devices". Current coldplug-like setups will likely add duplicate devices with the wrong subsystem. There are also bus-devices with have a dev_t, and that will make them show up in /sys/class, which might confuse some tools too.
I guess we will need to find some other solution as a /sys/class/ for that. And we must prefix the links with 'c' and 'b' because dev_t is not unique across char and block devices.
Thanks,
Kay
Quoted 15 lines · Kay Sievers
Quoted 2 lines
Probably something like /sys/class/block/MAJ:MIN
"Devices directories" are not supposed to contain duplicate entries. It would slow-down, or may even break things.
Quoted 1 line
or /sys/class/devnums/bMAJ:MIN?
These are no devices belonging to the class "devnums", so it may confuse things which crawl these directories to get "all devices". Current coldplug-like setups will likely add duplicate devices with the wrong subsystem. There are also bus-devices with have a
dev_t, and that will make them show up in/sys/class, which might confuse some tools too.
Quoted 4 lines
I guess we will need to find some other solution as a
/sys/class/for that. And we must prefix the links with 'c' and 'b' becausedev_tis not unique across char and block devices.
It doesn't really seem to be to belong under class at all. I would suggest /sys/dev/char/ and /sys/dev/block/, for char and block respectively.
-hpa
Quoted 21 lines · H. Peter Anvin
Quoted 19 lines · Kay Sievers
Quoted 4 lines
Probably something like /sys/class/block/MAJ:MIN
"Devices directories" are not supposed to contain duplicate entries. It would slow-down, or may even break things.
Quoted 2 lines
or /sys/class/devnums/bMAJ:MIN?
These are no devices belonging to the class "devnums", so it may confuse things which crawl these directories to get "all devices". Current coldplug-like setups will likely add duplicate devices with the wrong subsystem. There are also bus-devices with have a
dev_t, and that will make them show up in/sys/class, which might confuse some tools too.
Quoted 9 lines
Quoted 5 lines
I guess we will need to find some other solution as a
/sys/class/for that. And we must prefix the links with 'c' and 'b' becausedev_tis not unique across char and block devices.It doesn't really seem to be to belong under class at all. I would suggest
/sys/dev/char/and/sys/dev/block/, for char and block respectively.
This thread fizzled out without a patch... here goes:
[ note: I'm replying via gmail, so if it has whitespace mangled the patch please see the attachment ]
-----snip---->
sysfs: add /sys/dev/{char,block} to lookup sysfs path by major:minor
From: Dan Williams <[email protected]>
Why?:
There are occasions where userspace would like to access sysfs attributes for a device but it may not know how sysfs has named the device or the path. For example what is the sysfs path for /dev/disk/by-id/ata-ST3160827AS_5MT004CK? With this change a call to stat(2) returns the major:minor then userspace can see that /sys/dev/block/8:32 links to /sys/block/sdc.
What are the alternatives?:
1/ Add an ioctl to return the path: Doable, but sysfs is meant to reduce
the need to proliferate ioctl interfaces into the kernel, so this seems counter productive.
2/ Use udev to create these symlinks: Also doable, but it adds a
udev dependency to utilities that might be running in a limited environment like an initramfs.
---
drivers/base/core.c | 37 ++++++++++++++++++++++++++++++++++++- 1 files changed, 36 insertions(+), 1 deletions(-)
drivers/base/core.c +3 -0
@@ -27,6 +27,9 @@int (*platform_notify)(struct device *dev) =NULL ;int (*platform_notify_remove)(struct device *dev) =NULL ; +static struct kobject *dev_kobj; +static struct kobject *char_kobj; +static struct kobject *block_kobj;#ifdef CONFIG_BLOCK static inline int device_is_not_partition (struct device *dev)
@@ -759,6 +762,11 @@ static void device_remove_class_symlinks(struct
device *dev)
sysfs_remove_link(&dev->kobj, "subsystem");
}
+static struct kobject *device_to_dev_kobj(struct device *dev) +{
+ return dev->class == &block_class ? block_kobj : char_kobj;
+}
+
/**
* device_add - add device to device hierarchy.
* @dev: device.
@@ -775,6 +783,7 @@ int device_add(struct device *dev)
struct device *parent = NULL;
struct class_interface *class_intf;
int error;
+ char devt_str[25];
dev = get_device(dev);
if (!dev || !strlen(dev->bus_id)) {
@@ -806,9 +815,16 @@ int device_add(struct device *dev)
goto attrError;
if (MAJOR(dev->devt)) {
+ struct kobject *kobj = device_to_dev_kobj(dev);
+
error = device_create_file(dev, &devt_attr); if (error)
goto ueventattrError;
+
+ format_dev_t(devt_str, dev->devt);
+ error = sysfs_create_link(kobj, &dev->kobj, devt_str);
+ if (error)
+ goto devtattrError;
}
error = device_add_class_symlinks(dev);
@@ -854,6 +870,9 @@ int device_add(struct device *dev)
device_remove_class_symlinks(dev);
SymlinkError:
if (MAJOR(dev->devt))
+ sysfs_remove_link(device_to_dev_kobj(dev), devt_str);
+ devtattrError:
+ if (MAJOR(dev->devt))
device_remove_file(dev, &devt_attr);
ueventattrError:
device_remove_file(dev, &uevent_attr);
@@ -925,12 +944,16 @@ void device_del(struct device *dev)
{
struct device *parent = dev->parent; struct class_interface *class_intf;
+ char devt_str[25];
device_pm_remove(dev);
if (parent)
klist_del(&dev->knode_parent);
- if (MAJOR(dev->devt))
+ if (MAJOR(dev->devt)) {
+ format_dev_t(devt_str, dev->devt);
+ sysfs_remove_link(device_to_dev_kobj(dev), devt_str);
device_remove_file(dev, &devt_attr);
+ }
if (dev->class) {
device_remove_class_symlinks(dev);
@@ -1055,6 +1078,15 @@ int __init devices_init(void)
devices_kset = kset_create_and_add("devices", &device_uevent_ops, NULL); if (!devices_kset)
return -ENOMEM;
+ dev_kobj = kobject_create_and_add("dev", NULL);
+ if (!dev_kobj)
+ return -ENOMEM;
+ block_kobj = kobject_create_and_add("block", dev_kobj);
+ if (!block_kobj)
+ return -ENOMEM;
+ char_kobj = kobject_create_and_add("char", dev_kobj);
+ if (!char_kobj)
+ return -ENOMEM;
return 0;
}
@@ -1380,4 +1412,7 @@ void device_shutdown(void)
dev->driver->shutdown(dev);
}
}
+ kobject_put(char_kobj);
+ kobject_put(block_kobj);
+ kobject_put(dev_kobj);
}
sysfs: add /sys/dev/{char,block} to lookup sysfs path by major:minor
From: Dan Williams <[email protected]>
Why?:
There are occasions where userspace would like to access sysfs attributes for a device but it may not know how sysfs has named the device or the path. For example what is the sysfs path for /dev/disk/by-id/ata-ST3160827AS_5MT004CK? With this change a call to stat(2) returns the major:minor then userspace can see that /sys/dev/block/8:32 links to /sys/block/sdc.
What are the alternatives?:
1/ Add an ioctl to return the path: Doable, but sysfs is meant to reduce
the need to proliferate ioctl interfaces into the kernel, so this seems counter productive.
2/ Use udev to create these symlinks: Also doable, but it adds a
udev dependency to utilities that might be running in a limited environment like an initramfs.
---
drivers/base/core.c | 37 ++++++++++++++++++++++++++++++++++++- 1 files changed, 36 insertions(+), 1 deletions(-)
drivers/base/core.c +36 -1
@@ -27,6 +27,9 @@int (*platform_notify)(struct device *dev) =NULL ;int (*platform_notify_remove)(struct device *dev) =NULL ; +static struct kobject *dev_kobj; +static struct kobject *char_kobj; +static struct kobject *block_kobj;#ifdef CONFIG_BLOCK static inline int device_is_not_partition (struct device *dev)
@@ -759,6 +762,11 @@ static void device_remove_class_symlinks(struct device *dev)sysfs_remove_link (&dev->kobj,"subsystem" ); } +static struct kobject *device_to_dev_kobj (struct device *dev) +{ +return dev->class == &block_class ? block_kobj : char_kobj; +} +/** * device_add - add device to device hierarchy. * @dev: device.
@@ -775,6 +783,7 @@ int device_add(struct device *dev)struct device *parent =NULL ;struct class_interface *class_intf;int error; +char devt_str[25 ]; dev =get_device (dev);if (!dev || !strlen (dev->bus_id)) {
@@ -806,9 +815,16 @@ int device_add(struct device *dev) goto attrError;if (MAJOR (dev->devt)) { +struct kobject *kobj =device_to_dev_kobj (dev); + error =device_create_file (dev, &devt_attr);if (error) goto ueventattrError; + +format_dev_t (devt_str, dev->devt); + error =sysfs_create_link (kobj, &dev->kobj, devt_str); +if (error) + goto devtattrError; } error =device_add_class_symlinks (dev);
@@ -854,6 +870,9 @@ int device_add(struct device *dev)device_remove_class_symlinks (dev); SymlinkError:if (MAJOR (dev->devt)) +sysfs_remove_link (device_to_dev_kobj (dev), devt_str); + devtattrError: +if (MAJOR (dev->devt))device_remove_file (dev, &devt_attr); ueventattrError:device_remove_file (dev, &uevent_attr);
@@ -925,12 +944,16 @@ void device_del(struct device *dev) {struct device *parent = dev->parent;struct class_interface *class_intf; +char devt_str[25 ];device_pm_remove (dev);if (parent)klist_del (&dev->knode_parent); -if (MAJOR (dev->devt)) +if (MAJOR (dev->devt)) { +format_dev_t (devt_str, dev->devt); +sysfs_remove_link (device_to_dev_kobj (dev), devt_str);device_remove_file (dev, &devt_attr); + }if (dev->class) {device_remove_class_symlinks (dev);
@@ -1055,6 +1078,15 @@ int __init devices_init(void) devices_kset =kset_create_and_add ("devices" , &device_uevent_ops,NULL );if (!devices_kset)return -ENOMEM ; + dev_kobj =kobject_create_and_add ("dev" ,NULL ); +if (!dev_kobj) +return -ENOMEM ; + block_kobj =kobject_create_and_add ("block" , dev_kobj); +if (!block_kobj) +return -ENOMEM ; + char_kobj =kobject_create_and_add ("char" , dev_kobj); +if (!char_kobj) +return -ENOMEM ;return 0 ; }
@@ -1380,4 +1412,7 @@ void device_shutdown(void) dev->driver->shutdown (dev); } } +kobject_put (char_kobj); +kobject_put (block_kobj); +kobject_put (dev_kobj); }
Quoted 9 lines · On Fri, 11 Apr 2008 16:25:32 -0700 "Dan Williams" <[email protected]>…
Quoted 3 lines
It doesn't really seem to be to belong under class at all. I would suggest
/sys/dev/char/and/sys/dev/block/, for char and block respectively.This thread fizzled out without a patch... here goes:
...
sysfs: add
/sys/dev/{char,block}to lookup sysfs path by major:minor
Crickets are chirping and I can't remember what the conclusion to all this was. In fact the thread was more than ten-deep so I probably fell asleep.
I queued it up so that others cannot do the same ;)
Quoted 5 lines · Andrew Morton
Quoted 3 lines · On Fri, 11 Apr 2008 16:25:32 -0700 "Dan Williams" <[email protected]>…
Quoted 3 lines
It doesn't really seem to be to belong under class at all. I would suggest
/sys/dev/char/and/sys/dev/block/, for char and block respectively.
Quoted 8 lines
Quoted 5 lines
This thread fizzled out without a patch... here goes:
...
sysfs: add
/sys/dev/{char,block}to lookup sysfs path by major:minorCrickets are chirping and I can't remember what the conclusion to all this was. In fact the thread was more than ten-deep so I probably fell asleep.
..
Last I recall, Greg was vehemently opposed to having direct path access by device number in sysfs, but many other people saw benefit.
Myself (the originator), I simply decided that my sysfs access code has to work with older kernels too, so for now I'm just doing a brute force tree search to find things in sysfs. I did get the code size down smaller for it, but it's still a pain.
When the direct access feature goes in, I'll just change my code to try it first, and then still fall back to the tree search method on failure.
Quoted 1 line
I queued it up so that others cannot do the same ;)
..
Good!
James Bottomley
maintainerQuoted 8 lines · Andrew Morton
Quoted 6 lines · On Fri, 11 Apr 2008 16:25:32 -0700 "Dan Williams" <[email protected]>…
Quoted 3 lines
It doesn't really seem to be to belong under class at all. I would suggest
/sys/dev/char/and/sys/dev/block/, for char and block respectively.This thread fizzled out without a patch... here goes:
Quoted 8 lines
Quoted 3 lines
...
sysfs: add
/sys/dev/{char,block}to lookup sysfs path by major:minorCrickets are chirping and I can't remember what the conclusion to all this was. In fact the thread was more than ten-deep so I probably fell asleep.
I queued it up so that others cannot do the same ;)
The expressed preference was simply to expand the ioctl (or add a new one that got the required information without having to go through the old HDIOGETGEO path to extract the value from a fictitious geometry).
Greg was a bit sceptical of the value of the above proposal ...
James
Quoted 7 lines · James Bottomley
The expressed preference was simply to expand the ioctl (or add a new one that got the required information without having to go through the old HDIOGETGEO path to extract the value from a fictitious geometry).
Greg was a bit sceptical of the value of the above proposal ...
However, you have to admit that kind of defeats the whole point of having this information in sysfs. IMNSHO, even scanning sysfs is better than keep adding binary ioctls.
-hpa
Subject: sysfs: add /sys/dev/{char,block} to lookup sysfs path by major:minor
From: Dan Williams <[email protected]>
Why?:
There are occasions where userspace would like to access sysfs attributes for a device but it may not know how sysfs has named the device or the path. For example what is the sysfs path for /dev/disk/by-id/ata-ST3160827AS_5MT004CK? With this change a call to stat(2) returns the major:minor then userspace can see that /sys/dev/block/8:32 links to /sys/block/sdc.
What are the alternatives?:
1/ Add an ioctl to return the path: Doable, but sysfs is meant to reduce
the need to proliferate ioctl interfaces into the kernel, so this seems counter productive.
2/ Use udev to create these symlinks: Also doable, but it adds a
udev dependency to utilities that might be running in a limited environment like an initramfs.
3/ Do a full-tree search of sysfs.
---
Andrew here is an updated patch with some presumptive acked-by's from Mark and hpa.
* fixed up ENOMEM handling in devices_init()
* added a short blurb in Documentation/filesystems/sysfs.txt
* dropped the size of the buffer passed to format_dev_t a bit
Documentation/filesystems/sysfs.txt | 6 +++++ drivers/base/core.c | 46 ++++++++++++++++++++++++++++++++++- 2 files changed, 51 insertions(+), 1 deletions(-)
Documentation/filesystems/sysfs.txt +6 -0
@@ -248,6 +248,7 @@ The top level sysfs directory looks like: block/ bus/ class/ +dev/ devices/ firmware/ net/
@@ -274,6 +275,11 @@ fs/ contains a directory for some filesystems. Currently each filesystem wanting to export attributes must create its own hierarchy below fs/ (see ./fuse.txt for an example). +dev/ contains two directories char/ and block/. Inside these two +directories there are symlinks named <major>:<minor>. These symlinks +point to the sysfs directory for the given device. /sys/dev provides a +quick way to lookup the sysfs interface for a device from the result of +a stat(2) operation. More information can driver-model specific features can be found in Documentation/driver-model/.
drivers/base/core.c +45 -1
101 lines not on this page · raw message
[email protected]
Appliedpatch sysfs-add-sys-dev-char-block-to-lookup-sysfs-path-by-major-minor.patch added to gregkh-2.6 tree
This is a note to let you know that I've just added the patch titled
Subject: sysfs: add /sys/dev/{char,block} to lookup sysfs path by major:minor
to my gregkh-2.6 tree. Its filename is
sysfs-add-sys-dev-char-block-to-lookup-sysfs-path-by-major-minor.patch
This tree can be found at
http://www.kernel.org/pub/linux/kernel/people/gregkh/gregkh-2.6/patches/
Quoted 1 line
From [email protected] Wed Apr 16 13:49:38 2008
From: Dan Williams <[email protected]>
Date: Tue, 15 Apr 2008 16:43:15 -0700
Subject: sysfs: add /sys/dev/{char,block} to lookup sysfs path by major:minor
To: Andrew Morton <[email protected]>
Cc: James Bottomley <[email protected]>, Kay Sievers <[email protected]>, Tejun Heo <[email protected]>, Mark Lord <[email protected]>, Greg KH <[email protected]>, Jens Axboe <[email protected]>, Jeff Garzik <[email protected]>, Linus Torvalds <[email protected]>, Linux Kernel <[email protected]>, IDE/ATA development list <[email protected]>, linux-scsi <[email protected]>, "H. Peter Anvin" <[email protected]>
Message-ID: <[email protected]>
From: Dan Williams <[email protected]>
Why?:
There are occasions where userspace would like to access sysfs attributes for a device but it may not know how sysfs has named the device or the path. For example what is the sysfs path for /dev/disk/by-id/ata-ST3160827AS_5MT004CK? With this change a call to stat(2) returns the major:minor then userspace can see that /sys/dev/block/8:32 links to /sys/block/sdc.
What are the alternatives?:
1/ Add an ioctl to return the path: Doable, but sysfs is meant to reduce
the need to proliferate ioctl interfaces into the kernel, so this seems counter productive.
2/ Use udev to create these symlinks: Also doable, but it adds a
udev dependency to utilities that might be running in a limited environment like an initramfs.
3/ Do a full-tree search of sysfs.
--- Documentation/filesystems/sysfs.txt | 6 ++++ drivers/base/core.c | 46 +++++++++++++++++++++++++++++++++++- 2 files changed, 51 insertions(+), 1 deletion(-)
Documentation/filesystems/sysfs.txt +6 -0
@@ -248,6 +248,7 @@ The top level sysfs directory looks like block/ bus/ class/ +dev/ devices/ firmware/ net/
@@ -274,6 +275,11 @@ fs/ contains a directory for some filesy filesystem wanting to export attributes must create its own hierarchy below fs/ (see ./fuse.txt for an example). +dev/ contains two directories char/ and block/. Inside these two +directories there are symlinks named <major>:<minor>. These symlinks +point to the sysfs directory for the given device. /sys/dev provides a +quick way to lookup the sysfs interface for a device from the result of +a stat(2) operation. More information can driver-model specific features can be found in Documentation/driver-model/.
drivers/base/core.c +45 -1
101 lines not on this page · raw message
Patches currently in gregkh-2.6 which might be from [email protected] are
driver-core/sysfs-refill-attribute-buffer-when-reading-from-offset-0.patch driver-core/sysfs-add-sys-dev-char-block-to-lookup-sysfs-path-by-major-minor.patch
[email protected]
Appliedpatch sysfs-add-sys-dev-char-block-to-lookup-sysfs-path-by-major-minor.patch added to gregkh-2.6 tree
In reply to Dan WilliamsThis is a note to let you know that I've just added the patch titled
Subject: sysfs: add /sys/dev/{char,block} to lookup sysfs path by major:minor
to my gregkh-2.6 tree. Its filename is
sysfs-add-sys-dev-char-block-to-lookup-sysfs-path-by-major-minor.patch
This tree can be found at
http://www.kernel.org/pub/linux/kernel/people/gregkh/gregkh-2.6/patches/
Quoted 1 line
From [email protected] Wed Apr 16 13:49:38 2008
From: Dan Williams <[email protected]>
Date: Tue, 15 Apr 2008 16:43:15 -0700
Subject: sysfs: add /sys/dev/{char,block} to lookup sysfs path by major:minor
To: Andrew Morton <[email protected]>
Cc: James Bottomley <[email protected]>, Kay Sievers <[email protected]>, Tejun Heo <[email protected]>, Mark Lord <[email protected]>, Greg KH <[email protected]>, Jens Axboe <[email protected]>, Jeff Garzik <[email protected]>, Linus Torvalds <[email protected]>, Linux Kernel <[email protected]>, IDE/ATA development list <[email protected]>, linux-scsi <[email protected]>, "H. Peter Anvin" <[email protected]>
Message-ID: <[email protected]>
From: Dan Williams <[email protected]>
Why?:
There are occasions where userspace would like to access sysfs attributes for a device but it may not know how sysfs has named the device or the path. For example what is the sysfs path for /dev/disk/by-id/ata-ST3160827AS_5MT004CK? With this change a call to stat(2) returns the major:minor then userspace can see that /sys/dev/block/8:32 links to /sys/block/sdc.
What are the alternatives?:
1/ Add an ioctl to return the path: Doable, but sysfs is meant to reduce
the need to proliferate ioctl interfaces into the kernel, so this seems counter productive.
2/ Use udev to create these symlinks: Also doable, but it adds a
udev dependency to utilities that might be running in a limited environment like an initramfs.
3/ Do a full-tree search of sysfs.
--- Documentation/filesystems/sysfs.txt | 6 ++++ drivers/base/core.c | 46 +++++++++++++++++++++++++++++++++++- 2 files changed, 51 insertions(+), 1 deletion(-)
Documentation/filesystems/sysfs.txt +6 -0
@@ -248,6 +248,7 @@ The top level sysfs directory looks like block/ bus/ class/ +dev/ devices/ firmware/ net/
@@ -274,6 +275,11 @@ fs/ contains a directory for some filesy filesystem wanting to export attributes must create its own hierarchy below fs/ (see ./fuse.txt for an example). +dev/ contains two directories char/ and block/. Inside these two +directories there are symlinks named <major>:<minor>. These symlinks +point to the sysfs directory for the given device. /sys/dev provides a +quick way to lookup the sysfs interface for a device from the result of +a stat(2) operation. More information can driver-model specific features can be found in Documentation/driver-model/.
drivers/base/core.c +45 -1
101 lines not on this page · raw message
Patches currently in gregkh-2.6 which might be from [email protected] are
driver-core/sysfs-refill-attribute-buffer-when-reading-from-offset-0.patch driver-core/sysfs-add-sys-dev-char-block-to-lookup-sysfs-path-by-major-minor.patch
Quoted 14 lines · Mark Lord
Quoted 5 lines · Greg KH
Quoted 1 line · Mark Lord
Perhaps Greg will chime in.
I've been waiting to see if sanity will take hold of anyone here.
..
So have we. sysfs is a total nightmare to extract information from under program / script control. The idea presented in this thread, is to have it cross-index the contents with a method that actually makes it easy to access in many common scenarios, without requiring huge gobs of code in user space. Or in kernel space.
Quoted 3 lines
And it's not just a few 10s of lines of code currently, but rather about 80-100 lines just to find the correct device subdir, and *then* a few more 10s of lines of code to retrieve the value.
Hmm, 100 lines? What else do you need?
$ grep -l 8:3 /sys/class/block/*/dev /sys/class/block/sdc/dev
Kay
Quoted 10 lines · Kay Sievers
Quoted 4 lines
And it's not just a few 10s of lines of code currently, but rather about 80-100 lines just to find the correct device subdir, and *then* a few more 10s of lines of code to retrieve the value.
Hmm, 100 lines? What else do you need?
$ grep -l 8:3 /sys/class/block/*/dev /sys/class/block/sdc/dev
That's particularly funny, because your very own example gives the wrong result -- sdc is 8:32 not 8:3 (which is sdc3, which is also excluded by your search.)
Not to mention the fact that it is still O(n).
-hpa
Quoted 8 lines · H. Peter Anvin
Quoted 7 lines · Kay Sievers
Quoted 4 lines
And it's not just a few 10s of lines of code currently, but rather about 80-100 lines just to find the correct device subdir, and *then* a few more 10s of lines of code to retrieve the value.
Hmm, 100 lines? What else do you need?
Quoted 7 lines
Quoted 3 lines
$ grep -l 8:3 /sys/class/block/*/dev /sys/class/block/sdc/devThat's particularly funny, because your very own example gives the wrong result -- sdc is 8:32 not 8:3 (which is sdc3, which is also excluded by your search.)
Very true, but I guess you get the idea, and know how to add the proper string match to grep. :)
Quoted 1 line
Not to mention the fact that it is still O(n).
Any real numbers from a large setup, which show that we want to have a reverse devnum map in sysfs?
Thanks,
Kay
James Bottomley
maintainerQuoted 22 lines · Mark Lord
(resending .. forgot to copy the lists originally)
We have a problem coming down the pipeline.
Practically all utilities that care about it, use
ioctl(fd, HDIO_GETGEO)to determine the starting sector offset of a hard disk partition.SCSI, libata, IDE, USB, Firewire.. you name it.
The return value uses "unsigned long",
which on a 32-bit system limits drive offsets to 2TB.There will be single drives exceeding this limit within the next 12 months or less, and we already have RAID arrays that exceed 2TB.
So.. what's the replacement for
HDIO_GETGEOon 32-bits ?One candidate might seem to be the existing
/sys/block/dev/partition/startwhich I expect is already 64-bit friendly.
Quoted 6 lines
But this requires about 150 lines of somewhat complex C code to access, using only the
dev_t(from stat(2) on a file) as a starting point, or less if one relies upon the udev device name matching the sysfs device name.Is it time now for
HDIO_GETGEO64to make an appearance? Similar to how the existing BLKGETSIZE64 is supplanting BLKGETSIZE ?
Perhaps I've missed something, but surely geometry doesn't make sense on a >2TB drive does it? The only reason we use it on modern disks (which usually make it up specially for us) is that the DOS partition scheme requires it. Once we're over 2TB, isn't it impossible to use DOS partitions (well, OK, unless you increase the sector size, but that's only delaying the inevitable), so we can just go with a proper disk labelling scheme and use BLKGETSIZE64 all the time.
James
Quoted 5 lines · James Bottomley
On Tue, 2008-03-25 at 00:02 -0400, Mark Lord wrote:
..Quoted 3 lines
Practically all utilities that care about it, use
ioctl(fd, HDIO_GETGEO)to determine the starting sector offset of a hard disk partition.
..
Quoted 7 lines
Perhaps I've missed something, but surely geometry doesn't make sense on a >2TB drive does it? The only reason we use it on modern disks (which usually make it up specially for us) is that the DOS partition scheme requires it. Once we're over 2TB, isn't it impossible to use DOS partitions (well, OK, unless you increase the sector size, but that's only delaying the inevitable), so we can just go with a proper disk labelling scheme and use BLKGETSIZE64 all the time.
..
I haven't thought much about problems with the virtual geometry, because, as you say, we really don't care about it for the most part. We use LBA values from the partition tables rather than CHS. I suppose those also likely to be 32-bit limited.
The "partition offset", or "starting sector" is the important bit of information for most things. And that's currently available from HDIO_GETGEO, and from /sys/block/XXX/XXXn/start, if sysfs is mounted.
We just need an easy way to get it, given a dev_t from stat(2). Currently there isn't an easy way, and HDIO_GETGEO returns only 32-bits on a 32-bit system.
Cheers
James Bottomley
maintainerQuoted 21 lines · Mark Lord
Quoted 5 lines · James Bottomley
On Tue, 2008-03-25 at 00:02 -0400, Mark Lord wrote:
..Quoted 3 lines
Practically all utilities that care about it, use
ioctl(fd, HDIO_GETGEO)to determine the starting sector offset of a hard disk partition...
Quoted 7 lines
Perhaps I've missed something, but surely geometry doesn't make sense on a >2TB drive does it? The only reason we use it on modern disks (which usually make it up specially for us) is that the DOS partition scheme requires it. Once we're over 2TB, isn't it impossible to use DOS partitions (well, OK, unless you increase the sector size, but that's only delaying the inevitable), so we can just go with a proper disk labelling scheme and use BLKGETSIZE64 all the time.
..
I haven't thought much about problems with the virtual geometry, because, as you say, we really don't care about it for the most part. We use LBA values from the partition tables rather than CHS. I suppose those also likely to be 32-bit limited.
Quoted 7 lines
The "partition offset", or "starting sector" is the important bit of information for most things. And that's currently available from
HDIO_GETGEO, and from/sys/block/XXX/XXXn/start, if sysfs is mounted.We just need an easy way to get it, given a
dev_tfrom stat(2). Currently there isn't an easy way, andHDIO_GETGEOreturns only 32-bits on a 32-bit system.
But I think where this is leading is that you've been using the geometry call, but all you really want to know is the actual partition start in sector units, so a new BLKGETPARTSTART (or something) ioctl that was designed to return a u64 would work for you? That sounds reasonable to me; so not a HDIO_GETGEO64 which gets us into trouble with geometries, but a simple ioctl that gives you exactly what you're looking for.
James
Quoted 30 lines · James Bottomley
Quoted 22 lines · Mark Lord
(resending .. forgot to copy the lists originally)
We have a problem coming down the pipeline.
Practically all utilities that care about it, use
ioctl(fd, HDIO_GETGEO)to determine the starting sector offset of a hard disk partition.SCSI, libata, IDE, USB, Firewire.. you name it.
The return value uses "unsigned long",
which on a 32-bit system limits drive offsets to 2TB.There will be single drives exceeding this limit within the next 12 months or less, and we already have RAID arrays that exceed 2TB.
So.. what's the replacement for
HDIO_GETGEOon 32-bits ?One candidate might seem to be the existing
/sys/block/dev/partition/startwhich I expect is already 64-bit friendly.Quoted 6 lines
But this requires about 150 lines of somewhat complex C code to access, using only the
dev_t(from stat(2) on a file) as a starting point, or less if one relies upon the udev device name matching the sysfs device name.Is it time now for
HDIO_GETGEO64to make an appearance? Similar to how the existing BLKGETSIZE64 is supplanting BLKGETSIZE ?
Quoted 8 lines
Perhaps I've missed something, but surely geometry doesn't make sense on a >2TB drive does it? The only reason we use it on modern disks (which usually make it up specially for us) is that the DOS partition scheme requires it. Once we're over 2TB, isn't it impossible to use DOS partitions (well, OK, unless you increase the sector size, but that's only delaying the inevitable), so we can just go with a proper disk labelling scheme and use BLKGETSIZE64 all the time.
I believe GUID Partition Tables (GPTs) are the answer.
I believe one of the features of GPT is the elimination of the 32-bit sector restrictions.
http://en.wikipedia.org/wiki/GUID_Partition_Table
Windows VISTA 64-bit supports GPTs on data disks and new Mac OS based systems have been using it on internal drives for a couple years at least.
GPTs are part of the Extensible Firmware Interface (EFI), so they should be usable for PC bootable disks at some point. (Maybe now in some cases?)
I'm not sure what the Linux Kernel support is for GPTs.
Greg
Signature · 9 lines
Greg Freemyer Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer First 99 Days Litigation White Paper - http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf The Norcross Group The Intersection of Evidence & Technology http://www.norcrossgroup.com
Quoted 52 lines · On Tue, 25 Mar 2008 13:45:35 -0400 Greg Freemyer wrote:
Quoted 30 lines · James Bottomley
Quoted 22 lines · Mark Lord
(resending .. forgot to copy the lists originally)
We have a problem coming down the pipeline.
Practically all utilities that care about it, use
ioctl(fd, HDIO_GETGEO)to determine the starting sector offset of a hard disk partition.SCSI, libata, IDE, USB, Firewire.. you name it.
The return value uses "unsigned long",
which on a 32-bit system limits drive offsets to 2TB.There will be single drives exceeding this limit within the next 12 months or less, and we already have RAID arrays that exceed 2TB.
So.. what's the replacement for
HDIO_GETGEOon 32-bits ?One candidate might seem to be the existing
/sys/block/dev/partition/startwhich I expect is already 64-bit friendly.Quoted 6 lines
But this requires about 150 lines of somewhat complex C code to access, using only the
dev_t(from stat(2) on a file) as a starting point, or less if one relies upon the udev device name matching the sysfs device name.Is it time now for
HDIO_GETGEO64to make an appearance? Similar to how the existing BLKGETSIZE64 is supplanting BLKGETSIZE ?Quoted 8 lines
Perhaps I've missed something, but surely geometry doesn't make sense on a >2TB drive does it? The only reason we use it on modern disks (which usually make it up specially for us) is that the DOS partition scheme requires it. Once we're over 2TB, isn't it impossible to use DOS partitions (well, OK, unless you increase the sector size, but that's only delaying the inevitable), so we can just go with a proper disk labelling scheme and use BLKGETSIZE64 all the time.
I believe GUID Partition Tables (GPTs) are the answer.
I believe one of the features of GPT is the elimination of the 32-bit sector restrictions.
http://en.wikipedia.org/wiki/GUID_Partition_Table
Windows VISTA 64-bit supports GPTs on data disks and new Mac OS based systems have been using it on internal drives for a couple years at least.
Quoted 5 lines
GPTs are part of the Extensible Firmware Interface (EFI), so they should be usable for PC bootable disks at some point. (Maybe now in some cases?)
I'm not sure what the Linux Kernel support is for GPTs.
It's implemented. Not sure about how well used/tested it is.
config EFI_PARTITION
bool "EFI GUID Partition support"
depends on PARTITION_ADVANCED
select CRC32
help
Say Y here if you would like to use hard disks under Linux which were partitioned using EFI GPT.
---
~Randy
Quoted 3 lines · Randy Dunlap
Quoted 1 line
I'm not sure what the Linux Kernel support is for GPTs.
It's implemented. Not sure about how well used/tested it is.
ia64 uses it exclusively ... at least on discs that you want to use from EFI.
Signature · 4 lines
Intel are signing my paycheques ... these opinions are still mine "Bill, look, we understand that you're interested in selling us this operating system, but compare it to ours. We can't possibly take such a retrograde step."
Quoted 7 lines · Matthew Wilcox
Quoted 2 lines · Randy Dunlap
Quoted 1 line
I'm not sure what the Linux Kernel support is for GPTs.
It's implemented. Not sure about how well used/tested it is.
ia64 uses it exclusively ... at least on discs that you want to use from EFI.
I thinks intel-Macs do too.
Boaz
Quoted 11 lines · Greg Freemyer
I believe GUID Partition Tables (GPTs) are the answer.
I believe one of the features of GPT is the elimination of the 32-bit sector restrictions.
http://en.wikipedia.org/wiki/GUID_Partition_Table
Windows VISTA 64-bit supports GPTs on data disks and new Mac OS based systems have been using it on internal drives for a couple years at least.
Quoted 5 lines
GPTs are part of the Extensible Firmware Interface (EFI), so they should be usable for PC bootable disks at some point. (Maybe now in some cases?)
I'm not sure what the Linux Kernel support is for GPTs.
It has been supported since the first Itanium systems shipped. It's the first code I wrote 7+ years before it was really needed. :-) Most distributions have it enabled, as do userspace tools like GNU Parted.
Signature · 3 lines
Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 10 lines · Linus Torvalds
Quoted 2 lines · Mark Lord
The return value uses "unsigned long",
which on a 32-bit system limits drive offsets to 2TB.One relevant question is: does anybody seriously care about the combination of "32 bit" and "huge modern drives" any more?
Sure, we can add a 64-bit version that ends up being used only on 32-bit systems, but quite frankly, I think the solution here is to just ignore the issue and see if anybody really even cares.
Because quite frankly, the kind of people who buy modern 2TB drives generally don't then couple them to CPU's that are five+ years old.
..
Yeah. Except Dell will undoubtedly have them in desktops
within 2 years, and tons of people (myself included) still use
32-bit (K)Ubuntu on our systems, simply for the better binary compatibility that it is perceived to give with things like browser plugins and stuff.
Using sysfs interfaces might be a good alternative, if they were easier to use, but drives are not directly accessible there using the dev_t value from stat(2).
Instead, software has to search everything inside /sys/block/ looking for a "dev" file whose contents match, rather than just trying to access something like this:
/sys/block/8:1/start
or
/sys/block/majors/8/minors/1/start
Or any one of a number of similar ways to arrange it.
Cheers
Quoted 10 lines · Mark Lord
Quoted 8 lines · Linus Torvalds
Quoted 2 lines · Mark Lord
The return value uses "unsigned long",
which on a 32-bit system limits drive offsets to 2TB.One relevant question is: does anybody seriously care about the combination of "32 bit" and "huge modern drives" any more?
Quoted 6 lines
Quoted 6 lines
Sure, we can add a 64-bit version that ends up being used only on 32-bit systems, but quite frankly, I think the solution here is to just ignore the issue and see if anybody really even cares.
Because quite frankly, the kind of people who buy modern 2TB drives generally don't then couple them to CPU's that are five+ years old.
We provide data services to our clients. We are already seeing USB enclosures routinely provided to us by our clients with 1TB. 1.5TB on occasion. 2TB usb enclosures can't be far behind.
For usb a bigger factor than anything is when will MS offer compatibility/supportfor 2TB+ drives. As soon as they become readily supported in MS, our clients will start buying them and filling them up and we will need to be able to access them from all of our systems. (old and new).
Greg
Signature · 9 lines
Greg Freemyer Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer First 99 Days Litigation White Paper - http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf The Norcross Group The Intersection of Evidence & Technology http://www.norcrossgroup.com
Quoted 10 lines · Mark Lord
Quoted 9 lines · Linus Torvalds
Quoted 2 lines · Mark Lord
The return value uses "unsigned long",
which on a 32-bit system limits drive offsets to 2TB.One relevant question is: does anybody seriously care about the combination of "32 bit" and "huge modern drives" any more?
Sure, we can add a 64-bit version that ends up being used only on 32-bit systems, but quite frankly, I think the solution here is to just ignore the issue and see if anybody really even cares.
Quoted 7 lines
Quoted 1 line
Because quite frankly, the kind of people who buy modern 2TB drives generally don't then couple them to CPU's that are five+ years old.
..
Yeah. Except Dell will undoubtedly have them in desktops
within 2 years, and tons of people (myself included) still use
32-bit (K)Ubuntu on our systems, simply for the better binary compatibility that it is perceived to give with things like browser plugins and stuff.
I think that there are many embedded applications (lots of them linux based) which have large amounts of storage behind low power, low cost 32 bit CPU's.
Think of the home/small office NAS boxes that you can get from bestbuy or other big box stores. Those devices today have 4 S-ATA drives (each of which can be 1TB in size).
Also, if you have a very low end box, it can still access really large storage over iSCSI or a SAN which will present as a local, large device.
Over time, even these low end CPU's will migrate towards 64 bits, but we are not there yet...
ric
Quoted 6 lines · Ric Wheeler
Quoted 5 lines
Yeah. Except Dell will undoubtedly have them in desktops within 2 years, and tons of people (myself included) still use 32-bit (K)Ubuntu on our systems, simply for the better binary compatibility that it is perceived to give with things like browser plugins and stuff.
Quoted 6 lines
I think that there are many embedded applications (lots of them linux based) which have large amounts of storage behind low power, low cost 32 bit CPU's.
Think of the home/small office NAS boxes that you can get from bestbuy or other big box stores. Those devices today have 4 S-ATA drives (each of which can be 1TB in size).
I can attest to this. I hear from a reliable source (manufacturer) that 2TB 3.5" disks will be out no later than first half of 2009 (possibly even sooner, or at least 1.5TB). I currently use 1TB disks with a Geode LX based motherboard for SATA RAID and I plan on upgrading/consolidating to larger sizes once they become available in the market. 64-bit is not an option for me on this hardware.
-Andrew
Quoted 10 lines · Ric Wheeler
I think that there are many embedded applications (lots of them linux based) which have large amounts of storage behind low power, low cost 32 bit CPU's.
Think of the home/small office NAS boxes that you can get from bestbuy or other big box stores. Those devices today have 4 S-ATA drives (each of which can be 1TB in size).
Also, if you have a very low end box, it can still access really large storage
over iSCSI or a SAN which will present as a local, large device.
Don't those devices run into trouble with fsck? The amount of memory you need to fsck a device is obviously going to depend on the filesystem, but it has to grow with device size, and I'm not sure that 4GB is enough virtual address space to fsck 2TB.
Signature · 4 lines
Intel are signing my paycheques ... these opinions are still mine "Bill, look, we understand that you're interested in selling us this operating system, but compare it to ours. We can't possibly take such a retrograde step."
Quoted 6 lines · Matthew Wilcox
Quoted 5 lines · Ric Wheeler
I think that there are many embedded applications (lots of them linux based) which have large amounts of storage behind low power, low cost 32 bit CPU's.
Think of the home/small office NAS boxes that you can get from bestbuy or other big box stores. Those devices today have 4 S-ATA drives (each of which can be 1TB in size).
Quoted 7 lines
Quoted 2 lines
Also, if you have a very low end box, it can still access really large storage over iSCSI or a SAN which will present as a local, large device.
Don't those devices run into trouble with fsck? The amount of memory you need to fsck a device is obviously going to depend on the filesystem, but it has to grow with device size, and I'm not sure that 4GB is enough virtual address space to fsck 2TB.
Absolutely - they more or less hit a stonewall once the disk has any trouble and you need to fsck. On the other hand, this might be merciful since on 64 bit boxes, we will let you run the fsck and watch it run for a week or so before you despair ;-)
On a serious note, fsck time tends to track more the number of active inodes, so you can fsck a large file system if you use it to store large files (especially if you use a file system with dynamic inode creation or something like the uninitialized ext4 inodes).
ric
Quoted 4 lines · Ric Wheeler
Quoted 4 lines
Don't those devices run into trouble with fsck? The amount of memory you need to fsck a device is obviously going to depend on the filesystem, but it has to grow with device size, and I'm not sure that 4GB is enough virtual address space to fsck 2TB.
Well 2TB, assuming a 4k blocksize, means a block bitmap is 512 megs. So at least for ext3, 4GB should be just enough, unless you hit certainly really nasty complicated corruptions (i.e. large number of blocks claimed by more than one inode, which can happen if an inode table is written to the wrong location on disk --- on top of some other portion of the inode table), or if the filesystem has a large number of files with hard links (such as the case with certain backup programs).
The plan is to implement some kind of run-length encoding to compress the in-memory requirements for storing the bitmaps, but that hasn't been coded yet. If someone is a staff programmer for one of these bookshelf NAS manufacturers is interested in implementing such a beast, they should talk to me; I've thought quite a bit about the design, and I just need a minion to implement it. :-)
Quoted 9 lines
Absolutely - they more or less hit a stonewall once the disk has any trouble and you need to fsck. On the other hand, this might be merciful since on 64 bit boxes, we will let you run the fsck and watch it run for a week or so before you despair ;-)
On a serious note, fsck time tends to track more the number of active inodes, so you can fsck a large file system if you use it to store large files (especially if you use a file system with dynamic inode creation or something like the uninitialized ext4 inodes).
And ext4 extents will help because it reduces the number of indirect blocks you have to read, which will significantly reduce the fsck time. So there will be improvements on the horizon.
- Ted
Quoted 9 lines · Theodore Tso
Well 2TB, assuming a 4k blocksize, means a block bitmap is 512 megs. So at least for ext3, 4GB should be just enough, unless you hit certainly really nasty complicated corruptions (i.e. large number of blocks claimed by more than one inode, which can happen if an inode table is written to the wrong location on disk --- on top of some other portion of the inode table), or if the filesystem has a large number of files with hard links (such as the case with certain backup programs).
Whoops, screwed up my math. The block bitmap for a 2TB filesystem is 64 megs, not 512 megs. 2*41 / 2**12 / 2**3 == 2**26, or 64mb. E2fsck in the worst case will allocate 5 inode bitmaps and 3 block bitmaps, plus various arrays for directory blocks and keeping track of refcounts (which are optimized for counnts of 0 and 1, so lots of hard links will blow up your memory usage, although we do have a tdb option which helps in that particular case). So I'd say that most of the time 3GB of address space should really be enough for a 2TB raid array, unless you get really pathalogical corruption cases.
- Ted
Quoted 9 lines · Theodore Tso
Well 2TB, assuming a 4k blocksize, means a block bitmap is 512 megs. So at least for ext3, 4GB should be just enough, unless you hit certainly really nasty complicated corruptions (i.e. large number of blocks claimed by more than one inode, which can happen if an inode table is written to the wrong location on disk --- on top of some other portion of the inode table), or if the filesystem has a large number of files with hard links (such as the case with certain backup programs).
Whoops, screwed up my math. The block bitmap for a 2TB filesystem is 64 megs, not 512 megs. 2*41 / 2**12 / 2**3 == 2**26, or 64mb. E2fsck in the worst case will allocate 5 inode bitmaps and 3 block bitmaps, plus various arrays for directory blocks and keeping track of refcounts (which are optimized for counnts of 0 and 1, so lots of hard links will blow up your memory usage, although we do have a tdb option which helps in that particular case). So I'd say that most of the time 3GB of address space should really be enough for a 2TB raid array, unless you get really pathalogical corruption cases.
- Ted