Dynamic device.map
RFC 1/1 Dynamic device.map
As Robert said recently, we're trying to get rid of our reliance on device.map. Right now, it is still necessary to at least have entries in device.map for any disks you wish to use with GRUB, although they don't have to be particularly sensible; any bidirectional mapping will do (at least as long as the 'search' command works).
I'd like to make it possible to run with no device.map at all, since that makes it possible to plug in new disks without having to reconfigure GRUB. Here's a draft patch to do this. I haven't supplied a ChangeLog entry yet since it isn't suitable for commit yet, as noted in TODO comments, but I'd welcome comments on the approach here.
This does slow down (e.g.) grub-probe a little: on my laptop with a single hard disk, the median time for three runs of './grub-probe --target=drive /' is 0.09 seconds, while the median time for three runs of './grub-probe --target=drive -m /boot/grub/device.map.invalid /' is 0.18 seconds. I expect it would be worse on machines with many disks. For the most part, I doubt this is a significant problem (certainly not in e.g. grub-setup), but it will probably make a difference in scripts that call grub-probe many times, such as grub-mkconfig. I suggest that those should check whether the default device.map exists, and, if not, send the output of grub-mkdevicemap to a temporary file and use that as an ephemeral device.map cache.
=== modified file 'conf/any-emu.rmk'
any-emu.rmk +1 -1
@@ -40,7 +40,7 @@ grub_emu_SOURCES = commands/minicmd.c co fs/befs.c fs/befs_be.c fs/tar.c \ \ util/console.c util/hostfs.c util/grub-emu.c util/misc.c \ - util/hostdisk.c util/getroot.c \ + util/hostdisk.c util/getroot.c util/deviceiter.c \ \ disk/raid.c disk/raid5_recover.c disk/raid6_recover.c \ disk/mdraid_linux.c disk/dmraid_nvidia.c disk/lvm.c \
=== modified file 'conf/common.rmk'
common.rmk +1 -0
@@ -17,6 +17,7 @@ sbin_UTILITIES += grub-probe util/grub-probe.c_DEPENDENCIES = grub_probe_init.h grub_probe_SOURCES = gnulib/progname.c util/grub-probe.c \ util/hostdisk.c util/misc.c util/getroot.c \ + util/deviceiter.c \ kern/device.c kern/disk.c kern/err.c kern/misc.c \ kern/parser.c kern/partition.c kern/file.c \ \
=== modified file 'conf/i386-pc.rmk'
i386-pc.rmk +1 -0
@@ -93,6 +93,7 @@ util/i386/pc/grub-mkimage.c_DEPENDENCIES util/i386/pc/grub-setup.c_DEPENDENCIES = grub_setup_init.h grub_setup_SOURCES = gnulib/progname.c \ util/i386/pc/grub-setup.c util/hostdisk.c \ + util/deviceiter.c \ util/misc.c util/getroot.c kern/device.c kern/disk.c \ kern/err.c kern/misc.c kern/parser.c kern/partition.c \ kern/file.c kern/fs.c kern/env.c fs/fshelp.c \
=== modified file 'conf/sparc64-ieee1275.rmk'
sparc64-ieee1275.rmk +1 -0
@@ -68,6 +68,7 @@ grub_mkimage_SOURCES = util/sparc64/ieee # For grub-setup. util/sparc64/ieee1275/grub-setup.c_DEPENDENCIES = grub_setup_init.h grub_setup_SOURCES = util/sparc64/ieee1275/grub-setup.c util/hostdisk.c \ + util/deviceiter.c \ util/misc.c util/getroot.c kern/device.c kern/disk.c \ kern/err.c kern/misc.c kern/parser.c kern/partition.c \ kern/file.c kern/fs.c kern/env.c fs/fshelp.c \
=== modified file 'util/hostdisk.c'
hostdisk.c +55 -0
73 lines not on this page · raw message
Thanks,
Signature · 1 line
Colin Watson [[email protected]]
Quoted 11 lines · Colin Watson
As Robert said recently, we're trying to get rid of our reliance on device.map. Right now, it is still necessary to at least have entries in device.map for any disks you wish to use with GRUB, although they don't have to be particularly sensible; any bidirectional mapping will do (at least as long as the 'search' command works).
I'd like to make it possible to run with no device.map at all, since that makes it possible to plug in new disks without having to reconfigure GRUB. Here's a draft patch to do this. I haven't supplied a ChangeLog entry yet since it isn't suitable for commit yet, as noted in TODO comments, but I'd welcome comments on the approach here.
Ah, I hadn't noticed r1870, sorry. Am I right in believing that my patch is obsolete, then?
Signature · 1 line
Colin Watson [[email protected]]
Quoted 7 lines · Colin Watson
Quoted 6 lines · Colin Watson
As Robert said recently, we're trying to get rid of our reliance on device.map. Right now, it is still necessary to at least have entries in device.map for any disks you wish to use with GRUB, although they don't have to be particularly sensible; any bidirectional mapping will do (at least as long as the 'search' command works).
Quoted 8 lines
Quoted 5 lines
I'd like to make it possible to run with no device.map at all, since that makes it possible to plug in new disks without having to reconfigure GRUB. Here's a draft patch to do this. I haven't supplied a ChangeLog entry yet since it isn't suitable for commit yet, as noted in TODO comments, but I'd welcome comments on the approach here.
Ah, I hadn't noticed r1870, sorry. Am I right in believing that my patch is obsolete, then?
I'm not completely sure, but it seems so. Sorry about that :-(
Anyhow, if you find more issues with systems that have no device.map, help is welcome fixing those of course.
Signature · 5 lines
Robert Millan The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and how) you may access your data; but nobody's threatening your freedom: we still allow you to remove your data and not access it at all."
Quoted 5 lines · Robert Millan
Quoted 2 lines · Colin Watson
Ah, I hadn't noticed r1870, sorry. Am I right in believing that my patch is obsolete, then?
I'm not completely sure, but it seems so. Sorry about that :-(
Don't apologise for doing something before I got round to it! :-)
Quoted 2 lines
Anyhow, if you find more issues with systems that have no device.map, help is welcome fixing those of course.
I'm trying to figure out how to make Debian's grub-installer operate without a device.map; it has various legacy bits and pieces that need conversion, and I'm working on these.
Along the way, though, I noticed that grub-install still unconditionally creates a device.map. This seems likely to become confusing if devices are changing around a lot, since it's never updated. I'd like to get to the point where it doesn't do this by default. How about we add support for an option to disable this, and make grub-installer and the Debian maintainer scripts pass it once they're ready? Some time later, we can flip the default value.
Signature · 1 line
Colin Watson [[email protected]]
Quoted 5 lines · Colin Watson
I'm trying to figure out how to make Debian's grub-installer operate without a device.map; it has various legacy bits and pieces that need conversion, and I'm working on these.
Quoted 7 lines
Along the way, though, I noticed that grub-install still unconditionally creates a device.map. This seems likely to become confusing if devices are changing around a lot, since it's never updated. I'd like to get to the point where it doesn't do this by default. How about we add support for an option to disable this, and make grub-installer and the Debian maintainer scripts pass it once they're ready? Some time later, we can flip the default value.
Why is this option necessary? If we removed automated generation of device.map, user can still call grub-mkdevicemap manually.
But first we'd need to figure out what we do with the "set root=xxx" backward compatibility hack. Has it been a while long enough that we can remove support for GRUB installs that didn't come with UUID support?
If we don't do anything on grub-mkconfig side and remove device.map, we'll end up with things like "set root=(/dev/sda1)" which is not just useless but also utterly confusing.
Signature · 5 lines
Robert Millan The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and how) you may access your data; but nobody's threatening your freedom: we still allow you to remove your data and not access it at all."
Quoted 13 lines · Robert Millan
Quoted 11 lines · Colin Watson
I'm trying to figure out how to make Debian's grub-installer operate without a device.map; it has various legacy bits and pieces that need conversion, and I'm working on these.
Along the way, though, I noticed that grub-install still unconditionally creates a device.map. This seems likely to become confusing if devices are changing around a lot, since it's never updated. I'd like to get to the point where it doesn't do this by default. How about we add support for an option to disable this, and make grub-installer and the Debian maintainer scripts pass it once they're ready? Some time later, we can flip the default value.
Quoted 2 lines
Why is this option necessary? If we removed automated generation of device.map, user can still call grub-mkdevicemap manually.
Certainly, but I would like to be able to deliver a system that does not generate a device.map by default.
Quoted 8 lines
But first we'd need to figure out what we do with the "set root=xxx" backward compatibility hack. Has it been a while long enough that we can remove support for GRUB installs that didn't come with UUID support?
If we don't do anything on grub-mkconfig side and remove device.map, we'll end up with things like "set root=(/dev/sda1)" which is not just useless but also utterly confusing.
No, I don't think we will. grub-probe is perfectly capable of mapping /dev/sda1 to (hd0,1) even without device.map; it's just that that mapping will be solely dependent on device iteration order at the time you run grub-mkconfig. (As opposed to whatever device iteration order happened to be the first time you installed GRUB, for the nearly-everyone who never edited device.map by hand ...)
Signature · 1 line
Colin Watson [[email protected]]
Quoted 14 lines · Colin Watson
Quoted 13 lines · Robert Millan
Quoted 11 lines · Colin Watson
I'm trying to figure out how to make Debian's grub-installer operate without a device.map; it has various legacy bits and pieces that need conversion, and I'm working on these.
Along the way, though, I noticed that grub-install still unconditionally creates a device.map. This seems likely to become confusing if devices are changing around a lot, since it's never updated. I'd like to get to the point where it doesn't do this by default. How about we add support for an option to disable this, and make grub-installer and the Debian maintainer scripts pass it once they're ready? Some time later, we can flip the default value.
Quoted 5 lines
Quoted 2 lines
Why is this option necessary? If we removed automated generation of device.map, user can still call grub-mkdevicemap manually.
Certainly, but I would like to be able to deliver a system that does not generate a device.map by default.
We don't need a new option for that. We'd just not generate device.map at all, and provide grub-mkdevicemap for users who want to mess with it.
Quoted 2 lines
No, I don't think we will. grub-probe is perfectly capable of mapping
/dev/sda1to (hd0,1) even without device.map;
I wasn't very fond of this because it's BIOS-specific. But I guess we can live with it.
Signature · 3 lines
Robert Millan "Be the change you want to see in the world" -- Gandhi
Quoted 5 lines · Robert Millan
Quoted 2 lines
No, I don't think we will. grub-probe is perfectly capable of mapping
/dev/sda1to (hd0,1) even without device.map;I wasn't very fond of this because it's BIOS-specific. But I guess we can live with it.
On second thought, I'm really not convinced we need this kind of logic. It seems a lot like a workaround.
Just leave it with (/dev/foo). If you remove device.map generation from grub-install, you can remove the backward compatibility hack while at it. Then we rely only on UUIDs.
In the future, we can rely on the initial value of $root as well, but this would require some rework (disk / partition split).
Signature · 3 lines
Robert Millan "Be the change you want to see in the world" -- Gandhi
From: Robert Millan <[email protected]>
Date: Sat, 26 Dec 2009 14:25:51 +0100
Quoted 1 line
Then we rely only on UUIDs.
This is exactly what I suggested we should avoid.
On OpenFirmware we scan every device alias, %80 of them point to non-existing devices. The aliases are just an enumeration of the set of devices that could be behind a controller, not what has actually been detected by the firmware or anything like that.
So every scan for a UUID can eat up an enormous amount of time, poking devices that simply are not there.
Quoted 7 lines · David Miller
From: Robert Millan <[email protected]>
Date: Sat, 26 Dec 2009 14:25:51 +0100Quoted 1 line
Then we rely only on UUIDs.
This is exactly what I suggested we should avoid.
Quoted 8 lines
On OpenFirmware we scan every device alias, %80 of them point to non-existing devices. The aliases are just an enumeration of the set of devices that could be behind a controller, not what has actually been detected by the firmware or anything like that.
So every scan for a UUID can eat up an enormous amount of time, poking devices that simply are not there.
I understand. It's not a good idea to use UUIDs exclussively even
Quoted 2 lines · outside the context of OpenFirmware. This is why I also
In the future, we can rely on the initial value of $root as well, but this would require some rework (disk / partition split).
I'm fine with delaying removal of devices.map untill this replacement is ready.
Signature · 3 lines
Robert Millan "Be the change you want to see in the world" -- Gandhi
Quoted 5 lines · Robert Millan
Quoted 4 lines · Robert Millan
Quoted 2 lines · Colin Watson
No, I don't think we will. grub-probe is perfectly capable of mapping
/dev/sda1to (hd0,1) even without device.map;
Quoted 7 lines
Quoted 2 lines
I wasn't very fond of this because it's BIOS-specific. But I guess we can live with it.
On second thought, I'm really not convinced we need this kind of logic. It seems a lot like a workaround.
Just leave it with (
/dev/foo).
You mean literally with the parentheses? I don't understand, since /dev/ names will be unintelligible to GRUB when running outside an operating system.
Quoted 2 lines
If you remove device.map generation from grub-install, you can remove the backward compatibility hack while at it.
Which backward compatibility hack?
Quoted 2 lines
In the future, we can rely on the initial value of $root as well, but this would require some rework (disk / partition split).
Would you mind elaborating on this? I would like to get this sorted out and am willing to work on it.
Sorry to be stupid,
Signature · 1 line
Colin Watson [[email protected]]
Quoted 5 lines · Colin Watson
Quoted 1 line
Just leave it with (
/dev/foo).You mean literally with the parentheses? I don't understand, since
/dev/names will be unintelligible to GRUB when running outside an operating system.
Yes. This just means we'd have "set root=(/dev/foo)" statements in grub.cfg, but those are just meant as a backward compatibility hack for pre-UUID GRUB installs.
Which I guess addresses your next question:
Quoted 4 lines
Quoted 2 lines
If you remove device.map generation from grub-install, you can remove the backward compatibility hack while at it.
Which backward compatibility hack?
:-)
However, notice that after David Miller's remark about Sparc support, it's probably better to:
- Keep the backward compatibility hack.
- Not remove device.map on Sparc yet (untill a better solution is available)
Quoted 5 lines
Quoted 2 lines
In the future, we can rely on the initial value of $root as well, but this would require some rework (disk / partition split).
Would you mind elaborating on this? I would like to get this sorted out and am willing to work on it.
Uhm actually I'm not completely sure this would work, as multi-disk setups have multiple values for initial $root.
Better just ignore this part ;-)
Quoted 1 line
Sorry to be stupid,
Not at all!
Signature · 3 lines
Robert Millan "Be the change you want to see in the world" -- Gandhi
Quoted 9 lines · Robert Millan
Quoted 4 lines · Colin Watson
Quoted 1 line
Just leave it with (
/dev/foo).You mean literally with the parentheses? I don't understand, since
/dev/names will be unintelligible to GRUB when running outside an operating system.Yes. This just means we'd have "set root=(/dev/foo)" statements in grub.cfg, but those are just meant as a backward compatibility hack for pre-UUID GRUB installs.
Are you are implying that UUID will be the only way? I don't use initrd's on my systems so I need root=(hd0,x) or root=(/dev/foo). AFAIK initrd is the only way to load with UUIDs.
-- Bruce
Quoted 7 lines · Bruce Dubbs
Quoted 6 lines · Robert Millan
Quoted 4 lines · Colin Watson
Quoted 1 line
Just leave it with (
/dev/foo).You mean literally with the parentheses? I don't understand, since
/dev/names will be unintelligible to GRUB when running outside an operating system.
Quoted 7 lines
Quoted 3 lines
Yes. This just means we'd have "set root=(/dev/foo)" statements in grub.cfg, but those are just meant as a backward compatibility hack for pre-UUID GRUB installs.
Are you are implying that UUID will be the only way? I don't use initrd's on my systems so I need root=(hd0,x) or root=(/dev/foo). AFAIK initrd is the only way to load with UUIDs.
I think you're mixing up two different things. There are two 'root' variables involved:
1) GRUB's 'root' variable, its base for filesystem operations
2) The root= parameter passed to the Linux kernel, which identifies the desired root filesystem
Robert is talking about 1), but whether you use an initrd/initramfs is only relevant to 2).
Signature · 1 line
Colin Watson [[email protected]]
Quoted 16 lines · Colin Watson
Quoted 11 lines · Bruce Dubbs
Quoted 8 lines · Robert Millan
Quoted 4 lines · Colin Watson
Quoted 1 line
Just leave it with (
/dev/foo).You mean literally with the parentheses? I don't understand, since
/dev/names will be unintelligible to GRUB when running outside an operating system.Yes. This just means we'd have "set root=(/dev/foo)" statements in grub.cfg, but those are just meant as a backward compatibility hack for pre-UUID GRUB installs.
Are you are implying that UUID will be the only way? I don't use initrd's on my systems so I need root=(hd0,x) or root=(/dev/foo). AFAIK initrd is the only way to load with UUIDs.
I think you're mixing up two different things. There are two 'root' variables involved:
Quoted 6 lines
1) GRUB's 'root' variable, its base for filesystem operations
2) The root= parameter passed to the Linux kernel, which identifies the desired root filesystemRobert is talking about 1), but whether you use an initrd/initramfs is only relevant to 2).
OK, I didn't realize set root was capable of using UUIDs. I did know that the two root entries were different. I got that mixed up with the search command combined with the root=UUID=... which I think needs initrd.
Do I have it right now?
Should 'set root' be renamed to 'set grubroot'? I think something like that would prevent some confusion.
-- Bruce
Am Donnerstag, den 07.01.2010, 23:27 -0600 schrieb Bruce Dubbs:
Quoted 31 lines
Quoted 5 lines · Colin Watson
Quoted 4 lines · Bruce Dubbs
Quoted 3 lines · Robert Millan
Quoted 2 lines · Colin Watson
Quoted 1 line
Just leave it with (
/dev/foo).You mean literally with the parentheses? I don't understand,
since
/dev/Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
names will be unintelligible to GRUB when running outside an
operating
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
system.
Yes. This just means we'd have "set root=(/dev/foo)" statements
in grub.cfg,
Quoted 1 line
Quoted 1 line
Quoted 1 line
but those are just meant as a backward compatibility hack for
pre-UUID GRUB
Quoted 10 lines
Quoted 4 lines
Quoted 1 line
installs.
Are you are implying that UUID will be the only way? I don't use initrd's on my systems so I need root=(hd0,x) or root=(/dev/foo). AFAIK initrd is the only way to load with UUIDs.
I think you're mixing up two different things. There are two 'root' variables involved:
1) GRUB's 'root' variable, its base for filesystem operations
2) The root= parameter passed to the Linux kernel, whichidentifies
Quoted 3 lines
the desired root filesystem
Robert is talking about 1), but whether you use an initrd/initramfs
is
Quoted 1 line
only relevant to 2).
Quoted 7 lines
OK, I didn't realize set root was capable of using UUIDs. I did know that the two root entries were different. I got that mixed up with the
search command combined with the root=UUID=... which I think needs initrd.Do I have it right now?
set root only supports plain GRUB devices.
search --set sets $root (for GRUB, not for Linux) to the first device found with that UUID, LABEL or file. Depending on the other argument to search command.
Quoted 5 lines
Should 'set root' be renamed to 'set grubroot'? I think something like
that would prevent some confusion.-- Bruce
I don't get why there's a confusion at all.
It should be clear that the value specified with the linux command is for it and not for GRUB.
Signature · 2 lines
Felix Zielcke Proud Debian Maintainer and GNU GRUB developer
Quoted 13 lines · Felix Zielcke
Am Donnerstag, den 07.01.2010, 23:27 -0600 schrieb Bruce Dubbs:
Quoted 5 lines
OK, I didn't realize set root was capable of using UUIDs. I did know that the two root entries were different. I got that mixed up with the search command combined with the root=UUID=... which I think needs
initrd.Do I have it right now?
set root only supports plain GRUB devices.
search --set sets $root (for GRUB, not for Linux) to the first device found with that UUID, LABEL or file. Depending on the other argument to search command.
Quoted 6 lines
Quoted 2 lines
Should 'set root' be renamed to 'set grubroot'? I think something like that would prevent some confusion.
I don't get why there's a confusion at all.
It should be clear that the value specified with the linux command is for it and not for GRUB.
It's not confusing if you've been around GRUB for a while, but it is confusing for new users. The term 'root' is overloaded and only distinguished by context. The two contexts do have similar meanings so the differences are really fairly subtle.
-- Bruce
Am Donnerstag, den 10.12.2009, 01:55 +0100 schrieb Robert Millan:
Quoted 4 lines
But first we'd need to figure out what we do with the "set root=xxx" backward compatibility hack. Has it been a while long enough that we can remove support for GRUB installs that didn't come with UUID support?
Well we still have Arthur Marsh' bug open that search --fs-uuid fails with right UUID even though ls (hdx,y) shows it.
https://savannah.gnu.org/bugs/?26834
Also on Debian BTS and Launchpad.
The good thing though is that only a grub_errno = GRUB_ERR_NONE; needs to be commited to have this fixed.
Unfortunately he attached the full kern/device.c to the report not the patch Vladimir made for him.
Signature · 2 lines
Felix Zielcke Proud Debian Maintainer and GNU GRUB developer
Quoted 8 lines · Felix Zielcke
Am Donnerstag, den 10.12.2009, 01:55 +0100 schrieb Robert Millan:
Quoted 4 lines
But first we'd need to figure out what we do with the "set root=xxx" backward compatibility hack. Has it been a while long enough that we can remove support for GRUB installs that didn't come with UUID support?
Well we still have Arthur Marsh' bug open that search --fs-uuid fails with right UUID even though ls (hdx,y) shows it.
Uhm ISTR Vladimir fixed this one.
Signature · 3 lines
Robert Millan "Be the change you want to see in the world" -- Gandhi
Am Donnerstag, den 24.12.2009, 22:17 +0100 schrieb Robert Millan:
Quoted 8 lines · Felix Zielcke
Quoted 2 lines
Am Donnerstag, den 10.12.2009, 01:55 +0100 schrieb Robert Millan:
Quoted 1 line
But first we'd need to figure out what we do with the "set
root=xxx"
Quoted 4 lines
Quoted 3 lines
backward compatibility hack. Has it been a while long enough that we can remove support for GRUB installs that didn't come with UUID support?
Quoted 5 lines
Quoted 1 line
Well we still have Arthur Marsh' bug open that search --fs-uuid
fails
Quoted 1 line
with right UUID even though ls (hdx,y) shows it.
Uhm ISTR Vladimir fixed this one.
Yes in the meanwhile he commited the fix.
Signature · 2 lines
Felix Zielcke Proud Debian Maintainer and GNU GRUB developer