Environment block support for grub2

Hi,

This patch add environment block support, which can be used to locate root device using uuid or label.

The previous prefix is stored in variable rdir.

Different platform use grub_machine_get_envblk to return a pointer to the name=value area of environment block. In main.c, it uses grub_parse_envblk to walk through the list and assign values to variables.

I also replace grub_machine_set_prefix with grub_machine_set_root. As it seems strange to compute the prefix in grub_machine_set_prefix, and then split it in grub_set_root_dev to get the root device. Now, grub_machine_set_root set the root directly, and in grub_set_root_dev, it uses root and rdir to generate the prefix.

grub-mkimage will initialize the environment block with one item:
rdir=/boot/grub

Second, use grub-editenv to set the uuid or label of the root device. For example:

Quoted 10 lines

Different platform use grub_machine_get_envblk to return a pointer to the name=value area of environment block. In main.c, it uses grub_parse_envblk to walk through the list and assign values to variables.

I also replace grub_machine_set_prefix with grub_machine_set_root. As it seems strange to compute the prefix in grub_machine_set_prefix, and then split it in grub_set_root_dev to get the root device. Now, grub_machine_set_root set the root directly, and in grub_set_root_dev, it uses root and rdir to generate the prefix.

Note that root and prefix don't necessarily have to point at the same device.

prefix always points at the directory containing GRUB modules etc, and is needed at initial stage. root is used later on by grub.cfg to access our payload (I think our current use of root to initialise prefix is gratuitous and could better be avoided).

Since for initialising root we don't have any problem, as we can use all the fancy stuff grub.cfg allows, unless I missed something I think it's better if at this stage we just focus on prefix.

Quoted 5 lines

grub-mkimage will initialize the environment block with one item:
rdir=/boot/grub

Second, use grub-editenv to set the uuid or label of the root device. For example:

Would it be simpler if grub-mkimage embedded an initialised environment block directly?

The root is assigned in the initialization phraze. Currently, it's done this way:

machine_machine_init -> calculate root device, then set variable prefix using root and grub_prefix.

modules initialization

grub_set_root_dev -> Use variable prefix to get the root variable.

Load normal.mod

As you can see, it first create prefix, then split it to find root. At the end of grub_set_root_dev, prefix and root will point to the same device. This also make it different to change the root setting in findroot, which is run in modules initialization stage. We must split prefix to find the root directory, and use the new root to form a new prefix. The prefix would later be split again to find root, what a mess.

The new method:

machine_machine_init -> calculate root device, then set root variable directly.

Load variable in the environment block

modules initialization

findroot

more modules initialization

grub_set_root_dev -> Use variable root to get the root prefix.

Load normal.mod

The end result would be the same, but the new method is cleaner, and allows overwrite of the root and rdir variable using environment block and findroot.

I think the method is sound; what I'm complaining about (and it's not something specific to your patch) is that we're referring to two different things by the same name ("root"), and even put them in the same variable.

In the initialization phase, "root" is the device that contains our GRUB directory.

Afterwards, "root" is the device we're currently accessing (be it for loading fonts, backgrounds, Linux images, whatever).

If "root" means "just a placeholder for whatever device we're acessing at the moment", then it would make sense, but in our code (i.e. in the names we're giving to commands and functions) it's assumed to mean "the device containing /boot/grub".

So what do we want to do with this? Should we have different variables for each thing (and in that case, is "root" for initial stage or for grub.cfg) or should we use "root" as a placeholder for any path reference, and adjust our function names etc to reflect that?

I've been thinking a bit more about this, and I think it should be possible to solve the problem at hand without extending the existing framework. I'm concerned about finding a solution that is as simple (and small) as possible.

Do you mind waiting a few days? I'll send a proposal tomorrow or so.

Quoted 7 lines · Robert Millan

I've been thinking a bit more about this, and I think it should be possible to solve the problem at hand without extending the existing framework. I'm concerned about finding a solution that is as simple (and small) as possible.

Do you mind waiting a few days? I'll send a proposal tomorrow or so.

Ok, no hurry.

Sorry for the delay; it was quite a bit of work as I got hit by a number of bugs here and there :-)

Considering that the problem was quite similar to the "boot from LVM" issue we already knew about, I thought it'd be a good idea to start with that.

Although not directly related, the patch at in "boot from LVM / RAID" mail I just sent would allow to easily boot via UUID as well.

If we add a UUID-based disk driver (see "disk/fs_uuid.c" mail), then it's just a matter of increasing the grub_prefix size so that an UUID can fit in it, and adjusting the install scripts to do it.

Here's a sample (untested) patch to illustrate it.

Then for the Vista use case, I suppose the remaining questions are if NTFS supports UUIDs (or otherwise we have to ressort to labels?), and if we can extract filesystem metadata from Windows (if possible without requiring that a Cygwin installation is present).

Quoted 13 lines · Robert Millan

Sorry for the delay; it was quite a bit of work as I got hit by a number of bugs here and there :-)

Considering that the problem was quite similar to the "boot from LVM" issue we already knew about, I thought it'd be a good idea to start with that.

Although not directly related, the patch at in "boot from LVM / RAID" mail I just sent would allow to easily boot via UUID as well.

If we add a UUID-based disk driver (see "disk/fs_uuid.c" mail), then it's just a matter of increasing the grub_prefix size so that an UUID can fit in it, and adjusting the install scripts to do it.

Quoted 1 line

Here's a sample (untested) patch to illustrate it.

I take a quick look at your patch, the method should work, but i remember okuji oppose embedding device name in prefix, don't know how he look at it now.

Quoted 16 lines · Bean

Quoted 14 lines · Robert Millan

Sorry for the delay; it was quite a bit of work as I got hit by a number of bugs here and there :-)

Considering that the problem was quite similar to the "boot from LVM" issue we already knew about, I thought it'd be a good idea to start with that.

Although not directly related, the patch at in "boot from LVM / RAID" mail I just sent would allow to easily boot via UUID as well.

If we add a UUID-based disk driver (see "disk/fs_uuid.c" mail), then it's just a matter of increasing the grub_prefix size so that an UUID can fit in it, and adjusting the install scripts to do it.

Here's a sample (untested) patch to illustrate it.

Quoted 3 lines

I take a quick look at your patch, the method should work, but i remember okuji oppose embedding device name in prefix, don't know how he look at it now.

Do you remember why? (or where it was discussed)

Anyway, CCing him. Okuji, please say something! :-)

Hi,

I find the post:

http://lists.gnu.org/archive/html/grub-devel/2008-02/msg00542.html

Quoted 14 lines · Bean

Quoted 13 lines · Robert Millan

Sorry for the delay; it was quite a bit of work as I got hit by a number of bugs here and there :-)

Considering that the problem was quite similar to the "boot from LVM" issue we already knew about, I thought it'd be a good idea to start with that.

Although not directly related, the patch at in "boot from LVM / RAID" mail I just sent would allow to easily boot via UUID as well.

If we add a UUID-based disk driver (see "disk/fs_uuid.c" mail), then it's just a matter of increasing the grub_prefix size so that an UUID can fit in it, and adjusting the install scripts to do it.

Quoted 4 lines

Quoted 1 line

Here's a sample (untested) patch to illustrate it.

I take a quick look at your patch, the method should work, but i remember okuji oppose embedding device name in prefix, don't know how he look at it now.

Do you remember why? (or where it was discussed)

Anyway, CCing him. Okuji, please say something! :-)

Quoted 4 lines

Then for the Vista use case, I suppose the remaining questions are if NTFS supports UUIDs (or otherwise we have to ressort to labels?), and if we can extract filesystem metadata from Windows (if possible without requiring that a Cygwin installation is present).

ntfs has a 64-bit serial number, it's not quite the same as uuid, but it can be used to identify the partition.

This patch adds the 64-bit serial number as UUID. But I'm not sure what is the "standard" method to stringify this number. Apparently, Windows only prints half of it in the dir command.

Dne 7 Červen 2008, 01:31, Robert Millan napsal(a):

Quoted 3 lines

This patch adds the 64-bit serial number as UUID. But I'm not sure what is the "standard" method to stringify this number. Apparently, Windows only prints half of it in the dir command.

That seems to me as some backward (pronounce: awkward) compatibility of M$ Volume ID. Volume ID in M$ windows is (as you say) 'xxxx-xxxx' two 16 bit hexadecimal numbers. And all programs I tested report the number this way. So if we want user to enter these numbers manually we should ignore the remaining four bits. Alternatively we can provide utility that prints out whole serial number.

Quoted 11 lines · Tomáš Ebenlendr

Dne 7 Červen 2008, 01:31, Robert Millan napsal(a):

Quoted 3 lines

This patch adds the 64-bit serial number as UUID. But I'm not sure what is the "standard" method to stringify this number. Apparently, Windows only prints half of it in the dir command.

That seems to me as some backward (pronounce: awkward) compatibility of M$ Volume ID. Volume ID in M$ windows is (as you say) 'xxxx-xxxx' two 16 bit hexadecimal numbers. And all programs I tested report the number this way. So if we want user to enter these numbers manually we should ignore the remaining four bits. Alternatively we can provide utility that prints out whole serial number.

I suggest that you check udev sources:
http://www.us.kernel.org/pub/linux/utils/kernel/hotplug/

It calculates UUID for many filesystems (see extras/lib/volime_id/lib). It would be nice to stay compatible with udev.

Quoted 16 lines · Robert Millan

I think the method is sound; what I'm complaining about (and it's not something specific to your patch) is that we're referring to two different things by the same name ("root"), and even put them in the same variable.

In the initialization phase, "root" is the device that contains our GRUB directory.

Afterwards, "root" is the device we're currently accessing (be it for loading fonts, backgrounds, Linux images, whatever).

If "root" means "just a placeholder for whatever device we're acessing at the moment", then it would make sense, but in our code (i.e. in the names we're giving to commands and functions) it's assumed to mean "the device containing /boot/grub".

Quoted 4 lines

So what do we want to do with this? Should we have different variables for each thing (and in that case, is "root" for initial stage or for grub.cfg) or should we use "root" as a placeholder for any path reference, and adjust our function names etc to reflect that?

root is used in loaders as well, we shouldn't change its name. We could use another variable to store the root device at initial stage, it could actually be useful as it always points to the boot media. Perhaps we can name it "boot". What's your idea ?

Quoted 22 lines · Bean

Quoted 20 lines · Robert Millan

I think the method is sound; what I'm complaining about (and it's not something specific to your patch) is that we're referring to two different things by the same name ("root"), and even put them in the same variable.

In the initialization phase, "root" is the device that contains our GRUB directory.

Afterwards, "root" is the device we're currently accessing (be it for loading fonts, backgrounds, Linux images, whatever).

If "root" means "just a placeholder for whatever device we're acessing at the moment", then it would make sense, but in our code (i.e. in the names we're giving to commands and functions) it's assumed to mean "the device containing /boot/grub".

So what do we want to do with this? Should we have different variables for each thing (and in that case, is "root" for initial stage or for grub.cfg) or should we use "root" as a placeholder for any path reference, and adjust our function names etc to reflect that?

Quoted 4 lines

root is used in loaders as well, we shouldn't change its name. We could use another variable to store the root device at initial stage, it could actually be useful as it always points to the boot media. Perhaps we can name it "boot". What's your idea ?

I'm fine with "boot". I'm CCing Okuji; would like to make sure he doesn't have any objections (I think it was he who gave it this layout).

Quoted 28 lines · On Saturday 31 May 2008 17:21:25 Robert Millan wrote:

Quoted 26 lines · Bean

Quoted 16 lines · Robert Millan

I think the method is sound; what I'm complaining about (and it's not something specific to your patch) is that we're referring to two different things by the same name ("root"), and even put them in the same variable.

In the initialization phase, "root" is the device that contains our GRUB directory.

Afterwards, "root" is the device we're currently accessing (be it for loading fonts, backgrounds, Linux images, whatever).

If "root" means "just a placeholder for whatever device we're acessing at the moment", then it would make sense, but in our code (i.e. in the names we're giving to commands and functions) it's assumed to mean "the device containing /boot/grub".

Quoted 4 lines

So what do we want to do with this? Should we have different variables for each thing (and in that case, is "root" for initial stage or for grub.cfg) or should we use "root" as a placeholder for any path reference, and adjust our function names etc to reflect that?

root is used in loaders as well, we shouldn't change its name. We could use another variable to store the root device at initial stage, it could actually be useful as it always points to the boot media. Perhaps we can name it "boot". What's your idea ?

Quoted 2 lines

I'm fine with "boot". I'm CCing Okuji; would like to make sure he doesn't have any objections (I think it was he who gave it this layout).

"boot" is used for a command, so it is very confusing to use the same name for a variable. If you want to have this kind of variable, please find another name. I myself have no idea.

Hi,

No problem, I can find another name.

BTW, there are now two schemes for locating device at boot, the environment block and robert's uuid device, which one do you favor, or is there other scheme you prefer ?

Quoted 4 lines · Bean

Hi,

No problem, I can find another name.

Sorry, my mistake. I complained about $root being inappropiate for early stage use, but after that I noticed that $root is used in every stage to refer to "whatever drive we're currently accessing"; even before $prefix is initialised!

I'm fine with $root if nobody objects.

Quoted 3 lines

BTW, there are now two schemes for locating device at boot, the environment block and robert's uuid device, which one do you favor, or is there other scheme you prefer ?

I think there's no problem with having two schemes as long as they don't take extra core.img space (except for the rare use cases they're intended to) and don't conflict with each other.

As for the "findroot" approach, I have two comments/ideas:

- Since findroot.mod is basicaly an heuristic, and it's possible we may want different kinds of methods that work similarly, how about calling it "findroot_by_file" or something like that? Then we could have a separate module for "findroot_by_uuid", etc.

- I think your idea has the potential of obsoleting some parts of our init routines, maybe (if we're lucky) making kernel.img smaller. For example, we currently make grub_prefix a static length string (could it be dynamic? -> less space); then try to extract $root from it, since we embedded it in the same place (if we had a scheme to specify it separately -> less code to parse it).