Skip to contentSkip to conversation

fastboot/async and initramfs: How am I supposed to know when devices are finished initializing?

DISCUSSION Thomas Bächler 5 participants 12 messages Mar 29, 2009 – Mar 30, 2009 2 messages not in the archive

lore
Share & tools
lore
#0 lore raw

Okay, I have this problem in 2.6.29 with the "fastboot" option and in today's linux-2.6.git.

In initramfs, we run the following commands:

    /sbin/udevd --daemon
    /sbin/udevadm trigger
    /sbin/udevadm settle

Among other things, this loads the ata_piix on my machine. On older kernels I could assume that after these commands, /dev/sda* existed and I could immediately access them (in my case, I run cryptsetup on /dev/sda6). But now, the devices don't exist here, but are only created a second or so later. As a result, the initramfs script has already bailed out as it couldn't find /dev/sda6 and assumed that the hard drive didn't exist.

So my question is, if udevadm settle doesn't wait for the devices anymore, how am I supposed to know when the devices have been created? If there is no way to know, I would consider this a regression, as it is no longer possible to cleanly write initramfs scripts.

Thanks for you replies
Thomas Bächler
Arch Linux Developer

#1 lore raw
Quoted 9 lines · On Sun, 29 Mar 2009 18:02:41 +0200 Thomas Bächler <[email protected]> wrote:

Okay, I have this problem in 2.6.29 with the "fastboot" option and in today's linux-2.6.git.

In initramfs, we run the following commands:

     /sbin/udevd --daemon
     /sbin/udevadm trigger
     /sbin/udevadm settle

Among other things, this loads the ata_piix on my machine. On older kernels I could assume that after these commands, /dev/sda* existed

not if you have CONFIG_SCSI_SCAN_ASYNC set though..

Quoted 5 lines

and I could immediately access them (in my case, I run cryptsetup on /dev/sda6). But now, the devices don't exist here, but are only created a second or so later. As a result, the initramfs script has already bailed out as it couldn't find /dev/sda6 and assumed that the hard drive didn't exist.

the CONFIG_SCSI_WAIT_SCAN method (basically loading that module to wait for the scans to finish) will work for you......

Signature · 3 lines
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
#2 lore raw
Quoted 11 lines · Arjan van de Ven
Quoted 1 line

Among other things, this loads the ata_piix on my machine. On older kernels I could assume that after these commands, /dev/sda* existed

not if you have CONFIG_SCSI_SCAN_ASYNC set though..

Quoted 4 lines

and I could immediately access them (in my case, I run cryptsetup on /dev/sda6). But now, the devices don't exist here, but are only created a second or so later. As a result, the initramfs script has
already bailed out as it couldn't find /dev/sda6 and assumed that the
hard drive didn't exist.

the CONFIG_SCSI_WAIT_SCAN method (basically loading that module to wait for the scans to finish) will work for you......

Thanks, I will integrate that into our initramfs. Will that also work for USB mass storage (which was already problematic with older kernels, you load the module you don't know how long it takes until the sdX devices are created)? Can I load that module before loading the actual SCSI/SATA/PATA driver?

#13 lore raw
In reply to a message not in the archive
Quoted 8 lines · Arjan van de Ven
Quoted 3 lines

Thanks, I will integrate that into our initramfs. Will that also work for USB mass storage (which was already problematic with older kernels, you load the module you don't know how long it takes until
the sdX devices are created)?

in part.

The problem with USB probing is that devices come online whenever they feel like it; there is no way to wait for that.

Yes, that is a problem when trying to make bootable USB pen drives.

Quoted 11 lines
Quoted 2 lines

Can I load that module before loading the actual SCSI/SATA/PATA driver?

it's the loading of the driver that causes the wait. Not the act of having the driver loaded.

so the way to wait for things to settle is to load the driver...

... I take it the answer to your question is "no". Yes you can load it earlier, but no it has no effect. You can however load/unload it multiple times.

I understand the logic now, works as expected on my machine. Many thanks for your quick help.

#3 lore raw
In reply to Thomas Bächler
Quoted 13 lines · Thomas Bächler
Quoted 12 lines · Arjan van de Ven
Quoted 3 lines

Among other things, this loads the ata_piix on my machine. On older kernels I could assume that after these commands, /dev/sda* existed

not if you have CONFIG_SCSI_SCAN_ASYNC set though..

Quoted 5 lines

and I could immediately access them (in my case, I run cryptsetup on /dev/sda6). But now, the devices don't exist here, but are only created a second or so later. As a result, the initramfs script has already bailed out as it couldn't find /dev/sda6 and assumed that the hard drive didn't exist.

Quoted 7 lines
Quoted 2 lines

the CONFIG_SCSI_WAIT_SCAN method (basically loading that module to wait for the scans to finish) will work for you......

Thanks, I will integrate that into our initramfs. Will that also work for USB mass storage (which was already problematic with older kernels, you load the module you don't know how long it takes until the sdX devices are created)?

USB is interrupt driven, there is and will never be such a thing as "scanned" or "settled" -- anything can come and go at any time.

In initramfs, you have to wait until the device shows up, not for a random module to initialize, or a bus to be scanned -- that can never work correctly, it's pure luck, that your logic was always slower than the kernel.

You need a block device -- so you should just wait for the block device, instead of making assumptions about initialization of drivers or buses. :)

Thanks,
Kay

#4 lore raw
Quoted 8 lines · Kay Sievers

In initramfs, you have to wait until the device shows up, not for a random module to initialize, or a bus to be scanned -- that can never work correctly, it's pure luck, that your logic was always slower than the kernel.

You need a block device -- so you should just wait for the block device, instead of making assumptions about initialization of drivers or buses. :)

I am planning to do that but there is one problem: Usually, the user specifies a root device, or a device that I want to open with cryptsetup, or anything similar and I can wait until it shows up.

For lvm however, the user does not specify a specific block device that I can wait for, instead lvm scans all available block devices. Now, how do I know that the block device that contains my physical volume is already there? Maybe I have several hard drives, and several volume groups, so should I now call vgchange -ay again and again until the right volume group shows up?

#5 lore raw
Quoted 7 lines · Thomas Bächler
Quoted 6 lines · Kay Sievers

In initramfs, you have to wait until the device shows up, not for a random module to initialize, or a bus to be scanned -- that can never work correctly, it's pure luck, that your logic was always slower than the kernel.

Quoted 7 lines
Quoted 3 lines

You need a block device -- so you should just wait for the block device, instead of making assumptions about initialization of drivers or buses. :)

I am planning to do that but there is one problem: Usually, the user specifies a root device, or a device that I want to open with cryptsetup, or anything similar and I can wait until it shows up.

Right, you have to run the tools until the have found all what they need to give you your root device. Please ask the device mapper developers to port their stuff to the year 2009. :)

Quoted 6 lines

For lvm however, the user does not specify a specific block device that I can wait for, instead lvm scans all available block devices. Now, how do I know that the block device that contains my physical volume is already there? Maybe I have several hard drives, and several volume groups, so should I now call vgchange -ay again and again until the right volume group shows up?

Yeah, today, you just call the "broken" tools in a loop until they give you the device you wait for. :(

Assembling meta devices, or multi-volume devices should be triggered directly when the underlying devices show up. If auto-assembly (which kind of works today with md's --incremental) is properly implemented, you should still be able to just wait for the specified root device to show up, but the dm/lvm guys need to get the stuff done they talk about for years now. :)

Thanks,
Kay

#9 lore raw

Re: fastboot/async and initramfs: How am I supposed to know when?devices are finished initializing?

In reply to Thomas Bächler
Quoted 15 lines · Thomas Bächler
Quoted 8 lines · Kay Sievers

In initramfs, you have to wait until the device shows up, not for a random module to initialize, or a bus to be scanned -- that can never work correctly, it's pure luck, that your logic was always slower than the kernel.

You need a block device -- so you should just wait for the block device, instead of making assumptions about initialization of drivers or buses. :)

I am planning to do that but there is one problem: Usually, the user specifies a root device, or a device that I want to open with cryptsetup, or anything similar and I can wait until it shows up.

Quoted 7 lines

For lvm however, the user does not specify a specific block device that I can wait for, instead lvm scans all available block devices. Now, how do I know that the block device that contains my physical volume is already there? Maybe I have several hard drives, and several volume groups, so should I now call vgchange -ay again and again until the right volume group shows up?

If the user is specifying the boot device, could you not 'teach' them to LABEL the boot device and live off something slightly more fancy (but akin to) the following:

----
while true
do
mount LABEL=root /whatever

if [ $? -eq 0 ]; then
break
fi

  # do some md and lvm magic dance here

sleep 1
done
----

Your initramfs could do extra steps everytime the loop...erm...loops.

Now of course the downside is LABELing mounted devices, XFS for example, is not an easy thing to do... :-/

Cheers

Signature · 2 lines
Alexander Clouter
.sigmonster says: Am I ranting?  I hope so.  My ranting gets raves.
#6 lore raw
In reply to Kay Sievers
Quoted 5 lines · Kay Sievers

Assembling meta devices, or multi-volume devices should be triggered directly when the underlying devices show up. If auto-assembly (which kind of works today with md's --incremental) is properly implemented, you should still be able to just wait for the specified root device to show up...

Incremental assembly mostly works but it would be nice to have some event to signal "proceed with degraded assembly because I do not think any more disks are coming". With the below patch my thought is that we can have a rule that:
1/ checks if incremental assembly is in progress (to prevent adding timeouts to the non-raid path)
2/ checks if the udev event queue is empty
3/ upon finding an idle queue pause to allow last chance discovery 4/ assuming the queue remains idle proceed with degraded assembly

Thoughts? Perhaps (3) can be eliminated after loading scsi_wait_scan.

Thanks,
Dan

---ws damaged patch---
settle: allow --timeout=0 to test for "queue busy"

From: Dan Williams <[email protected]>

Allow udevadm settle to be used as a "queue busy" detector. For use in a early raid discovery where we want to hold off committing to the current list of disks while devices might still be arriving.

---
 udev/udevadm-settle.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)
udev/udevadm-settle.c +3 -1
index 867bbed..2fa85a4 100644
@@ -85,6 +85,9 @@ int udevadm_settle(struct udev *udev, int argc, char *argv[])
 		usleep(1000 * 1000 / LOOP_PER_SECOND);
 	}
 
+	if (!udev_queue_get_queue_is_empty(udev_queue))
+		rc = 1;
+
 	/* if we reached the timeout, print the list of remaining events */
 	if (loop <= 0) {
 		struct udev_list_entry *list_entry;
@@ -97,7 +100,6 @@ int udevadm_settle(struct udev *udev, int argc, char *argv[])
 				       udev_list_entry_get_name(list_entry),
 				       udev_list_entry_get_value(list_entry));
 		}
-		rc = 1;
 	}
 exit:
 	udev_queue_unref(udev_queue);
#7 lore raw
Quoted 4 lines · Dan Williams

settle: allow --timeout=0 to test for "queue busy"

+       if (!udev_queue_get_queue_is_empty(udev_queue))
+               rc = 1;

You mean, we should not wait? Then we need to check for timeout == 0 and return immediately instead of entering the checking loop at all, right?

Thanks,
Kay

#8 lore raw
Quoted 11 lines · Kay Sievers
Quoted 4 lines · Dan Williams

settle: allow --timeout=0 to test for "queue busy"

+       if (!udev_queue_get_queue_is_empty(udev_queue))
+               rc = 1;

You mean, we should not wait? Then we need to check for timeout == 0 and return immediately instead of entering the checking loop at all, right?

We already skip if loop==0...?

</me checks latest git>

Oh, this patch (against 138) needs to be reworked.

Thanks,
Dan

#10 lore raw

Re: fastboot/async and initramfs: How am I supposed to know when?devices are finished initializing?

In reply to Alexander Clouter
Quoted 13 lines · Alexander Clouter

If the user is specifying the boot device, could you not 'teach' them to LABEL the boot device and live off something slightly more fancy (but akin to) the following:

----
while true
do
mount LABEL=root /whatever

if [ $? -eq 0 ]; then
break
fi

  # do some md and lvm magic dance here
Quoted 7 lines

sleep 1
done
----

Your initramfs could do extra steps everytime the loop...erm...loops.

Now of course the downside is LABELing mounted devices, XFS for example, is not an easy thing to do... :-/

We do use UUIDs (or LABELs), but that doesn't solve the LVM problem, because the given device "/dev/somevg/somelv" only appears after LVM has been activated! I am thinking about using udev scan for LVM volume groups each time a hard drive appear that contains an LVM device. But that is a topic for another place.