Weird behaviour with lsblk and freshly created loop device
Hello,
Right after associating a loop device with a disk image file, I'm using lsblk to retrieve some info about the partitions.
If I'm doing this as root:
$ losetup /dev/loop0 /tmp/installer-disks/disk0.img && lsblk -n --raw --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d /dev/loop0
then the output is missing the parttypes:
/dev/loop0
/dev/loop0p1
/dev/loop0p2
However adding a sleep or 'udevadm settle' after setting the loop device the parttypes are correctly showed:
$ losetup /dev/loop0 /tmp/installer-disks/disk0.img && udevadm settle && lsblk -n --raw --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d /dev/loop0 /dev/loop0 /dev/loop0p1 0657fd6d-a4ab-43c4-84e5-0933c84b4f4f /dev/loop0p2 4f68bce3-e8cd-4db1-96e7-fbcaf984b709
Could anybody tell me if that's expected ?
Thanks.
BTW, I'm using lsblk from util-linux 2.25 on Archlinux.
Quoted 30 lines · Francis Moreau
Hello,
Right after associating a loop device with a disk image file, I'm using lsblk to retrieve some info about the partitions.
If I'm doing this as root:
$ losetup /dev/loop0 /tmp/installer-disks/disk0.img && lsblk -n --raw --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d /dev/loop0then the output is missing the parttypes:
/dev/loop0
/dev/loop0p1
/dev/loop0p2However adding a sleep or 'udevadm settle' after setting the loop device the parttypes are correctly showed:
$ losetup /dev/loop0 /tmp/installer-disks/disk0.img && udevadm settle && lsblk -n --raw --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d /dev/loop0 /dev/loop0 /dev/loop0p1 0657fd6d-a4ab-43c4-84e5-0933c84b4f4f /dev/loop0p2 4f68bce3-e8cd-4db1-96e7-fbcaf984b709Could anybody tell me if that's expected ?
Thanks.
Karel Zak
maintainerQuoted 19 lines · Francis Moreau
Hello,
Right after associating a loop device with a disk image file, I'm using lsblk to retrieve some info about the partitions.
If I'm doing this as root:
$ losetup /dev/loop0 /tmp/installer-disks/disk0.img && lsblk -n --raw --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d /dev/loop0then the output is missing the parttypes:
/dev/loop0
/dev/loop0p1
/dev/loop0p2However adding a sleep or 'udevadm settle' after setting the loop device the parttypes are correctly showed:
Quoted 8 lines
$ losetup /dev/loop0 /tmp/installer-disks/disk0.img && udevadm settle && lsblk -n --raw --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d /dev/loop0 /dev/loop0 /dev/loop0p1 0657fd6d-a4ab-43c4-84e5-0933c84b4f4f /dev/loop0p2 4f68bce3-e8cd-4db1-96e7-fbcaf984b709Could anybody tell me if that's expected ?
Sure, lsblk reads the information from udev db.
You can compile --without-udev, the it will use libblkid only.
Maybe we can add something like "lsblk --no-udev" to bypass udev without recompilation, but is it really necessary?
Karel
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com
Quoted 27 lines · Karel Zak
Quoted 26 lines · Francis Moreau
Hello,
Right after associating a loop device with a disk image file, I'm using lsblk to retrieve some info about the partitions.
If I'm doing this as root:
$ losetup /dev/loop0 /tmp/installer-disks/disk0.img && lsblk -n --raw --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d /dev/loop0then the output is missing the parttypes:
/dev/loop0
/dev/loop0p1
/dev/loop0p2However adding a sleep or 'udevadm settle' after setting the loop device the parttypes are correctly showed:
$ losetup /dev/loop0 /tmp/installer-disks/disk0.img && udevadm settle && lsblk -n --raw --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d /dev/loop0 /dev/loop0 /dev/loop0p1 0657fd6d-a4ab-43c4-84e5-0933c84b4f4f /dev/loop0p2 4f68bce3-e8cd-4db1-96e7-fbcaf984b709
Quoted 8 lines
Quoted 1 line
Could anybody tell me if that's expected ?
Sure, lsblk reads the information from udev db.
You can compile --without-udev, the it will use libblkid only.
Maybe we can add something like "lsblk --no-udev" to bypass udev without recompilation, but is it really necessary?
It seems that lsblk uses udev to get some block device metadata and asks some others to the kernel. If so, it makes the whole process racy because udev might not have handled the events sent by the kernel yet.
I'm not sure why udev is used by default in the first place, what are the benefits ? Using libblkid, at least by default seems the right thing to do.
Otherwise maybe lsblk should do the equivalent of 'udevadm settle' to handle correctly freshly created devices ?
Thanks.
Karel Zak
maintainerQuoted 5 lines · Francis Moreau
It seems that lsblk uses udev to get some block device metadata and asks some others to the kernel. If so, it makes the whole process racy because udev might not have handled the events sent by the kernel yet.
I'm not sure why udev is used by default in the first place, what are
* info from udev is accessible for non-root users
* it's better to scan devices only once on one place only
* udev is able to gather information from more sources (for example libblkid does not provide WWN)
Quoted 5 lines
the benefits ? Using libblkid, at least by default seems the right thing to do.
Otherwise maybe lsblk should do the equivalent of 'udevadm settle' to handle correctly freshly created devices ?
That's question, now (because it's not hardcoded to lsblk) everyone is able to control this behavior, all you need is to add 'udevadm settle' to your use-case.
The problem I see is that there is no any hint about this behavior in lsblk man page.
Karel
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com
Quoted 15 lines · Karel Zak
Quoted 5 lines · Francis Moreau
It seems that lsblk uses udev to get some block device metadata and asks some others to the kernel. If so, it makes the whole process racy because udev might not have handled the events sent by the kernel yet.
I'm not sure why udev is used by default in the first place, what are
* info from udev is accessible for non-root users
* it's better to scan devices only once on one place only
* udev is able to gather information from more sources (for example libblkid does not provide WWN)Quoted 3 lines
the benefits ? Using libblkid, at least by default seems the right thing to do.
Quoted 6 lines
Quoted 2 lines
Otherwise maybe lsblk should do the equivalent of 'udevadm settle' to handle correctly freshly created devices ?
That's question, now (because it's not hardcoded to lsblk) everyone is able to control this behavior, all you need is to add 'udevadm settle' to your use-case.
The question is more why let the user do that ?
If lsblk reports the partitions, shouldn't it do that only when it's sure to fully have retrieve the partition's metadata ?
BTW, is the user supposed to know that lsblk relies on udev or is this implementation detail ?
Thanks
Karel Zak
maintainerQuoted 5 lines · Francis Moreau
Quoted 3 lines
That's question, now (because it's not hardcoded to lsblk) everyone is able to control this behavior, all you need is to add 'udevadm settle' to your use-case.
The question is more why let the user do that ?
because we're talking about small, simple command line util, not about complex high-level UI
Quoted 2 lines
If lsblk reports the partitions, shouldn't it do that only when it's sure to fully have retrieve the partition's metadata ?
if you do not have exclusive access to the device then you cannot be sure at all. All is asynchronous...
Quoted 2 lines
BTW, is the user supposed to know that lsblk relies on udev or is this implementation detail ?
This is very generic system feature, for example if you want to mount a device by /dev/disk/by-* symlinks then nowhere is guarantee that mount(8) is not faster than udev+blkid, etc.
Yes, we can add "udevadm settle" functionality into all system utils, but it will increase complexity and degrade performance. So from my point of view it seems better to explain the problem in man page, keep lsblk simple and stupid and assume that users who really care will use "udevadm settle".
(The another disadvantage is that libudev is not able to wait for
specific device, it waits for all active event.)
Karel
Signature · 2 lines
Karel Zak <[email protected]> http://karelzak.blogspot.com
Quoted 23 lines · Karel Zak
Quoted 5 lines · Francis Moreau
Quoted 3 lines
That's question, now (because it's not hardcoded to lsblk) everyone is able to control this behavior, all you need is to add 'udevadm settle' to your use-case.
The question is more why let the user do that ?
because we're talking about small, simple command line util, not about complex high-level UI
Quoted 2 lines
If lsblk reports the partitions, shouldn't it do that only when it's sure to fully have retrieve the partition's metadata ?
if you do not have exclusive access to the device then you cannot be sure at all. All is asynchronous...
Quoted 2 lines
BTW, is the user supposed to know that lsblk relies on udev or is this implementation detail ?
This is very generic system feature, for example if you want to mount a device by
/dev/disk/by-*symlinks then nowhere is guarantee that mount(8) is not faster than udev+blkid, etc.
Quoted 5 lines
Yes, we can add "udevadm settle" functionality into all system utils, but it will increase complexity and degrade performance. So from my point of view it seems better to explain the problem in man page, keep lsblk simple and stupid and assume that users who really care will use "udevadm settle".
Also you might want to add a new option for forcing lsblk to wait for udev ?
Thanks for your answers.
Quoted 10 lines
From: Francis Moreau <[email protected]>
Right after associating a loop device with a disk image file, I'm using lsblk to retrieve some info about the partitions.
If I'm doing this as root:
$ losetup /dev/loop0 /tmp/installer-disks/disk0.img && lsblk -n --raw --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d /dev/loop0
Quoted 8 lines
then the output is missing the parttypes:
/dev/loop0
/dev/loop0p1
/dev/loop0p2However adding a sleep or 'udevadm settle' after setting the loop device the parttypes are correctly showed:
It looks like the process of determining the partitions is asynchronous, and is not assured of being completed when losetup returns.
What happens when you add -P to losetup? The manual page (for Fedora
19) says "force kernel to scan partition table on newly created loop device", which suggests that the partition scan will be completed by the time losetup returns.
Dale
Quoted 16 lines · Dale R. Worley
Quoted 16 lines
From: Francis Moreau <[email protected]>
Right after associating a loop device with a disk image file, I'm using lsblk to retrieve some info about the partitions.
If I'm doing this as root:
$ losetup /dev/loop0 /tmp/installer-disks/disk0.img && lsblk -n --raw --paths -o NAME,PARTTYPE /dev/loop0 && losetup -d /dev/loop0then the output is missing the parttypes:
/dev/loop0
/dev/loop0p1
/dev/loop0p2
Quoted 6 lines
Quoted 2 lines
However adding a sleep or 'udevadm settle' after setting the loop device the parttypes are correctly showed:
It looks like the process of determining the partitions is asynchronous, and is not assured of being completed when losetup returns.
Still lsblk lists the partitions.
I'm not sure how lsblk works. It seems that it relies on udev to get certain information, parttype is an example but doesn't for listing partitions of block devices.
That seems to make the whole process racy.
If this is correct, I'm wondering why lsblk relies on udev to retrieve block device metadata. Can't it find them out directly from the kernel ?
Quoted 5 lines
What happens when you add -P to losetup? The manual page (for Fedora
19) says "force kernel to scan partition table on newly created loop device", which suggests that the partition scan will be completed by the time losetup returns.
Adding -P doesn't help. Actually the kernel correctly detected the partitions embedded in the loop device. But udev hasn't handled the associeted events yet when lsblk is called.
Thanks