losetup on a image file containing (GPT) partition doesn't create partition devices

From: Francis Moreau <[email protected]>

I have a disk image file which is using GPT and embeds 2 partitions.

I'm setting up a loop device with:

 $ losetup -P -f disk.img

and expect the partition devices (loop0p[12] for example) to be automatically created.

From: [email protected] (Dale R. Worley)

Then I detached the loopback device, re-attached it, and this time, the partition devices were *not* created, and /var/log/messages did not have the "loop0: p1 p2" message.

Given that the message starts "kernel:", I suspect that the udev behavior is triggered by the kernel detecting the partition table, and that is not happening. So it looks like it's a kernel bug.

Quoted 6 lines

From: [email protected] (Dale R. Worley)

Then I detached the loopback device, re-attached it, and this time, the partition devices were *not* created, and /var/log/messages did not have the "loop0: p1 p2" message.

Quoted 3 lines

Given that the message starts "kernel:", I suspect that the udev behavior is triggered by the kernel detecting the partition table, and that is not happening. So it looks like it's a kernel bug.

I tried that again, after updating to kernel 3.14.19-100.fc19.x86_64, and it worked this time.

Quoted 6 lines · Dale R. Worley

Quoted 6 lines

From: [email protected] (Dale R. Worley)

Then I detached the loopback device, re-attached it, and this time, the partition devices were *not* created, and /var/log/messages did not have the "loop0: p1 p2" message.

Quoted 6 lines

Quoted 3 lines

Given that the message starts "kernel:", I suspect that the udev behavior is triggered by the kernel detecting the partition table, and that is not happening. So it looks like it's a kernel bug.

I tried that again, after updating to kernel 3.14.19-100.fc19.x86_64, and it worked this time.

Well I'm already running 3.14.18, and the problem is still present but not systematic.

Here's a script which should trigger the problem:

-----
while :; do

    losetup -P /dev/loop0 disk.raw ||
    exit

    test -e /dev/loop0p1 || {
        echo "missing loop partition"
        losetup -d /dev/loop0
        break
    }
    echo -n "losetup worked: "
    ls /dev/loop0*

    losetup -d /dev/loop0 ||
    exit

    sleep 1

done
----

After looking at the kernel code, the kernel can silently fail to scan the partition table. This can happen if another process is already using the device and I think udev is the process that can interfere with 'losetup -P'.

What do you think ?

From: Francis Moreau <[email protected]>

Ok, so IMHO I think that losetup(8) should issue its own ioctl(BLKRRPART) and shouldn't rely on the kernel during the loop setup since it can fail silently. This way it can check the return value of ioctl() and moght choose to do it again in case the return value is -EBUSY.

Or indicates in the man page that the -P can fail silently.

Quoted 7 lines

From: Francis Moreau <[email protected]>

Ok, so IMHO I think that losetup(8) should issue its own ioctl(BLKRRPART) and shouldn't rely on the kernel during the loop setup since it can fail silently. This way it can check the return value of ioctl() and moght choose to do it again in case the return value is -EBUSY.

Quoted 1 line

Or indicates in the man page that the -P can fail silently.

It's not clear to me why the kernel would be expected to check itself for partitions in this manner. If losetup does not get the -P option, it seems like the kernel should not check.

Quoted 5 lines · Dale R. Worley

From: Francis Moreau <[email protected]>

Quoted 4 lines

Ok, so IMHO I think that losetup(8) should issue its own ioctl(BLKRRPART) and shouldn't rely on the kernel during the loop setup since it can fail silently. This way it can check the return value of ioctl() and moght choose to do it again in case the return value is -EBUSY.

Hm I was probably not clear, let me phase it again:

Ok, so IMHO I think that losetup(8) should issue its own ioctl(BLKRRPART) and shouldn't rely on the kernel during the loop setup since the kernel can fail silently. Using ioctl(BLKRRPART) allows losetup(8) to check the return value of ioctl() and might choose to do it again in case the return value is -EBUSY.

we've been tracking the same bug in Chromium OS:
http://crbug.com/411693 basically, under load, using losetup -P randomly fails to create the relevant partition nodes. atm we have hacks in place to call blockdev --rereadpt when it looks like the kernel has failed us.

it would be nice if the kernel didn't ignore the result of the ioctl thus allowing all errors to go unnoticed. you can see this in drivers/block/loop.c where it calls ioctl_by_bdev multiple times and doesn't check the return value.

since all current kernels are broken though, and might be for the foreseeable future, having losetup issue the ioctl itself might be a good tradeoff. i think it'd trigger unnecessary overhead (rescanning the device after it already worked), but seems better than having losetup try and check other sources (like if arbitrary partition nodes were created).

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 11 lines · Mike Frysinger

we've been tracking the same bug in Chromium OS:
http://crbug.com/411693 basically, under load, using losetup -P randomly fails to create the relevant partition nodes. atm we have hacks in place to call blockdev --rereadpt when it looks like the kernel has failed us.

it would be nice if the kernel didn't ignore the result of the ioctl thus allowing all errors to go unnoticed. you can see this in drivers/block/loop.c where it calls ioctl_by_bdev multiple times and doesn't check the return value.

Quoted 6 lines

since all current kernels are broken though, and might be for the foreseeable future, having losetup issue the ioctl itself might be a good tradeoff. i think it'd trigger unnecessary overhead (rescanning the device after it already worked), but seems better than having losetup try and check other sources (like if arbitrary partition nodes were created).

As long as the kernel has a flag that is supposed to cause it to interpret the partition table, then it should *work* and not silently fail. This is a bug in the kernel that should be fixed.

Yes, I don't like the idea with the ioctl workaround in losetup. It's fine to use such workarounds in stable branches or in distro specific patches, but I really don't want to use workarounds for bugs in regular releases (master branch).

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/3/2014 3:36 AM, Karel Zak wrote:

Quoted 4 lines

Yes, I don't like the idea with the ioctl workaround in losetup. It's fine to use such workarounds in stable branches or in distro specific patches, but I really don't want to use workarounds for bugs in regular releases (master branch).

Then again, maybe the whole idea of the kernel handling the partition table should be abandoned ( and the feature removed from the kernel ) and losetup should just call partx and be done with it.

Quoted 11 lines · Mike Frysinger

we've been tracking the same bug in Chromium OS:
http://crbug.com/411693 basically, under load, using losetup -P randomly fails to create the relevant partition nodes. atm we have hacks in place to call blockdev --rereadpt when it looks like the kernel has failed us.

it would be nice if the kernel didn't ignore the result of the ioctl thus allowing all errors to go unnoticed. you can see this in drivers/block/loop.c where it calls ioctl_by_bdev multiple times and doesn't check the return value.

Quoted 6 lines

since all current kernels are broken though, and might be for the foreseeable future, having losetup issue the ioctl itself might be a good tradeoff. i think it'd trigger unnecessary overhead (rescanning the device after it already worked), but seems better than having losetup try and check other sources (like if arbitrary partition nodes were created).

As long as the kernel has a flag that is supposed to cause it to interpret the partition table, then it should *work* and not silently fail. This is a bug in the kernel that should be fixed.

As a workaround though, you might consider just not using losetup -P in the first place and just run partx when you want to activate partitions. This also has the advantage of working even when the loop module is not loaded with the max_parts argument.