NILFS2: double uuid

Hi,

a nilfs2 formatted disk fails to mount via fstab due to double uuid's. See lsblk output below. The logs indicate that the system attempts to mount /dev/sdb rather than /dev/sdb1, which of course fails. In addition, /dev/sdb should not have any uuid at all. Don't know why that happens.

The phenomenon is easily reproducible: format a partition with nilfs2, register it with the proper uuid in fstab and reboot. Tried both with USB memory and real HDD.

[root@keera ~]# lsblk -f
NAME FSTYPE LABEL UUID MOUNTPOINT
sdb ff17dda9-fcae-42e7-a438-9087de58902e
`-sdb1 xfs ff17dda9-fcae-42e7-a438-9087de58902e

Thanks,
Heinz

Copy error: replace xfs with nilfs2. Sorry!

(CCed to [email protected])
Hi Heinz,

I couldn't reproduce the issue (in a CentOS 7 environment).

Could you tell us the version information of distro, lsblk, libblkid, nilfs-utils, and kernel you are using ?

The following is an example of mine:

$ lsblk -f
NAME   FSTYPE LABEL  UUID                                 MOUNTPOINT
sda
└─sda1 nilfs2        9dcd01c0-2bc8-41bf-a400-8ad8755aac6a
$ lsblk --version
lsblk from util-linux 2.23.2

$ lscp -V
lscp (nilfs-utils 2.2.3)

$ rpm -q libblkid util-linux
libblkid-2.23.2-22.el7_1.x86_64
util-linux-2.23.2-22.el7_1.x86_64

$ uname -r
4.1.0-rc7

Regards,
Ryusuke Konishi

It happens on two different systems:

The first one is bog-standard Fedora 21:

[htd@chiara ~]$ rpm -qa | egrep 'lsblk|libblkid|nilfs' libblkid-devel-2.25.2-3.fc21.x86_64 libblkid-2.25.2-3.fc21.x86_64 nilfs-utils-2.2.3-1.fc21.x86_64

[htd@chiara ~]$ lscp -V
lscp (nilfs-utils 2.2.3)

[htd@chiara ~]$ lsblk --version
lsblk from util-linux 2.25.2

[htd@chiara ~]$ uname -a
Linux chiara.fritha.org 4.0.5-rc1 #1 SMP PREEMPT Wed Jun 3 20:41:46 CEST 2015 x86_64 x86_64 x86_64 GNU/Linux

The second one is Arch Linux on a Raspberry Pi:

[root@keera ~]# pacman -Q | egrep 'libutil|nilfs'
libutil-linux 2.26.2-1
nilfs-utils 2.2.3-1

[root@keera ~]# lsblk --version
lsblk from util-linux 2.26.2

[root@keera ~]# lscp -V
lscp (nilfs-utils 2.2.3)

[root@keera ~]# uname -a
Linux keera 3.18.14-2-ARCH #1 SMP PREEMPT Thu May 28 07:19:33 MDT 2015 armv7l GNU/Linux

Tried to re-format the home partition on a Fedora 21 laptop with nilfs2 and ran into the same problem as with USB memory as a samba share on the Pi: the system tried to mount the block device rather than the partition due to the uuid as shown in my previous email.

Thanks, Heinz.

OpenPGP Fingerprint: 531E 8E75 4395 ED38 CEC2  FD75 A1EB 6944 60F4 A92C
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 08.06.2015, Heinz Diehl wrote:

To be more precise, here's what works and what don't, in detail (and after a fresh install of Arch):

The USB memory is xfs formatted and works fine:

[root@alarmpi /]# lsblk -f
NAME FSTYPE LABEL UUID MOUNTPOINT
sda
`-sda1 xfs ff17dda9-fcae-42e7-a438-9087de58902e
mmcblk0
|-mmcblk0p1 vfat EA5B-4477 /boot
`-mmcblk0p2 ext4 c4ddc925-15ab-4465-ac78-967a845e98d5 /

Now, it's nilfs2 formatted:

[root@alarmpi /]# mkfs.nilfs2 /dev/sda1
WARNING: Device /dev/sda1 appears to contain an existing xfs superblock.
WARNING: All data will be lost after format!

DO YOU REALLY WANT TO FORMAT DEVICE /dev/sda1?

Continue? [y/N] y
mkfs.nilfs2 (nilfs-utils 2.2.3)
Start writing file system initial data to the device

       Blocksize:4096  Device:/dev/sda1  Device Size:32026656768

File system initialization succeeded !!

After that, all seems to be ok. lsblk shown no double uuid:

[root@alarmpi /]# lsblk -f
NAME FSTYPE LABEL UUID MOUNTPOINT
sda
`-sda1 nilfs2 98da384c-392e-4551-98c0-d076524f5d8b
mmcblk0
|-mmcblk0p1 vfat EA5B-4477 /boot
`-mmcblk0p2 ext4 c4ddc925-15ab-4465-ac78-967a845e98d5 /
[root@alarmpi /]#

Now the USB drive gets manually mounted, all is ok:

[root@alarmpi /]# mount /dev/sda1 /USBDRIVE
[root@alarmpi /]# lsblk -f
NAME FSTYPE LABEL UUID MOUNTPOINT
sda
`-sda1 nilfs2 98da384c-392e-4551-98c0-d076524f5d8b /USBDRIVE
mmcblk0
|-mmcblk0p1 vfat EA5B-4477 /boot
`-mmcblk0p2 ext4 c4ddc925-15ab-4465-ac78-967a845e98d5 /

Now, the newly formatted drive is registered in fstab to be automatically mounted on boot:

UUID=ff17dda9-fcae-42e7-a438-9087de58902e /USBDRIVE nilfs2 defaults 0 0

After rebooting the machine, nothing is mounted, and lsblk shows the double uuid:

[root@alarmpi /]# lsblk -f
NAME FSTYPE LABEL UUID MOUNTPOINT
sda 98da384c-392e-4551-98c0-d076524f5d8b
`-sda1 nilfs2 98da384c-392e-4551-98c0-d076524f5d8b
mmcblk0
|-mmcblk0p1 vfat EA5B-4477 /boot
`-mmcblk0p2 ext4 c4ddc925-15ab-4465-ac78-967a845e98d5 /

The logs say:

Jun 08 11:23:47 alarmpi mount: mount.nilfs2: Error while mounting /dev/sda on /USBDRIVE: Device or resource busy
Jun 08 11:23:47 alarmpi systemd: Failed to mount /USBDRIVE.

Here it becomes clear what happens: the system wants to mount /dev/sda rather than /dev/sda1, and thus fails.

Out of curiosity, I tried both xfs, ext4 and btrfs, and all of them just work.

Thanks,
Heinz.
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Sorry for the noise, but it's quite difficult to sum up all these uuids when cop&paste doesn't work: the above uuid is wrong, but it's a simple copy error by me. So just assume it is correct. Sorry again!

The point is that the doube uuid occurs after booting, when trying to mount from fstab.

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi,

I've tested the same steps as you wrote above (first created an xfs partition, overrode it with a nilfs2 partition, wrote a similar entry to fstab, and reboot), but didn't reproduce the issue.

On my CentOS 7 environment, lsblk and default mount are perfectly working.

So, it may be a version dependent issue of util-linux.
I will try to reproduce and nallow down the issue with newer util-linux packages.

Thanks,
Ryusuke Konishi

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

(CCed to Karel Zak)
Hi,

I succeeded to reproduce this issue on Fedora 20, 21, 22 and Debian jessie. Also, I could narrow down the issue.

This turned out to be an issue of libblkid in util-linux and introduced by the commit 5f77ce6f3269 ("libblkid: (nilfs2) check also backup superblock"):

* commit 1a38ad5c3271a59c7e51580242a2fbd3b0f16495 --> OK

$ sudo LD_LIBRARY_PATH=/usr/local/lib lsblk --version
lsblk from util-linux 2.24.153-1a38
$ sudo LD_LIBRARY_PATH=/usr/local/lib lsblk -f
NAME   FSTYPE  LABEL           UUID                                 MOUNTPOINT
[...]
sdb
`-sdb1 nilfs2                  c6cd2c9c-0291-4f9f-be9b-10ff8e2acbe6 /test

* commit 5f77ce6f32692b473ffcec4c6f63dbd38cd5eeda --> NG

$ sudo LD_LIBRARY_PATH=/usr/local/lib lsblk --version
lsblk from util-linux 2.24.154-5f77c
$ sudo LD_LIBRARY_PATH=/usr/local/lib lsblk -f
NAME   FSTYPE  LABEL           UUID                                 MOUNTPOINT
[...]
sdb    nilfs2                  c6cd2c9c-0291-4f9f-be9b-10ff8e2acbe6
`-sdb1 nilfs2                  c6cd2c9c-0291-4f9f-be9b-10ff8e2acbe6 /test

Here, the backup super block of /dev/sdb1 got detected also for /dev/sdb by the commit 5f77ce6f3269.

This change has been applied between v2.24 and v2.24.1 of util-linux, and not yet fixed in the mainline.

It causes the duplicate uuid and leads the UUID mount written in the fstab file to mount the device itself (i.e. /dev/sdb in this example). Thus the mount failure happens.

It looks like the backup super block should be dropped from candidates if its device size (sbp->s_dev_size) doesn't match the partition size.

Regards,
Ryusuke Konishi

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Thanks a lot for your effort!

Just in case this matters, there is also a report from the util-linux mailing list pointing to this particular commit:

http://permalink.gmane.org/gmane.linux.utilities.util-linux-ng/10766

Ok, I'll try to identify and revert this commit in the meantime.

Thanks, Heinz.
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

The patch doesn't revert cleanly. Is there a workaround I could use until the problem is properly fixed?

Thanks,
Heinz

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Yeah, fixed:
http://git.kernel.org/cgit/utils/util-linux/util-linux.git/commit/?id=00817742ce360119e079a33e12cf84118ff7c63e

Note that workaround is to not use nilfs2 on the last partition or have a tiny gap (1 sector is enough) between last partition and the end of the whole-disk.

Karel

 Karel Zak  <[email protected]>
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[....]

Thanks a lot to both of you! Tried both solutions and can confirm that they work both.

Hi,

Thanks for your quick work!

I tested the patch. It almost worked fine.
One issue I found is a transient state after fs-resizing.

After shrinking the file system, both superblocks dropped and lsblk failed to detect the filesystem:

$ sudo LD_LIBRARY_PATH=/usr/local/lib lsblk -f
NAME   FSTYPE  LABEL           UUID MOUNTPOINT
[...]
sdb
`-sdb1 nilfs2                  2d7cd130-82a0-4a3c-b8a8-4ac5a26f5703 /test

$ sudo nilfs-resize -y /dev/sdb1 1G
Partition size = 2146435072 bytes.
Shrink the filesystem size from 2146435072 bytes to 1073741824 bytes.
128 segments will be truncated from segnum 127.
Moving 103 in-use segments.
progress |***********************************************|
Done.

$ sudo umount /test
$ sudo mount /dev/sdb1 /test
$ sudo LD_LIBRARY_PATH=/usr/local/lib lsblk -f
NAME   FSTYPE  LABEL           UUID MOUNTPOINT
[...]
sdb
`-sdb1                                                              /test

This blank state continued until I shrank the partition or re-extended the filesystem to the partition size.

Could you consider confining the s_dev_size test only to the backup superblock ?

It seems that we don't have to drop the primary super block even if s_dev_size doesn't fit to the partition size.

Regards,
Ryusuke Konishi

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Hmm... why nilfs-resize does not update the size in the superblock? It seems like nilfs-resize bug.

Yes, fixed. I have also enabled the s_dev_size check for whole-disk devices only to minimize number of situations when we rely on the s_dev_size.

Karel

 Karel Zak  <[email protected]>
 http://karelzak.blogspot.com
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

nilfs-resize (to be exact, RESIZE ioctl of nilfs2) updates s_dev_size in both superblocks. What nilfs-resize doesn't change is the partition size. (It needs help of a partitioning tool)

Thanks again. The updated libblkid/lsblk works frawlessly.

Regards,
Ryusuke Konishi
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html