Skip to contentSkip to conversation

Multidevice f2fs mount after disk rearrangement

DISCUSSION Oleksandr Natalenko 3 participants 9 messages Jan 4, 2020 – Jan 8, 2020

lore
Share & tools
lore
#0 lore raw

Hello.

I was brave enough to create f2fs filesystem spanning through 2 physical device using this command:

# mkfs.f2fs -t 0 /dev/sdc -c /dev/sdd

It worked fine until I removed /dev/sdb from my system, so f2fs devices became:

/dev/sdc -> /dev/sdb
/dev/sdd -> /dev/sdc

Now, when I try to mount it, I get the following:

# mount -t f2fs /dev/sdb /mnt/fs
mount: /mnt/fs: mount(2) system call failed: No such file or directory.

In dmesg:

[Jan 4 17:25] F2FS-fs (sdb): Mount Device [ 0]: /dev/sdc, 59063, 0 - 1cd6fff
[ +0,000024] F2FS-fs (sdb): Failed to find devices

fsck also fails with the following assertion:

[ASSERT] (init_sb_info: 908) !strcmp((char *)sb->devs[i].path, (char *)c.devices[i].path)

Am I doing something obviously stupid, and the device path can be (somehow) changed so that the mount succeeds, or this is unfixable, and f2fs relies on persistent device naming?

Please suggest.

Thank you.

Signature · 1 line
  Oleksandr Natalenko (post-factum)
#1 lore raw

Hi.

Quoted 26 lines · Oleksandr Natalenko

I was brave enough to create f2fs filesystem spanning through 2 physical device using this command:

# mkfs.f2fs -t 0 /dev/sdc -c /dev/sdd

It worked fine until I removed /dev/sdb from my system, so f2fs devices became:

/dev/sdc -> /dev/sdb
/dev/sdd -> /dev/sdc

Now, when I try to mount it, I get the following:

# mount -t f2fs /dev/sdb /mnt/fs
mount: /mnt/fs: mount(2) system call failed: No such file or directory.

In dmesg:

[Jan 4 17:25] F2FS-fs (sdb): Mount Device [ 0]: /dev/sdc,
59063, 0 - 1cd6fff
[ +0,000024] F2FS-fs (sdb): Failed to find devices

fsck also fails with the following assertion:

[ASSERT] (init_sb_info: 908) !strcmp((char *)sb->devs[i].path, (char *)c.devices[i].path)

Quoted 7 lines

Am I doing something obviously stupid, and the device path can be (somehow) changed so that the mount succeeds, or this is unfixable, and f2fs relies on persistent device naming?

Please suggest.

Thank you.

Erm, fine. I studied f2fs-tools code a little bit and discovered that superblock indeed had /dev/sdX paths saved as strings. So I fired up hexedit and just changed the superblock directly on the first device, substituting sdc with sdb and sdd with sdc (I did it twice; I guess there are 2 copies of superblock), and after this the mount worked.

Am I really supposed to do this manually ;)?

Signature · 1 line
  Oleksandr Natalenko (post-factum)
#2 lore raw

Hello,

Thanks for the report. :)

Quoted 36 lines · On 2020/1/5 5:52, Oleksandr Natalenko via Linux-f2fs-devel wrote:

Hi.

Quoted 33 lines · Oleksandr Natalenko

I was brave enough to create f2fs filesystem spanning through 2 physical device using this command:

# mkfs.f2fs -t 0 /dev/sdc -c /dev/sdd

It worked fine until I removed /dev/sdb from my system, so f2fs devices became:

/dev/sdc -> /dev/sdb
/dev/sdd -> /dev/sdc

Now, when I try to mount it, I get the following:

# mount -t f2fs /dev/sdb /mnt/fs
mount: /mnt/fs: mount(2) system call failed: No such file or directory.

In dmesg:

[Jan 4 17:25] F2FS-fs (sdb): Mount Device [ 0]: /dev/sdc,
59063, 0 - 1cd6fff
[ +0,000024] F2FS-fs (sdb): Failed to find devices

fsck also fails with the following assertion:

[ASSERT] (init_sb_info: 908) !strcmp((char *)sb->devs[i].path, (char *)c.devices[i].path)

Am I doing something obviously stupid, and the device path can be (somehow) changed so that the mount succeeds, or this is unfixable, and f2fs relies on persistent device naming?

Please suggest.

Quoted 7 lines
Quoted 1 line

Thank you.

Erm, fine. I studied f2fs-tools code a little bit and discovered that superblock indeed had /dev/sdX paths saved as strings. So I fired up hexedit and just changed the superblock directly on the first device, substituting sdc with sdb and sdd with sdc (I did it twice; I guess there are 2 copies of superblock), and after this the mount worked.

Alright, it works if superblock checksum feature is off...

Quoted 2 lines

Am I really supposed to do this manually ;)?

We'd better add that ability in tune.f2fs. And I guess we need to let kernel/fsck to notice that case, and give hint to run tune.f2fs to reconfigure primary/secondary/... device paths.

Thanks,

Quoted 1 line
#3 lore raw
Quoted 49 lines · Chao Yu

Hello,

Thanks for the report. :)

Quoted 36 lines · On 2020/1/5 5:52, Oleksandr Natalenko via Linux-f2fs-devel wrote:

Hi.

Quoted 33 lines · Oleksandr Natalenko

I was brave enough to create f2fs filesystem spanning through 2 physical device using this command:

# mkfs.f2fs -t 0 /dev/sdc -c /dev/sdd

It worked fine until I removed /dev/sdb from my system, so f2fs devices became:

/dev/sdc -> /dev/sdb
/dev/sdd -> /dev/sdc

Now, when I try to mount it, I get the following:

# mount -t f2fs /dev/sdb /mnt/fs
mount: /mnt/fs: mount(2) system call failed: No such file or directory.

In dmesg:

[Jan 4 17:25] F2FS-fs (sdb): Mount Device [ 0]: /dev/sdc,
59063, 0 - 1cd6fff
[ +0,000024] F2FS-fs (sdb): Failed to find devices

fsck also fails with the following assertion:

[ASSERT] (init_sb_info: 908) !strcmp((char *)sb->devs[i].path, (char *)c.devices[i].path)

Am I doing something obviously stupid, and the device path can be (somehow) changed so that the mount succeeds, or this is unfixable, and f2fs relies on persistent device naming?

Please suggest.

Quoted 7 lines
Quoted 1 line

Thank you.

Erm, fine. I studied f2fs-tools code a little bit and discovered that superblock indeed had /dev/sdX paths saved as strings. So I fired up hexedit and just changed the superblock directly on the first device, substituting sdc with sdb and sdd with sdc (I did it twice; I guess there are 2 copies of superblock), and after this the mount worked.

Quoted 8 lines

Alright, it works if superblock checksum feature is off...

Quoted 2 lines

Am I really supposed to do this manually ;)?

We'd better add that ability in tune.f2fs. And I guess we need to let kernel/fsck to notice that case, and give hint to run tune.f2fs to reconfigure primary/secondary/... device paths.

I'm thinking to add tunesb.f2fs to edit superblock explicitly, since it has to edit it without getting superblock/checkpoint and other f2fs metadata.

For example,

# tunesb.f2fs -c /dev/sdb -c /dev/sdc /dev/sda
.. superblock info ..
.. device list ..
.. hot/cold extensions ..

Will modify the device list, if it's different from parameter.

Quoted 4 lines

Thanks,

Quoted 1 line
#4 lore raw
Quoted 62 lines · Jaegeuk Kim
Quoted 49 lines · Chao Yu

Hello,

Thanks for the report. :)

Quoted 36 lines · On 2020/1/5 5:52, Oleksandr Natalenko via Linux-f2fs-devel wrote:

Hi.

Quoted 33 lines · Oleksandr Natalenko

I was brave enough to create f2fs filesystem spanning through 2 physical device using this command:

# mkfs.f2fs -t 0 /dev/sdc -c /dev/sdd

It worked fine until I removed /dev/sdb from my system, so f2fs devices became:

/dev/sdc -> /dev/sdb
/dev/sdd -> /dev/sdc

Now, when I try to mount it, I get the following:

# mount -t f2fs /dev/sdb /mnt/fs
mount: /mnt/fs: mount(2) system call failed: No such file or directory.

In dmesg:

[Jan 4 17:25] F2FS-fs (sdb): Mount Device [ 0]: /dev/sdc,
59063, 0 - 1cd6fff
[ +0,000024] F2FS-fs (sdb): Failed to find devices

fsck also fails with the following assertion:

[ASSERT] (init_sb_info: 908) !strcmp((char *)sb->devs[i].path, (char *)c.devices[i].path)

Am I doing something obviously stupid, and the device path can be (somehow) changed so that the mount succeeds, or this is unfixable, and f2fs relies on persistent device naming?

Please suggest.

Quoted 7 lines
Quoted 1 line

Thank you.

Erm, fine. I studied f2fs-tools code a little bit and discovered that superblock indeed had /dev/sdX paths saved as strings. So I fired up hexedit and just changed the superblock directly on the first device, substituting sdc with sdb and sdd with sdc (I did it twice; I guess there are 2 copies of superblock), and after this the mount worked.

Quoted 8 lines

Alright, it works if superblock checksum feature is off...

Quoted 2 lines

Am I really supposed to do this manually ;)?

We'd better add that ability in tune.f2fs. And I guess we need to let kernel/fsck to notice that case, and give hint to run tune.f2fs to reconfigure primary/secondary/... device paths.

I'm thinking to add tunesb.f2fs to edit superblock explicitly, since it has to edit it without getting superblock/checkpoint and other f2fs metadata.

Quoted 7 lines

For example,

# tunesb.f2fs -c /dev/sdb -c /dev/sdc /dev/sda
.. superblock info ..
.. device list ..
.. hot/cold extensions ..

Will modify the device list, if it's different from parameter.

Looks good to me.

Thanks,

Quoted 7 lines
Quoted 5 lines

Thanks,

Quoted 1 line

.

#5 lore raw
In reply to Oleksandr Natalenko
Quoted 38 lines · Oleksandr Natalenko

Hi.

Quoted 27 lines · Oleksandr Natalenko

I was brave enough to create f2fs filesystem spanning through 2 physical device using this command:

# mkfs.f2fs -t 0 /dev/sdc -c /dev/sdd

It worked fine until I removed /dev/sdb from my system, so f2fs devices became:

/dev/sdc -> /dev/sdb
/dev/sdd -> /dev/sdc

Now, when I try to mount it, I get the following:

# mount -t f2fs /dev/sdb /mnt/fs
mount: /mnt/fs: mount(2) system call failed: No such file or directory.

In dmesg:

[Jan 4 17:25] F2FS-fs (sdb): Mount Device [ 0]: /dev/sdc,
59063, 0 - 1cd6fff
[ +0,000024] F2FS-fs (sdb): Failed to find devices

fsck also fails with the following assertion:

[ASSERT] (init_sb_info: 908) !strcmp((char *)sb->devs[i].path, (char *)c.devices[i].path)

Quoted 7 lines

Am I doing something obviously stupid, and the device path can be (somehow) changed so that the mount succeeds, or this is unfixable, and f2fs relies on persistent device naming?

Please suggest.

Thank you.

Quoted 7 lines

Erm, fine. I studied f2fs-tools code a little bit and discovered that superblock indeed had /dev/sdX paths saved as strings. So I fired up hexedit and just changed the superblock directly on the first device, substituting sdc with sdb and sdd with sdc (I did it twice; I guess there are 2 copies of superblock), and after this the mount worked.

Am I really supposed to do this manually ;)?

Thank you for investigating this ahead of me. :) Yes, the device list is stored in superblock, so hacking it manually should work.

Let me think about a tool to tune that.

Thanks,

Quoted 3 lines
Signature · 2 lines
-- 
  Oleksandr Natalenko (post-factum)
#6 lore raw

Hi.

Quoted 4 lines · Jaegeuk Kim

Thank you for investigating this ahead of me. :) Yes, the device list is stored in superblock, so hacking it manually should work.

Let me think about a tool to tune that.

Thank you both for the replies.

IIUC, tune.f2fs is not there yet. I saw a submission, but I do not see it as accepted, right?

Having this in tune.f2fs would be fine (assuming the assertion is replaced with some meaningful hint message), but wouldn't it be more convenient for an ordinary user to have implemented something like:

# mount -t f2fs /dev/sdb -o nextdev=/dev/sdc /mnt/fs

Hm?

Signature · 1 line
  Oleksandr Natalenko (post-factum)
#7 lore raw
Quoted 11 lines · On 2020/1/7 2:40, Oleksandr Natalenko via Linux-f2fs-devel wrote:

Hi.

Quoted 5 lines · Jaegeuk Kim

Thank you for investigating this ahead of me. :) Yes, the device list is stored
in superblock, so hacking it manually should work.

Let me think about a tool to tune that.

Thank you both for the replies.

Quoted 8 lines

IIUC, tune.f2fs is not there yet. I saw a submission, but I do not see it as accepted, right?

Having this in tune.f2fs would be fine (assuming the assertion is replaced with some meaningful hint message), but wouldn't it be more convenient for an ordinary user to have implemented something like:

# mount -t f2fs /dev/sdb -o nextdev=/dev/sdc /mnt/fs

Hmm... sounds reasonable, however, the risk is obvious, if we mount with wrong primary device, filesystem can be aware that with metadata sanity check, if we mount with wrong secondary/... devices by mistake (or intentionally, people may think filesystem should be aware illegal parameters....), filesystem won't be aware of that, then metadata/data will be inconsistent...

Although that may also happen when we use tunesb.f2fs, but fsck.f2fs can be followed to verify the modification of tunesb.f2fs, that would be much safer.

So I suggest we can do that in tools first, maybe implement nextdev mount option if we have added metadata in secondary/... device.

Thanks,

Quoted 3 lines

Hm?

#8 lore raw
Quoted 27 lines · Chao Yu
Quoted 11 lines · On 2020/1/7 2:40, Oleksandr Natalenko via Linux-f2fs-devel wrote:

Hi.

Quoted 5 lines · Jaegeuk Kim

Thank you for investigating this ahead of me. :) Yes, the device list is stored
in superblock, so hacking it manually should work.

Let me think about a tool to tune that.

Thank you both for the replies.

Quoted 8 lines

IIUC, tune.f2fs is not there yet. I saw a submission, but I do not see it as accepted, right?

Having this in tune.f2fs would be fine (assuming the assertion is replaced with some meaningful hint message), but wouldn't it be more convenient for an ordinary user to have implemented something like:

# mount -t f2fs /dev/sdb -o nextdev=/dev/sdc /mnt/fs

Hmm... sounds reasonable, however, the risk is obvious, if we mount with wrong primary device, filesystem can be aware that with metadata sanity check, if we mount with wrong secondary/... devices by mistake (or intentionally, people may think filesystem should be aware illegal parameters....), filesystem won't be aware of that, then metadata/data will be inconsistent...

Quoted 5 lines

Although that may also happen when we use tunesb.f2fs, but fsck.f2fs can be followed to verify the modification of tunesb.f2fs, that would be much safer.

So I suggest we can do that in tools first, maybe implement nextdev mount option if we have added metadata in secondary/... device.

+1, it'd be risky for user to give the device list whenever mounting the filesystem. There'll be subtle corner cases where f2fs needs to deal with given ambiguous sets between superblock and mount option.

Quoted 6 lines

Thanks,

Quoted 3 lines

Hm?