New: Issue with file system image created with mke2fs parameter -E offset

https://bugzilla.kernel.org/show_bug.cgi?id=216714

Bug ID: 216714
Summary: Issue with file system image created with mke2fs
parameter -E offset
Product: File System
Version: 2.5
Kernel Version: Linux 5.4.0-131-generic
Hardware: Intel
OS: Linux
Tree: Mainline
Status: NEW
Severity: normal
Priority: P1
Component: ext4
Assignee: [email protected]
Reporter: [email protected]
Regression: No

Overview:
We encountered an issue while using e2fsck to check the file system image created with the command “mke2fs -t ext4 -E offset=10”.
The issue was observed in e2fsprogs 1.46.5 and 1.45.6.

According to the manpage of mke2fs, the parameter “-E offset” is used to create the file system at an offset from the beginning of the device or file. This can be useful when creating disk images for virtual machines. No range is specified for this parameter.

Below are the steps to reproduce:

truncate -s 500M image
mke2fs -t ext4 -E offset=10 image 500M
e2fsck -yf image

mke2fs output log:
mke2fs 1.46.5 (30-Dec-2021)
Discarding device blocks: done
Creating filesystem with 512000 1k blocks and 128016 inodes Filesystem UUID: 6154bed2-c92b-45fe-b6db-97192ae46155 Superblock backups stored on blocks:

        8193, 24577, 40961, 57345, 73729, 204801, 221185, 401409

Allocating group tables: done
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

e2fsck output log:
e2fsck 1.46.5 (30-Dec-2021)
ext2fs_open2: Bad magic number in super-block
e2fsck: Superblock invalid, trying backup blocks...
e2fsck: Bad magic number in super-block while trying to open image

The superblock could not be read or does not describe a valid ext2/ext3/ext4 filesystem. If the device is valid and it really contains an ext2/ext3/ext4 filesystem (and not swap or ufs or something else), then the superblock is corrupt, and you might try running e2fsck with an alternate superblock:

    e2fsck -b 8193 <device>
 or
    e2fsck -b 32768 <device>

You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=216714

Lukas Czerner ([email protected]) changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #1 from Lukas Czerner ([email protected]) --- Hi,

the -E offset=10 creates a file system that is offset by 10 bytes from the beginning of the device or a file (as is written in the man page). It is no wonder then that the tools can't recognize the file system on the device or a file because the super block is not where it should be - it is offset by 10 bytes.

If you want to use such a file system you have to present it to the other tools, or even kernel, with the correct offset. It is not going to be automatic.

So for example this works:

mke2fs -q -t ext4 -E offset=10 image 500M
sudo losetup -o 10 --show -f image
/dev/loop0
sudo e2fsck -f /dev/loop0
e2fsck 1.46.6-rc1 (12-Sep-2022)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/loop0: 11/128016 files (0.0% non-contiguous), 26684/512000 blocks

I hope it helps.

You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=216714

Artem S. Tashkinov ([email protected]) changed:

What |Removed |Added
----------------------------------------------------------------------------

             Status|NEW                         |RESOLVED
         Resolution|---                         |INVALID

You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=216714

--- Comment #2 from [email protected] ---
Hi Lukas,

Thank you for your reply. It makes sense to me now. I tried using the offset parameter with losetup and it works for me.

I have another question related to this. Is there any way to find out the offset after creating the file system image? For example, after creating the image with mke2fs, I again have to provide the offset to losetup. Is there any tool or utility that tells us the offset value after creating the file system image? It would be very helpful if you could help me regarding this.

Thank you again for your time.

You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=216714

--- Comment #3 from Lukas Czerner ([email protected]) --- Hi,

that's a good question. I don't know of any existing tool that could do the scanning for you. You can either put something together yourself, or you could use blkid with --offset argument and try to guess the offset. I don't think there is any better way other than scan the beginning of the device looking for file system signature stored in superblock.

-Lukas

You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.kernel.org/show_bug.cgi?id=216714

Theodore Tso ([email protected]) changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected]

--- Comment #4 from Theodore Tso ([email protected]) ---
I'm curious --- *why* are you wanting to create file systems with an offset to
begin with? The original reason why this feature was added was in a highly
specialized case where someone was creating a system image for some embedded system or for a virtual machine. In this use case, the image contained a partition table, and the offset feature was used to create a filesystem at the appropriate location as specified by the partition table.

A typical use case is as part of an automated build procedure where the system image (say, for an Android mobile device, or some ARM development board, such as a Beaglebone, Arduino, etc., or some Virtual machine), using the mke2fs -d option to pre-populate the file system with the root file system, or some data
partition, etc. Since Best Practices for such automated build systems involve
creating a reproducible build, there is nothing precious on the file system
that can't be replicated by re-running the the automated build. So if the
offset is wrong (which is to say, inconsistent with the partition table which was laid down using the same automated build system), the developer will just curse to themselves, and can determine the offset by looking at the build image creation script, and then adjust that offset to match with the offset that was set in the partition table.

So I'm a bit perplexed about why you were using a random starting offset for the file system, and why you can't seem to figure out the offset afterwards. The typical approach is to RTFS (Read The Fine Shellscript) to determine the offset, and then to fix the perhaps not-so-fine shell script. :-)

You may reply to this email to add a comment.

You are receiving this mail because:
You are watching the assignee of the bug.