Skip to contentSkip to conversation

wic: updated fstab does not preserve metadata of the original file

PATCH In review [email protected] 5 participants 20 messages Sep 3, 2025 – Oct 23, 2025 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

cover wic: updated fstab does not preserve metadata of the original file

From: Daniel Andrade <[email protected]>

Using install in the rootfs plugin forces fstab to be replaced entirely, meaning that even its Inodes will change, leading xattrs and SELinux context stored by pseudo not to be applied.
The fix just uses cp without preserving attributes from the temporary fstab since none of them are needed, just the content.
Same thing happens with the predefined mechanisms for ext4 and msdos. Using debugfs there is no way to replace contents while maintaining metadata, so the approach taken on the path was to remove the different fstab logic for those fstypes and also use the same modified cp command. Reviewing the builds I did it seems to work for all of the fstypes.

Another problem is that the timestamp applied to fstab is not the same as every other file. It seems like the SOURCE_DATE_EPOCH variable goes to the fallback timestamp (SOURCE_DATE_EPOCH_FALLBACK). Since you are using that variable everywhere, it is not the same value as REPRODUCIBLE_TIMESTAMP_ROOTFS under poky/meta/conf/bitbake.conf that is applied in every other file.

Daniel Andrade (1):
wic: Content of the temporary updated fstab should be copied into the
original not replacing it entirely.

 meta/conf/bitbake.conf                   |  4 +++-
 scripts/lib/wic/partition.py             | 15 +--------------
 scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
 3 files changed, 6 insertions(+), 17 deletions(-)
#1 lore raw

PATCH 1/1 wic: Content of the temporary updated fstab should be copied into the original not replacing it entirely.

From: Daniel Andrade <[email protected]>

Fixes [15947]
The current functionality to update fstab generates a new temporary fstab with the new partition configuration.
However, this file does not retain any metadata being it a completely new file.
Because of this, when the rootfs plugin under poky/scripts/lib/wic/plugins/source/rootfs.py copies the file, it overrides the original fstab metadata. The patch removes implementation of msdos/ext code for fstab since it is not possible to append content without rewriting the file to preserve metadata.

The timestamp applied to fstab is not the same as every other file.
It seems like the SOURCE_DATE_EPOCH variable goes to the fallback timestamp SOURCE_DATE_EPOCH_FALLBACK.
Since that variable is used everywhere, it is not the same value as REPRODUCIBLE_TIMESTAMP_ROOTFS under poky/meta/conf/bitbake.conf that is applied in every other file.

---
 meta/conf/bitbake.conf                   |  4 +++-
 scripts/lib/wic/partition.py             | 15 +--------------
 scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
 3 files changed, 6 insertions(+), 17 deletions(-)
meta/conf/bitbake.conf +3 -1
index acf4e2d153..0581fc3564 100644
@@ -684,8 +684,10 @@ export PYTHONHASHSEED = "0"
 export PERL_HASH_SEED = "0"
 export SOURCE_DATE_EPOCH ?= "${@get_source_date_epoch_value(d)}"
 # A SOURCE_DATE_EPOCH of '0' might be misinterpreted as no SDE
+# If these 2 are misaligned, when generating fstab later on wic, the timestamps will never match because
+# The repeatibility uses the REPRODUCIBLE_TIMESTAMP_ROOTFS while wic uses SOURCE_DATE_EPOCH (therefore its fallback)
 SOURCE_DATE_EPOCH_FALLBACK ??= "1302044400"
-REPRODUCIBLE_TIMESTAMP_ROOTFS ??= "1520598896"
+REPRODUCIBLE_TIMESTAMP_ROOTFS ??= "1302044400"
 
 ##################################################################
 # Settings used by bitbake-layers.
scripts/lib/wic/partition.py +1 -14
index bf2c34d594..82d754835c 100644
@@ -131,7 +131,7 @@ class Partition():
         partition command parameters.
         """
         self.updated_fstab_path = updated_fstab_path
-        if self.updated_fstab_path and not (self.fstype.startswith("ext") or self.fstype == "msdos"):
+        if self.updated_fstab_path:
             self.update_fstab_in_rootfs = True
 
         if not self.source:
@@ -295,15 +295,6 @@ class Partition():
             (self.fstype, extraopts, rootfs, label_str, self.fsuuid, rootfs_dir)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
-        if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
-            debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
-            with open(debugfs_script_path, "w") as f:
-                f.write("cd etc\n")
-                f.write("rm fstab\n")
-                f.write("write %s fstab\n" % (self.updated_fstab_path))
-            debugfs_cmd = "debugfs -w -f %s %s" % (debugfs_script_path, rootfs)
-            exec_native_cmd(debugfs_cmd, native_sysroot)
-
         mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
@@ -400,10 +391,6 @@ class Partition():
         mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
         exec_native_cmd(mcopy_cmd, native_sysroot)
 
-        if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
-            mcopy_cmd = "mcopy -m -i %s %s ::/etc/fstab" % (rootfs, self.updated_fstab_path)
-            exec_native_cmd(mcopy_cmd, native_sysroot)
-
         chmod_cmd = "chmod 644 %s" % rootfs
         exec_cmd(chmod_cmd)
 
scripts/lib/wic/plugins/source/rootfs.py +2 -2
index e29f3a4c2f..3848af2a91 100644
@@ -223,8 +223,8 @@ class RootfsPlugin(SourcePlugin):
             part.has_fstab = os.path.exists(os.path.join(new_rootfs, "etc/fstab"))
             if part.update_fstab_in_rootfs and part.has_fstab and not part.no_fstab_update:
                 fstab_path = os.path.join(new_rootfs, "etc/fstab")
-                # Assume that fstab should always be owned by root with fixed permissions
-                install_cmd = "install -m 0644 -p %s %s" % (part.updated_fstab_path, fstab_path)
+                # We dont want any metadata of the updated fstab, just the one that already existed
+                install_cmd = "cp --no-preserve=all %s %s" % (part.updated_fstab_path, fstab_path)
                 if new_pseudo:
                     pseudo = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo)
                 else:
#2 lore raw

Re: [OE-core] [PATCH 1/1] wic: Content of the temporary updated fstab should be copied into the original not replacing it entirely.

Quoted 8 lines · On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via lists.openembedded.org…

From: Daniel Andrade <[email protected]>

Fixes [15947]
The current functionality to update fstab generates a new temporary fstab with the new partition configuration.
However, this file does not retain any metadata being it a completely new file.
Because of this, when the rootfs plugin under poky/scripts/lib/wic/plugins/source/rootfs.py copies the file, it overrides the original fstab metadata. The patch removes implementation of msdos/ext code for fstab since it is not possible to append content without rewriting the file to preserve metadata.

Quoted 6 lines

The timestamp applied to fstab is not the same as every other file.
It seems like the SOURCE_DATE_EPOCH variable goes to the fallback timestamp SOURCE_DATE_EPOCH_FALLBACK.
Since that variable is used everywhere, it is not the same value as REPRODUCIBLE_TIMESTAMP_ROOTFS under poky/meta/conf/bitbake.conf that is applied in every other file.

---

Hi Daniel,

I finally had a bit of time to have a deeper look at these changes and at the wic.Wic.test_no_fstab_update test. It turns out the test is correct and your patch is indeed introducing a regression.

So our first assumption was the test was wrong, as changing the filesystem resulted in a test fail. But actually, if you change the filesystem in the wic command file, you also have to change the command extracting the file from the filesystem. E.g. for squashfs, you need to replace "debugfs -R cat" with "sqfscat".

Now talking about your patch, my comments below.

Quoted 30 lines
scripts/lib/wic/partition.py +1 -10
index bf2c34d594..82d754835c 100644
@@ -131,7 +131,7 @@ class Partition():
         partition command parameters.
         """
         self.updated_fstab_path = updated_fstab_path
-        if self.updated_fstab_path and not (self.fstype.startswith("ext") or self.fstype == "msdos"):
+        if self.updated_fstab_path:
             self.update_fstab_in_rootfs = True
 
         if not self.source:
@@ -295,15 +295,6 @@ class Partition():
             (self.fstype, extraopts, rootfs, label_str, self.fsuuid, rootfs_dir)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
-        if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
-            debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
-            with open(debugfs_script_path, "w") as f:
-                f.write("cd etc\n")
-                f.write("rm fstab\n")
-                f.write("write %s fstab\n" % (self.updated_fstab_path))
-            debugfs_cmd = "debugfs -w -f %s %s" % (debugfs_script_path, rootfs)
-            exec_native_cmd(debugfs_cmd, native_sysroot)
-
         mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 

scripts/lib/wic/partition.py class Partition(): · line 396

Quoted 10 lines
scripts/lib/wic/partition.py +0 -4
@@ -400,10 +391,6 @@ class Partition():
         mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
         exec_native_cmd(mcopy_cmd, native_sysroot)
 
-        if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
-            mcopy_cmd = "mcopy -m -i %s %s ::/etc/fstab" % (rootfs, self.updated_fstab_path)
-            exec_native_cmd(mcopy_cmd, native_sysroot)
-
         chmod_cmd = "chmod 644 %s" % rootfs
         exec_cmd(chmod_cmd)
 

So you are removing an optimisation on ext and fat filesystems. This might raise further questions from other reviewers, but if that's needed, why not.

scripts/lib/wic/plugins/source/rootfs.py class RootfsPlugin(SourcePlugin): · line 227

Quoted 12 lines
scripts/lib/wic/plugins/source/rootfs.py +2 -2
index e29f3a4c2f..3848af2a91 100644
@@ -223,8 +223,8 @@ class RootfsPlugin(SourcePlugin):
             part.has_fstab = os.path.exists(os.path.join(new_rootfs, "etc/fstab"))
             if part.update_fstab_in_rootfs and part.has_fstab and not part.no_fstab_update:
                 fstab_path = os.path.join(new_rootfs, "etc/fstab")
-                # Assume that fstab should always be owned by root with fixed permissions
-                install_cmd = "install -m 0644 -p %s %s" % (part.updated_fstab_path, fstab_path)
+                # We dont want any metadata of the updated fstab, just the one that already existed
+                install_cmd = "cp --no-preserve=all %s %s" % (part.updated_fstab_path, fstab_path)

Now this is what makes the test fail. So I understand the switch from install to cp, as it will reuse the same inode instead of creating a new one. But this is also what causes the test regression.

When creating a filesystem image with a modified fstab, a copy of the filesystem content is first made, so file modifications only affect this specific image. But in order to gain time and disk space, this copy is made using hardlinks: see use of copyhardlinktree() in do_prepare_partition() from scripts/lib/wic/plugins/source/rootfs.py. So as cp will keep the same inode, it means you will modify the content on both the copy and the reference, and so all further generated image.

So I'm sorry, we will have to find another way to implement this change.

Thanks,
Mathieu

Signature · 3 lines
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#3 lore raw

Re: [OE-core] [PATCH 1/1] wic: Content of the temporary updated fstab should be copied into the original not replacing it entirely.

Hi Mathieu,

Thank you for the you time testing this thoroughly.
After some test, I was able to provide a new alternative (attached to this message if everything goes well).

For the performance part of ext* and msdos fstype I have nothing to counter argument. I did not find a way to add xattrs using those methods and, according to what seemed to be the natural way of the code design of wic, I let all the fstype flal under rootfs.py. Any other solution is welcome!

For the second part, this time, instead of using cp, I let the install be like it was before. However, instead of running it under the pseudo environment, I executed it as the host machine. This way, I was able to recheck the database using pseudo, which will fail and therefore led the files.db to be rebuilt. This happens because he is able to detect the inodes of fstab entry on all the tables do not match and update them accordingly. This way we don't run the risk of modifying the original content of fstab under the normal yocto output files because the file is actually not the same.

I also ran oe-selftest -r wic.Wic.test_no_fstab_update on a core-image-minimal image with the patch and it seemed to have passed.

Best Regards,
Daniel

#4 lore raw

PATCH 1/1 Fstab: Fix xattrs not being maintained on fstab file when using wic fstab update funtionalities

From: Daniel Andrade <[email protected]>

The wic fstab generation capability was removing the xattrs of the /etc/fstab, possibly rendering the produced image unusable in some case (e.g.: SELinux enforced on a read-only rootfs). The file appears as unlabeled no matter the fstype chosen.
To bypass this, some late procedures done on ext* and msdos fstype have been removed and all the fstab logic was passed to rootfs.py.
On this one, the idea is to still use the install command, but in this case, executed outside the pseudo environment. After the file as sucessfully been place, the idea was to check again for inconsistencies in the database (now the fstab inode is different) and force it to rebuild with the correct inode, ensure fstab inode is correctly set on all the tables, including the xattrs one.

---
 scripts/lib/wic/partition.py             | 15 +--------------
 scripts/lib/wic/plugins/source/rootfs.py |  8 ++++----
 2 files changed, 5 insertions(+), 18 deletions(-)
scripts/lib/wic/partition.py +1 -14
index bf2c34d594..82d754835c 100644
@@ -131,7 +131,7 @@ class Partition():
         partition command parameters.
         """
         self.updated_fstab_path = updated_fstab_path
-        if self.updated_fstab_path and not (self.fstype.startswith("ext") or self.fstype == "msdos"):
+        if self.updated_fstab_path:
             self.update_fstab_in_rootfs = True
 
         if not self.source:
@@ -295,15 +295,6 @@ class Partition():
             (self.fstype, extraopts, rootfs, label_str, self.fsuuid, rootfs_dir)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
-        if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
-            debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
-            with open(debugfs_script_path, "w") as f:
-                f.write("cd etc\n")
-                f.write("rm fstab\n")
-                f.write("write %s fstab\n" % (self.updated_fstab_path))
-            debugfs_cmd = "debugfs -w -f %s %s" % (debugfs_script_path, rootfs)
-            exec_native_cmd(debugfs_cmd, native_sysroot)
-
         mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
@@ -400,10 +391,6 @@ class Partition():
         mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
         exec_native_cmd(mcopy_cmd, native_sysroot)
 
-        if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
-            mcopy_cmd = "mcopy -m -i %s %s ::/etc/fstab" % (rootfs, self.updated_fstab_path)
-            exec_native_cmd(mcopy_cmd, native_sysroot)
-
         chmod_cmd = "chmod 644 %s" % rootfs
         exec_cmd(chmod_cmd)
 
scripts/lib/wic/plugins/source/rootfs.py +4 -4
index e29f3a4c2f..c0d9a28b91 100644
@@ -225,11 +225,11 @@ class RootfsPlugin(SourcePlugin):
                 fstab_path = os.path.join(new_rootfs, "etc/fstab")
                 # Assume that fstab should always be owned by root with fixed permissions
                 install_cmd = "install -m 0644 -p %s %s" % (part.updated_fstab_path, fstab_path)
+                exec_native_cmd(install_cmd, native_sysroot)
                 if new_pseudo:
-                    pseudo = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo)
-                else:
-                    pseudo = None
-                exec_native_cmd(install_cmd, native_sysroot, pseudo)
+                    pseudo_cmd = "%s -B " % (cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo))
+                    exec_native_cmd(pseudo_cmd, native_sysroot)
+
 
         part.prepare_rootfs(cr_workdir, oe_builddir,
                             new_rootfs or part.rootfs_dir, native_sysroot,
#5 lore raw

Re: [PATCH 1/1] Fstab: Fix xattrs not being maintained on fstab file when using wic fstab update funtionalities

Quoted 10 lines · On Fri Oct 17, 2025 at 1:04 PM CEST, dani.barra25 wrote:

From: Daniel Andrade <[email protected]>

The wic fstab generation capability was removing the xattrs of the /etc/fstab, possibly rendering the produced image unusable in some case (e.g.: SELinux enforced on a read-only rootfs). The file appears as unlabeled no matter the fstype chosen.
To bypass this, some late procedures done on ext* and msdos fstype have been removed and all the fstab logic was passed to rootfs.py.
On this one, the idea is to still use the install command, but in this case, executed outside the pseudo environment. After the file as sucessfully been place, the idea was to check again for inconsistencies in the database (now the fstab inode is different) and force it to rebuild with the correct inode, ensure fstab inode is correctly set on all the tables, including the xattrs one.

---

Hi Daniel,

Thanks for the new version. It looks like this one is now breaking wic.Wic.test_exclude_path and wic.Wic.test_include_path tests:

2025-10-19 07:55:18,010 - oe-selftest - INFO - wic.Wic.test_exclude_path (subunit.RemotedTestCase)
2025-10-19 07:55:18,011 - oe-selftest - INFO -  ... FAIL

...
AssertionError: Command 'wic create temp.wks -e core-image-minimal -o /srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/wic-tmp' returned non-zero exit status 1:
...
ERROR: _exec_cmd: export PATH=/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/sbin:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/usr/sbin:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/usr/bin:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/bin:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/hosttools:$PATH;export PSEUDO_PREFIX=/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/usr;export PSEUDO_LOCALSTATEDIR=/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/wic-tmp/tmp.wic.dhlnwwia/pseudo1;export PSEUDO_PASSWD=/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/wic-tmp/tmp.wic.dhlnwwia/rootfs1;export PSEUDO_NOSYMLINKEXP=1;/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/sysroots-components/x86_64/pseudo-native/usr/bin/pseudo -B returned '1' instead of 0
output: PRAGMA journal_mode = OFF;: database is locked
PRAGMA synchronous = OFF;: database is locked
Failed: database is locked
Error getting 'logs.db' database.
PRAGMA journal_mode = OFF;: database is locked
PRAGMA synchronous = OFF;: database is locked
Failed: database is locked
Error getting 'files.db' database.
error during load from disk: database is locked
pdb_files: database error.
Couldn't start file list, can't scan.
db cleanup for server shutdown, 07:55:17.956
memory-to-file backup complete, 07:55:17.956.
db cleanup finished, 07:55:17.956
considering table files
considering table xattrs
considering table migrations

https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2574

Can you have a look at these errors?

Thanks,
Mathieu

Signature · 3 lines
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

A message that is not in the archive (sent off-list or not yet received): [email protected]

#7 lore raw

PATCH 1/1 Fstab xattrs: This update ensures that fstab xattrs are correctly updated without adding a lock to the database.

From: Daniel Andrade <[email protected]>

The process of repairing the database using -B flag of pseudo sometimes causes errors because the database lock file is present. Therefore, to fix it, first we ensure that any connection to files.db is closed using pseudo flag -S followed by the actual command to repair it after fstab was successfully updated.

---
 scripts/lib/wic/partition.py             | 15 +--------------
 scripts/lib/wic/plugins/source/rootfs.py | 19 ++++++++++++++-----
 2 files changed, 15 insertions(+), 19 deletions(-)
scripts/lib/wic/partition.py +1 -14
index bf2c34d594..82d754835c 100644
@@ -131,7 +131,7 @@ class Partition():
         partition command parameters.
         """
         self.updated_fstab_path = updated_fstab_path
-        if self.updated_fstab_path and not (self.fstype.startswith("ext") or self.fstype == "msdos"):
+        if self.updated_fstab_path:
             self.update_fstab_in_rootfs = True
 
         if not self.source:
@@ -295,15 +295,6 @@ class Partition():
             (self.fstype, extraopts, rootfs, label_str, self.fsuuid, rootfs_dir)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
-        if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
-            debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
-            with open(debugfs_script_path, "w") as f:
-                f.write("cd etc\n")
-                f.write("rm fstab\n")
-                f.write("write %s fstab\n" % (self.updated_fstab_path))
-            debugfs_cmd = "debugfs -w -f %s %s" % (debugfs_script_path, rootfs)
-            exec_native_cmd(debugfs_cmd, native_sysroot)
-
         mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
         exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
 
@@ -400,10 +391,6 @@ class Partition():
         mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
         exec_native_cmd(mcopy_cmd, native_sysroot)
 
-        if self.updated_fstab_path and self.has_fstab and not self.no_fstab_update:
-            mcopy_cmd = "mcopy -m -i %s %s ::/etc/fstab" % (rootfs, self.updated_fstab_path)
-            exec_native_cmd(mcopy_cmd, native_sysroot)
-
         chmod_cmd = "chmod 644 %s" % rootfs
         exec_cmd(chmod_cmd)
 
scripts/lib/wic/plugins/source/rootfs.py +14 -5
index e29f3a4c2f..09446baef2 100644
@@ -225,12 +225,21 @@ class RootfsPlugin(SourcePlugin):
                 fstab_path = os.path.join(new_rootfs, "etc/fstab")
                 # Assume that fstab should always be owned by root with fixed permissions
                 install_cmd = "install -m 0644 -p %s %s" % (part.updated_fstab_path, fstab_path)
+                exec_native_cmd(install_cmd, native_sysroot)
+ 
                 if new_pseudo:
-                    pseudo = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo)
-                else:
-                    pseudo = None
-                exec_native_cmd(install_cmd, native_sysroot, pseudo)
-
+                    pseudo_prefix = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo)
+                   
+                    # Shutdown any existing pseudo server
+                    # This ensures no other processes are using the database
+                    shutdown_cmd = "%s -S" % pseudo_prefix
+                    exec_native_cmd(shutdown_cmd, native_sysroot)
+                   
+                    # Database is not locked anymore
+                    # Repairs inodes related to fstab. This way xattrs and metadata is correctly applied to the new file
+                    repair_cmd = "%s -B" % pseudo_prefix
+                    exec_native_cmd(repair_cmd, native_sysroot)
+ 
         part.prepare_rootfs(cr_workdir, oe_builddir,
                             new_rootfs or part.rootfs_dir, native_sysroot,
                             pseudo_dir = new_pseudo or pseudo_dir)
#8 lore raw

Re: [OE-core] [PATCH] Fstab xattrs: This update ensures that fstab xattrs are correctly updated without adding a lock to the database.

scripts/lib/wic/plugins/source/rootfs.py class RootfsPlugin(SourcePlugin): · line 242

Quoted 12 lines · On Sun, 19 Oct 2025 at 23:45, dani.barra25 via lists.openembedded.org…
scripts/lib/wic/plugins/source/rootfs.py +12 -0
+                    pseudo_prefix = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo)
+
+                    # Shutdown any existing pseudo server
+                    # This ensures no other processes are using the database
+                    shutdown_cmd = "%s -S" % pseudo_prefix
+                    exec_native_cmd(shutdown_cmd, native_sysroot)
+
+                    # Database is not locked anymore
+                    # Repairs inodes related to fstab. This way xattrs and metadata is correctly applied to the new file
+                    repair_cmd = "%s -B" % pseudo_prefix
+                    exec_native_cmd(repair_cmd, native_sysroot)
+

Is it possible to describe how to reproduce and observe the issue that this patch aims to fix? This snippet makes me *very* nervous as pseudo is used by mulitple bitbake tasks in parallel, and doing shutdowns and repairs in the middle of it can blow up spectacularly.

Alex

#9 lore raw

Re: [OE-core] [PATCH] Fstab xattrs: This update ensures that fstab xattrs are correctly updated without adding a lock to the database.

Hello Alex,
Yes I can. Maybe I am doing something wrong since the patch is an alternative to past threads but somehow it gets separated without context.

Starting from the beginning.
My original patch aimed to fix a simple thing. No matter what fstype you were using, no xattrs from the build process were assigned to /etc/fstab if you had multiple partitions. And this was particularly problematic when you have read-only rootfs like squashfs and you need to have all the metadata available during build, like the SELinux file labels, since they cannot be set during boot time.
This is because it creates a temp folder and then creates a new fstab that is later replaced in the image.

The replacement is made via 2 options:
1. If it was ext* or msdos, it will fall under specific methods for those types that will open the already built image and remove the fstab and writing the new one, losing all the metadata of that file.
2. It falls under the "install" cmd of rootfs.py and this one was being executed inside the pseudo environment. This means the original entry of fstab on files db is completely replace with this new file and all the xattrs are gone

To avoid this, my first patch removed the debugfs specifics of ext* and msdos, since those did not leveragy any "write only content" mechanism without replacing the actual file. Then, all the fstab modifications would lie under rootfs.py
which seemed to be the common point for everything. On this I simply replaced "install" with "cp" since I only wanted to modify the contents.
https://lists.openembedded.org/g/openembedded-core/topic/115043355
Later in the discussion, Mathieu correctly pointed out that it could not be used because it fails a test due to the usage of copyhardlinktree() for this temporary rootfs folder. This meant that I was not only modifying the temporary fstab, but also the original one,
leading to a crash of wic.Wic.test_no_fstab_update.
https://lists.openembedded.org/g/openembedded-core/topic/115043356

The second approach I took was looking at how pseudo behaved. This time, I decided to maintain the original install cmd, but instead of running it under fakeroot, I ran it on the host system. This meant that, if a later check of the db occurred, it would correctly detect
an inode change on fstab and therefore it could fix it using the -B option. And that is what I did :
https://lists.openembedded.org/g/openembedded-core/topic/115805391

I only neglected the fact that, in reality, other tests could have failed because of this. And, in fact, it happened. Because of this, tests regarding including and excluding paths were failing because the db was locked.
That's when I reached the current patch alternative.
I knew it could cause problems but honestly I never considered it would and when I run wic tests, I didn't see any failure. In reality, I was trying to replicate what was being done a few lines above with the options -B -m and -M. The difference
is that pseudo doesn't have a flag to repair a single file, therefore this was the only solution found.

A good alternative to this would be pseudo actually having a functionality to repair single file inodes, avoiding to completely check the whole database and therefore better management on this side.
In the end this bug rendered all my images that enforced SELinux with read only rootfs (squashfs) useless and I'd assume I might not be the only one with this issue.

Another thing I notice is that, for repeatability purposes, there are multiple variables being used to timestamp files. This is just a recommendation/question. Shouldn't all those variable timestamps be aligned also?

In any case, if you have some other idea on how to approach this, let me know!

Daniel

Alexander Kanavin <[email protected]> escreveu (segunda, 20/10/2025 à(s) 13:09):

Quoted 26 lines
Quoted 1 line · On Sun, 19 Oct 2025 at 23:45, dani.barra25 via lists.openembedded.org…

+ pseudo_prefix = cls.__get_pseudo(native_sysroot,

new_rootfs, new_pseudo)

scripts/lib/wic/plugins/source/rootfs.py class RootfsPlugin(SourcePlugin):

Quoted 3 lines
scripts/lib/wic/plugins/source/rootfs.py +3 -0
+
+                    # Shutdown any existing pseudo server
+                    # This ensures no other processes are using the

database

scripts/lib/wic/plugins/source/rootfs.py class RootfsPlugin(SourcePlugin):

Quoted 5 lines
scripts/lib/wic/plugins/source/rootfs.py +5 -0
+                    shutdown_cmd = "%s -S" % pseudo_prefix
+                    exec_native_cmd(shutdown_cmd, native_sysroot)
+
+                    # Database is not locked anymore
+                    # Repairs inodes related to fstab. This way xattrs

and metadata is correctly applied to the new file

scripts/lib/wic/plugins/source/rootfs.py class RootfsPlugin(SourcePlugin): · line 242

Quoted 3 lines
scripts/lib/wic/plugins/source/rootfs.py +3 -0
+                    repair_cmd = "%s -B" % pseudo_prefix
+                    exec_native_cmd(repair_cmd, native_sysroot)
+

Is it possible to describe how to reproduce and observe the issue that this patch aims to fix? This snippet makes me *very* nervous as pseudo is used by mulitple bitbake tasks in parallel, and doing shutdowns and repairs in the middle of it can blow up spectacularly.

Alex

#10 lore raw

Re: [OE-core] [PATCH] Fstab xattrs: This update ensures that fstab xattrs are correctly updated without adding a lock to the database.

In reply to Alexander Kanavin
Quoted 5 lines · Alexander Kanavin via lists.openembedded.org

Is it possible to describe how to reproduce and observe the issue that this patch aims to fix? This snippet makes me *very* nervous as pseudo is used by mulitple bitbake tasks in parallel, and doing shutdowns and repairs in the middle of it can blow up spectacularly.

Agreed. If you’re building a single image then yes, the wic tasks will typically happen at the end with nothing else in parallel. But this is not a guarantee and killing pseudo whilst other recipes might be mid-build is not acceptable.

Ross

#11 lore raw

Re: [PATCH 1/1] Fstab: Fix xattrs not being maintained on fstab file when using wic fstab update funtionalities

In reply to Mathieu Dubois-Briand

Hi Mathieu,

I did not notice that it was failing other tests. It seemed like the error was caused due to a lock file. To fix it, before issuing the repair of the database, I first shutdown any open connection using -S. Doing this I saw that the 3 tests passed and fstab was correctly labeled.
The other thing I noticed was the different timestamps, but as I said a while ago, it was caused due to using SOURCE_DATE_EPOCH,

REPRODUCIBLE_TIMESTAMP_ROOTFS, and SOURCE_DATE_EPOCH_FALLBACK. This should also be reviewed since it might compromise build repeatability.

In any case, it was not the topic of the patch, I just wanted to state this in case someone checked the contents of the produced image.

Best Regards,
Daniel

A domingo, 19/10/2025, 11:02, Mathieu Dubois-Briand < [email protected]> escreveu:

Quoted 70 lines · On Fri Oct 17, 2025 at 1:04 PM CEST, dani.barra25 wrote:
Quoted 3 lines

From: Daniel Andrade <[email protected]>

The wic fstab generation capability was removing the xattrs of the

/etc/fstab, possibly rendering the produced image unusable in some case (e.g.: SELinux enforced on a read-only rootfs).

Quoted 2 lines

The file appears as unlabeled no matter the fstype chosen. To bypass this, some late procedures done on ext* and msdos fstype have

been removed and all the fstab logic was passed to rootfs.py.

Quoted 1 line

On this one, the idea is to still use the install command, but in this

case, executed outside the pseudo environment. After the file as sucessfully been place, the idea was to check again for inconsistencies

Quoted 1 line

in the database (now the fstab inode is different) and force it to

rebuild with the correct inode, ensure fstab inode is correctly set on all the tables, including the xattrs one.

Quoted 3 lines

---

Hi Daniel,

Thanks for the new version. It looks like this one is now breaking wic.Wic.test_exclude_path and wic.Wic.test_include_path tests:

2025-10-19 07:55:18,010 - oe-selftest - INFO - wic.Wic.test_exclude_path
(subunit.RemotedTestCase)
2025-10-19 07:55:18,011 - oe-selftest - INFO -  ... FAIL

...
AssertionError: Command 'wic create temp.wks -e core-image-minimal -o
/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/wic-tmp'
returned non-zero exit status 1:
...
ERROR: _exec_cmd: export PATH=/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/sbin:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/usr/sbin:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/usr/bin:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/usr/bin/x86_64-poky-linux:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/bin:/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/hosttools:$PATH;export PSEUDO_PREFIX=/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/work/x86-64-v3-poky-linux/wic-tools/1.0/recipe-sysroot-native/usr;export
PSEUDO_LOCALSTATEDIR=/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/wic-tmp/tmp.wic.dhlnwwia/pseudo1;export
PSEUDO_PASSWD=/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/wic-tmp/tmp.wic.dhlnwwia/rootfs1;export
PSEUDO_NOSYMLINKEXP=1;/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3133304/tmp/sysroots-components/x86_64/pseudo-native/usr/bin/pseudo
-B returned '1' instead of 0
output: PRAGMA journal_mode = OFF;: database is locked
PRAGMA synchronous = OFF;: database is locked
Failed: database is locked
Error getting 'logs.db' database.
PRAGMA journal_mode = OFF;: database is locked
PRAGMA synchronous = OFF;: database is locked
Failed: database is locked
Error getting 'files.db' database.
error during load from disk: database is locked
pdb_files: database error.
Couldn't start file list, can't scan.
db cleanup for server shutdown, 07:55:17.956
memory-to-file backup complete, 07:55:17.956.
db cleanup finished, 07:55:17.956
considering table files
considering table xattrs
considering table migrations

https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2574

Can you have a look at these errors?

Thanks,
Mathieu

--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

#12 lore raw
In reply to [email protected]
Quoted 9 lines · On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via lists.openembedded.org…

From: Daniel Andrade <[email protected]>

Using install in the rootfs plugin forces fstab to be replaced entirely, meaning that even its Inodes will change, leading xattrs and SELinux context stored by pseudo not to be applied.
The fix just uses cp without preserving attributes from the temporary fstab since none of them are needed, just the content.
Same thing happens with the predefined mechanisms for ext4 and msdos. Using debugfs there is no way to replace contents while maintaining metadata, so the approach taken on the path was to remove the different fstab logic for those fstypes and also use the same modified cp command. Reviewing the builds I did it seems to work for all of the fstypes.

Another problem is that the timestamp applied to fstab is not the same as every other file. It seems like the SOURCE_DATE_EPOCH variable goes to the fallback timestamp (SOURCE_DATE_EPOCH_FALLBACK). Since you are using that variable everywhere, it is not the same value as REPRODUCIBLE_TIMESTAMP_ROOTFS under poky/meta/conf/bitbake.conf that is applied in every other file.

Quoted 8 lines

Daniel Andrade (1):
wic: Content of the temporary updated fstab should be copied into the
original not replacing it entirely.

 meta/conf/bitbake.conf                   |  4 +++-
 scripts/lib/wic/partition.py             | 15 +--------------
 scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
 3 files changed, 6 insertions(+), 17 deletions(-)

Hi Daniel,

Thanks for your patch.

It looks like it is breaking a test:

2025-09-04 15:28:11,112 - oe-selftest - INFO - wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)
2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
...
2025-09-04 15:28:11,114 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py", line 859, in test_no_fstab_update
    self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
  File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python3.12/unittest/case.py", line 1251, in assertMultiLineEqual
    self.fail(self._formatMessage(msg, standardMsg))
  File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
    raise self.failureException(msg)

AssertionError: 'af3c087d6c9131735c8d1f270a226892' != '9edb8255abd217fdb20e118833afb856'
- af3c087d6c9131735c8d1f270a226892
+ 9edb8255abd217fdb20e118833afb856

https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268

Can you fix it please?

Thanks,
Mathieu

Signature · 3 lines
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#13 lore raw
Quoted 36 lines · Mathieu Dubois-Briand via lists.openembedded.org
Quoted 9 lines · On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via lists.openembedded.org…

From: Daniel Andrade<[email protected]>

Using install in the rootfs plugin forces fstab to be replaced entirely, meaning that even its Inodes will change, leading xattrs and SELinux context stored by pseudo not to be applied.
The fix just uses cp without preserving attributes from the temporary fstab since none of them are needed, just the content.
Same thing happens with the predefined mechanisms for ext4 and msdos. Using debugfs there is no way to replace contents while maintaining metadata, so the approach taken on the path was to remove the different fstab logic for those fstypes and also use the same modified cp command. Reviewing the builds I did it seems to work for all of the fstypes.

Another problem is that the timestamp applied to fstab is not the same as every other file. It seems like the SOURCE_DATE_EPOCH variable goes to the fallback timestamp (SOURCE_DATE_EPOCH_FALLBACK). Since you are using that variable everywhere, it is not the same value as REPRODUCIBLE_TIMESTAMP_ROOTFS under poky/meta/conf/bitbake.conf that is applied in every other file.

Quoted 8 lines

Daniel Andrade (1):
wic: Content of the temporary updated fstab should be copied into the
original not replacing it entirely.

  meta/conf/bitbake.conf                   |  4 +++-
  scripts/lib/wic/partition.py             | 15 +--------------
  scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
  3 files changed, 6 insertions(+), 17 deletions(-)

Hi Daniel,

Thanks for your patch.

It looks like it is breaking a test:

2025-09-04 15:28:11,112 - oe-selftest - INFO - wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)
2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
...
2025-09-04 15:28:11,114 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
   File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py", line 859, in test_no_fstab_update
     self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
   File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
     assertion_func(first, second, msg=msg)
   File "/usr/lib/python3.12/unittest/case.py", line 1251, in assertMultiLineEqual
     self.fail(self._formatMessage(msg, standardMsg))
   File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
     raise self.failureException(msg)
Quoted 8 lines

AssertionError: 'af3c087d6c9131735c8d1f270a226892' != '9edb8255abd217fdb20e118833afb856'
- af3c087d6c9131735c8d1f270a226892
+ 9edb8255abd217fdb20e118833afb856

https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268

Can you fix it please?

Ping?

I think this is being tracked by:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15947

"WIC does not preserve metadata when updating fstab"

Btw, we were just following the "Need Info" process during the bug review meeting so that's why I'm sending this email.

../Randy

Quoted 13 lines

Thanks,
Mathieu

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#222962):https://lists.openembedded.org/g/openembedded-core/message/222962 Mute This Topic:https://lists.openembedded.org/mt/115043355/3616765
Group Owner:[email protected] Unsubscribe:https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 2 lines
# Randy MacLeod
# Wind River Linux
#14 lore raw

Hello guys, sorry for the delay.
I have been busy, but I hope to provide you with a fix in the next few days.

Sorry about that,
Daniel

Randy MacLeod <[email protected]> escreveu (quinta, 11/09/2025 à(s) 17:56):

Quoted 76 lines

On 2025-09-04 11:58 a.m., Mathieu Dubois-Briand via lists.openembedded.org wrote:

On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via lists.openembedded.org wrote:

From: Daniel Andrade <[email protected]> <[email protected]>

Using install in the rootfs plugin forces fstab to be replaced entirely, meaning that even its Inodes will change, leading xattrs and SELinux context stored by pseudo not to be applied.
The fix just uses cp without preserving attributes from the temporary fstab since none of them are needed, just the content.
Same thing happens with the predefined mechanisms for ext4 and msdos. Using debugfs there is no way to replace contents while maintaining metadata, so the approach taken on the path was to remove the different fstab logic for those fstypes and also use the same modified cp command. Reviewing the builds I did it seems to work for all of the fstypes.

Another problem is that the timestamp applied to fstab is not the same as every other file. It seems like the SOURCE_DATE_EPOCH variable goes to the fallback timestamp (SOURCE_DATE_EPOCH_FALLBACK). Since you are using that variable everywhere, it is not the same value as REPRODUCIBLE_TIMESTAMP_ROOTFS under poky/meta/conf/bitbake.conf that is applied in every other file.

Daniel Andrade (1):
wic: Content of the temporary updated fstab should be copied into the
original not replacing it entirely.

 meta/conf/bitbake.conf                   |  4 +++-
 scripts/lib/wic/partition.py             | 15 +--------------
 scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
 3 files changed, 6 insertions(+), 17 deletions(-)

Hi Daniel,

Thanks for your patch.

It looks like it is breaking a test:

2025-09-04 15:28:11,112 - oe-selftest - INFO - wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)
2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
...
2025-09-04 15:28:11,114 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py", line 859, in test_no_fstab_update
    self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
  File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python3.12/unittest/case.py", line 1251, in assertMultiLineEqual
    self.fail(self._formatMessage(msg, standardMsg))
  File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
    raise self.failureException(msg)

AssertionError: 'af3c087d6c9131735c8d1f270a226892' != '9edb8255abd217fdb20e118833afb856'
- af3c087d6c9131735c8d1f270a226892
+ 9edb8255abd217fdb20e118833afb856
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268

Can you fix it please?

Ping?

I think this is being tracked by:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15947

"WIC does not preserve metadata when updating fstab"

Btw, we were just following the "Need Info" process during the bug review meeting
so that's why I'm sending this email.

../Randy

Thanks,
Mathieu

--
# Randy MacLeod
# Wind River Linux

#15 lore raw

Good Afternoon,

I have been trying to figure it out and something seems off. The specific test you refer to may be hiding a problem or maybe I have my testbench compromised and I would like your help to test.
The test test_no_fstab_update verifies if the fstab was updated or not because on the wks file used to do it there is a --no-fstab-update. The thing is, I think this test is passing because what is forcing it to not update is a constraint that I removed previously on the patch (the one that checks if the system is ext* or msdos) and not because the flag is triggering its intended functionality.
The crosscheck I did was modifying the fstype from ext4 to squashfs and indeed the test failed.
Can someone also verify this?

Daniel

Daniel Andrade <[email protected]> escreveu (sábado, 13/09/2025 à(s) 11:02):

Quoted 86 lines

Hello guys, sorry for the delay.
I have been busy, but I hope to provide you with a fix in the next few days.

Sorry about that,
Daniel

Randy MacLeod <[email protected]> escreveu (quinta, 11/09/2025 à(s) 17:56):

Quoted 76 lines

On 2025-09-04 11:58 a.m., Mathieu Dubois-Briand via lists.openembedded.org wrote:

On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via lists.openembedded.org wrote:

From: Daniel Andrade <[email protected]> <[email protected]>

Using install in the rootfs plugin forces fstab to be replaced entirely, meaning that even its Inodes will change, leading xattrs and SELinux context stored by pseudo not to be applied.
The fix just uses cp without preserving attributes from the temporary fstab since none of them are needed, just the content.
Same thing happens with the predefined mechanisms for ext4 and msdos. Using debugfs there is no way to replace contents while maintaining metadata, so the approach taken on the path was to remove the different fstab logic for those fstypes and also use the same modified cp command. Reviewing the builds I did it seems to work for all of the fstypes.

Another problem is that the timestamp applied to fstab is not the same as every other file. It seems like the SOURCE_DATE_EPOCH variable goes to the fallback timestamp (SOURCE_DATE_EPOCH_FALLBACK). Since you are using that variable everywhere, it is not the same value as REPRODUCIBLE_TIMESTAMP_ROOTFS under poky/meta/conf/bitbake.conf that is applied in every other file.

Daniel Andrade (1):
wic: Content of the temporary updated fstab should be copied into the
original not replacing it entirely.

 meta/conf/bitbake.conf                   |  4 +++-
 scripts/lib/wic/partition.py             | 15 +--------------
 scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
 3 files changed, 6 insertions(+), 17 deletions(-)

Hi Daniel,

Thanks for your patch.

It looks like it is breaking a test:

2025-09-04 15:28:11,112 - oe-selftest - INFO - wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)
2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
...
2025-09-04 15:28:11,114 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py", line 859, in test_no_fstab_update
    self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
  File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python3.12/unittest/case.py", line 1251, in assertMultiLineEqual
    self.fail(self._formatMessage(msg, standardMsg))
  File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
    raise self.failureException(msg)

AssertionError: 'af3c087d6c9131735c8d1f270a226892' != '9edb8255abd217fdb20e118833afb856'
- af3c087d6c9131735c8d1f270a226892
+ 9edb8255abd217fdb20e118833afb856
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268

Can you fix it please?

Ping?

I think this is being tracked by:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15947

"WIC does not preserve metadata when updating fstab"

Btw, we were just following the "Need Info" process during the bug review meeting
so that's why I'm sending this email.

../Randy

Thanks,
Mathieu

--
# Randy MacLeod
# Wind River Linux

#16 lore raw

Hi Daniel,

Sorry, but I'm not sure to get your point.

I ran the test locally, and it does fail. It also fails if I modify it to use squashfs instead of ext4.

My reproduction procedure:
git clone https://git.yoctoproject.org/poky-ci-archive -b autobuilder.yoctoproject.org/valkyrie/a-full-2320 . oe-init-build-env
echo 'SANITY_TESTED_DISTROS = ""' >> conf/local.conf
oe-selftest -r wic.Wic.test_no_fstab_update

Thanks,
Mathieu

Quoted 112 lines · Daniel Andrade

Good Afternoon,

I have been trying to figure it out and something seems off. The specific test you refer to may be hiding a problem or maybe I have my testbench compromised and I would like your help to test.
The test test_no_fstab_update verifies if the fstab was updated or not because on the wks file used to do it there is a --no-fstab-update. The thing is, I think this test is passing because what is forcing it to not update is a constraint that I removed previously on the patch (the one that checks if the system is ext* or msdos) and not because the flag is triggering its intended functionality.
The crosscheck I did was modifying the fstype from ext4 to squashfs and indeed the test failed.
Can someone also verify this?

Daniel

Daniel Andrade <[email protected]> escreveu (sábado, 13/09/2025 à(s) 11:02):

Quoted 92 lines

Hello guys, sorry for the delay.
I have been busy, but I hope to provide you with a fix in the next few days.

Sorry about that,
Daniel

Randy MacLeod <[email protected]> escreveu (quinta, 11/09/2025 à(s) 17:56):

Quoted 82 lines

On 2025-09-04 11:58 a.m., Mathieu Dubois-Briand via lists.openembedded.org wrote:

On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via lists.openembedded.org wrote:

From: Daniel Andrade <[email protected]> <[email protected]>

Using install in the rootfs plugin forces fstab to be replaced entirely, meaning that even its Inodes will change, leading xattrs and SELinux context stored by pseudo not to be applied.
The fix just uses cp without preserving attributes from the temporary fstab since none of them are needed, just the content.
Same thing happens with the predefined mechanisms for ext4 and msdos. Using debugfs there is no way to replace contents while maintaining metadata, so the approach taken on the path was to remove the different fstab logic for those fstypes and also use the same modified cp command. Reviewing the builds I did it seems to work for all of the fstypes.

Another problem is that the timestamp applied to fstab is not the same as every other file. It seems like the SOURCE_DATE_EPOCH variable goes to the fallback timestamp (SOURCE_DATE_EPOCH_FALLBACK). Since you are using that variable everywhere, it is not the same value as REPRODUCIBLE_TIMESTAMP_ROOTFS under poky/meta/conf/bitbake.conf that is applied in every other file.

Daniel Andrade (1):
wic: Content of the temporary updated fstab should be copied into the
original not replacing it entirely.

 meta/conf/bitbake.conf                   |  4 +++-
 scripts/lib/wic/partition.py             | 15 +--------------
 scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
 3 files changed, 6 insertions(+), 17 deletions(-)

Hi Daniel,

Thanks for your patch.

It looks like it is breaking a test:

2025-09-04 15:28:11,112 - oe-selftest - INFO - wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)
2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
...
2025-09-04 15:28:11,114 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py", line 859, in test_no_fstab_update
    self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
  File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python3.12/unittest/case.py", line 1251, in assertMultiLineEqual
    self.fail(self._formatMessage(msg, standardMsg))
  File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
    raise self.failureException(msg)

AssertionError: 'af3c087d6c9131735c8d1f270a226892' != '9edb8255abd217fdb20e118833afb856'
- af3c087d6c9131735c8d1f270a226892
+ 9edb8255abd217fdb20e118833afb856
https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268

Can you fix it please?

Ping?

I think this is being tracked by:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15947

"WIC does not preserve metadata when updating fstab"

Btw, we were just following the "Need Info" process during the bug review meeting
so that's why I'm sending this email.

../Randy

Thanks,
Mathieu

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#222962): https://lists.openembedded.org/g/openembedded-core/message/222962
Mute This Topic: https://lists.openembedded.org/mt/115043355/3616765
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

--
# Randy MacLeod
# Wind River Linux

Signature · 3 lines
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#17 lore raw

Hello Mathieu,

Thank you for the quick answer.
Ok, I will start from the beginning.

The failing test is supposed to test --no-fstab-update flags on wks file. According to the understanding of the code, the flag is being passed across python scripts either as self.no_fstab_update, like in scripts/lib/wic/partition.py, or part.no_fstab_update, on scripts/lib/wic/plugins/source/rootfs.py.
The issue that is arising from the oe-selftest -r wic.Wic.test_no_fstab_update is that it is not actually testing it, because somehow the assigning of the previous variables is not correct, which leads to a later assumption.
The thing that is really doing the "no update" is the line 134 of scripts/lib/wic/partition.py where it excludes ext* and msdos partitions from falling in the common plugin for Rootfs.
Because of this, I suggested someone to, in a clean and a different environment compared to mine, change the wks file used to squashfs fstype (or any other supported wic fstype) instead of ext4 which indeed also made the test fail.
In summary, the test is failing not because of the patch (even though maybe something can be improved or fixed later), but because it is just checking for ext4 partitions that fall in a different way of approaching fstab while the real mechanism is not behaving as desired.

TLDR: The test is getting passed not because it is enforcing --no-fstab-update but because the partition is ext* or msdos. Changing it to squashfs, btrfs or any other wic supported partition, with or without the patch, will cause this to fail. This is an extra thing non-related with the proposed patch but that is preventing it to behave correctly as intended by the test.

Mathieu Dubois-Briand <[email protected]> escreveu (terça, 23/09/2025 à(s) 19:44):

Quoted 158 lines

Hi Daniel,

Sorry, but I'm not sure to get your point.

I ran the test locally, and it does fail. It also fails if I modify it to use squashfs instead of ext4.

My reproduction procedure:
git clone https://git.yoctoproject.org/poky-ci-archive -b autobuilder.yoctoproject.org/valkyrie/a-full-2320 . oe-init-build-env
echo 'SANITY_TESTED_DISTROS = ""' >> conf/local.conf oe-selftest -r wic.Wic.test_no_fstab_update

Thanks,
Mathieu

Quoted 9 lines · Daniel Andrade

Good Afternoon,

I have been trying to figure it out and something seems off. The specific test you refer to may be hiding a problem or maybe I have my testbench compromised and I would like your help to test.
The test test_no_fstab_update verifies if the fstab was updated or not because on the wks file used to do it there is a --no-fstab-update. The thing is, I think this test is passing because what is forcing it to not update is a constraint that I removed previously on the patch (the one

that

Quoted 9 lines

checks if the system is ext* or msdos) and not because the flag is triggering its intended functionality.
The crosscheck I did was modifying the fstype from ext4 to squashfs and indeed the test failed.
Can someone also verify this?

Daniel

Daniel Andrade <[email protected]> escreveu (sábado, 13/09/2025

à(s)

Quoted 10 lines

11:02):

Quoted 8 lines

Hello guys, sorry for the delay.
I have been busy, but I hope to provide you with a fix in the next few days.

Sorry about that,
Daniel

Randy MacLeod <[email protected]> escreveu (quinta,

11/09/2025

Quoted 6 lines
Quoted 6 lines

à(s) 17:56):

Quoted 4 lines

On 2025-09-04 11:58 a.m., Mathieu Dubois-Briand via lists.openembedded.org wrote:

On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via

Quoted 4 lines · lists.openembedded.org
Quoted 4 lines
Quoted 4 lines

From: Daniel Andrade <[email protected]> <[email protected]>

Using install in the rootfs plugin forces fstab to be replaced

entirely, meaning that even its Inodes will change, leading xattrs and SELinux context stored by pseudo not to be applied.

Quoted 1 line
Quoted 1 line
Quoted 1 line

The fix just uses cp without preserving attributes from the

temporary fstab since none of them are needed, just the content.

Quoted 1 line
Quoted 1 line
Quoted 1 line

Same thing happens with the predefined mechanisms for ext4 and msdos.

Using debugfs there is no way to replace contents while maintaining metadata, so the approach taken on the path was to remove the different fstab logic for those fstypes and also use the same modified cp command. Reviewing the builds I did it seems to work for all of the fstypes.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Another problem is that the timestamp applied to fstab is not the same

as every other file. It seems like the SOURCE_DATE_EPOCH variable goes to the fallback timestamp (SOURCE_DATE_EPOCH_FALLBACK).

Quoted 1 line
Quoted 1 line
Quoted 1 line

Since you are using that variable everywhere, it is not the same value

as REPRODUCIBLE_TIMESTAMP_ROOTFS under poky/meta/conf/bitbake.conf that is applied in every other file.

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

Daniel Andrade (1):
wic: Content of the temporary updated fstab should be copied into the
original not replacing it entirely.

 meta/conf/bitbake.conf                   |  4 +++-
 scripts/lib/wic/partition.py             | 15 +--------------
 scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
 3 files changed, 6 insertions(+), 17 deletions(-)
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Hi Daniel,

Thanks for your patch.

It looks like it is breaking a test:

2025-09-04 15:28:11,112 - oe-selftest - INFO -

wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
...
2025-09-04 15:28:11,114 - oe-selftest - INFO -

testtools.testresult.real._StringException: Traceback (most recent call last):

Quoted 1 line
Quoted 1 line
Quoted 1 line

File

"/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py", line 859, in test_no_fstab_update

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
assertion_func(first, second, msg=msg)
File "/usr/lib/python3.12/unittest/case.py", line 1251, in

assertMultiLineEqual

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

self.fail(self._formatMessage(msg, standardMsg))
File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
raise self.failureException(msg)
AssertionError: 'af3c087d6c9131735c8d1f270a226892' !=

'9edb8255abd217fdb20e118833afb856'

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- af3c087d6c9131735c8d1f270a226892
+ 9edb8255abd217fdb20e118833afb856

https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Can you fix it please?

Ping?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

I think this is being tracked by:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15947

"WIC does not preserve metadata when updating fstab"

Btw, we were just following the "Need Info" process during the bug

review

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

meeting
so that's why I'm sending this email.

../Randy

Thanks,
Mathieu

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

--
# Randy MacLeod
# Wind River Linux

--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

#18 lore raw

OK! So I misunderstood your previous mail.

I confirm that on master branch the test succeed, but fails if I change the filesystem to squashfs or btrfs.

On a first glance, it looks like you are right, but I can't say I am experienced with wic internals. We might want a second opinion here.

If the test is wrong, it has to be fixed. And so probably the tested code also have to be fixed. I this something you can do?

Thanks,
Mathieu

Quoted 202 lines · Daniel Andrade

Hello Mathieu,

Thank you for the quick answer.
Ok, I will start from the beginning.

The failing test is supposed to test --no-fstab-update flags on wks file. According to the understanding of the code, the flag is being passed across python scripts either as self.no_fstab_update, like in scripts/lib/wic/partition.py, or part.no_fstab_update, on scripts/lib/wic/plugins/source/rootfs.py.
The issue that is arising from the oe-selftest -r wic.Wic.test_no_fstab_update is that it is not actually testing it, because somehow the assigning of the previous variables is not correct, which leads to a later assumption.
The thing that is really doing the "no update" is the line 134 of scripts/lib/wic/partition.py where it excludes ext* and msdos partitions from falling in the common plugin for Rootfs.
Because of this, I suggested someone to, in a clean and a different environment compared to mine, change the wks file used to squashfs fstype (or any other supported wic fstype) instead of ext4 which indeed also made the test fail.
In summary, the test is failing not because of the patch (even though maybe something can be improved or fixed later), but because it is just checking for ext4 partitions that fall in a different way of approaching fstab while the real mechanism is not behaving as desired.

TLDR: The test is getting passed not because it is enforcing --no-fstab-update but because the partition is ext* or msdos. Changing it to squashfs, btrfs or any other wic supported partition, with or without the patch, will cause this to fail. This is an extra thing non-related with the proposed patch but that is preventing it to behave correctly as intended by the test.

Mathieu Dubois-Briand <[email protected]> escreveu (terça, 23/09/2025 à(s) 19:44):

Quoted 166 lines

Hi Daniel,

Sorry, but I'm not sure to get your point.

I ran the test locally, and it does fail. It also fails if I modify it to use squashfs instead of ext4.

My reproduction procedure:
git clone https://git.yoctoproject.org/poky-ci-archive -b autobuilder.yoctoproject.org/valkyrie/a-full-2320 . oe-init-build-env
echo 'SANITY_TESTED_DISTROS = ""' >> conf/local.conf oe-selftest -r wic.Wic.test_no_fstab_update

Thanks,
Mathieu

Quoted 9 lines · Daniel Andrade

Good Afternoon,

I have been trying to figure it out and something seems off. The specific test you refer to may be hiding a problem or maybe I have my testbench compromised and I would like your help to test.
The test test_no_fstab_update verifies if the fstab was updated or not because on the wks file used to do it there is a --no-fstab-update. The thing is, I think this test is passing because what is forcing it to not update is a constraint that I removed previously on the patch (the one

that

Quoted 9 lines

checks if the system is ext* or msdos) and not because the flag is triggering its intended functionality.
The crosscheck I did was modifying the fstype from ext4 to squashfs and indeed the test failed.
Can someone also verify this?

Daniel

Daniel Andrade <[email protected]> escreveu (sábado, 13/09/2025

à(s)

Quoted 10 lines

11:02):

Quoted 8 lines

Hello guys, sorry for the delay.
I have been busy, but I hope to provide you with a fix in the next few days.

Sorry about that,
Daniel

Randy MacLeod <[email protected]> escreveu (quinta,

11/09/2025

Quoted 6 lines
Quoted 6 lines

à(s) 17:56):

Quoted 4 lines

On 2025-09-04 11:58 a.m., Mathieu Dubois-Briand via lists.openembedded.org wrote:

On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via

Quoted 4 lines · lists.openembedded.org
Quoted 4 lines
Quoted 4 lines

From: Daniel Andrade <[email protected]> <[email protected]>

Using install in the rootfs plugin forces fstab to be replaced

entirely, meaning that even its Inodes will change, leading xattrs and SELinux context stored by pseudo not to be applied.

Quoted 1 line
Quoted 1 line
Quoted 1 line

The fix just uses cp without preserving attributes from the

temporary fstab since none of them are needed, just the content.

Quoted 1 line
Quoted 1 line
Quoted 1 line

Same thing happens with the predefined mechanisms for ext4 and msdos.

Using debugfs there is no way to replace contents while maintaining metadata, so the approach taken on the path was to remove the different fstab logic for those fstypes and also use the same modified cp command. Reviewing the builds I did it seems to work for all of the fstypes.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Another problem is that the timestamp applied to fstab is not the same

as every other file. It seems like the SOURCE_DATE_EPOCH variable goes to the fallback timestamp (SOURCE_DATE_EPOCH_FALLBACK).

Quoted 1 line
Quoted 1 line
Quoted 1 line

Since you are using that variable everywhere, it is not the same value

as REPRODUCIBLE_TIMESTAMP_ROOTFS under poky/meta/conf/bitbake.conf that is applied in every other file.

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

Daniel Andrade (1):
wic: Content of the temporary updated fstab should be copied into the
original not replacing it entirely.

 meta/conf/bitbake.conf                   |  4 +++-
 scripts/lib/wic/partition.py             | 15 +--------------
 scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
 3 files changed, 6 insertions(+), 17 deletions(-)
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Hi Daniel,

Thanks for your patch.

It looks like it is breaking a test:

2025-09-04 15:28:11,112 - oe-selftest - INFO -

wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
...
2025-09-04 15:28:11,114 - oe-selftest - INFO -

testtools.testresult.real._StringException: Traceback (most recent call last):

Quoted 1 line
Quoted 1 line
Quoted 1 line

File

"/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py", line 859, in test_no_fstab_update

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
assertion_func(first, second, msg=msg)
File "/usr/lib/python3.12/unittest/case.py", line 1251, in

assertMultiLineEqual

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

self.fail(self._formatMessage(msg, standardMsg))
File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
raise self.failureException(msg)
AssertionError: 'af3c087d6c9131735c8d1f270a226892' !=

'9edb8255abd217fdb20e118833afb856'

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- af3c087d6c9131735c8d1f270a226892
+ 9edb8255abd217fdb20e118833afb856

https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Can you fix it please?

Ping?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

I think this is being tracked by:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15947

"WIC does not preserve metadata when updating fstab"

Btw, we were just following the "Need Info" process during the bug

review

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

meeting
so that's why I'm sending this email.

../Randy

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Thanks,
Mathieu

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#222962):

https://lists.openembedded.org/g/openembedded-core/message/222962

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Mute This Topic: https://lists.openembedded.org/mt/115043355/3616765
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub

[[email protected]]

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

-=-=-=-=-=-=-=-=-=-=-=-

--
# Randy MacLeod
# Wind River Linux

--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Signature · 3 lines
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#19 lore raw

Hi Trevor,

I'm not sure you have seen the following mail thread. It was suggested we seek for your advices during today patch review call.

Do you have any opinion about this test, and if the behaviour we are seeing is expected?

Thanks,
Mathieu

Quoted 217 lines · Mathieu Dubois-Briand

OK! So I misunderstood your previous mail.

I confirm that on master branch the test succeed, but fails if I change the filesystem to squashfs or btrfs.

On a first glance, it looks like you are right, but I can't say I am experienced with wic internals. We might want a second opinion here.

If the test is wrong, it has to be fixed. And so probably the tested code also have to be fixed. I this something you can do?

Thanks,
Mathieu

Quoted 202 lines · Daniel Andrade

Hello Mathieu,

Thank you for the quick answer.
Ok, I will start from the beginning.

The failing test is supposed to test --no-fstab-update flags on wks file. According to the understanding of the code, the flag is being passed across python scripts either as self.no_fstab_update, like in scripts/lib/wic/partition.py, or part.no_fstab_update, on scripts/lib/wic/plugins/source/rootfs.py.
The issue that is arising from the oe-selftest -r wic.Wic.test_no_fstab_update is that it is not actually testing it, because somehow the assigning of the previous variables is not correct, which leads to a later assumption.
The thing that is really doing the "no update" is the line 134 of scripts/lib/wic/partition.py where it excludes ext* and msdos partitions from falling in the common plugin for Rootfs.
Because of this, I suggested someone to, in a clean and a different environment compared to mine, change the wks file used to squashfs fstype (or any other supported wic fstype) instead of ext4 which indeed also made the test fail.
In summary, the test is failing not because of the patch (even though maybe something can be improved or fixed later), but because it is just checking for ext4 partitions that fall in a different way of approaching fstab while the real mechanism is not behaving as desired.

TLDR: The test is getting passed not because it is enforcing --no-fstab-update but because the partition is ext* or msdos. Changing it to squashfs, btrfs or any other wic supported partition, with or without the patch, will cause this to fail. This is an extra thing non-related with the proposed patch but that is preventing it to behave correctly as intended by the test.

Mathieu Dubois-Briand <[email protected]> escreveu (terça, 23/09/2025 à(s) 19:44):

Quoted 166 lines

Hi Daniel,

Sorry, but I'm not sure to get your point.

I ran the test locally, and it does fail. It also fails if I modify it to use squashfs instead of ext4.

My reproduction procedure:
git clone https://git.yoctoproject.org/poky-ci-archive -b autobuilder.yoctoproject.org/valkyrie/a-full-2320 . oe-init-build-env
echo 'SANITY_TESTED_DISTROS = ""' >> conf/local.conf oe-selftest -r wic.Wic.test_no_fstab_update

Thanks,
Mathieu

Quoted 9 lines · Daniel Andrade

Good Afternoon,

I have been trying to figure it out and something seems off. The specific test you refer to may be hiding a problem or maybe I have my testbench compromised and I would like your help to test.
The test test_no_fstab_update verifies if the fstab was updated or not because on the wks file used to do it there is a --no-fstab-update. The thing is, I think this test is passing because what is forcing it to not update is a constraint that I removed previously on the patch (the one

that

Quoted 9 lines

checks if the system is ext* or msdos) and not because the flag is triggering its intended functionality.
The crosscheck I did was modifying the fstype from ext4 to squashfs and indeed the test failed.
Can someone also verify this?

Daniel

Daniel Andrade <[email protected]> escreveu (sábado, 13/09/2025

à(s)

Quoted 10 lines

11:02):

Quoted 8 lines

Hello guys, sorry for the delay.
I have been busy, but I hope to provide you with a fix in the next few days.

Sorry about that,
Daniel

Randy MacLeod <[email protected]> escreveu (quinta,

11/09/2025

Quoted 6 lines
Quoted 6 lines

à(s) 17:56):

Quoted 4 lines

On 2025-09-04 11:58 a.m., Mathieu Dubois-Briand via lists.openembedded.org wrote:

On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via

Quoted 4 lines · lists.openembedded.org
Quoted 4 lines
Quoted 4 lines

From: Daniel Andrade <[email protected]> <[email protected]>

Using install in the rootfs plugin forces fstab to be replaced

entirely, meaning that even its Inodes will change, leading xattrs and SELinux context stored by pseudo not to be applied.

Quoted 1 line
Quoted 1 line
Quoted 1 line

The fix just uses cp without preserving attributes from the

temporary fstab since none of them are needed, just the content.

Quoted 1 line
Quoted 1 line
Quoted 1 line

Same thing happens with the predefined mechanisms for ext4 and msdos.

Using debugfs there is no way to replace contents while maintaining metadata, so the approach taken on the path was to remove the different fstab logic for those fstypes and also use the same modified cp command. Reviewing the builds I did it seems to work for all of the fstypes.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Another problem is that the timestamp applied to fstab is not the same

as every other file. It seems like the SOURCE_DATE_EPOCH variable goes to the fallback timestamp (SOURCE_DATE_EPOCH_FALLBACK).

Quoted 1 line
Quoted 1 line
Quoted 1 line

Since you are using that variable everywhere, it is not the same value

as REPRODUCIBLE_TIMESTAMP_ROOTFS under poky/meta/conf/bitbake.conf that is applied in every other file.

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

Daniel Andrade (1):
wic: Content of the temporary updated fstab should be copied into the
original not replacing it entirely.

 meta/conf/bitbake.conf                   |  4 +++-
 scripts/lib/wic/partition.py             | 15 +--------------
 scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
 3 files changed, 6 insertions(+), 17 deletions(-)
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Hi Daniel,

Thanks for your patch.

It looks like it is breaking a test:

2025-09-04 15:28:11,112 - oe-selftest - INFO -

wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
...
2025-09-04 15:28:11,114 - oe-selftest - INFO -

testtools.testresult.real._StringException: Traceback (most recent call last):

Quoted 1 line
Quoted 1 line
Quoted 1 line

File

"/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py", line 859, in test_no_fstab_update

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
File "/usr/lib/python3.12/unittest/case.py", line 885, in assertEqual
assertion_func(first, second, msg=msg)
File "/usr/lib/python3.12/unittest/case.py", line 1251, in

assertMultiLineEqual

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

self.fail(self._formatMessage(msg, standardMsg))
File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
raise self.failureException(msg)
AssertionError: 'af3c087d6c9131735c8d1f270a226892' !=

'9edb8255abd217fdb20e118833afb856'

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- af3c087d6c9131735c8d1f270a226892
+ 9edb8255abd217fdb20e118833afb856

https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Can you fix it please?

Ping?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

I think this is being tracked by:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15947

"WIC does not preserve metadata when updating fstab"

Btw, we were just following the "Need Info" process during the bug

review

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

meeting
so that's why I'm sending this email.

../Randy

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Thanks,
Mathieu

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#222962):

https://lists.openembedded.org/g/openembedded-core/message/222962

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Mute This Topic: https://lists.openembedded.org/mt/115043355/3616765
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub

[[email protected]]

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

-=-=-=-=-=-=-=-=-=-=-=-

--
# Randy MacLeod
# Wind River Linux

--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Signature · 3 lines
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#20 lore raw

Hi Trevor and Mathieu,

If, by any chance, you need more information, goal or objective with what I am trying to accomplish, or any other thing you might find useful, please let me know.
I am available to provide more explanation, either via this thread or via a separate email or meeting.
For the moment I will halt the research/fixing I was looking/doing until you provide further confirmation on the behaviour of that mechanism.

Have a nice day,
Daniel

Mathieu Dubois-Briand <[email protected]> escreveu (quinta, 25/09/2025 à(s) 12:11):

Quoted 258 lines

Hi Trevor,

I'm not sure you have seen the following mail thread. It was suggested we seek for your advices during today patch review call.

Do you have any opinion about this test, and if the behaviour we are seeing is expected?

Thanks,
Mathieu

Quoted 8 lines · Mathieu Dubois-Briand

OK! So I misunderstood your previous mail.

I confirm that on master branch the test succeed, but fails if I change the filesystem to squashfs or btrfs.

On a first glance, it looks like you are right, but I can't say I am experienced with wic internals. We might want a second opinion here.

Quoted 13 lines

If the test is wrong, it has to be fixed. And so probably the tested code also have to be fixed. I this something you can do?

Thanks,
Mathieu

Quoted 6 lines · Daniel Andrade

Hello Mathieu,

Thank you for the quick answer.
Ok, I will start from the beginning.

The failing test is supposed to test --no-fstab-update flags on wks

file.

Quoted 1 line
Quoted 1 line

According to the understanding of the code, the flag is being passed

across

Quoted 9 lines
Quoted 9 lines

python scripts either as self.no_fstab_update, like in scripts/lib/wic/partition.py, or part.no_fstab_update, on scripts/lib/wic/plugins/source/rootfs.py.
The issue that is arising from the oe-selftest -r wic.Wic.test_no_fstab_update is that it is not actually testing it, because somehow the assigning of the previous variables is not correct, which leads to a later assumption.
The thing that is really doing the "no update" is the line 134 of scripts/lib/wic/partition.py where it excludes ext* and msdos

partitions

Quoted 3 lines
Quoted 3 lines

from falling in the common plugin for Rootfs.
Because of this, I suggested someone to, in a clean and a different environment compared to mine, change the wks file used to squashfs

fstype

Quoted 3 lines
Quoted 3 lines

(or any other supported wic fstype) instead of ext4 which indeed also made the test fail.
In summary, the test is failing not because of the patch (even though

maybe

Quoted 1 line
Quoted 1 line

something can be improved or fixed later), but because it is just

checking

Quoted 1 line
Quoted 1 line

for ext4 partitions that fall in a different way of approaching fstab

while

Quoted 4 lines
Quoted 4 lines

the real mechanism is not behaving as desired.

TLDR: The test is getting passed not because it is enforcing --no-fstab-update but because the partition is ext* or msdos.

Changing it

Quoted 2 lines
Quoted 2 lines

to squashfs, btrfs or any other wic supported partition, with or without the patch, will cause this to fail. This is an extra thing non-related

with

Quoted 4 lines
Quoted 4 lines

the proposed patch but that is preventing it to behave correctly as intended by the test.

Mathieu Dubois-Briand <[email protected]> escreveu

(terça,

Quoted 16 lines
Quoted 16 lines

23/09/2025 à(s) 19:44):

Quoted 14 lines

Hi Daniel,

Sorry, but I'm not sure to get your point.

I ran the test locally, and it does fail. It also fails if I modify it to use squashfs instead of ext4.

My reproduction procedure:
git clone https://git.yoctoproject.org/poky-ci-archive -b autobuilder.yoctoproject.org/valkyrie/a-full-2320 . oe-init-build-env
echo 'SANITY_TESTED_DISTROS = ""' >> conf/local.conf oe-selftest -r wic.Wic.test_no_fstab_update

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Thanks,
Mathieu

Quoted 4 lines · Daniel Andrade

Good Afternoon,

I have been trying to figure it out and something seems off. The specific test you refer to may be hiding a problem or maybe I

have my

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

testbench compromised and I would like your help to test. The test test_no_fstab_update verifies if the fstab was updated

or not

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

because on the wks file used to do it there is a --no-fstab-update.

The

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

thing is, I think this test is passing because what is forcing it to

not

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

update is a constraint that I removed previously on the patch (the

one

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

that

Quoted 3 lines

checks if the system is ext* or msdos) and not because the flag is triggering its intended functionality.
The crosscheck I did was modifying the fstype from ext4 to squashfs

and

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 6 lines

indeed the test failed.
Can someone also verify this?

Daniel

Daniel Andrade <[email protected]> escreveu (sábado, 13/09/2025

à(s)

Quoted 4 lines

11:02):

Quoted 2 lines

Hello guys, sorry for the delay.
I have been busy, but I hope to provide you with a fix in the next

few

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines
Quoted 6 lines

days.

Sorry about that,
Daniel

Randy MacLeod <[email protected]> escreveu (quinta,

11/09/2025

Quoted 2 lines
Quoted 2 lines

à(s) 17:56):

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

On 2025-09-04 11:58 a.m., Mathieu Dubois-Briand via lists.openembedded.org wrote:

On Tue Sep 2, 2025 at 9:05 PM CEST, dani.barra25 via

Quoted 2 lines · lists.openembedded.org
Quoted 2 lines
Quoted 2 lines

From: Daniel Andrade <[email protected]> <

[email protected]>

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Using install in the rootfs plugin forces fstab to be replaced

entirely, meaning that even its Inodes will change, leading xattrs and SELinux context stored by pseudo not to be applied.

Quoted 1 line
Quoted 1 line
Quoted 1 line

The fix just uses cp without preserving attributes from the

temporary fstab since none of them are needed, just the content.

Quoted 1 line
Quoted 1 line
Quoted 1 line

Same thing happens with the predefined mechanisms for ext4 and

msdos.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Using debugfs there is no way to replace contents while maintaining metadata, so the approach taken on the path was to remove the different fstab logic for those fstypes and also use the same modified cp

command.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Reviewing the builds I did it seems to work for all of the fstypes.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Another problem is that the timestamp applied to fstab is not the

same

Quoted 1 line
Quoted 1 line
Quoted 1 line

as every other file. It seems like the SOURCE_DATE_EPOCH variable

goes to

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

the fallback timestamp (SOURCE_DATE_EPOCH_FALLBACK).

Quoted 1 line
Quoted 1 line
Quoted 1 line

Since you are using that variable everywhere, it is not the same

value

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

as REPRODUCIBLE_TIMESTAMP_ROOTFS under poky/meta/conf/bitbake.conf that is applied in every other file.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Daniel Andrade (1):
wic: Content of the temporary updated fstab should be copied

into the

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

original not replacing it entirely.

 meta/conf/bitbake.conf                   |  4 +++-
 scripts/lib/wic/partition.py             | 15 +--------------
 scripts/lib/wic/plugins/source/rootfs.py |  4 ++--
 3 files changed, 6 insertions(+), 17 deletions(-)

Hi Daniel,

Thanks for your patch.

It looks like it is breaking a test:

2025-09-04 15:28:11,112 - oe-selftest - INFO -
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

wic.Wic.test_no_fstab_update (subunit.RemotedTestCase)

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
2025-09-04 15:28:11,113 - oe-selftest - INFO -  ... FAIL
...
2025-09-04 15:28:11,114 - oe-selftest - INFO -

testtools.testresult.real._StringException: Traceback (most recent call last):

Quoted 1 line
Quoted 1 line
Quoted 1 line

File

"/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py",

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

line 859, in test_no_fstab_update

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

self.assertEqual(bf_fstab_md5sum, part_fstab_md5sum[1])
File "/usr/lib/python3.12/unittest/case.py", line 885, in

assertEqual

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

assertion_func(first, second, msg=msg)
File "/usr/lib/python3.12/unittest/case.py", line 1251, in

assertMultiLineEqual

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

self.fail(self._formatMessage(msg, standardMsg))
File "/usr/lib/python3.12/unittest/case.py", line 715, in fail
raise self.failureException(msg)
AssertionError: 'af3c087d6c9131735c8d1f270a226892' !=

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

'9edb8255abd217fdb20e118833afb856'

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

- af3c087d6c9131735c8d1f270a226892
+ 9edb8255abd217fdb20e118833afb856

https://autobuilder.yoctoproject.org/valkyrie/#/builders/23/builds/2412https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2268

Quoted 36 lines
Quoted 36 lines
Quoted 36 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Can you fix it please?

Ping?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

I think this is being tracked by:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15947

"WIC does not preserve metadata when updating fstab"

Btw, we were just following the "Need Info" process during the bug

review

Quoted 19 lines
Quoted 19 lines
Quoted 19 lines

meeting
so that's why I'm sending this email.

../Randy

Thanks,
Mathieu

--
# Randy MacLeod
# Wind River Linux

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com