Skip to content

Patch review - pseudo file mode corruption fix V3

Steve Sakoman openembedded-core 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest46 patches 2 repliesIn review
  2. v147 patches 6 repliesSuperseded

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/46pseudo: Switch to oe-core branch in git repometa/recipes-devtools/pseudo/files/0001-Add-statx.patch meta/recipes-devtools/pseudo/files/0001-maketables-wrappers-use-Python-3.patch meta/recipes-devtools/pseudo/files/0001-pseudo-On-a-DB-fixup-remove-files-that-do-not-exist-.patch +8 more+2 -6180 replies
2/46pseudo: merge in fixes for setfacl issuemeta/recipes-devtools/pseudo/pseudo_git.bb+1 -10 replies
3/46pseudo: Update to add OFC fcntl lock updatesmeta/recipes-devtools/pseudo/pseudo_git.bb+1 -10 replies
4/46pseudo: fix renaming to selfmeta/recipes-devtools/pseudo/pseudo_git.bb+1 -10 replies
5/46pseudo: Ignore mismatched inodes from the dbmeta/recipes-devtools/pseudo/files/delete_mismatches.patch meta/recipes-devtools/pseudo/pseudo_git.bb+52 -00 replies
6/46pseudo: Add support for ignoring paths from the pseudo DBmeta/recipes-devtools/pseudo/files/add_ignore_paths.patch meta/recipes-devtools/pseudo/pseudo_git.bb+299 -00 replies
7/46pseudo: Abort on mismatch patchmeta/recipes-devtools/pseudo/files/abort_on_mismatch.patch meta/recipes-devtools/pseudo/pseudo_git.bb+65 -00 replies
8/46psuedo: Add tracking of linked files for fdsmeta/recipes-devtools/pseudo/files/track_link_fds.patch meta/recipes-devtools/pseudo/pseudo_git.bb+156 -00 replies
9/46pseudo: Fix xattr segfaultmeta/recipes-devtools/pseudo/files/xattr_fix.patch meta/recipes-devtools/pseudo/pseudo_git.bb+41 -00 replies
10/46pseudo: Add may unlink patchmeta/recipes-devtools/pseudo/files/mayunlink.patch meta/recipes-devtools/pseudo/pseudo_git.bb+38 -00 replies
11/46pseudo: Add pathfix patchmeta/recipes-devtools/pseudo/files/pathfix.patch meta/recipes-devtools/pseudo/pseudo_git.bb+26 -00 replies
12/46base/bitbake.conf: Enable pseudo path filteringmeta/classes/archiver.bbclass meta/classes/base.bbclass meta/classes/image_types_wic.bbclass +4 more+22 -60 replies
13/46wic: Fix permissions when using exclude or include pathscripts/lib/wic/partition.py scripts/lib/wic/plugins/source/rootfs.py+38 -60 replies
14/46wic: Fix multi images .wks with bitbakemeta/classes/image_types_wic.bbclass scripts/lib/wic/partition.py+9 -80 replies
15/46wic: Avoid creating invalid pseudo directoryscripts/lib/wic/partition.py+8 -70 replies
16/46wic: Handle new PSEUDO_IGNORE_PATHS variablemeta/classes/image_types_wic.bbclass scripts/lib/wic/partition.py+8 -60 replies
17/46pseudo: Fix statx function usagemeta/recipes-devtools/pseudo/files/statxfix.patch meta/recipes-devtools/pseudo/pseudo_git.bb+57 -00 replies
18/46bitbake.conf: Extend PSEUDO_IGNORE_PATHS to ${COREBASE}/metameta/conf/bitbake.conf+1 -10 replies
19/46abi_version,sanity: Tell users TMPDIR must be clean after pseudo changesmeta/classes/sanity.bbclass meta/conf/abi_version.conf+4 -10 replies
20/46pseudo: Update to account for patches merged on branchmeta/recipes-devtools/pseudo/files/abort_on_mismatch.patch meta/recipes-devtools/pseudo/files/add_ignore_paths.patch meta/recipes-devtools/pseudo/files/delete_mismatches.patch +6 more+1 -7350 replies
21/46pseudo: Upgrade to include mkostemp64 wrappermeta/recipes-devtools/pseudo/pseudo_git.bb+1 -10 replies
22/46base.bbclass: use os.path.normpath instead of just comparing WORKDIR and S as stringsmeta/classes/base.bbclass+2 -20 replies
23/46oeqa/selftest/runtime_test: Exclude gpg directory from pseudo databasemeta/lib/oeqa/selftest/cases/runtime_test.py+1 -00 replies
24/46tcl: adapt to potential pseudo changesmeta/recipes-devtools/tcltk/tcl_8.6.10.bb+1 -00 replies
25/46bitbake.conf: Exclude ${CCACHE_DIR} from pseudo databasemeta/conf/bitbake.conf+1 -10 replies
26/46uninative: Don't use single sstate for pseudo-nativemeta/classes/base.bbclass meta/classes/sstate.bbclass meta/lib/oe/sstatesig.py+6 -10 replies
27/46pseudo: Simplify pseudo_client_ignore_path_chroot()meta/recipes-devtools/pseudo/files/0002-pseudo_client-Lessen-indentation-of-pseudo_client_ig.patch meta/recipes-devtools/pseudo/files/0003-pseudo_client-Simplify-pseudo_client_ignore_path_chr.patch meta/recipes-devtools/pseudo/pseudo_git.bb+121 -00 replies
28/46pseudo: Update to print PSEUDO_LOGFILE in abort message on path mismatchesmeta/recipes-devtools/pseudo/pseudo_git.bb+1 -10 replies
29/46distutils3: allow setup.py to be run from a different directory to ${S}meta/classes/distutils3.bbclass+6 -40 replies
30/46pseudo: Drop patches merged into upstream branchmeta/recipes-devtools/pseudo/files/0002-pseudo_client-Lessen-indentation-of-pseudo_client_ig.patch meta/recipes-devtools/pseudo/files/0003-pseudo_client-Simplify-pseudo_client_ignore_path_chr.patch meta/recipes-devtools/pseudo/pseudo_git.bb+1 -1220 replies
31/46lib/oe/path: Add canonicalize()meta/lib/oe/path.py+21 -00 replies
32/46bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHSmeta/conf/bitbake.conf+2 -20 replies
33/46wic: Pass canonicalized paths in PSEUDO_IGNORE_PATHSscripts/lib/wic/partition.py+11 -10 replies
34/46bitbake.conf: Prevent pyc file generation in pseudo contextmeta-selftest/lib/pseudo_pyc_test1.py meta-selftest/lib/pseudo_pyc_test2.py meta-selftest/recipes-test/pseudo-pyc-test/pseudo-pyc-test.bb +2 more+46 -20 replies
35/46license.bbclass: Add COMMON_LICENSE_DIR and LICENSE_PATH dirs to PSEUDO_IGNORE_PATHSmeta/classes/license.bbclass+1 -00 replies
36/46bitbake.conf: Add /run/ to PSEUDO_IGNORE_PATHSmeta/conf/bitbake.conf+1 -10 replies
37/46pseudo: Add lchmod wrappermeta/recipes-devtools/pseudo/pseudo_git.bb+1 -10 replies
38/46pseudo: Update for arm host and memleak fixes/cleanupmeta/recipes-devtools/pseudo/pseudo_git.bb+1 -10 replies
39/46wic: Add workdir argumentscripts/lib/wic/plugins/imager/direct.py scripts/wic+3 -10 replies
40/46wic: Allow exec_native_cmd to run HOSTTOOLSmeta/classes/image_types_wic.bbclass scripts/lib/wic/misc.py+6 -30 replies
41/46wic: Ensure internal workdir is not reusedscripts/lib/wic/plugins/imager/direct.py+11 -10 replies
42/46image_types_wic: Move wic working directorymeta/classes/image_types_wic.bbclass+7 -10 replies
43/46wic: Update pseudo db when excluding content from rootfsscripts/lib/wic/plugins/source/rootfs.py+8 -50 replies
44/46wic: Add --change-directory argumentscripts/lib/wic/help.py scripts/lib/wic/ksparser.py scripts/lib/wic/partition.py +1 more+26 -30 replies
45/46wic: Copy rootfs dir if fstab needs updatingscripts/lib/wic/partition.py scripts/lib/wic/plugins/imager/direct.py scripts/lib/wic/plugins/source/rootfs.py+29 -290 replies
46/46wic: Optimise fstab modification for ext2/3/4 and msdos partitionsscripts/lib/wic/partition.py scripts/lib/wic/plugins/source/rootfs.py+28 -80 replies

Reviews trailers per patch

None of the 46 patches has a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchPaul Barker
SeriesA-b
1/46
2/46
3/46
4/46
5/46
6/46
7/46
8/46
9/46
10/46
11/46
12/46
13/46
14/46
15/46
16/46
17/46
18/46
19/46
20/46
21/46
22/46
23/46
24/46
25/46
26/46
27/46
28/46
29/46
30/46
31/46
32/46
33/46
34/46
35/46
36/46
37/46
38/46
39/46
40/46
41/46
42/46
43/46
44/46
45/46
46/46

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v3 thread

Cover letter [OE-core][dunfell V3 00/46] Patch review - pseudo file mode corruption fix V3

Issues with undetected file mode corruption in pseudo have been identified.

Fixes have been merged into master and gatesgarth over the past several months and things seem to have stabilized enough that we can consider backporting these fixes to dunfell.

This is a somewhat more invasive change than normal. Specifically the user will be required to clean TMPDIR after this merge. It may also expose issues with other layers, though master and gategarth have probably helped to pave the way for this change.

Personally I think that this fixes a serious enough issue that we should consider this series for dunfell.

This third version removes:

bitbake.conf: Remove TERM from default BB_HASHBASE_WHITELIST

and hence the need for any bitbake changes.

Let me know if you you are aware of any other pseudo issues which would necessitate delaying the merge of this series.

Please have comments back by Thursday evening.

Passed a-full on autobuilder:

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/1784

The following changes since commit 897822233faef0f8f35dc1d8a39e1c4bc0550f1e:

ppp: Whitelist CVE-2020-15704 (2021-01-19 04:22:10 -1000)

are available in the Git repository at:

git://git.openembedded.org/openembedded-core-contrib stable/dunfell-nut
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=stable/dunfell-nut

Jack Mitchell (1):
distutils3: allow setup.py to be run from a different directory to

    ${S}

Joe Slater (1):
pseudo: fix renaming to self

Martin Jansa (2):
base.bbclass: use os.path.normpath instead of just comparing WORKDIR
and S as strings
license.bbclass: Add COMMON_LICENSE_DIR and LICENSE_PATH dirs to
PSEUDO_IGNORE_PATHS

Mingli Yu (2):
tcl: adapt to potential pseudo changes bitbake.conf: Exclude ${CCACHE_DIR} from pseudo database

Paul Barker (8):
bitbake.conf: Prevent pyc file generation in pseudo context
wic: Add workdir argument
wic: Allow exec_native_cmd to run HOSTTOOLS
wic: Ensure internal workdir is not reused
image_types_wic: Move wic working directory
wic: Update pseudo db when excluding content from rootfs
wic: Copy rootfs dir if fstab needs updating
wic: Optimise fstab modification for ext2/3/4 and msdos partitions

Peter Kjellerstedt (4):
pseudo: Simplify pseudo_client_ignore_path_chroot() lib/oe/path: Add canonicalize()
bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS
wic: Pass canonicalized paths in PSEUDO_IGNORE_PATHS

Ricardo Ribalda Delgado (4):
wic: Fix permissions when using exclude or include path
wic: Fix multi images .wks with bitbake
wic: Avoid creating invalid pseudo directory
wic: Add --change-directory argument

Richard Purdie (23):
pseudo: Switch to oe-core branch in git repo
pseudo: merge in fixes for setfacl issue
pseudo: Update to add OFC fcntl lock updates
pseudo: Ignore mismatched inodes from the db
pseudo: Add support for ignoring paths from the pseudo DB
pseudo: Abort on mismatch patch
psuedo: Add tracking of linked files for fds
pseudo: Fix xattr segfault
pseudo: Add may unlink patch
pseudo: Add pathfix patch
base/bitbake.conf: Enable pseudo path filtering
wic: Handle new PSEUDO_IGNORE_PATHS variable
pseudo: Fix statx function usage
bitbake.conf: Extend PSEUDO_IGNORE_PATHS to ${COREBASE}/meta abi_version,sanity: Tell users TMPDIR must be clean after pseudo
changes
pseudo: Update to account for patches merged on branch
pseudo: Upgrade to include mkostemp64 wrapper oeqa/selftest/runtime_test: Exclude gpg directory from pseudo database
uninative: Don't use single sstate for pseudo-native
pseudo: Drop patches merged into upstream branch bitbake.conf: Add /run/ to PSEUDO_IGNORE_PATHS
pseudo: Add lchmod wrapper
pseudo: Update for arm host and memleak fixes/cleanup

Tomasz Dziendzielski (1):
pseudo: Update to print PSEUDO_LOGFILE in abort message on path
mismatches

 meta-selftest/lib/pseudo_pyc_test1.py         |   1 +
 meta-selftest/lib/pseudo_pyc_test2.py         |   1 +
 .../pseudo-pyc-test/pseudo-pyc-test.bb        |  15 ++
 meta/classes/archiver.bbclass                 |   2 +-
 meta/classes/base.bbclass                     |   6 +
 meta/classes/distutils3.bbclass               |  10 +-
 meta/classes/image_types_wic.bbclass          |  20 ++-
 meta/classes/license.bbclass                  |   1 +
 meta/classes/populate_sdk_base.bbclass        |   2 +
 meta/classes/sanity.bbclass                   |   3 +
 meta/classes/sstate.bbclass                   |   4 +
 meta/conf/abi_version.conf                    |   2 +-
 meta/conf/bitbake.conf                        |  13 +-
 meta/lib/oe/path.py                           |  21 +++
 meta/lib/oe/sstatesig.py                      |   4 +-
 meta/lib/oeqa/selftest/cases/pseudo.py        |  27 ++++
 meta/lib/oeqa/selftest/cases/runtime_test.py  |   1 +
 .../pseudo/files/0001-Add-statx.patch         | 106 --------------
 ...001-maketables-wrappers-use-Python-3.patch |  34 -----
 ...ixup-remove-files-that-do-not-exist-.patch |  49 -------
 .../0001-pseudo_ipc.h-Fix-enum-typedef.patch  |  31 ----
 ...1-realpath.c-Remove-trailing-slashes.patch |  57 --------
 ...xattr-adjust-for-attr-2.4.48-release.patch |  48 ------
 .../pseudo/files/moreretries.patch            |  19 ---
 .../pseudo/files/seccomp.patch                | 137 ------------------
 .../pseudo/files/toomanyfiles.patch           |  71 ---------
 .../pseudo/files/xattr_version.patch          |  54 -------
 meta/recipes-devtools/pseudo/pseudo_git.bb    |  14 +-
 meta/recipes-devtools/tcltk/tcl_8.6.10.bb     |   1 +
 scripts/lib/wic/help.py                       |   6 +
 scripts/lib/wic/ksparser.py                   |   1 +
 scripts/lib/wic/misc.py                       |   7 +-
 scripts/lib/wic/partition.py                  |  66 ++++++---
 scripts/lib/wic/plugins/imager/direct.py      |  48 +++---
 scripts/lib/wic/plugins/source/rootfs.py      |  85 +++++++++--
 scripts/postinst-intercepts/update_font_cache |   2 +
 scripts/wic                                   |   2 +
 37 files changed, 281 insertions(+), 690 deletions(-)
 create mode 100644 meta-selftest/lib/pseudo_pyc_test1.py
 create mode 100644 meta-selftest/lib/pseudo_pyc_test2.py
 create mode 100644 meta-selftest/recipes-test/pseudo-pyc-test/pseudo-pyc-test.bb
 create mode 100644 meta/lib/oeqa/selftest/cases/pseudo.py
 delete mode 100644 meta/recipes-devtools/pseudo/files/0001-Add-statx.patch
 delete mode 100644 meta/recipes-devtools/pseudo/files/0001-maketables-wrappers-use-Python-3.patch
 delete mode 100644 meta/recipes-devtools/pseudo/files/0001-pseudo-On-a-DB-fixup-remove-files-that-do-not-exist-.patch
 delete mode 100644 meta/recipes-devtools/pseudo/files/0001-pseudo_ipc.h-Fix-enum-typedef.patch
 delete mode 100644 meta/recipes-devtools/pseudo/files/0001-realpath.c-Remove-trailing-slashes.patch
 delete mode 100644 meta/recipes-devtools/pseudo/files/0006-xattr-adjust-for-attr-2.4.48-release.patch
 delete mode 100644 meta/recipes-devtools/pseudo/files/moreretries.patch
 delete mode 100644 meta/recipes-devtools/pseudo/files/seccomp.patch
 delete mode 100644 meta/recipes-devtools/pseudo/files/toomanyfiles.patch
 delete mode 100644 meta/recipes-devtools/pseudo/files/xattr_version.patch
Signature · 1 line
2.25.1