Skip to contentSkip to conversation

tests/docker: tolerate chmod EPERM when unpacking hexagon toolchain

PATCH In review Brian Cain 2 participants 3 messages Jun 19 – Jul 11

lore
Share & tools
lore

Brian Cain

author
#0 lore raw

PATCH v2 1/1 tests/docker: tolerate chmod EPERM when unpacking hexagon toolchain

The debian-hexagon-cross image unpacks the codelinaro clang+llvm hexagon toolchain tarball into /opt. The archive contains symlinks (the .so/.a library aliases) and directories whose stored modes GNU tar restores via chmod()/lchmod(). Under docker, where the build runs as real root, those calls succeed. Under rootless podman the build runs in a user namespace on overlay storage, which rejects the chmod()/lchmod() calls with EPERM:

  tar: .../libclang_rt.builtins.a: Cannot change mode to rwxrwxrwx: \
       Operation not permitted
  tar: .../x86_64-linux-gnu: Cannot change mode to rwxr-xr-x: \
       Operation not permitted
  tar: Exiting with failure status due to previous errors

tar then exits non-zero and aborts the build. The mode metadata on these symlinks and directories is irrelevant for our use, and neither --no-same-permissions nor extracting as a non-root user stops GNU tar 1.35 from attempting the lchmod on the symlinks.

Download the tarball to a file and ignore tar's exit status during extraction with --no-same-owner and --no-same-permissions. Rather than trying to distinguish that known-benign failure from a genuinely broken download or extraction by pattern-matching tar's stderr text (fragile across tar versions and locales), verify the toolchain actually works by running hexagon-clang --version afterwards, and pass curl --fail so a failed download surfaces immediately instead of writing an error page into the tarball.

---
 tests/docker/dockerfiles/debian-hexagon-cross.docker | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
tests/docker/dockerfiles/debian-hexagon-cross.docker +10 -1
index 23e8bb2fb26..fc1e3ef4e37 100644
@@ -49,7 +49,16 @@ ENV TOOLCHAIN_BASENAME=clang+llvm-${TOOLCHAIN_RELEASE}-cross-hexagon-unknown-lin
 ENV TOOLCHAIN_URL=https://artifacts.codelinaro.org/artifactory/codelinaro-toolchain-for-hexagon/${TOOLCHAIN_RELEASE}_/${TOOLCHAIN_BASENAME}.tar.zst
 ENV CCACHE_WRAPPERSDIR=/usr/libexec/ccache-wrappers
 
-RUN curl -#SL "$TOOLCHAIN_URL" | tar --zstd -xC "$TOOLCHAIN_INSTALL"
+# Rootless podman's overlay storage rejects the chmod()/lchmod() calls GNU
+# tar makes while restoring the toolchain symlinks' and directories' stored
+# modes, with EPERM (the mode metadata itself is harmless and unused here).
+# Ignore tar's exit status, but verify the toolchain actually extracted and
+# runs, so a genuinely broken download or extraction still fails the build.
+RUN curl --fail -#SLo /tmp/toolchain.tar.zst "$TOOLCHAIN_URL" && \
+    { tar --zstd --no-same-owner --no-same-permissions \
+          -xf /tmp/toolchain.tar.zst -C "$TOOLCHAIN_INSTALL" || true; } && \
+    rm -f /tmp/toolchain.tar.zst && \
+    "$TOOLCHAIN_INSTALL/$TOOLCHAIN_BASENAME/x86_64-linux-gnu/bin/hexagon-clang" --version
 ENV PATH=$PATH:${TOOLCHAIN_INSTALL}/${TOOLCHAIN_BASENAME}/x86_64-linux-gnu/bin
 ENV MAKE=/usr/bin/make
 # As a final step configure the user (if env is defined)
Signature · 1 line
2.34.1
#1 lore raw
Quoted 18 lines · Brian Cain

The debian-hexagon-cross image unpacks the codelinaro clang+llvm hexagon toolchain tarball into /opt. The archive contains symlinks (the .so/.a library aliases) and directories whose stored modes GNU tar restores via chmod()/lchmod(). Under docker, where the build runs as real root, those calls succeed. Under rootless podman the build runs in a user namespace on overlay storage, which rejects the chmod()/lchmod() calls with EPERM:

  tar: .../libclang_rt.builtins.a: Cannot change mode to rwxrwxrwx: \
       Operation not permitted
  tar: .../x86_64-linux-gnu: Cannot change mode to rwxr-xr-x: \
       Operation not permitted
  tar: Exiting with failure status due to previous errors

tar then exits non-zero and aborts the build. The mode metadata on these symlinks and directories is irrelevant for our use, and neither --no-same-permissions nor extracting as a non-root user stops GNU tar 1.35 from attempting the lchmod on the symlinks.

Quoted 9 lines

Download the tarball to a file and ignore tar's exit status during extraction with --no-same-owner and --no-same-permissions. Rather than trying to distinguish that known-benign failure from a genuinely broken download or extraction by pattern-matching tar's stderr text (fragile across tar versions and locales), verify the toolchain actually works by running hexagon-clang --version afterwards, and pass curl --fail so a failed download surfaces immediately instead of writing an error page into the tarball.

With those options, I don't think there is any reason for tar to fail. Also, as Matheus mentions, it can cover some other issues (disk full, process killed). So it would be better to remove || true. You can leave the hexagon-clang check anyway.

Quoted 27 lines
---
 tests/docker/dockerfiles/debian-hexagon-cross.docker | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
tests/docker/dockerfiles/debian-hexagon-cross.docker +10 -1
index 23e8bb2fb26..fc1e3ef4e37 100644
@@ -49,7 +49,16 @@ ENV TOOLCHAIN_BASENAME=clang+llvm-${TOOLCHAIN_RELEASE}-cross-hexagon-unknown-lin
 ENV TOOLCHAIN_URL=https://artifacts.codelinaro.org/artifactory/codelinaro-toolchain-for-hexagon/${TOOLCHAIN_RELEASE}_/${TOOLCHAIN_BASENAME}.tar.zst
 ENV CCACHE_WRAPPERSDIR=/usr/libexec/ccache-wrappers
 
-RUN curl -#SL "$TOOLCHAIN_URL" | tar --zstd -xC "$TOOLCHAIN_INSTALL"
+# Rootless podman's overlay storage rejects the chmod()/lchmod() calls GNU
+# tar makes while restoring the toolchain symlinks' and directories' stored
+# modes, with EPERM (the mode metadata itself is harmless and unused here).
+# Ignore tar's exit status, but verify the toolchain actually extracted and
+# runs, so a genuinely broken download or extraction still fails the build.
+RUN curl --fail -#SLo /tmp/toolchain.tar.zst "$TOOLCHAIN_URL" && \
+    { tar --zstd --no-same-owner --no-same-permissions \
+          -xf /tmp/toolchain.tar.zst -C "$TOOLCHAIN_INSTALL" || true; } && \
+    rm -f /tmp/toolchain.tar.zst && \
+    "$TOOLCHAIN_INSTALL/$TOOLCHAIN_BASENAME/x86_64-linux-gnu/bin/hexagon-clang" --version
 ENV PATH=$PATH:${TOOLCHAIN_INSTALL}/${TOOLCHAIN_BASENAME}/x86_64-linux-gnu/bin
 ENV MAKE=/usr/bin/make
 # As a final step configure the user (if env is defined)
author
#2 lore raw
Quoted 28 lines · Pierrick Bouvier
Quoted 18 lines · Brian Cain

The debian-hexagon-cross image unpacks the codelinaro clang+llvm hexagon toolchain tarball into /opt. The archive contains symlinks (the .so/.a library aliases) and directories whose stored modes GNU tar restores via chmod()/lchmod(). Under docker, where the build runs as real root, those calls succeed. Under rootless podman the build runs in a user namespace on overlay storage, which rejects the chmod()/lchmod() calls with EPERM:

   tar: .../libclang_rt.builtins.a: Cannot change mode to rwxrwxrwx: \
        Operation not permitted
   tar: .../x86_64-linux-gnu: Cannot change mode to rwxr-xr-x: \
        Operation not permitted
   tar: Exiting with failure status due to previous errors

tar then exits non-zero and aborts the build. The mode metadata on these symlinks and directories is irrelevant for our use, and neither --no-same-permissions nor extracting as a non-root user stops GNU tar 1.35 from attempting the lchmod on the symlinks.

Quoted 9 lines

Download the tarball to a file and ignore tar's exit status during extraction with --no-same-owner and --no-same-permissions. Rather than trying to distinguish that known-benign failure from a genuinely broken download or extraction by pattern-matching tar's stderr text (fragile across tar versions and locales), verify the toolchain actually works by running hexagon-clang --version afterwards, and pass curl --fail so a failed download surfaces immediately instead of writing an error page into the tarball.

Quoted 4 lines

With those options, I don't think there is any reason for tar to fail. Also, as Matheus mentions, it can cover some other issues (disk full, process killed). So it would be better to remove || true. You can leave the hexagon-clang check anyway.

tar's failing because of the lchown for the symlinks.  Agreed: masking with || true is not the right solution.  Instead, let's try a different approach using .deb packages:

v3: https://lore.kernel.org/qemu-devel/[email protected]/

This should also have a nice side-effect of decreasing the container size significantly.

Quoted 28 lines
Quoted 28 lines
---
  tests/docker/dockerfiles/debian-hexagon-cross.docker | 11 ++++++++++-
  1 file changed, 10 insertions(+), 1 deletion(-)
tests/docker/dockerfiles/debian-hexagon-cross.docker +10 -1
index 23e8bb2fb26..fc1e3ef4e37 100644
@@ -49,7 +49,16 @@ ENV TOOLCHAIN_BASENAME=clang+llvm-${TOOLCHAIN_RELEASE}-cross-hexagon-unknown-lin
  ENV TOOLCHAIN_URL=https://artifacts.codelinaro.org/artifactory/codelinaro-toolchain-for-hexagon/${TOOLCHAIN_RELEASE}_/${TOOLCHAIN_BASENAME}.tar.zst
  ENV CCACHE_WRAPPERSDIR=/usr/libexec/ccache-wrappers
 
-RUN curl -#SL "$TOOLCHAIN_URL" | tar --zstd -xC "$TOOLCHAIN_INSTALL"
+# Rootless podman's overlay storage rejects the chmod()/lchmod() calls GNU
+# tar makes while restoring the toolchain symlinks' and directories' stored
+# modes, with EPERM (the mode metadata itself is harmless and unused here).
+# Ignore tar's exit status, but verify the toolchain actually extracted and
+# runs, so a genuinely broken download or extraction still fails the build.
+RUN curl --fail -#SLo /tmp/toolchain.tar.zst "$TOOLCHAIN_URL" && \
+    { tar --zstd --no-same-owner --no-same-permissions \
+          -xf /tmp/toolchain.tar.zst -C "$TOOLCHAIN_INSTALL" || true; } && \
+    rm -f /tmp/toolchain.tar.zst && \
+    "$TOOLCHAIN_INSTALL/$TOOLCHAIN_BASENAME/x86_64-linux-gnu/bin/hexagon-clang" --version
  ENV PATH=$PATH:${TOOLCHAIN_INSTALL}/${TOOLCHAIN_BASENAME}/x86_64-linux-gnu/bin
  ENV MAKE=/usr/bin/make
  # As a final step configure the user (if env is defined)