tests/docker: tolerate chmod EPERM when unpacking hexagon toolchain
PATCH v2 1/1 tests/docker: tolerate chmod EPERM when unpacking hexagon toolchain
The debian-hexagon-cross image unpacks the codelinaro clang+llvm hexagon toolchain tarball into /opt. The archive contains symlinks (the .so/.a library aliases) and directories whose stored modes GNU tar restores via chmod()/lchmod(). Under docker, where the build runs as real root, those calls succeed. Under rootless podman the build runs in a user namespace on overlay storage, which rejects the chmod()/lchmod() calls with EPERM:
tar: .../libclang_rt.builtins.a: Cannot change mode to rwxrwxrwx: \
Operation not permitted
tar: .../x86_64-linux-gnu: Cannot change mode to rwxr-xr-x: \
Operation not permitted
tar: Exiting with failure status due to previous errorstar then exits non-zero and aborts the build. The mode metadata on these symlinks and directories is irrelevant for our use, and neither --no-same-permissions nor extracting as a non-root user stops GNU tar 1.35 from attempting the lchmod on the symlinks.
Download the tarball to a file and ignore tar's exit status during extraction with --no-same-owner and --no-same-permissions. Rather than trying to distinguish that known-benign failure from a genuinely broken download or extraction by pattern-matching tar's stderr text (fragile across tar versions and locales), verify the toolchain actually works by running hexagon-clang --version afterwards, and pass curl --fail so a failed download surfaces immediately instead of writing an error page into the tarball.
--- tests/docker/dockerfiles/debian-hexagon-cross.docker | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
tests/docker/dockerfiles/debian-hexagon-cross.docker +10 -1
@@ -49,7 +49,16 @@ ENV TOOLCHAIN_BASENAME=clang+llvm-${TOOLCHAIN_RELEASE}-cross-hexagon-unknown-lin ENV TOOLCHAIN_URL=https://artifacts.codelinaro.org/artifactory/codelinaro-toolchain-for-hexagon/${TOOLCHAIN_RELEASE}_/${TOOLCHAIN_BASENAME}.tar.zst ENV CCACHE_WRAPPERSDIR=/usr/libexec/ccache-wrappers -RUN curl -#SL "$TOOLCHAIN_URL" | tar --zstd -xC "$TOOLCHAIN_INSTALL" +# Rootless podman's overlay storage rejects the chmod()/lchmod() calls GNU +# tar makes while restoring the toolchain symlinks' and directories' stored +# modes, with EPERM (the mode metadata itself is harmless and unused here). +# Ignore tar's exit status, but verify the toolchain actually extracted and +# runs, so a genuinely broken download or extraction still fails the build. +RUN curl --fail -#SLo /tmp/toolchain.tar.zst "$TOOLCHAIN_URL" && \ + { tar --zstd --no-same-owner --no-same-permissions \ + -xf /tmp/toolchain.tar.zst -C "$TOOLCHAIN_INSTALL" || true; } && \ + rm -f /tmp/toolchain.tar.zst && \ + "$TOOLCHAIN_INSTALL/$TOOLCHAIN_BASENAME/x86_64-linux-gnu/bin/hexagon-clang" --version ENV PATH=$PATH:${TOOLCHAIN_INSTALL}/${TOOLCHAIN_BASENAME}/x86_64-linux-gnu/bin ENV MAKE=/usr/bin/make # As a final step configure the user (if env is defined)
Signature · 1 line
2.34.1
Quoted 18 lines · Brian Cain
The debian-hexagon-cross image unpacks the codelinaro clang+llvm hexagon toolchain tarball into /opt. The archive contains symlinks (the .so/.a library aliases) and directories whose stored modes GNU tar restores via chmod()/lchmod(). Under docker, where the build runs as real root, those calls succeed. Under rootless podman the build runs in a user namespace on overlay storage, which rejects the chmod()/lchmod() calls with EPERM:
tar: .../libclang_rt.builtins.a: Cannot change mode to rwxrwxrwx: \ Operation not permitted tar: .../x86_64-linux-gnu: Cannot change mode to rwxr-xr-x: \ Operation not permitted tar: Exiting with failure status due to previous errorstar then exits non-zero and aborts the build. The mode metadata on these symlinks and directories is irrelevant for our use, and neither --no-same-permissions nor extracting as a non-root user stops GNU tar 1.35 from attempting the lchmod on the symlinks.
Quoted 9 lines
Download the tarball to a file and ignore tar's exit status during extraction with --no-same-owner and --no-same-permissions. Rather than trying to distinguish that known-benign failure from a genuinely broken download or extraction by pattern-matching tar's stderr text (fragile across tar versions and locales), verify the toolchain actually works by running hexagon-clang --version afterwards, and pass curl --fail so a failed download surfaces immediately instead of writing an error page into the tarball.
With those options, I don't think there is any reason for tar to fail. Also, as Matheus mentions, it can cover some other issues (disk full, process killed). So it would be better to remove || true. You can leave the hexagon-clang check anyway.
Quoted 27 lines
Signed-off-by Brian Cain <[email protected]>--- tests/docker/dockerfiles/debian-hexagon-cross.docker | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)tests/docker/dockerfiles/debian-hexagon-cross.docker +10
-1@@ -49,7 +49,16 @@ ENV TOOLCHAIN_BASENAME=clang+llvm-${TOOLCHAIN_RELEASE}-cross-hexagon-unknown-lin ENV TOOLCHAIN_URL=https://artifacts.codelinaro.org/artifactory/codelinaro-toolchain-for-hexagon/${TOOLCHAIN_RELEASE}_/${TOOLCHAIN_BASENAME}.tar.zst ENV CCACHE_WRAPPERSDIR=/usr/libexec/ccache-wrappers -RUN curl -#SL "$TOOLCHAIN_URL" | tar --zstd -xC "$TOOLCHAIN_INSTALL" +# Rootless podman's overlay storage rejects the chmod()/lchmod() calls GNU +# tar makes while restoring the toolchain symlinks' and directories' stored +# modes, with EPERM (the mode metadata itself is harmless and unused here). +# Ignore tar's exit status, but verify the toolchain actually extracted and +# runs, so a genuinely broken download or extraction still fails the build. +RUN curl --fail -#SLo /tmp/toolchain.tar.zst "$TOOLCHAIN_URL" && \ + { tar --zstd --no-same-owner --no-same-permissions \ + -xf /tmp/toolchain.tar.zst -C "$TOOLCHAIN_INSTALL" || true; } && \ + rm -f /tmp/toolchain.tar.zst && \ + "$TOOLCHAIN_INSTALL/$TOOLCHAIN_BASENAME/x86_64-linux-gnu/bin/hexagon-clang" --version ENV PATH=$PATH:${TOOLCHAIN_INSTALL}/${TOOLCHAIN_BASENAME}/x86_64-linux-gnu/bin ENV MAKE=/usr/bin/make # As a final step configure the user (if env is defined)
Quoted 28 lines · Pierrick Bouvier
Quoted 18 lines · Brian Cain
The debian-hexagon-cross image unpacks the codelinaro clang+llvm hexagon toolchain tarball into /opt. The archive contains symlinks (the .so/.a library aliases) and directories whose stored modes GNU tar restores via chmod()/lchmod(). Under docker, where the build runs as real root, those calls succeed. Under rootless podman the build runs in a user namespace on overlay storage, which rejects the chmod()/lchmod() calls with EPERM:
tar: .../libclang_rt.builtins.a: Cannot change mode to rwxrwxrwx: \ Operation not permitted tar: .../x86_64-linux-gnu: Cannot change mode to rwxr-xr-x: \ Operation not permitted tar: Exiting with failure status due to previous errorstar then exits non-zero and aborts the build. The mode metadata on these symlinks and directories is irrelevant for our use, and neither --no-same-permissions nor extracting as a non-root user stops GNU tar 1.35 from attempting the lchmod on the symlinks.
Quoted 9 lines
Download the tarball to a file and ignore tar's exit status during extraction with --no-same-owner and --no-same-permissions. Rather than trying to distinguish that known-benign failure from a genuinely broken download or extraction by pattern-matching tar's stderr text (fragile across tar versions and locales), verify the toolchain actually works by running hexagon-clang --version afterwards, and pass curl --fail so a failed download surfaces immediately instead of writing an error page into the tarball.
Quoted 4 lines
With those options, I don't think there is any reason for tar to fail. Also, as Matheus mentions, it can cover some other issues (disk full, process killed). So it would be better to remove || true. You can leave the hexagon-clang check anyway.
tar's failing because of the lchown for the symlinks. Agreed: masking with || true is not the right solution. Instead, let's try a different approach using .deb packages:
v3: https://lore.kernel.org/qemu-devel/[email protected]/
This should also have a nice side-effect of decreasing the container size significantly.
Quoted 28 lines
Quoted 28 lines
Signed-off-by Brian Cain <[email protected]>--- tests/docker/dockerfiles/debian-hexagon-cross.docker | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)tests/docker/dockerfiles/debian-hexagon-cross.docker +10
-1@@ -49,7 +49,16 @@ ENV TOOLCHAIN_BASENAME=clang+llvm-${TOOLCHAIN_RELEASE}-cross-hexagon-unknown-lin ENV TOOLCHAIN_URL=https://artifacts.codelinaro.org/artifactory/codelinaro-toolchain-for-hexagon/${TOOLCHAIN_RELEASE}_/${TOOLCHAIN_BASENAME}.tar.zst ENV CCACHE_WRAPPERSDIR=/usr/libexec/ccache-wrappers -RUN curl -#SL "$TOOLCHAIN_URL" | tar --zstd -xC "$TOOLCHAIN_INSTALL" +# Rootless podman's overlay storage rejects the chmod()/lchmod() calls GNU +# tar makes while restoring the toolchain symlinks' and directories' stored +# modes, with EPERM (the mode metadata itself is harmless and unused here). +# Ignore tar's exit status, but verify the toolchain actually extracted and +# runs, so a genuinely broken download or extraction still fails the build. +RUN curl --fail -#SLo /tmp/toolchain.tar.zst "$TOOLCHAIN_URL" && \ + { tar --zstd --no-same-owner --no-same-permissions \ + -xf /tmp/toolchain.tar.zst -C "$TOOLCHAIN_INSTALL" || true; } && \ + rm -f /tmp/toolchain.tar.zst && \ + "$TOOLCHAIN_INSTALL/$TOOLCHAIN_BASENAME/x86_64-linux-gnu/bin/hexagon-clang" --version ENV PATH=$PATH:${TOOLCHAIN_INSTALL}/${TOOLCHAIN_BASENAME}/x86_64-linux-gnu/bin ENV MAKE=/usr/bin/make # As a final step configure the user (if env is defined)