rpm2cpio.sh: update to the last 4.x version
PATCH v2 1/2 rpm2cpio.sh: update to the last 4.x version
From: Alberto Planas <[email protected]>
openSUSE RPMs are compressing the RPM payload using zstd, that correspond to the magic ID 0x28, 0xb5, 0x2f.
This patch update the script to the last version from the rpm project, and add support to this compression format, and extract the cpio payload using the "unzstd" binary.
--- scripts/rpm2cpio.sh | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) mode change 100755 => 100644 scripts/rpm2cpio.sh
scripts/rpm2cpio.sh +20 -10
@@ -7,7 +7,7 @@ fatal() { } pkg="$1" -[ -n "$pkg" -a -e "$pkg" ] || +[ -n "$pkg" ] && [ -e "$pkg" ] || fatal "No package supplied" _dd() {
@@ -16,14 +16,23 @@ _dd() { } calcsize() { + + case "$(_dd $1 bs=4 count=1 | tr -d '\0')" in + "$(printf '\216\255\350')"*) ;; # '\x8e\xad\xe8' + *) fatal "File doesn't look like rpm: $pkg" ;; + esac + offset=$(($1 + 8)) local i b b0 b1 b2 b3 b4 b5 b6 b7 i=0 while [ $i -lt 8 ]; do - b=$(_dd $(($offset + $i)) bs=1 count=1; echo X) - b=${b%X} + # add . to not loose \n + # strip \0 as it gets dropped with warning otherwise + b="$(_dd $(($offset + $i)) bs=1 count=1 | tr -d '\0' ; echo .)" + b=${b%.} # strip . again + [ -z "$b" ] && b="0" || b="$(exec printf '%u\n' "'$b")"
@@ -35,7 +44,7 @@ calcsize() { offset=$(($offset + $rsize)) } -case "$(_dd 0 bs=8 count=1)" in +case "$(_dd 0 bs=4 count=1 | tr -d '\0')" in "$(printf '\355\253\356\333')"*) ;; # '\xed\xab\xee\xdb' *) fatal "File doesn't look like rpm: $pkg" ;; esac
@@ -46,10 +55,11 @@ sigsize=$rsize calcsize $(($offset + (8 - ($sigsize % 8)) % 8)) hdrsize=$rsize -case "$(_dd $offset bs=3 count=1)" in - "$(printf '\102\132')"*) _dd $offset | bzip2 -d ;; # '\x42\x5a' - "$(printf '\037\213')"*) _dd $offset | gunzip ;; # '\x1f\x8b' - "$(printf '\375\067')"*) _dd $offset | xzcat ;; # '\xfd\x37' - "$(printf '\135\000')"*) _dd $offset | unlzma ;; # '\x5d\x00' - *) fatal "Unrecognized rpm file: $pkg" ;; +case "$(_dd $offset bs=2 count=1 | tr -d '\0')" in + "$(printf '\102\132')") _dd $offset | bunzip2 ;; # '\x42\x5a' + "$(printf '\037\213')") _dd $offset | gunzip ;; # '\x1f\x8b' + "$(printf '\375\067')") _dd $offset | xzcat ;; # '\xfd\x37' + "$(printf '\135')") _dd $offset | unlzma ;; # '\x5d\x00' + "$(printf '\050\265')") _dd $offset | unzstd ;; # '\x28\xb5' + *) fatal "Unrecognized payload compression format in rpm file: $pkg" ;; esac
Signature · 1 line
2.41.0
PATCH v2 2/2 bitbake.conf: add unzstd in HOSTTOOLS
From: Alberto Planas <[email protected]>
rpm2cpio.sh can make calls to unzstd to uncompress the RPM payload that conform the cpio file.
zstd is already part of HOSTTOOLS, as a link to the system installed zstd.
This patch add unzstd in HOSTOOLS list as a non-optional binary, so is available to rpm2cpio.sh when it is required.
--- meta/conf/bitbake.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/conf/bitbake.conf +1 -1
@@ -524,7 +524,7 @@ HOSTTOOLS += " \ python3 pzstd ranlib readelf readlink realpath rm rmdir rpcgen sed seq sh \ sha1sum sha224sum sha256sum sha384sum sha512sum \ sleep sort split stat strings strip tail tar tee test touch tr true uname \ - uniq wc wget which xargs zstd \ + uniq unzstd wc wget which xargs zstd \ " # Tools needed to run testimage runtime image testing
Signature · 1 line
2.41.0
Re: [OE-core] [PATCH v2 2/2] bitbake.conf: add unzstd in HOSTTOOLS
Quoted 21 lines · Alberto Planas via lists.openembedded.org
From: Alberto Planas <[email protected]>
rpm2cpio.shcan make calls to unzstd to uncompress the RPM payload that conform the cpio file.zstd is already part of HOSTTOOLS, as a link to the system installed zstd.
This patch add unzstd in HOSTOOLS list as a non-optional binary, so is available to
rpm2cpio.shwhen it is required.Signed-off-by Alberto Planas <[email protected]>--- meta/conf/bitbake.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)meta/conf/bitbake.conf +0
-0
meta/conf/bitbake.conf HOSTTOOLS += " \ · line 530
Quoted 8 lines
meta/conf/bitbake.conf +1
-1@@ -524,7 +524,7 @@ HOSTTOOLS += " \ python3 pzstd ranlib readelf readlink realpath rm rmdir rpcgen sed seq sh \ sha1sum sha224sum sha256sum sha384sum sha512sum \ sleep sort split stat strings strip tail tar tee test touch tr true uname \ - uniq wc wget which xargs zstd \ + uniq unzstd wc wget which xargs zstd \ " # Tools needed to run testimage runtime image testing
Couldn't we just use zstd -d in the script? They're always the same binary anyway as far as I understand it...
Cheers,
Richard
Re: [OE-core] [PATCH v2 2/2] bitbake.conf: add unzstd in HOSTTOOLS
Quoted 4 lines
From: Richard Purdie <[email protected]>
Couldn't we just use zstd -d in the script? They're always the same binary anyway as far as I understand it...
Yes, this should work. I decided to include the second patch to avoid changing the rpm2cpio.sh code, that is a verbatim copy of the upstream script (minus the added comment from the old version, that reference where to find the script)
If this is the preferred way I can do a v3 that drop this patch and changes rpm2cpio.sh.
Re: [OE-core] [PATCH v2 2/2] bitbake.conf: add unzstd in HOSTTOOLS
Quoted 5 lines · Alberto Planas Dominguez
Quoted 4 lines
From: Richard Purdie <[email protected]>
Couldn't we just use zstd -d in the script? They're always the same binary anyway as far as I understand it...
Quoted 7 lines
Yes, this should work. I decided to include the second patch to avoid changing the
rpm2cpio.shcode, that is a verbatim copy of the upstream script (minus the added comment from the old version, that reference where to find the script)If this is the preferred way I can do a v3 that drop this patch and changes
rpm2cpio.sh.
I guess we can stay in sync then, I was just hoping not to add more to HOSTTOOLS. Thanks for the context.
Cheers,
Richard
Re: [OE-core] [PATCH v2 2/2] bitbake.conf: add unzstd in HOSTTOOLS
I am seeing some race condition see
https://errors.yoctoproject.org/Errors/Details/708923/
Quoted 30 lines · Richard Purdie
Quoted 5 lines · Alberto Planas Dominguez
Quoted 4 lines
From: Richard Purdie <[email protected]>
Couldn't we just use zstd -d in the script? They're always the same binary anyway as far as I understand it...
Quoted 7 lines
Yes, this should work. I decided to include the second patch to avoid changing the
rpm2cpio.shcode, that is a verbatim copy of the upstream script (minus the added comment from the old version, that reference where to find the script)If this is the preferred way I can do a v3 that drop this patch and changes
rpm2cpio.sh.I guess we can stay in sync then, I was just hoping not to add more to HOSTTOOLS. Thanks for the context.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182861): https://lists.openembedded.org/g/openembedded-core/message/182861
Mute This Topic: https://lists.openembedded.org/mt/99546093/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
I am seeing some race condition see
https://errors.yoctoproject.org/Errors/Details/708923/
Quoted 96 lines · Alberto Planas via lists.openembedded.org
From: Alberto Planas <[email protected]>
openSUSE RPMs are compressing the RPM payload using zstd, that correspond to the magic ID
0x28,0xb5,0x2f.This patch update the script to the last version from the rpm project, and add support to this compression format, and extract the cpio payload using the "unzstd" binary.
Signed-off-by Alberto Planas <[email protected]>--- scripts/rpm2cpio.sh | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) mode change 100755 => 100644 scripts/rpm2cpio.shscripts/rpm2cpio.sh +20
-11@@ -7,7 +7,7 @@ fatal() { } pkg="$1" -[ -n "$pkg" -a -e "$pkg" ] || +[ -n "$pkg" ] && [ -e "$pkg" ] || fatal "No package supplied" _dd() {@@ -16,14 +16,23 @@ _dd() { } calcsize() { + + case "$(_dd $1 bs=4 count=1 | tr -d '\0')" in + "$(printf '\216\255\350')"*) ;; # '\x8e\xad\xe8' + *) fatal "File doesn't look like rpm: $pkg" ;; + esac + offset=$(($1 + 8)) local i b b0 b1 b2 b3 b4 b5 b6 b7 i=0 while [ $i -lt 8 ]; do - b=$(_dd $(($offset + $i)) bs=1 count=1; echo X) - b=${b%X} + # add . to not loose \n + # strip \0 as it gets dropped with warning otherwise + b="$(_dd $(($offset + $i)) bs=1 count=1 | tr -d '\0' ; echo .)" + b=${b%.} # strip . again + [ -z "$b" ] && b="0" || b="$(exec printf '%u\n' "'$b")"@@ -35,7 +44,7 @@ calcsize() { offset=$(($offset + $rsize)) } -case "$(_dd 0 bs=8 count=1)" in +case "$(_dd 0 bs=4 count=1 | tr -d '\0')" in "$(printf '\355\253\356\333')"*) ;; # '\xed\xab\xee\xdb' *) fatal "File doesn't look like rpm: $pkg" ;; esac@@ -46,10 +55,11 @@ sigsize=$rsize calcsize $(($offset + (8 - ($sigsize % 8)) % 8)) hdrsize=$rsize -case "$(_dd $offset bs=3 count=1)" in - "$(printf '\102\132')"*) _dd $offset | bzip2 -d ;; # '\x42\x5a' - "$(printf '\037\213')"*) _dd $offset | gunzip ;; # '\x1f\x8b' - "$(printf '\375\067')"*) _dd $offset | xzcat ;; # '\xfd\x37' - "$(printf '\135\000')"*) _dd $offset | unlzma ;; # '\x5d\x00' - *) fatal "Unrecognized rpm file: $pkg" ;; +case "$(_dd $offset bs=2 count=1 | tr -d '\0')" in + "$(printf '\102\132')") _dd $offset | bunzip2 ;; # '\x42\x5a' + "$(printf '\037\213')") _dd $offset | gunzip ;; # '\x1f\x8b' + "$(printf '\375\067')") _dd $offset | xzcat ;; # '\xfd\x37' + "$(printf '\135')") _dd $offset | unlzma ;; # '\x5d\x00' + "$(printf '\050\265')") _dd $offset | unzstd ;; # '\x28\xb5' + *) fatal "Unrecognized payload compression format in rpm file: $pkg" ;; esac --2.41.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#182844): https://lists.openembedded.org/g/openembedded-core/message/182844
Mute This Topic: https://lists.openembedded.org/mt/99546092/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
From: Khem Raj <[email protected]>
Quoted 2 lines
I am seeing some race condition see
https://errors.yoctoproject.org/Errors/Details/708923/
Confirmed. It is the second time that I do the same mistake. I hope v3 finally fix it.