Skip to contentSkip to conversation

create-spdx: ignore packing control files from ipk and deb

PATCH Jose Quaresma 2 participants 2 messages Aug 9, 2022 – Aug 10, 2022 1 message not in the archive

lore
Share & tools
lore
#1 lore raw
In reply to a message not in the archive

Hi Steve,

Can this patch be backported to kirkstone?

Jose

Jose Quaresma via lists.openembedded.org <quaresma.jose= [email protected]> escreveu no dia terça, 2/08/2022 à(s) 10:29:

Quoted 110 lines

Otherwise spdx can have references for data that is not packed in the package delivered because this contol data is temporarly and only exist while the package is been write.

During do_package_write_ipk task in do_package_ipk the control files is cleaned up at the end. This can create a race condiction when the do_create_spdx task runs the add_package_files function and these files is been deleted at same time in the task do_package_write_ipk.

ERROR: alsa-topology-conf-1.2.5.1-r0 do_create_spdx: Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>

     0001:
 *** 0002:do_create_spdx(d)
     0003:

File:
'/srv/oe/build/conf/../../layers/openembedded-core/meta/classes/create-spdx.bbclass',
lineno: 567, function: do_create_spdx
0563: package_doc.add_relationship(package_doc,
"DESCRIBES", spdx_package)

     0564:
     0565:            package_archive = deploy_dir_spdx / "packages" / (

package_doc.name + ".tar.zst")
0566: with optional_tarfile(package_archive,
archive_packaged) as archive:
*** 0567: package_files = add_package_files(

     0568:                    d,
     0569:                    package_doc,
     0570:                    spdx_package,
     0571:                    pkgdest / package,

File:
'/srv/oe/build/conf/../../layers/openembedded-core/meta/classes/create-spdx.bbclass',
lineno: 234, function: add_package_files
0230: info.mtime = source_date_epoch
0231:
0232: archive.addfile(info, f)
0233:

 *** 0234:                sha1 = bb.utils.sha1_file(filepath)
     0235:                sha1s.append(sha1)
     0236:                spdx_file.checksums.append(oe.spdx.SPDXChecksum(

0237: algorithm="SHA1",
0238: checksumValue=sha1,
File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 559, function: sha1_file
0555: """
0556: Return the hex string representation of the SHA1 checksum of
the filename

     0557:    """
     0558:    import hashlib
 *** 0559:    return _hasher(hashlib.sha1(), filename)
     0560:
     0561:def sha384_file(filename):
     0562:    """
     0563:    Return the hex string representation of the SHA384 checksum
of the filename
File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 528, function: _hasher
     0524:
     0525:def _hasher(method, filename):
     0526:    import mmap
     0527:
 *** 0528:    with open(filename, "rb") as f:
     0529:        try:
     0530:            with mmap.mmap(f.fileno(), 0,
access=mmap.ACCESS_READ) as mm:
     0531:                for chunk in iter(lambda: mm.read(8192), b''):
     0532:                    method.update(chunk)

Exception: FileNotFoundError: [Errno 2] No such file or directory:
'/srv/oe/build/tmp-lmp/work/all-lmp-linux/alsa-topology-conf/1.2.5.1-r0/packages-split/alsa-topology-conf/CONTROL/control'

ERROR: Logfile of failure stored in:
/srv/oe/build/tmp-lmp/work/all-lmp-linux/alsa-topology-conf/1.2.5.1-r0/temp/log.do_create_spdx.998864
INFO: recipe alsa-topology-conf-1.2.5.1-r0: task do_create_spdx: Failed

---
 meta/classes/create-spdx.bbclass | 1 +
 1 file changed, 1 insertion(+)
meta/classes/create-spdx.bbclass +1 -1
index 37b6b569a1..08ef7fd409 100644
@@ -571,6 +571,7 @@ python do_create_spdx() {
                     pkgdest / package,
                     lambda file_counter:
 oe.sbom.get_packaged_file_spdxid(pkg_name, file_counter),
                     lambda filepath: ["BINARY"],
+                    ignore_top_level_dirs=['CONTROL', 'DEBIAN'],
                     archive=archive,
                 )
 
--

2.37.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#168773):
https://lists.openembedded.org/g/openembedded-core/message/168773
Mute This Topic: https://lists.openembedded.org/mt/92767075/5052612
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 3 lines
Best regards,

José Quaresma
#2 lore raw
Quoted 4 lines · Jose Quaresma

Hi Steve,

Can this patch be backported to kirkstone?

Yes, will do!

Steve

Quoted 99 lines

Jose Quaresma via lists.openembedded.org <[email protected]> escreveu no dia terça, 2/08/2022 à(s) 10:29:

Quoted 84 lines

Otherwise spdx can have references for data that is not packed in the package delivered because this contol data is temporarly and only exist while the package is been write.

During do_package_write_ipk task in do_package_ipk the control files is cleaned up at the end. This can create a race condiction when the do_create_spdx task runs the add_package_files function and these files is been deleted at same time in the task do_package_write_ipk.

ERROR: alsa-topology-conf-1.2.5.1-r0 do_create_spdx: Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>

     0001:
 *** 0002:do_create_spdx(d)
     0003:

File: '/srv/oe/build/conf/../../layers/openembedded-core/meta/classes/create-spdx.bbclass', lineno: 567, function: do_create_spdx
0563: package_doc.add_relationship(package_doc, "DESCRIBES", spdx_package)
0564:
0565: package_archive = deploy_dir_spdx / "packages" / (package_doc.name + ".tar.zst")
0566: with optional_tarfile(package_archive, archive_packaged) as archive:
*** 0567: package_files = add_package_files(

     0568:                    d,
     0569:                    package_doc,
     0570:                    spdx_package,
     0571:                    pkgdest / package,

File: '/srv/oe/build/conf/../../layers/openembedded-core/meta/classes/create-spdx.bbclass', lineno: 234, function: add_package_files
0230: info.mtime = source_date_epoch
0231:
0232: archive.addfile(info, f)
0233:

 *** 0234:                sha1 = bb.utils.sha1_file(filepath)
     0235:                sha1s.append(sha1)
     0236:                spdx_file.checksums.append(oe.spdx.SPDXChecksum(
     0237:                        algorithm="SHA1",
     0238:                        checksumValue=sha1,
File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 559, function: sha1_file
     0555:    """
     0556:    Return the hex string representation of the SHA1 checksum of the filename
     0557:    """
     0558:    import hashlib
 *** 0559:    return _hasher(hashlib.sha1(), filename)
     0560:
     0561:def sha384_file(filename):
     0562:    """
     0563:    Return the hex string representation of the SHA384 checksum of the filename
File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 528, function: _hasher
     0524:
     0525:def _hasher(method, filename):
     0526:    import mmap
     0527:
 *** 0528:    with open(filename, "rb") as f:
     0529:        try:
     0530:            with mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) as mm:
     0531:                for chunk in iter(lambda: mm.read(8192), b''):
     0532:                    method.update(chunk)

Exception: FileNotFoundError: [Errno 2] No such file or directory: '/srv/oe/build/tmp-lmp/work/all-lmp-linux/alsa-topology-conf/1.2.5.1-r0/packages-split/alsa-topology-conf/CONTROL/control'

ERROR: Logfile of failure stored in: /srv/oe/build/tmp-lmp/work/all-lmp-linux/alsa-topology-conf/1.2.5.1-r0/temp/log.do_create_spdx.998864
INFO: recipe alsa-topology-conf-1.2.5.1-r0: task do_create_spdx: Failed

---
 meta/classes/create-spdx.bbclass | 1 +
 1 file changed, 1 insertion(+)
meta/classes/create-spdx.bbclass +1 -1
index 37b6b569a1..08ef7fd409 100644
@@ -571,6 +571,7 @@ python do_create_spdx() {
                     pkgdest / package,
                     lambda file_counter: oe.sbom.get_packaged_file_spdxid(pkg_name, file_counter),
                     lambda filepath: ["BINARY"],
+                    ignore_top_level_dirs=['CONTROL', 'DEBIAN'],
                     archive=archive,
                 )
 
--

2.37.1

Quoted 8 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#168773): https://lists.openembedded.org/g/openembedded-core/message/168773
Mute This Topic: https://lists.openembedded.org/mt/92767075/5052612
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

--
Best regards,

José Quaresma