Skip to contentSkip to conversation

cargo-update-recipe-crates: Don't fail for partially empty Cargo.lock

PATCH Superseded Martin Schwan 3 participants 4 messages Nov 25, 2025 – Dec 11, 2025

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 cargo-update-recipe-crates: Don't fail for partially empty Cargo.lock

Do not fail if only some Cargo.lock files are empty. Only fail, if there are absolutely no dependencies found in any Cargo.lock.

This fixes the following error message, which would occur with "bitbake -c update_crates python3-orjson":

ERROR: python3-orjson-3.10.17-r0 do_update_crates: Execution of '.../python3-orjson/3.10.17/temp/run.do_update_crates.70693' failed with exit code 1
ERROR: Logfile of failure stored in: .../python3-orjson/3.10.17/temp/log.do_update_crates.70693
Log data follows:
| DEBUG: Executing python function extend_recipe_sysroot
| NOTE: Direct dependencies are ['.../sources/oe-core/../oe-core/meta/recipes-devtools/quilt/quilt-native_0.69.bb:do_populate_sysroot', 'virtual:native:.../sources/oe-core/../oe-core/meta/recipes-devtools/patch/patch_2.8.bb:do_populate_sysroot', 'virtual:native:.../sources/oe-core/../oe-core/meta/recipes-devtools/python/python3_3.13.9.bb:do_populate_sysroot'] | NOTE: Installed into sysroot: []
| NOTE: Skipping as already exists in sysroot: ['gettext-minimal-native', 'cmake-native', 'libtool-native', 'quilt-native', 'texinfo-dummy-native', 'openssl-native', 'expat-native', 'ncurses-native', 'util-linux-libuuid-native', 'zlib-native', 'libedit-native', 'make-native', 'patch-native', 'perl-native', 'python3-native', 'bzip2-native', 'xz-native', 'zstd-native', 'attr-native', 'gdbm-native', 'libffi-native', 'sqlite3-native'] | DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function do_update_crates
| Traceback (most recent call last):
| File "<stdin>", line 41, in <module>
| File "<stdin>", line 12, in get_crates
| ValueError: Unable to find any candidate crates that use crates.io
|
| The above exception was the direct cause of the following exception:
|
| Traceback (most recent call last):
| File "<stdin>", line 43, in <module>
| ValueError: Cannot parse '.../python3-orjson/3.10.17/sources/orjson-3.10.17/include/cargo/simdutf8-0.1.5/Cargo.lock'
| WARNING: exit code 1 from a shell command.
ERROR: Task (.../sources/oe-core/../meta-openembedded/meta-python/recipes-devtools/python/python3-orjson_3.10.17.bb:do_update_crates) failed with exit code '1'

After this change, a working crates include file is produced and warning messages are output instead to inform about potential issues in the corresponding project:

WARNING: python3-orjson-3.10.17-r0 do_update_crates: Unable to find any candidate crates that use crates.io
WARNING: python3-orjson-3.10.17-r0 do_update_crates: Cannot parse '.../python3-orjson/3.10.17/sources/orjson-3.10.17/include/cargo/simdutf8-0.1.5/Cargo.lock'

While at it, restructure do_update_crates() to be a real Python function and use the corresponding code directly inline.

---
 .../cargo-update-recipe-crates.bbclass        | 91 +++++++++----------
 1 file changed, 45 insertions(+), 46 deletions(-)
meta/classes-recipe/cargo-update-recipe-crates.bbclass +45 -46
index 3251d5ef2e..589f497680 100644
@@ -23,59 +23,58 @@ RECIPE_UPGRADE_EXTRA_TASKS += "do_update_crates"
 # The directory where to search for Cargo.lock files
 CARGO_LOCK_SRC_DIR ??= "${S}"
 
-do_update_crates() {
-    TARGET_FILE="${THISDIR}/${BPN}-crates.inc"
+python do_update_crates() {
+    import tomllib
 
-    nativepython3 - <<EOF
+    target_file = d.expand('${THISDIR}/${BPN}-crates.inc')
+    cargo_lock_src_dir = d.getVar('CARGO_LOCK_SRC_DIR')
 
-def get_crates(f):
-    import tomllib
-    c_list = '# from %s' % os.path.relpath(f, '${CARGO_LOCK_SRC_DIR}')
-    c_list += '\nSRC_URI += " \\\'
-    crates = tomllib.load(open(f, 'rb'))
+    def get_crates(f):
+        c_list = '# from %s' % os.path.relpath(f, cargo_lock_src_dir)
+        c_list += '\nSRC_URI += " \\'
+        crates = tomllib.load(open(f, 'rb'))
 
-    # Build a list with crates info that have crates.io in the source
-    crates_candidates = list(filter(lambda c: 'crates.io' in c.get('source', ''), crates['package']))
+        # Build a list with crates info that have crates.io in the source
+        crates_candidates = list(filter(lambda c: 'crates.io' in c.get('source', ''), crates['package']))
 
-    if not crates_candidates:
-        raise ValueError("Unable to find any candidate crates that use crates.io")
+        if not crates_candidates:
+            bb.warn('Unable to find any candidate crates that use crates.io')
+            return None
 
-    # Update crates uri and their checksum, to avoid name clashing on the checksum
-    # we need to rename crates with name and version to have a unique key
-    cksum_list = ''
-    for c in crates_candidates:
-        rename = "%s-%s" % (c['name'], c['version'])
-        c_list += '\n    crate://crates.io/%s/%s \\\' % (c['name'], c['version'])
-        if 'checksum' in c:
-            cksum_list += '\nSRC_URI[%s.sha256sum] = "%s"' % (rename, c['checksum'])
+        # Update crates uri and their checksum, to avoid name clashing on the checksum
+        # we need to rename crates with name and version to have a unique key
+        cksum_list = ''
+        for c in crates_candidates:
+            rename = "%s-%s" % (c['name'], c['version'])
+            c_list += '\n    crate://crates.io/%s/%s \\' % (c['name'], c['version'])
+            if 'checksum' in c:
+                cksum_list += '\nSRC_URI[%s.sha256sum] = "%s"' % (rename, c['checksum'])
 
-    c_list += '\n"\n'
-    c_list += cksum_list
-    c_list += '\n'
-    return c_list
+        c_list += '\n"\n'
+        c_list += cksum_list
+        c_list += '\n'
+        return c_list
 
-import os
-crates = "# Autogenerated with 'bitbake -c update_crates ${PN}'\n\n"
-found = False
-for root, dirs, files in os.walk('${CARGO_LOCK_SRC_DIR}'):
-    # ignore git and patches directories
-    if root.startswith(os.path.join('${CARGO_LOCK_SRC_DIR}', '.pc')):
-        continue
-    if root.startswith(os.path.join('${CARGO_LOCK_SRC_DIR}', '.git')):
-        continue
-    for file in files:
-        if file == 'Cargo.lock':
-            try:
+    crates = d.expand('# Autogenerated with \'bitbake -c update_crates ${PN}\'\n\n')
+    for root, dirs, files in os.walk(cargo_lock_src_dir):
+        # ignore git and patches directories
+        if root.startswith(os.path.join(cargo_lock_src_dir, '.pc')):
+            continue
+        if root.startswith(os.path.join(cargo_lock_src_dir, '.git')):
+            continue
+        for file in files:
+            if file == 'Cargo.lock':
                 cargo_lock_path = os.path.join(root, file)
-                crates += get_crates(os.path.join(root, file))
-            except Exception as e:
-                raise ValueError("Cannot parse '%s'" % cargo_lock_path) from e
-            else:
-                found = True
-if not found:
-    raise ValueError("Unable to find any Cargo.lock in ${CARGO_LOCK_SRC_DIR}")
-open("${TARGET_FILE}", 'w').write(crates)
-EOF
+                c = get_crates(cargo_lock_path)
+                if c is None:
+                    bb.warn('Cannot parse \'%s\'' % cargo_lock_path)
+                else:
+                    crates += c
+    if crates is None:
+        bb.error(f'Unable to find any Cargo.lock in {cargo_lock_src_dir}')
+
+    with open(target_file, 'w') as f:
+        f.write(crates)
 
-    bbnote "Successfully update crates inside '${TARGET_FILE}'"
+    bb.note(f'Successfully updated crates inside \'{target_file}\'')
 }
Signature · 1 line
2.43.0
#1 lore raw
Quoted 3 lines · Martin Schwan via lists.openembedded.org

Do not fail if only some Cargo.lock files are empty. Only fail, if there are absolutely no dependencies found in any Cargo.lock.

...

Quoted 2 lines

While at it, restructure do_update_crates() to be a real Python function and use the corresponding code directly inline.

This, as a diff on the mailing list, is incredibly hard to review, and needs to be split up. Refactoring in one commit (explaining the benefits), functional changes in another please. Even better if you can split things further, and explain what the particular modifications are (rather than use terms like 'restructure').

Aelx

#2 lore raw
Quoted 14 lines · Alexander Kanavin
Quoted 4 lines · Martin Schwan via lists.openembedded.org

Do not fail if only some Cargo.lock files are empty. Only fail, if there
are absolutely no dependencies found in any Cargo.lock.

...

Quoted 3 lines

While at it, restructure do_update_crates() to be a real Python function
and use the corresponding code directly inline.

Quoted 6 lines

This, as a diff on the mailing list, is incredibly hard to review, and
needs to be split up. Refactoring in one commit (explaining the benefits), functional changes in another please. Even better if you can split things further, and explain what the particular modifications are (rather than use terms like 'restructure').

Thanks for the feedback. Admittedly, I was a bit lazy with this one, as the changes were quite interwoven. I'll send a v2 and split things up, as best as I can.

Regards,
Martin

Quoted 2 lines

Aelx