Skip to contentSkip to conversation

fetch2: Display all missing checksum at once

PATCH In review [email protected] 5 participants 17 messages Apr 1, 2023 – Apr 5, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fetch2: Display all missing checksum at once

From: Frederic Martinsons <[email protected]>

Instead of only display the first and stop.
For recipe (crago based) that can contains several artifacts to fetch with their checksum, it will particularly handy to display all of missing one.

An example of error message would be

NOTE: Executing Tasks
ERROR: zvariant-3.12.0-r0 do_fetch: Missing SRC_URI checksum, please add those to the recipe:
SRC_URI[anyhow-1.0.70.sha256sum] = "7de8ce5e0f9f8d88245311066a578d72b7af3e7088f32783804676302df237e4" SRC_URI[chrono-0.4.24.sha256sum] = "4e3c5919066adf22df73762e50cffcde3a758f2a848b113b586d1f86728b673b" SRC_URI[serde-1.0.158.sha256sum] = "771d4d9c4163ee138805e12c710dd365e4f44be8be0503cb1bb9eb989425d9c9" SRC_URI[system-deps-1.3.2.sha256sum] = "0f3ecc17269a19353b3558b313bba738b25d82993e30d62a18406a24aba4649b"
ERROR: zvariant-3.12.0-r0 do_fetch: Bitbake Fetcher Error: BBFetchException('There was some missing checksums in the recipe')
ERROR: Logfile of failure stored in: /home/jenkins/yocto-poky-master/poky/build/tmp/work/core2-64-poky-linux/zvariant/3.12.0-r0/temp/log.do_fetch.1025
ERROR: Task (/home/jenkins/yocto-poky-master/poky/meta-selftest/recipes-extended/zvariant/zvariant_3.12.0.bb:do_fetch) failed with exit code '1'

---
 lib/bb/fetch2/__init__.py | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
lib/bb/fetch2/__init__.py +10 -8
index 31729885..1a86d8fd 100644
@@ -560,7 +560,6 @@ def verify_checksum(ud, d, precomputed={}, localpath=None, fatal_nochecksum=True
     file against those in the recipe each time, rather than only after
     downloading. See https://bugzilla.yoctoproject.org/show_bug.cgi?id=5571.
     """
-
     if ud.ignore_checksums or not ud.method.supports_checksum(ud):
         return {}
 
@@ -605,11 +604,7 @@ def verify_checksum(ud, d, precomputed={}, localpath=None, fatal_nochecksum=True
 
         # If strict checking enabled and neither sum defined, raise error
         if strict == "1":
-            messages.append("No checksum specified for '%s', please add at " \
-                            "least one to the recipe:" % ud.localpath)
-            messages.extend(checksum_lines)
-            logger.error("\n".join(messages))
-            raise NoChecksumError("Missing SRC_URI checksum", ud.url)
+            raise NoChecksumError("\n".join(checksum_lines))
 
         bb.event.fire(MissingChecksumEvent(ud.url, **checksum_event), d)
 
@@ -1728,6 +1723,7 @@ class Fetch(object):
         network = self.d.getVar("BB_NO_NETWORK")
         premirroronly = bb.utils.to_boolean(self.d.getVar("BB_FETCH_PREMIRRORONLY"))
 
+        checksum_missing_messages = []
         for u in urls:
             ud = self.ud[u]
             ud.setup_localpath(self.d)
@@ -1739,7 +1735,6 @@ class Fetch(object):
 
             try:
                 self.d.setVar("BB_NO_NETWORK", network)
-
                 if m.verify_donestamp(ud, self.d) and not m.need_update(ud, self.d):
                     done = True
                 elif m.try_premirror(ud, self.d):
@@ -1811,13 +1806,20 @@ class Fetch(object):
                     raise ChecksumError("Stale Error Detected")
 
             except BBFetchException as e:
-                if isinstance(e, ChecksumError):
+                if isinstance(e, NoChecksumError):
+                    (message, _) = e.args
+                    checksum_missing_messages.append(message)
+                    continue
+                elif isinstance(e, ChecksumError):
                     logger.error("Checksum failure fetching %s" % u)
                 raise
 
             finally:
                 if ud.lockfile:
                     bb.utils.unlockfile(lf)
+        if checksum_missing_messages:
+            logger.error("Missing SRC_URI checksum, please add those to the recipe: \n%s", "\n".join(checksum_missing_messages))
+            raise BBFetchException("There was some missing checksums in the recipe")
 
     def checkstatus(self, urls=None):
         """
Signature · 1 line
2.34.1
#1 lore raw
Quoted 8 lines · Frederic Martinsons

From: Frederic Martinsons <[email protected]>

Instead of only display the first and stop.
For recipe (crago based) that can contains several artifacts to fetch with their checksum, it will particularly handy to display all of missing one.

An example of error message would be

I am confused. Is this patch supposed to add the pkg version or not?  I am not seeing any. Any ideas?

I will be holding off branching meta-security until my 4 rust recipes can build again.

-armin

Quoted 88 lines

NOTE: Executing Tasks
ERROR: zvariant-3.12.0-r0 do_fetch: Missing SRC_URI checksum, please add those to the recipe:
SRC_URI[anyhow-1.0.70.sha256sum] = "7de8ce5e0f9f8d88245311066a578d72b7af3e7088f32783804676302df237e4" SRC_URI[chrono-0.4.24.sha256sum] = "4e3c5919066adf22df73762e50cffcde3a758f2a848b113b586d1f86728b673b" SRC_URI[serde-1.0.158.sha256sum] = "771d4d9c4163ee138805e12c710dd365e4f44be8be0503cb1bb9eb989425d9c9" SRC_URI[system-deps-1.3.2.sha256sum] = "0f3ecc17269a19353b3558b313bba738b25d82993e30d62a18406a24aba4649b"
ERROR: zvariant-3.12.0-r0 do_fetch: Bitbake Fetcher Error: BBFetchException('There was some missing checksums in the recipe')
ERROR: Logfile of failure stored in: /home/jenkins/yocto-poky-master/poky/build/tmp/work/core2-64-poky-linux/zvariant/3.12.0-r0/temp/log.do_fetch.1025
ERROR: Task (/home/jenkins/yocto-poky-master/poky/meta-selftest/recipes-extended/zvariant/zvariant_3.12.0.bb:do_fetch) failed with exit code '1'

---
  lib/bb/fetch2/__init__.py | 18 ++++++++++--------
  1 file changed, 10 insertions(+), 8 deletions(-)
lib/bb/fetch2/__init__.py +10 -9
index 31729885..1a86d8fd 100644
@@ -560,7 +560,6 @@ def verify_checksum(ud, d, precomputed={}, localpath=None, fatal_nochecksum=True
      file against those in the recipe each time, rather than only after
      downloading. See https://bugzilla.yoctoproject.org/show_bug.cgi?id=5571.
      """
-
      if ud.ignore_checksums or not ud.method.supports_checksum(ud):
          return {}
 
@@ -605,11 +604,7 @@ def verify_checksum(ud, d, precomputed={}, localpath=None, fatal_nochecksum=True
 
          # If strict checking enabled and neither sum defined, raise error
          if strict == "1":
-            messages.append("No checksum specified for '%s', please add at " \
-                            "least one to the recipe:" % ud.localpath)
-            messages.extend(checksum_lines)
-            logger.error("\n".join(messages))
-            raise NoChecksumError("Missing SRC_URI checksum", ud.url)
+            raise NoChecksumError("\n".join(checksum_lines))
 
          bb.event.fire(MissingChecksumEvent(ud.url, **checksum_event), d)
 
@@ -1728,6 +1723,7 @@ class Fetch(object):
          network = self.d.getVar("BB_NO_NETWORK")
          premirroronly = bb.utils.to_boolean(self.d.getVar("BB_FETCH_PREMIRRORONLY"))
 
+        checksum_missing_messages = []
          for u in urls:
              ud = self.ud[u]
              ud.setup_localpath(self.d)
@@ -1739,7 +1735,6 @@ class Fetch(object):
 
              try:
                  self.d.setVar("BB_NO_NETWORK", network)
-
                  if m.verify_donestamp(ud, self.d) and not m.need_update(ud, self.d):
                      done = True
                  elif m.try_premirror(ud, self.d):
@@ -1811,13 +1806,20 @@ class Fetch(object):
                      raise ChecksumError("Stale Error Detected")
 
              except BBFetchException as e:
-                if isinstance(e, ChecksumError):
+                if isinstance(e, NoChecksumError):
+                    (message, _) = e.args
+                    checksum_missing_messages.append(message)
+                    continue
+                elif isinstance(e, ChecksumError):
                      logger.error("Checksum failure fetching %s" % u)
                  raise
 
              finally:
                  if ud.lockfile:
                      bb.utils.unlockfile(lf)
+        if checksum_missing_messages:
+            logger.error("Missing SRC_URI checksum, please add those to the recipe: \n%s", "\n".join(checksum_missing_messages))
+            raise BBFetchException("There was some missing checksums in the recipe")
 
      def checkstatus(self, urls=None):
          """
 
-=-=-=-=-=-=-=-=-=-=-=-

Links: You receive all messages sent to this group.
View/Reply Online (#14629): https://lists.openembedded.org/g/bitbake-devel/message/14629
Mute This Topic: https://lists.openembedded.org/mt/97996454/1024635
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#2 lore raw
Quoted 12 lines · akuster
Quoted 8 lines · Frederic Martinsons

From: Frederic Martinsons <[email protected]>

Instead of only display the first and stop.
For recipe (crago based) that can contains several artifacts to fetch with their checksum, it will particularly handy to display all of missing one.

An example of error message would be

Quoted 9 lines

I am confused. Is this patch supposed to add the pkg version or not? I am not seeing any. Any ideas?

I will be holding off branching meta-security until my 4 rust recipes can build again.

-armin

Hello,

This patch is only for showing all missing checksum on crates for a recipe. To build the checskum list (with the version), you can use bitbake -c update_crates <your recipe>
and if you start from a pristine, you should have an error which will contain
all the data you want to copy/paste (if you have this present patch in bitbake)

The naming for pkg and version is already included in cargo-update-recipe-crates.bbclass
<https://git.openembedded.org/openembedded-core/tree/meta/classes-recipe/cargo-update-recipe-crates.bbclass?id=1795e98a04ad09b011afcc7cc3bf6dc49475b19a>
from oe-core
master:
see
https://git.openembedded.org/openembedded-core/commit/?id=1795e98a04ad09b011afcc7cc3bf6dc49475b19a

#3 lore raw
Quoted 23 lines · Frédéric Martinsons

On Tue, 4 Apr 2023 at 18:50, akuster <[email protected]> wrote:

On 4/1/23 11:21 AM, Frederic Martinsons wrote:
> From: Frederic Martinsons <[email protected]> >
> Instead of only display the first and stop.
> For recipe (crago based) that can contains several artifacts > to fetch with their checksum, it will particularly handy > to display all of missing one.
>
> An example of error message would be

I am confused. Is this patch supposed to add the pkg version or not?  I
am not seeing any. Any ideas?

I will be holding off branching meta-security until my 4 rust recipes can build again.

Quoted 7 lines

-armin

Hello,

This patch is only for showing all missing checksum on crates for a recipe.
To build the checskum list (with the version), you can use bitbake -c update_crates <your recipe>

update_crates fails when it starts to traverse  into the vendor dir. It can't parse the Cargo.toml files correctly.

Quoted 6 lines

and if you start from a pristine, you should have an error which will contain all the data you want to copy/paste (if you have this present patch in bitbake)

The naming for pkg and version is already included in cargo-update-recipe-crates.bbclass <https://git.openembedded.org/openembedded-core/tree/meta/classes-recipe/cargo-update-recipe-crates.bbclass?id=1795e98a04ad09b011afcc7cc3bf6dc49475b19a> from oe-core master:
see https://git.openembedded.org/openembedded-core/commit/?id=1795e98a04ad09b011afcc7cc3bf6dc49475b19a

#4 lore raw
In reply to akuster
Quoted 10 lines · akuster
Quoted 7 lines · Frederic Martinsons

From: Frederic Martinsons <[email protected]>

Instead of only display the first and stop.
For recipe (crago based) that can contains several artifacts to fetch with their checksum, it will particularly handy to display all of missing one.

Quoted 5 lines
Quoted 1 line

An example of error message would be

I am confused. Is this patch supposed to add the pkg version or not? I am not seeing any. Any ideas?

name parameter with pkg version was added to bbclass in:
https://git.openembedded.org/openembedded-core/commit/?id=1795e98a04ad09b011afcc7cc3bf6dc49475b19a

this change only allows all missing/incorrect checksums to be showsn at the same time, instead of one checksum per do_fetch failure.

#5 lore raw
Quoted 19 lines · Martin Jansa

On Tue, Apr 4, 2023 at 6:50 PM akuster <[email protected]> wrote:

On 4/1/23 11:21 AM, Frederic Martinsons wrote:
> From: Frederic Martinsons <[email protected]> >
> Instead of only display the first and stop.
> For recipe (crago based) that can contains several artifacts > to fetch with their checksum, it will particularly handy > to display all of missing one.
>
> An example of error message would be

I am confused. Is this patch supposed to add the pkg version or not?  I
am not seeing any. Any ideas?

Quoted 4 lines

name parameter with pkg version was added to bbclass in:
https://git.openembedded.org/openembedded-core/commit/?id=1795e98a04ad09b011afcc7cc3bf6dc49475b19a

this change only allows all missing/incorrect checksums to be showsn at the same time, instead of one checksum per do_fetch failure.

I get multiple hashes for the same package.

SRC_URI[wasi.sha256sum] = "1a143597ca7c7793eff794def352d41792a93c481eb1042423ff7ff72ba2c31f" SRC_URI[wasi.sha256sum] = "cccddf32554fecc6acb585f82a32a72e28b48f8c4c1883ddfeeeaa96f7d8e519"

so why no version info after the pkg name?

-armin

#6 lore raw
Quoted 29 lines · akuster
Quoted 12 lines · Martin Jansa

On Tue, Apr 4, 2023 at 6:50 PM akuster <[email protected]> wrote:

On 4/1/23 11:21 AM, Frederic Martinsons wrote:
> From: Frederic Martinsons <[email protected]> >
> Instead of only display the first and stop.
> For recipe (crago based) that can contains several artifacts > to fetch with their checksum, it will particularly handy > to display all of missing one.
>

Quoted 9 lines

> An example of error message would be

I am confused. Is this patch supposed to add the pkg version or not? I
am not seeing any. Any ideas?

name parameter with pkg version was added to bbclass in:

https://git.openembedded.org/openembedded-core/commit/?id=1795e98a04ad09b011afcc7cc3bf6dc49475b19a

Quoted 3 lines

this change only allows all missing/incorrect checksums to be showsn at the same time, instead of one checksum per do_fetch failure.

Quoted 9 lines

I get multiple hashes for the same package.

SRC_URI[wasi.sha256sum] = "1a143597ca7c7793eff794def352d41792a93c481eb1042423ff7ff72ba2c31f" SRC_URI[wasi.sha256sum] = "cccddf32554fecc6acb585f82a32a72e28b48f8c4c1883ddfeeeaa96f7d8e519"

so why no version info after the pkg name?

Did you regenerate the SRC_URIs with updated bbclass? It works for me as in:
https://github.com/shr-project/meta-webosose/commit/84dc9f3cc3dd85facde07ccddce0b9f2bcd90d39

#7 lore raw

I looked at the suricata recipe in meta-security and it didn't use cargo-update-recipes.bbclass

You should modify your rust recipes to use this class and have access to "bitbake -c update_crates" .

You can take some examples from , i.e python3-cryptography in openembedded-core or python3-pyruvate
in meta-python.

The first issue you'll encounter is some kind of bootstrapping (since update_crates needs to fetch
to be able to write checksum but the fetcher needs the checksum ...). With this error , you'll
see all the data you need to add to the suricata-crates.inc.

For curiosity, I tested locally the modification of suricata and it seems that this package embeds
unexpected Cargo.lock format (in rust/vendor/alloc-no-stdlib/Cargo.lock) which doesn't contain
a source entry (the package doesn't have any dependency).

I don't know how to handle such a file (simply ignore it in the class instead of raising an error ?)

Quoted 46 lines · Martin Jansa
Quoted 29 lines · akuster
Quoted 12 lines · Martin Jansa

On Tue, Apr 4, 2023 at 6:50 PM akuster <[email protected]> wrote:

On 4/1/23 11:21 AM, Frederic Martinsons wrote:
> From: Frederic Martinsons <[email protected]> >
> Instead of only display the first and stop.
> For recipe (crago based) that can contains several artifacts > to fetch with their checksum, it will particularly handy > to display all of missing one.
>

Quoted 9 lines

> An example of error message would be

I am confused. Is this patch supposed to add the pkg version or not? I
am not seeing any. Any ideas?

name parameter with pkg version was added to bbclass in:

https://git.openembedded.org/openembedded-core/commit/?id=1795e98a04ad09b011afcc7cc3bf6dc49475b19a

Quoted 3 lines

this change only allows all missing/incorrect checksums to be showsn at the same time, instead of one checksum per do_fetch failure.

Quoted 9 lines

I get multiple hashes for the same package.

SRC_URI[wasi.sha256sum] = "1a143597ca7c7793eff794def352d41792a93c481eb1042423ff7ff72ba2c31f" SRC_URI[wasi.sha256sum] = "cccddf32554fecc6acb585f82a32a72e28b48f8c4c1883ddfeeeaa96f7d8e519"

so why no version info after the pkg name?

Did you regenerate the SRC_URIs with updated bbclass? It works for me as in:

https://github.com/shr-project/meta-webosose/commit/84dc9f3cc3dd85facde07ccddce0b9f2bcd90d39

#8 lore raw
Quoted 5 lines · Frederic Martinsons

I looked at the suricata recipe in meta-security and it didn't use cargo-update-recipes.bbclass

You should modify your rust recipes to use this class and have access to "bitbake -c update_crates" .

Is this really the recommended way for rust packages? As you already mentioned, this has a big problem with bootstrapping: fetch (for downloading the crates) and compile (to create Cargo.lock) must be run before do_update_crates works.

IMO, the crate fetcher should be changed to create versioned 'name'. Then the 'cargo bitbake' output can be used directly and missing checksums can be copied & pasted.

Enrico

#9 lore raw

Hello,

I'm pretty new to cargo package (and even more with the support of it inside yocto) but I read from
that the update_crates bitbake task is the way to go for those recipes and
that
cargo bitbake is not supported in yocto (see
https://lists.openembedded.org/g/openembedded-core/message/178415)

Quoted 21 lines · Enrico Scholz
Quoted 5 lines · Frederic Martinsons

I looked at the suricata recipe in meta-security and it didn't use cargo-update-recipes.bbclass

You should modify your rust recipes to use this class and have access to "bitbake -c update_crates" .

Is this really the recommended way for rust packages? As you already mentioned, this has a big problem with bootstrapping: fetch (for downloading the crates) and compile (to create Cargo.lock) must be run before do_update_crates works.

IMO, the crate fetcher should be changed to create versioned 'name'. Then the 'cargo bitbake' output can be used directly and missing checksums can be copied & pasted.

Enrico

#10 lore raw
Quoted 5 lines · Frédéric Martinsons

I'm pretty new to cargo package (and even more with the support of it inside yocto) but I read from that the update_crates bitbake task is the way to go for those recipes and that cargo bitbake is not supported in yocto (see
https://lists.openembedded.org/g/openembedded-core/message/178415)

well, the problem is, that 'cargo-update-recipe-crates' can be only used to complete the list of crates with their sha256sum entries.

It is not suitable for creating the initial list of crates (which is not a trivial task). You have to use something like 'cargo bitbake' for it (regardless of its maintenance status).

Enrico

#11 lore raw

As a regular recipe which miss a checksum on its src uri, you can just let bitbake fetch and output the whole list of missing checksums, then copy paste that into your ${BPN}-crates.inc file.

Quoted 19 lines · Enrico Scholz
Quoted 5 lines · Frédéric Martinsons

I'm pretty new to cargo package (and even more with the support of it inside yocto) but I read from that the update_crates bitbake task is the way to go for those recipes and that cargo bitbake is not supported in yocto (see
https://lists.openembedded.org/g/openembedded-core/message/178415)

well, the problem is, that 'cargo-update-recipe-crates' can be only used to complete the list of crates with their sha256sum entries.

It is not suitable for creating the initial list of crates (which is not a trivial task). You have to use something like 'cargo bitbake' for it (regardless of its maintenance status).

Enrico

#12 lore raw
In reply to Frédéric Martinsons
Quoted 1 line · Frédéric Martinsons

I looked at the suricata recipe in meta-security and it didn't use cargo-update-recipes.bbclass

Right as the recipes existed long before this new class.

Quoted 6 lines

You should modify your rust recipes to use this class and have access to "bitbake -c update_crates" .

You can take some examples from , i.e python3-cryptography in openembedded-core or python3-pyruvate in meta-python.

Quoted 7 lines

The first issue you'll encounter is some kind of bootstrapping (since update_crates needs to fetch to be able to write checksum but the fetcher needs the checksum ...). With this error , you'll see all the data you need to add to the suricata-crates.inc.

For curiosity, I tested locally the modification of suricata and it seems that this package embeds unexpected Cargo.lock format (in rust/vendor/alloc-no-stdlib/Cargo.lock) which doesn't contain a source entry (the package doesn't have any dependency).

This is exactly where things stop for me and I start trying to work around it.  Suricata is working on .11 and it looks better in the Crate department. There git sources require bootstrapping which posses it own challenges

Quoted 1 line

I don't know how to handle such a file (simply ignore it in the class instead of raising an error ?)

There are mechanisms being used in other areas that allow for a recipe to skip or ignore a condition.

Thanks for taking a look, appreciate the support.

BR,
Armin

Quoted 53 lines · Martin Jansa

On Tue, Apr 4, 2023 at 11:08 PM akuster <[email protected]> wrote:

On 4/4/23 1:07 PM, Martin Jansa wrote:
> On Tue, Apr 4, 2023 at 6:50 PM akuster <[email protected]> wrote:
>
>
>
>     On 4/1/23 11:21 AM, Frederic Martinsons wrote:
>     > From: Frederic Martinsons <[email protected]> >     >
>     > Instead of only display the first and stop.
>     > For recipe (crago based) that can contains several artifacts
>     > to fetch with their checksum, it will particularly handy >     > to display all of missing one.
>     >
>     > An example of error message would be
>
>     I am confused. Is this patch supposed to add the pkg version or
>     not?  I
>     am not seeing any. Any ideas?
>
>
> name parameter with pkg version was added to bbclass in:
>
https://git.openembedded.org/openembedded-core/commit/?id=1795e98a04ad09b011afcc7cc3bf6dc49475b19a
>
> this change only allows all missing/incorrect checksums to be showsn
> at the same time, instead of one checksum per do_fetch failure.

I get multiple hashes for the same package.

SRC_URI[wasi.sha256sum] = "1a143597ca7c7793eff794def352d41792a93c481eb1042423ff7ff72ba2c31f" SRC_URI[wasi.sha256sum] = "cccddf32554fecc6acb585f82a32a72e28b48f8c4c1883ddfeeeaa96f7d8e519"

so why no version info after the pkg name?

Did you regenerate the SRC_URIs with updated bbclass? It works for me as in:
https://github.com/shr-project/meta-webosose/commit/84dc9f3cc3dd85facde07ccddce0b9f2bcd90d39

#13 lore raw
In reply to Frédéric Martinsons
Quoted 7 lines · Frédéric Martinsons

I looked at the suricata recipe in meta-security and it didn't use cargo-update-recipes.bbclass

You should modify your rust recipes to use this class and have access to "bitbake -c update_crates" .

You can take some examples from , i.e python3-cryptography in openembedded-core or python3-pyruvate in meta-python.

Quoted 7 lines

The first issue you'll encounter is some kind of bootstrapping (since update_crates needs to fetch to be able to write checksum but the fetcher needs the checksum ...). With this error , you'll see all the data you need to add to the suricata-crates.inc.

For curiosity, I tested locally the modification of suricata and it seems that this package embeds unexpected Cargo.lock format (in rust/vendor/alloc-no-stdlib/Cargo.lock) which doesn't contain a source entry (the package doesn't have any dependency).

I deleted the Cargo.* in the offending crates ( I found another) and I was able to finish running the update_creates. I notice "name" is now part of the crates URI and the SRC_URI hashes. It wont build now but I think I can work around that.

- armin

Quoted 55 lines

I don't know how to handle such a file (simply ignore it in the class instead of raising an error ?)

On Tue, 4 Apr 2023 at 23:35, Martin Jansa <[email protected]> wrote:

On Tue, Apr 4, 2023 at 11:08 PM akuster <[email protected]> wrote:

On 4/4/23 1:07 PM, Martin Jansa wrote:
> On Tue, Apr 4, 2023 at 6:50 PM akuster <[email protected]> wrote:
>
>
>
>     On 4/1/23 11:21 AM, Frederic Martinsons wrote:
>     > From: Frederic Martinsons <[email protected]> >     >
>     > Instead of only display the first and stop.
>     > For recipe (crago based) that can contains several artifacts
>     > to fetch with their checksum, it will particularly handy >     > to display all of missing one.
>     >
>     > An example of error message would be
>
>     I am confused. Is this patch supposed to add the pkg version or
>     not?  I
>     am not seeing any. Any ideas?
>
>
> name parameter with pkg version was added to bbclass in:
>
https://git.openembedded.org/openembedded-core/commit/?id=1795e98a04ad09b011afcc7cc3bf6dc49475b19a
>
> this change only allows all missing/incorrect checksums to be showsn
> at the same time, instead of one checksum per do_fetch failure.

I get multiple hashes for the same package.

SRC_URI[wasi.sha256sum] = "1a143597ca7c7793eff794def352d41792a93c481eb1042423ff7ff72ba2c31f" SRC_URI[wasi.sha256sum] = "cccddf32554fecc6acb585f82a32a72e28b48f8c4c1883ddfeeeaa96f7d8e519"

so why no version info after the pkg name?

Did you regenerate the SRC_URIs with updated bbclass? It works for me as in:
https://github.com/shr-project/meta-webosose/commit/84dc9f3cc3dd85facde07ccddce0b9f2bcd90d39

#14 lore raw

What is the problem of having name there?

The bitbake fetcher (master) handles that.

Quoted 95 lines · akuster
Quoted 16 lines · Frédéric Martinsons

I looked at the suricata recipe in meta-security and it didn't use cargo-update-recipes.bbclass

You should modify your rust recipes to use this class and have access to "bitbake -c update_crates" .

You can take some examples from , i.e python3-cryptography in openembedded-core or python3-pyruvate
in meta-python.

The first issue you'll encounter is some kind of bootstrapping (since update_crates needs to fetch
to be able to write checksum but the fetcher needs the checksum ...). With this error , you'll
see all the data you need to add to the suricata-crates.inc.

Quoted 5 lines

For curiosity, I tested locally the modification of suricata and it seems that this package embeds
unexpected Cargo.lock format (in rust/vendor/alloc-no-stdlib/Cargo.lock) which doesn't contain a source entry (the package doesn't have any dependency).

I deleted the Cargo.* in the offending crates ( I found another) and I was able to finish running the update_creates. I notice "name" is now part of the crates URI and the SRC_URI hashes. It wont build now but I think I can work around that.

- armin

Quoted 6 lines

I don't know how to handle such a file (simply ignore it in the class instead of raising an error ?)

On Tue, 4 Apr 2023 at 23:35, Martin Jansa <[email protected]>

Quoted 23 lines · wrote:
    On Tue, Apr 4, 2023 at 11:08 PM akuster <[email protected]> wrote:



        On 4/4/23 1:07 PM, Martin Jansa wrote:
        > On Tue, Apr 4, 2023 at 6:50 PM akuster <[email protected]>
        wrote:
        >
        >
        >
        >     On 4/1/23 11:21 AM, Frederic Martinsons wrote:
        >     > From: Frederic Martinsons <[email protected]

        >     >
        >     > Instead of only display the first and stop.
        >     > For recipe (crago based) that can contains several
        artifacts
        >     > to fetch with their checksum, it will particularly handy
        >     > to display all of missing one.
        >     >
        >     > An example of error message would be
        >
Quoted 9 lines
        >     I am confused. Is this patch supposed to add the pkg
        version or
        >     not?  I
        >     am not seeing any. Any ideas?
        >
        >
        > name parameter with pkg version was added to bbclass in:
        >

https://git.openembedded.org/openembedded-core/commit/?id=1795e98a04ad09b011afcc7cc3bf6dc49475b19a

Quoted 9 lines

>
> this change only allows all missing/incorrect checksums to be showsn
> at the same time, instead of one checksum per do_fetch failure.

I get multiple hashes for the same package.

SRC_URI[wasi.sha256sum] =

"1a143597ca7c7793eff794def352d41792a93c481eb1042423ff7ff72ba2c31f"

Quoted 2 lines
        SRC_URI[wasi.sha256sum] =

"cccddf32554fecc6acb585f82a32a72e28b48f8c4c1883ddfeeeaa96f7d8e519"

Quoted 7 lines

so why no version info after the pkg name?

Did you regenerate the SRC_URIs with updated bbclass? It works for me as in:

https://github.com/shr-project/meta-webosose/commit/84dc9f3cc3dd85facde07ccddce0b9f2bcd90d39

Quoted 1 line
#15 lore raw
Quoted 1 line · Frédéric Martinsons

What is the problem of having name there?

I deleted both alloc-no-stdlib and autocfg Cargo.* files and update_crates worked. I appears is missed a bunch that I manually added.

I got is building again.

Thanks for your help.

- Armin

Quoted 108 lines

The bitbake fetcher (master) handles that.

Le mer. 5 avr. 2023, 14:25, akuster <[email protected]> a écrit :

On 4/5/23 1:20 AM, Frédéric Martinsons wrote:
> I looked at the suricata recipe in meta-security and it didn't use > cargo-update-recipes.bbclass
>
> You should modify your rust recipes to use this class and have access
> to "bitbake -c update_crates" .
>
> You can take some examples from , i.e python3-cryptography in > openembedded-core or python3-pyruvate
> in meta-python.
>
> The first issue you'll encounter is some kind of bootstrapping (since
> update_crates needs to fetch
> to be able to write checksum but the fetcher needs the checksum ...).
> With this error , you'll
> see all the data you need to add to the suricata-crates.inc. >
> For curiosity, I tested locally the modification of suricata and it > seems that this package embeds
> unexpected Cargo.lock format (in
> rust/vendor/alloc-no-stdlib/Cargo.lock) which doesn't contain > a source entry (the package doesn't have any dependency).

I deleted the Cargo.* in the offending crates ( I found another) and I
was able to finish running the update_creates. I notice "name" is now part of the crates URI and the SRC_URI hashes. It wont build now but I
think I can work around that.

- armin
>
> I don't know how to handle such a file (simply ignore it in the class
> instead of raising an error ?)
>
>
> On Tue, 4 Apr 2023 at 23:35, Martin Jansa <[email protected]> wrote:
>
>     On Tue, Apr 4, 2023 at 11:08 PM akuster <[email protected]> wrote:
>
>
>
>         On 4/4/23 1:07 PM, Martin Jansa wrote:
>         > On Tue, Apr 4, 2023 at 6:50 PM akuster <[email protected]>
>         wrote:
>         >
>         >
>         >
>         >     On 4/1/23 11:21 AM, Frederic Martinsons wrote:
>         >     > From: Frederic Martinsons <[email protected]>
>         >     >
>         >     > Instead of only display the first and stop. >         >     > For recipe (crago based) that can contains several >         artifacts
>         >     > to fetch with their checksum, it will particularly handy
>         >     > to display all of missing one.
>         >     >
>         >     > An example of error message would be
>         >
>         >     I am confused. Is this patch supposed to add the pkg >         version or
>         >     not?  I
>         >     am not seeing any. Any ideas?
>         >
>         >
>         > name parameter with pkg version was added to bbclass in:
>         >
>
https://git.openembedded.org/openembedded-core/commit/?id=1795e98a04ad09b011afcc7cc3bf6dc49475b19a
>         >
>         > this change only allows all missing/incorrect checksums to >         be showsn
>         > at the same time, instead of one checksum per do_fetch failure.
>
>         I get multiple hashes for the same package.
>
>         SRC_URI[wasi.sha256sum] =
>  "1a143597ca7c7793eff794def352d41792a93c481eb1042423ff7ff72ba2c31f" >         SRC_URI[wasi.sha256sum] =
>  "cccddf32554fecc6acb585f82a32a72e28b48f8c4c1883ddfeeeaa96f7d8e519" >
>         so why no version info after the pkg name?
>
>
>     Did you regenerate the SRC_URIs with updated bbclass? It works for
>     me as in:
>
https://github.com/shr-project/meta-webosose/commit/84dc9f3cc3dd85facde07ccddce0b9f2bcd90d39
>

#16 lore raw
Quoted 5 lines · Le mer. 5 avr. 2023, 18:26, akuster808 <[email protected]> a écrit :
Quoted 1 line · Frédéric Martinsons

What is the problem of having name there?

Quoted 10 lines

I deleted both alloc-no-stdlib and autocfg Cargo.* files and update_crates worked. I appears is missed a bunch that I manually added.

I got is building again.

Thanks for your help.

- Armin

You're welcome, glad to see that you manage to build those recipes. Have a good day.