Skip to contentSkip to conversation

externalsrc: fix PKGV for devtool build

PATCH In review Paddy Chiu 4 participants 8 messages Sep 22 – Oct 14

lore
Share & tools
lore

Paddy Chiu

author
#0 lore raw

PATCH 1/1 externalsrc: fix PKGV for devtool build

Without this fix, PKGV will be the same as PV when using devtool build.
Since SRC_URI is set to local_srcuri in externalsrc.bbclass and consequently cause missing git commit hash in PKGV. This fix can prevent buildhistory package-version QA failures in devtool build.

---
 meta/classes-global/package.bbclass | 12 ++++++++++--
 meta/classes/externalsrc.bbclass    |  4 ++++
 2 files changed, 14 insertions(+), 2 deletions(-)
meta/classes-global/package.bbclass +10 -2
index 67148fc1ac..c5f8b2136f 100644
@@ -242,6 +242,14 @@ python () {
         d.appendVarFlag('do_package', 'deptask', " do_packagedata")
 }
 
+def package_get_srcdata(d):
+    srcuri = d.getVar("EXTERNALSRC_ORIG_SRC_URI")
+    if not srcuri:
+        return d
+
+    srcdata = d.createCopy()
+    srcdata.setVar("SRC_URI", srcuri)
+    return srcdata
 
 PRSERV_ACTIVE = "${@bool(d.getVar("PRSERV_HOST"))}"
 PRSERV_ACTIVE[vardepvalue] = "${PRSERV_ACTIVE}"
@@ -299,7 +307,7 @@ python package_get_auto_pr() {
                 pr = d.getVar('PR')
                 # Strip PR to make AUTOINC can increase when SRCREV is changed
                 base_ver = "AUTOINC-%s" % version[:-len(pr)]
-                srcpv = bb.fetch2.get_srcrev(d)
+                srcpv = bb.fetch2.get_srcrev(package_get_srcdata(d))
                 value = conn.getPR(base_ver, pkgarch, srcpv)
                 d.setVar("PRSERV_PV_AUTOINC", str(value))
 
@@ -319,7 +327,7 @@ python package_get_auto_pr() {
 python package_setup_pkgv() {
     pkgv = d.getVar("PKGV")
     # Expand SRCPV into PKGV if not present
-    srcpv = bb.fetch.get_pkgv_string(d)
+    srcpv = bb.fetch.get_pkgv_string(package_get_srcdata(d))
     if srcpv and "+" in pkgv:
         d.appendVar("PKGV", srcpv)
         pkgv = d.getVar("PKGV")
meta/classes/externalsrc.bbclass +4 -0
index 902ff2604f..a1bb92a418 100644
@@ -78,6 +78,10 @@ python () {
             d.setVar('B', '${WORKDIR}/${BPN}-${PV}')
 
         bb.fetch.get_hashvalue(d)
+        # Keep the original URI available for packaging. SCM URLs are removed
+        # below so externalsrc does not fetch or unpack the external source,
+        # but package_setup_pkgv() still needs them to derive PKGV.
+        d.setVar('EXTERNALSRC_ORIG_SRC_URI', d.getVar('SRC_URI'))
         local_srcuri = []
         fetch = bb.fetch2.Fetch((d.getVar('SRC_URI') or '').split(), d)
         for url in fetch.urls:
Signature · 1 line
2.34.1
#1 lore raw

Patchtest results for [PATCH] externalsrc: fix PKGV for devtool build

Thank you for your submission. Patchtest identified one or more issues with the patch. Please see the log below for more information:

---
Testing patch /home/patchtest/share/mboxes/externalsrc-fix-PKGV-for-devtool-build.patch

FAIL: test Signed-off-by presence: Mbox is missing Signed-off-by. Add it manually or with "git commit --amend -s" (test_mbox.TestMbox.test_signed_off_by_presence)

PASS: test auh changelog truncation notice (test_mbox.TestMbox.test_auh_changelog_truncation_notice)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence (test_mbox.TestMbox.test_commit_message_presence)
PASS: test commit message user tags (test_mbox.TestMbox.test_commit_message_user_tags)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
PASS: test target mailing list (test_mbox.TestMbox.test_target_mailing_list)

SKIP: pretest pylint: No python related patches, skipping test (test_python_pylint.PyLint.pretest_pylint)
SKIP: pretest src uri left files: Patch cannot be merged (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes or older target branch, skipping test (test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test CVE tag format: No new source patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new source patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new source patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test pylint: No python related patches, skipping test (test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: Patch cannot be merged (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this email with an explanation of why the patch should be accepted. If you believe these results are due to an error in patchtest, please submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category under 'Yocto Project Subprojects'). For more information on specific failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank you!

#2 lore raw
In reply to Paddy Chiu

Hi Paddy,

Quoted 6 lines · Paddy Chiu

Without this fix, PKGV will be the same as PV when using devtool build. Since SRC_URI is set to local_srcuri in externalsrc.bbclass and consequently cause missing git commit hash in PKGV.
This fix can prevent buildhistory package-version QA failures in devtool build.

Please use about 72 chars per line on the commit message and also add you Signed-off-by (git commit -s--amend)

Jose

Quoted 66 lines
---
 meta/classes-global/package.bbclass | 12 ++++++++++--
 meta/classes/externalsrc.bbclass    |  4 ++++
 2 files changed, 14 insertions(+), 2 deletions(-)
meta/classes-global/package.bbclass +8 -0
index 67148fc1ac..c5f8b2136f 100644
@@ -242,6 +242,14 @@ python () {
         d.appendVarFlag('do_package', 'deptask', " do_packagedata")
 }
 
+def package_get_srcdata(d):
+    srcuri = d.getVar("EXTERNALSRC_ORIG_SRC_URI")
+    if not srcuri:
+        return d
+
+    srcdata = d.createCopy()
+    srcdata.setVar("SRC_URI", srcuri)
+    return srcdata
 
 PRSERV_ACTIVE = "${@bool(d.getVar("PRSERV_HOST"))}"
 PRSERV_ACTIVE[vardepvalue] = "${PRSERV_ACTIVE}"
@@ -299,7 +307,7 @@ python package_get_auto_pr() {
                 pr = d.getVar('PR')
                 # Strip PR to make AUTOINC can increase when SRCREV is

changed

                 base_ver = "AUTOINC-%s" % version[:-len(pr)]
meta/classes-global/package.bbclass +2 -2
-                srcpv = bb.fetch2.get_srcrev(d)
+                srcpv = bb.fetch2.get_srcrev(package_get_srcdata(d))
                 value = conn.getPR(base_ver, pkgarch, srcpv)
                 d.setVar("PRSERV_PV_AUTOINC", str(value))
@@ -319,7 +327,7 @@ python package_get_auto_pr() {
 python package_setup_pkgv() {
     pkgv = d.getVar("PKGV")
     # Expand SRCPV into PKGV if not present
-    srcpv = bb.fetch.get_pkgv_string(d)
+    srcpv = bb.fetch.get_pkgv_string(package_get_srcdata(d))
     if srcpv and "+" in pkgv:
         d.appendVar("PKGV", srcpv)
         pkgv = d.getVar("PKGV")
meta/classes/externalsrc.bbclass +1 -0
index 902ff2604f..a1bb92a418 100644
@@ -78,6 +78,10 @@ python () {
             d.setVar('B', '${WORKDIR}/${BPN}-${PV}')
 
         bb.fetch.get_hashvalue(d)
+        # Keep the original URI available for packaging. SCM URLs are

removed
+ # below so externalsrc does not fetch or unpack the external
source,

meta/classes/externalsrc.bbclass +2 -1
+        # but package_setup_pkgv() still needs them to derive PKGV.
+        d.setVar('EXTERNALSRC_ORIG_SRC_URI', d.getVar('SRC_URI'))
         local_srcuri = []
         fetch = bb.fetch2.Fetch((d.getVar('SRC_URI') or '').split(), d)
         for url in fetch.urls:
--

2.34.1

#3 lore raw
In reply to Paddy Chiu
Quoted 52 lines · Paddy Chiu via lists.openembedded.org

Without this fix, PKGV will be the same as PV when using devtool build.
Since SRC_URI is set to local_srcuri in externalsrc.bbclass and consequently cause missing git commit hash in PKGV. This fix can prevent buildhistory package-version QA failures in devtool build.
---
 meta/classes-global/package.bbclass | 12 ++++++++++--
 meta/classes/externalsrc.bbclass    |  4 ++++

 2 files changed, 14 insertions(+), 2 deletions(-)
meta/classes-global/package.bbclass +0 -0
index 67148fc1ac..c5f8b2136f 100644
@@ -242,6 +242,14 @@ python () {
          d.appendVarFlag('do_package', 'deptask', " do_packagedata")
  }

meta/classes-global/package.bbclass +8 -0
+def package_get_srcdata(d):
+    srcuri = d.getVar("EXTERNALSRC_ORIG_SRC_URI")
+    if not srcuri:
+        return d
+
+    srcdata = d.createCopy()
+    srcdata.setVar("SRC_URI", srcuri)
+    return srcdata


 PRSERV_ACTIVE = "${@bool(d.getVar("PRSERV_HOST"))}" PRSERV_ACTIVE[vardepvalue] = "${PRSERV_ACTIVE}"

meta/classes-global/package.bbclass +0 -0
@@ -299,7 +307,7 @@ python package_get_auto_pr() {
                  pr = d.getVar('PR')
                  # Strip PR to make AUTOINC can increase when SRCREV is changed

                 base_ver = "AUTOINC-%s" % version[:-len(pr)] -                srcpv = bb.fetch2.get_srcrev(d)
+                srcpv = bb.fetch2.get_srcrev(package_get_srcdata(d)) value = conn.getPR(base_ver, pkgarch, srcpv) d.setVar("PRSERV_PV_AUTOINC", str(value))

meta/classes-global/package.bbclass +1 -1
@@ -319,7 +327,7 @@ python package_get_auto_pr() {
  python package_setup_pkgv() {
      pkgv = d.getVar("PKGV")
      # Expand SRCPV into PKGV if not present
-    srcpv = bb.fetch.get_pkgv_string(d)
+    srcpv = bb.fetch.get_pkgv_string(package_get_srcdata(d))

     if srcpv and "+" in pkgv:
         d.appendVar("PKGV", srcpv)
         pkgv = d.getVar("PKGV")

meta/classes/externalsrc.bbclass +0 -0
index 902ff2604f..a1bb92a418 100644
@@ -78,6 +78,10 @@ python () {
              d.setVar('B', '${WORKDIR}/${BPN}-${PV}')

Quoted 8 lines

         bb.fetch.get_hashvalue(d)
+        # Keep the original URI available for packaging. SCM URLs are removed +        # below so externalsrc does not fetch or unpack the external source, +        # but package_setup_pkgv() still needs them to derive PKGV. +        d.setVar('EXTERNALSRC_ORIG_SRC_URI', d.getVar('SRC_URI')) local_srcuri = []
         fetch = bb.fetch2.Fetch((d.getVar('SRC_URI') or '').split(), d) for url in fetch.urls:

We really need to find a different way to fix this. I don't doubt there is an issue here but change the datastore and it's variables around mid function like that is really a hacky workaround, not a proper fix.

If we did this each time we found a bug, the code would become unmaintainable.

Cheers,

Richard

author
#4 lore raw

Hi Richard,
Sorry for the late reply.

Patch v2 (Proposed fix) addresses the concern with the previous approach:
it no longer copies the datastore or rewrites SRC_URI during package version calculation.

Instead, externalsrc saves the formatted source-version suffix in EXTERNALSRC_PKGV_SUFFIX before filtering SCM URLs. The package and PR-service paths reuse it, while the normal fetcher path remains unchanged for non-externalsrc builds.

Validation used git-submodule-test for both requested transitions:

BitBake to devtool: both builds produce the same hash-bearing PKGV and no version-going-backwards QA error.

devtool to BitBake: both builds produce the same PKGV with no regression in the reverse transition.

Changes since v1:

- replace the copied datastore/SRC_URI override with a saved suffix;
- use the suffix for both PKGV and AUTOINC PR-service handling.

[PATCH v2 1/1] externalsrc: preserve source revision in PKGV

externalsrc removes SCM URLs from SRC_URI before packaging, so the packaging fetcher cannot derive the source revision for devtool builds.

Capture the formatted source-version suffix before filtering and reuse it for package version and AUTOINC PR-service handling. This keeps the normal fetcher path unchanged and avoids restoring a copied datastore.

---
 meta/classes-global/package.bbclass | 8 ++++++--
 meta/classes/externalsrc.bbclass    | 3 +++
 2 files changed, 9 insertions(+), 2 deletions(-)
(quoted diff) +0 -0

b/meta/classes-global/package.bbclass
index 67148fc1ac..f57397e83d 100644

meta/classes-global/package.bbclass +0 -0
@@ -299,7 +299,9 @@ python package_get_auto_pr() {
                 pr = d.getVar('PR')
                 # Strip PR to make AUTOINC can increase when SRCREV is

changed

                 base_ver = "AUTOINC-%s" % version[:-len(pr)]
-                srcpv = bb.fetch2.get_srcrev(d)
+                srcpv = d.getVar("EXTERNALSRC_PKGV_SUFFIX")
+                if srcpv is None:

+ srcpv = bb.fetch2.get_srcrev(d)
value = conn.getPR(base_ver, pkgarch, srcpv) d.setVar("PRSERV_PV_AUTOINC", str(value))

@@ -319,7 +321,9 @@ python package_get_auto_pr() {
python package_setup_pkgv() {
pkgv = d.getVar("PKGV")
# Expand SRCPV into PKGV if not present

-    srcpv = bb.fetch.get_pkgv_string(d)
+    srcpv = d.getVar("EXTERNALSRC_PKGV_SUFFIX")
+    if srcpv is None:

+ srcpv = bb.fetch.get_pkgv_string(d)
if srcpv and "+" in pkgv:
d.appendVar("PKGV", srcpv)
pkgv = d.getVar("PKGV")

(quoted diff) +0 -0

b/meta/classes/externalsrc.bbclass
index 902ff2604f..47c1b06788 100644

meta/classes/externalsrc.bbclass +1 -0
@@ -78,6 +78,9 @@ python () {
             d.setVar('B', '${WORKDIR}/${BPN}-${PV}')
 
         bb.fetch.get_hashvalue(d)
+        # Keep the source version available for packaging. SCM URLs are

removed
+ # below so externalsrc does not fetch or unpack the external
source.
+ d.setVar('EXTERNALSRC_PKGV_SUFFIX', bb.fetch.get_pkgv_string(d))

         local_srcuri = []
         fetch = bb.fetch2.Fetch((d.getVar('SRC_URI') or '').split(), d)
         for url in fetch.urls:

--
2.34.1

Please let me know if further changes or discussion are needed.

Best Regards,
Paddy

Quoted 82 lines · Richard Purdie
Quoted 2 lines · Paddy Chiu via lists.openembedded.org

Without this fix, PKGV will be the same as PV when using devtool build. Since SRC_URI is set to local_srcuri in externalsrc.bbclass and

consequently cause missing git commit hash in PKGV.

Quoted 1 line

This fix can prevent buildhistory package-version QA failures in devtool

build.

Review

Quoted 6 lines
---
 meta/classes-global/package.bbclass | 12 ++++++++++--
 meta/classes/externalsrc.bbclass    |  4 ++++
 2 files changed, 14 insertions(+), 2 deletions(-)
(quoted diff) +0 -0

b/meta/classes-global/package.bbclass

Quoted 19 lines

index 67148fc1ac..c5f8b2136f 100644

meta/classes-global/package.bbclass +8 -0
@@ -242,6 +242,14 @@ python () {
         d.appendVarFlag('do_package', 'deptask', " do_packagedata")
 }
 
+def package_get_srcdata(d):
+    srcuri = d.getVar("EXTERNALSRC_ORIG_SRC_URI")
+    if not srcuri:
+        return d
+
+    srcdata = d.createCopy()
+    srcdata.setVar("SRC_URI", srcuri)
+    return srcdata
 
 PRSERV_ACTIVE = "${@bool(d.getVar("PRSERV_HOST"))}"
 PRSERV_ACTIVE[vardepvalue] = "${PRSERV_ACTIVE}"

meta/classes-global/package.bbclass python package_get_auto_pr() { · line 308

Quoted 2 lines
meta/classes-global/package.bbclass +0 -0
@@ -299,7 +307,7 @@ python package_get_auto_pr() {
                 pr = d.getVar('PR')
                 # Strip PR to make AUTOINC can increase when SRCREV is

changed

Review

Quoted 16 lines
meta/classes-global/package.bbclass +2 -2
                 base_ver = "AUTOINC-%s" % version[:-len(pr)]
-                srcpv = bb.fetch2.get_srcrev(d)
+                srcpv = bb.fetch2.get_srcrev(package_get_srcdata(d))
                 value = conn.getPR(base_ver, pkgarch, srcpv)
                 d.setVar("PRSERV_PV_AUTOINC", str(value))
 
@@ -319,7 +327,7 @@ python package_get_auto_pr() {
 python package_setup_pkgv() {
     pkgv = d.getVar("PKGV")
     # Expand SRCPV into PKGV if not present
-    srcpv = bb.fetch.get_pkgv_string(d)
+    srcpv = bb.fetch.get_pkgv_string(package_get_srcdata(d))
     if srcpv and "+" in pkgv:
         d.appendVar("PKGV", srcpv)
         pkgv = d.getVar("PKGV")
(quoted diff) +0 -0

b/meta/classes/externalsrc.bbclass

meta/classes/externalsrc.bbclass python () { · line 81

Quoted 8 lines

index 902ff2604f..a1bb92a418 100644

meta/classes/externalsrc.bbclass +1 -0
@@ -78,6 +78,10 @@ python () {
             d.setVar('B', '${WORKDIR}/${BPN}-${PV}')
 
         bb.fetch.get_hashvalue(d)
+        # Keep the original URI available for packaging. SCM URLs are

removed

meta/classes/externalsrc.bbclass python () { · line 82

Quoted 1 line
meta/classes/externalsrc.bbclass +1 -0
+        # below so externalsrc does not fetch or unpack the external

source,

meta/classes/externalsrc.bbclass python () { · line 87

Quoted 5 lines
meta/classes/externalsrc.bbclass +2 -0
+        # but package_setup_pkgv() still needs them to derive PKGV.
+        d.setVar('EXTERNALSRC_ORIG_SRC_URI', d.getVar('SRC_URI'))
         local_srcuri = []
         fetch = bb.fetch2.Fetch((d.getVar('SRC_URI') or '').split(), d)
         for url in fetch.urls:

We really need to find a different way to fix this. I don't doubt there is an issue here but change the datastore and it's variables around mid function like that is really a hacky workaround, not a proper fix.

If we did this each time we found a bug, the code would become unmaintainable.

Cheers,

Richard

author
#5 lore raw

Hi Richard,
Is there any update?

Thanks,
Paddy

Quoted 180 lines · Paddy Chiu

Hi Richard,
Sorry for the late reply.

Patch v2 (Proposed fix) addresses the concern with the previous approach:
it no longer copies the datastore or rewrites SRC_URI during package version calculation.

Instead, externalsrc saves the formatted source-version suffix in EXTERNALSRC_PKGV_SUFFIX before filtering SCM URLs. The package and PR-service paths reuse it, while the normal fetcher path remains unchanged for non-externalsrc builds.

Validation used git-submodule-test for both requested transitions:

BitBake to devtool: both builds produce the same hash-bearing PKGV and no version-going-backwards QA error.

devtool to BitBake: both builds produce the same PKGV with no regression in the reverse transition.

Changes since v1:

- replace the copied datastore/SRC_URI override with a saved suffix;
- use the suffix for both PKGV and AUTOINC PR-service handling.

[PATCH v2 1/1] externalsrc: preserve source revision in PKGV

externalsrc removes SCM URLs from SRC_URI before packaging, so the packaging fetcher cannot derive the source revision for devtool builds.

Capture the formatted source-version suffix before filtering and reuse it for package version and AUTOINC PR-service handling. This keeps the normal fetcher path unchanged and avoids restoring a copied datastore.

---
 meta/classes-global/package.bbclass | 8 ++++++--
 meta/classes/externalsrc.bbclass    | 3 +++
 2 files changed, 9 insertions(+), 2 deletions(-)
meta/classes-global/package.bbclass +0 -0
index 67148fc1ac..f57397e83d 100644
@@ -299,7 +299,9 @@ python package_get_auto_pr() {
                 pr = d.getVar('PR')
                 # Strip PR to make AUTOINC can increase when SRCREV is

changed

                 base_ver = "AUTOINC-%s" % version[:-len(pr)]
meta/classes-global/package.bbclass +6 -2
-                srcpv = bb.fetch2.get_srcrev(d)
+                srcpv = d.getVar("EXTERNALSRC_PKGV_SUFFIX")
+                if srcpv is None:
+                    srcpv = bb.fetch2.get_srcrev(d)
                 value = conn.getPR(base_ver, pkgarch, srcpv)
                 d.setVar("PRSERV_PV_AUTOINC", str(value))
@@ -319,7 +321,9 @@ python package_get_auto_pr() {
 python package_setup_pkgv() {
     pkgv = d.getVar("PKGV")
     # Expand SRCPV into PKGV if not present
-    srcpv = bb.fetch.get_pkgv_string(d)
+    srcpv = d.getVar("EXTERNALSRC_PKGV_SUFFIX")
+    if srcpv is None:
+        srcpv = bb.fetch.get_pkgv_string(d)
     if srcpv and "+" in pkgv:
         d.appendVar("PKGV", srcpv)
         pkgv = d.getVar("PKGV")
meta/classes/externalsrc.bbclass +1 -0
index 902ff2604f..47c1b06788 100644
@@ -78,6 +78,9 @@ python () {
             d.setVar('B', '${WORKDIR}/${BPN}-${PV}')
 
         bb.fetch.get_hashvalue(d)
+        # Keep the source version available for packaging. SCM URLs are

removed
+ # below so externalsrc does not fetch or unpack the external
source.
+ d.setVar('EXTERNALSRC_PKGV_SUFFIX', bb.fetch.get_pkgv_string(d))

         local_srcuri = []
         fetch = bb.fetch2.Fetch((d.getVar('SRC_URI') or '').split(), d)
         for url in fetch.urls:

--
2.34.1

Please let me know if further changes or discussion are needed.

Best Regards,
Paddy

Quoted 82 lines · Richard Purdie
Quoted 2 lines · Paddy Chiu via lists.openembedded.org

Without this fix, PKGV will be the same as PV when using devtool build. Since SRC_URI is set to local_srcuri in externalsrc.bbclass and

consequently cause missing git commit hash in PKGV.

Quoted 1 line

This fix can prevent buildhistory package-version QA failures in

devtool build.

Review

Quoted 6 lines
---
 meta/classes-global/package.bbclass | 12 ++++++++++--
 meta/classes/externalsrc.bbclass    |  4 ++++
 2 files changed, 14 insertions(+), 2 deletions(-)
(quoted diff) +0 -0

b/meta/classes-global/package.bbclass

Quoted 19 lines

index 67148fc1ac..c5f8b2136f 100644

meta/classes-global/package.bbclass +8 -0
@@ -242,6 +242,14 @@ python () {
         d.appendVarFlag('do_package', 'deptask', " do_packagedata")
 }
 
+def package_get_srcdata(d):
+    srcuri = d.getVar("EXTERNALSRC_ORIG_SRC_URI")
+    if not srcuri:
+        return d
+
+    srcdata = d.createCopy()
+    srcdata.setVar("SRC_URI", srcuri)
+    return srcdata
 
 PRSERV_ACTIVE = "${@bool(d.getVar("PRSERV_HOST"))}"
 PRSERV_ACTIVE[vardepvalue] = "${PRSERV_ACTIVE}"

meta/classes-global/package.bbclass python package_get_auto_pr() { · line 308

Quoted 2 lines
meta/classes-global/package.bbclass +0 -0
@@ -299,7 +307,7 @@ python package_get_auto_pr() {
                 pr = d.getVar('PR')
                 # Strip PR to make AUTOINC can increase when SRCREV is

changed

Review

Quoted 16 lines
meta/classes-global/package.bbclass +2 -2
                 base_ver = "AUTOINC-%s" % version[:-len(pr)]
-                srcpv = bb.fetch2.get_srcrev(d)
+                srcpv = bb.fetch2.get_srcrev(package_get_srcdata(d))
                 value = conn.getPR(base_ver, pkgarch, srcpv)
                 d.setVar("PRSERV_PV_AUTOINC", str(value))
 
@@ -319,7 +327,7 @@ python package_get_auto_pr() {
 python package_setup_pkgv() {
     pkgv = d.getVar("PKGV")
     # Expand SRCPV into PKGV if not present
-    srcpv = bb.fetch.get_pkgv_string(d)
+    srcpv = bb.fetch.get_pkgv_string(package_get_srcdata(d))
     if srcpv and "+" in pkgv:
         d.appendVar("PKGV", srcpv)
         pkgv = d.getVar("PKGV")
(quoted diff) +0 -0

b/meta/classes/externalsrc.bbclass

meta/classes/externalsrc.bbclass python () { · line 81

Quoted 8 lines

index 902ff2604f..a1bb92a418 100644

meta/classes/externalsrc.bbclass +1 -0
@@ -78,6 +78,10 @@ python () {
             d.setVar('B', '${WORKDIR}/${BPN}-${PV}')
 
         bb.fetch.get_hashvalue(d)
+        # Keep the original URI available for packaging. SCM URLs are

removed

meta/classes/externalsrc.bbclass python () { · line 82

Quoted 1 line
meta/classes/externalsrc.bbclass +1 -0
+        # below so externalsrc does not fetch or unpack the external

source,

meta/classes/externalsrc.bbclass python () { · line 87

Quoted 5 lines
meta/classes/externalsrc.bbclass +2 -0
+        # but package_setup_pkgv() still needs them to derive PKGV.
+        d.setVar('EXTERNALSRC_ORIG_SRC_URI', d.getVar('SRC_URI'))
         local_srcuri = []
         fetch = bb.fetch2.Fetch((d.getVar('SRC_URI') or '').split(), d)
         for url in fetch.urls:

We really need to find a different way to fix this. I don't doubt there is an issue here but change the datastore and it's variables around mid function like that is really a hacky workaround, not a proper fix.

If we did this each time we found a bug, the code would become unmaintainable.

Cheers,

Richard

author
#6 lore raw

PATCH v2 1/1 externalsrc: preserve source revision in PKGV

In reply to Paddy Chiu

externalsrc removes SCM URLs from SRC_URI before packaging, so the packaging fetcher cannot derive the source revision for devtool builds.

Capture the formatted source-version suffix before filtering and reuse it for package version and AUTOINC PR-service handling. This keeps the normal fetcher path unchanged and avoids restoring a copied datastore.

---
 meta/classes-global/package.bbclass | 8 ++++++--
 meta/classes/externalsrc.bbclass    | 3 +++
 2 files changed, 9 insertions(+), 2 deletions(-)
meta/classes-global/package.bbclass +6 -2
index 67148fc1ac..f57397e83d 100644
@@ -299,7 +299,9 @@ python package_get_auto_pr() {
                 pr = d.getVar('PR')
                 # Strip PR to make AUTOINC can increase when SRCREV is changed
                 base_ver = "AUTOINC-%s" % version[:-len(pr)]
-                srcpv = bb.fetch2.get_srcrev(d)
+                srcpv = d.getVar("EXTERNALSRC_PKGV_SUFFIX")
+                if srcpv is None:
+                    srcpv = bb.fetch2.get_srcrev(d)
                 value = conn.getPR(base_ver, pkgarch, srcpv)
                 d.setVar("PRSERV_PV_AUTOINC", str(value))
 
@@ -319,7 +321,9 @@ python package_get_auto_pr() {
 python package_setup_pkgv() {
     pkgv = d.getVar("PKGV")
     # Expand SRCPV into PKGV if not present
-    srcpv = bb.fetch.get_pkgv_string(d)
+    srcpv = d.getVar("EXTERNALSRC_PKGV_SUFFIX")
+    if srcpv is None:
+        srcpv = bb.fetch.get_pkgv_string(d)
     if srcpv and "+" in pkgv:
         d.appendVar("PKGV", srcpv)
         pkgv = d.getVar("PKGV")
meta/classes/externalsrc.bbclass +3 -0
index 902ff2604f..47c1b06788 100644
@@ -78,6 +78,9 @@ python () {
             d.setVar('B', '${WORKDIR}/${BPN}-${PV}')
 
         bb.fetch.get_hashvalue(d)
+        # Keep the source version available for packaging. SCM URLs are removed
+        # below so externalsrc does not fetch or unpack the external source.
+        d.setVar('EXTERNALSRC_PKGV_SUFFIX', bb.fetch.get_pkgv_string(d))
         local_srcuri = []
         fetch = bb.fetch2.Fetch((d.getVar('SRC_URI') or '').split(), d)
         for url in fetch.urls:
Signature · 1 line
2.34.1
author
#7 lore raw

v2 cover externalsrc: preserve source revision in PKGV

In reply to Paddy Chiu

Hi Richard,

Patch v2 (Proposed fix) addresses the concern with the previous approach:
it no longer copies the datastore or rewrites SRC_URI during package version calculation.

Instead, externalsrc saves the formatted source-version suffix in EXTERNALSRC_PKGV_SUFFIX before filtering SCM URLs. The package and PR-service paths reuse it, while the normal fetcher path remains unchanged for non-externalsrc builds.

Validation used git-submodule-test for both requested transitions:

BitBake to devtool: both builds produce the same hash-bearing PKGV and no version-going-backwards QA error.

devtool to BitBake: both builds produce the same PKGV with no regression in the reverse transition.

Changes since v1:

- replace the copied datastore/SRC_URI override with a saved suffix;
- use the suffix for both PKGV and AUTOINC PR-service handling.

Signature · 1 line
2.34.1