Skip to contentSkip to conversation

bitbake: fetch2: correct PREMIRROR URI

PATCH In review Scott Weaver 2 participants 5 messages Aug 22, 2021

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 bitbake: fetch2: correct PREMIRROR URI

When downloadfilename is defined in a recipe's SRC_URI and a PREMIRROR is also defined for that URI, the downloadfilename is appended to the mirror URI and it should not be.

[YOCTO #13039]

---
 bitbake/lib/bb/fetch2/__init__.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
bitbake/lib/bb/fetch2/__init__.py +1 -1
index 914fa5c024..0bcd566e83 100644
@@ -467,7 +467,7 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
                     uri_decoded[5] = {}
                 elif ud.localpath and ud.method.supports_checksum(ud):
                     basename = os.path.basename(ud.localpath)
-                if basename and not result_decoded[loc].endswith(basename):
+                if basename and result_decoded[loc].endswith('/'):
                     result_decoded[loc] = os.path.join(result_decoded[loc], basename)
         else:
             return None
Signature · 1 line
2.25.1
#1 lore raw
Quoted 16 lines · Scott Weaver

When downloadfilename is defined in a recipe's SRC_URI and a PREMIRROR is also defined for that URI, the downloadfilename is appended to the mirror URI and it should not be.

[YOCTO #13039]

---
 bitbake/lib/bb/fetch2/__init__.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
bitbake/lib/bb/fetch2/__init__.py +0 -0
index 914fa5c024..0bcd566e83 100644

bitbake/lib/bb/fetch2/__init__.py def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None): · line 473

Quoted 8 lines
bitbake/lib/bb/fetch2/__init__.py +1 -1
@@ -467,7 +467,7 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
                     uri_decoded[5] = {}
                 elif ud.localpath and ud.method.supports_checksum(ud):
                     basename = os.path.basename(ud.localpath)
-                if basename and not result_decoded[loc].endswith(basename):
+                if basename and result_decoded[loc].endswith('/'):
                     result_decoded[loc] = os.path.join(result_decoded[loc], basename)
         else:
             return None

Thanks for the patch. Should we be adding a test for this to lib/bb/tests/fetch.py (run with bitbake-selftest)?

Cheers

Richard

#2 lore raw
Quoted 12 lines · Richard Purdie
Quoted 11 lines · Scott Weaver

When downloadfilename is defined in a recipe's SRC_URI and a PREMIRROR is also defined for that URI, the downloadfilename is appended to the mirror URI and it should not be.

[YOCTO #13039]

---
 bitbake/lib/bb/fetch2/__init__.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 16 lines

bitbake/lib/bb/fetch2/__init__.py def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None): · line 473

Quoted 13 lines
bitbake/lib/bb/fetch2/__init__.py +1 -1
index 914fa5c024..0bcd566e83 100644
@@ -467,7 +467,7 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
                     uri_decoded[5] = {}
                 elif ud.localpath and ud.method.supports_checksum(ud):
                     basename = os.path.basename(ud.localpath)
-                if basename and not result_decoded[loc].endswith(basename):
+                if basename and result_decoded[loc].endswith('/'):
                     result_decoded[loc] = os.path.join(result_decoded[loc], basename)
         else:
             return None

Thanks for the patch. Should we be adding a test for this to lib/bb/tests/fetch.py (run with bitbake-selftest)?

I'll look into that. I'm not familiar with the selftest but it's a good opportunity to learn. :)

Quoted 5 lines

Cheers

Richard

Signature · 1 line
  - Scott
#3 lore raw
Quoted 30 lines · Scott Weaver
Quoted 12 lines · Richard Purdie
Quoted 11 lines · Scott Weaver

When downloadfilename is defined in a recipe's SRC_URI and a PREMIRROR is also defined for that URI, the downloadfilename is appended to the mirror URI and it should not be.

[YOCTO #13039]

---
 bitbake/lib/bb/fetch2/__init__.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 16 lines

bitbake/lib/bb/fetch2/__init__.py def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None): · line 473

Quoted 13 lines
bitbake/lib/bb/fetch2/__init__.py +1 -1
index 914fa5c024..0bcd566e83 100644
@@ -467,7 +467,7 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
                     uri_decoded[5] = {}
                 elif ud.localpath and ud.method.supports_checksum(ud):
                     basename = os.path.basename(ud.localpath)
-                if basename and not result_decoded[loc].endswith(basename):
+                if basename and result_decoded[loc].endswith('/'):
                     result_decoded[loc] = os.path.join(result_decoded[loc], basename)
         else:
             return None

Thanks for the patch. Should we be adding a test for this to lib/bb/tests/fetch.py (run with bitbake-selftest)?

Quoted 2 lines

I'll look into that. I'm not familiar with the selftest but it's a good opportunity to learn. :)

Well, I've learned that this patch breaks the selftest.
I was wondering what the usecase was for the original line I changed but it seems this code is sensitive to whether PREMIRRORS is defined with a trailing '/' or not.
I'll dig deeper.

Quoted 7 lines
Quoted 6 lines

Cheers

Richard

Signature · 1 line
  - Scott
#4 lore raw
Quoted 31 lines · Scott Weaver
Quoted 30 lines · Scott Weaver
Quoted 28 lines · Richard Purdie
Quoted 16 lines · Scott Weaver

When downloadfilename is defined in a recipe's SRC_URI and a PREMIRROR is also defined for that URI, the downloadfilename is appended to the mirror URI and it should not be.

[YOCTO #13039]

---
 bitbake/lib/bb/fetch2/__init__.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
bitbake/lib/bb/fetch2/__init__.py +0 -0
index 914fa5c024..0bcd566e83 100644

bitbake/lib/bb/fetch2/__init__.py def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None): · line 473

Quoted 8 lines
bitbake/lib/bb/fetch2/__init__.py +1 -1
@@ -467,7 +467,7 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
                     uri_decoded[5] = {}
                 elif ud.localpath and ud.method.supports_checksum(ud):
                     basename = os.path.basename(ud.localpath)
-                if basename and not result_decoded[loc].endswith(basename):
+                if basename and result_decoded[loc].endswith('/'):
                     result_decoded[loc] = os.path.join(result_decoded[loc], basename)
         else:
             return None

Thanks for the patch. Should we be adding a test for this to lib/bb/tests/fetch.py (run with bitbake-selftest)?

Quoted 4 lines
Quoted 2 lines

I'll look into that. I'm not familiar with the selftest but it's a good opportunity to learn. :)

Well, I've learned that this patch breaks the selftest.

I had wondered but hadn't got to testing that yet. Where we find corner cases we do try and teach the test suite so we try and avoid regressions. We've had a lot of issues with this kind of thing in the past.

Quoted 4 lines

I was wondering what the usecase was for the original line I changed but it seems this code is sensitive to whether PREMIRRORS is defined with a trailing '/' or not.
I'll dig deeper.

Thanks, I don't doubt there are issues in there and hopefully the testsuite will show it's value and help figure through things!

Cheers,

Richard