Is http://downloads.yoctoproject.org/ down?
I can't reach http://downloads.yoctoproject.org/ to download http://downloads.yoctoproject.org/releases/uninative/2.7/x86_64-nativesdk-libc.tar.xz .
Did somebody know anything?
Quoted 6 lines · Zwicker Severin
I can’t reach http://downloads.yoctoproject.org/ to download http://downloads.yoctoproject.org/releases/uninative/2.7/x86_64-nativesdk-libc.tar.xz .
Did somebody know anything?
Yes, it currently appears to be down. The relevant folks will be looking at it but it's still early in the US timezones.
Quoted 11 lines · Paul Barker
Quoted 6 lines · Zwicker Severin
I can’t reach http://downloads.yoctoproject.org/ to download http://downloads.yoctoproject.org/releases/uninative/2.7/x86_64-nativesdk-libc.tar.xz .
Did somebody know anything?
Yes, it currently appears to be down. The relevant folks will be looking at it but it's still early in the US timezones.
Could it be Europe-specific problem?
Seems slow, but still working here in the US:
$ wget http://downloads.yoctoproject.org/releases/uninative/2.7/x86_64-nativesdk-libc.tar.xz --2020-05-12 09:08:15-- http://downloads.yoctoproject.org/releases/uninative/2.7/x86_64-nativesdk-libc.tar.xz Resolving downloads.yoctoproject.org... 198.145.29.63 Connecting to downloads.yoctoproject.org|198.145.29.63|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 5450224 (5.2M) [application/octet-stream] Saving to: ‘x86_64-nativesdk-libc.tar.xz’
x86_64-nativesdk-libc.tar.xz 100%[==========================================================================>] 5.20M 540KB/s in 59s
2020-05-12 09:09:27 (89.5 KB/s) - ‘x86_64-nativesdk-libc.tar.xz’ saved [5450224/5450224]
With downloads.yoctoproject.org down again this begs the question:
Why does uninative tarball get fetched by default from the internet every time?
Why is it in not cached like every other tarball that is downloaded by recipes?
The default behaviour for the yocto project should be to have the tarball stored in the yocto cache such that it is not fetched every time?
Quoted 9 lines · Scott Branden via lists.yoctoproject.org
With downloads.yoctoproject.org down again this begs the question:
Why does uninative tarball get fetched by default from the internet every time?
Why is it in not cached like every other tarball that is downloaded by recipes?
The default behaviour for the yocto project should be to have the tarball stored in the yocto cache such that it is not fetched every time?
Those are good questions.
As far as I know on my own local builds, the uninative tarball *is* cached locally in DL_DIR. We don't see repeated downloads on the autobuilder either so it appears to work there too.
If you're not seeing that it would definitely be something to investigate.
Cheers,
Richard
Hi Richard,
Quoted 11 lines · Richard Purdie
Quoted 9 lines · Scott Branden via lists.yoctoproject.org
With downloads.yoctoproject.org down again this begs the question:
Why does uninative tarball get fetched by default from the internet every time?
Why is it in not cached like every other tarball that is downloaded by recipes?
The default behaviour for the yocto project should be to have the tarball stored in the yocto cache such that it is not fetched every time?
Quoted 5 lines
Those are good questions.
As far as I know on my own local builds, the uninative tarball *is* cached locally in
DL_DIR. We don't see repeated downloads on the autobuilder either so it appears to work there too.
Yes, the uninative tarball *is* cached but I see repeated downloads.
Quoted 3 lines
If you're not seeing that it would definitely be something to investigate.
I just did a clone of poky.git and the repeated downloads do NOT happen. So I now need to investigate what is different when adding layers and conf settings that is causing the issue.
Quoted 4 lines
Cheers,
Richard
Thanks for confirming the uninative caching should work, but it is not in my environment and I will have to investigate.
Scott
We found the source of the uninative fetch problem:
SOURCE_MIRROR_URL needs to be set in order for uninative to be picked up from local mirror.
The standard PREMIRRORS mechanism doesn't work for this single tarball.
So every time a clean poky build took place it always went to the internet to fetch the uninative tarball the first time and store it in the local build download.
I think this should be clearly documented somewhere in the yocto manual?
Quoted 33 lines · Scott Branden
Hi Richard,
Quoted 11 lines · Richard Purdie
Quoted 9 lines · Scott Branden via lists.yoctoproject.org
With downloads.yoctoproject.org down again this begs the question:
Why does uninative tarball get fetched by default from the internet every time?
Why is it in not cached like every other tarball that is downloaded by recipes?
The default behaviour for the yocto project should be to have the tarball stored in the yocto cache such that it is not fetched every time?Quoted 5 lines
Those are good questions.
As far as I know on my own local builds, the uninative tarball *is* cached locally in
DL_DIR. We don't see repeated downloads on the autobuilder either so it appears to work there too.Yes, the uninative tarball *is* cached but I see repeated downloads.
Quoted 2 lines
If you're not seeing that it would definitely be something to investigate.
I just did a clone of poky.git and the repeated downloads do NOT happen. So I now need to investigate what is different when adding layers and conf settings that is causing the issue.
Quoted 4 lines
Cheers,
Richard
Thanks for confirming the uninative caching should work, but it is not in my environment and I will have to investigate.
Scott
Quoted 5 lines · Scott Branden
We found the source of the uninative fetch problem:
SOURCE_MIRROR_URLneeds to be set in order for uninative to be picked up from local mirror.
Quoted 8 lines
The standard PREMIRRORS mechanism doesn't work for this single tarball.
So every time a clean poky build took place it always went to the internet to fetch the uninative tarball
the first time and store it in the local build download.I think this should be clearly documented somewhere in the yocto manual?
That definitely explains things. I think its assumed that people would set appropriate PREMIRRORS to cover uninative in this case but that isn't documented. I'm not even sure we want for people to have to care about this :/.
Can you try this change:
meta/classes/uninative.bbclass +8 -3
@@ -56,9 +56,14 @@ python uninative_event_fetchloader() { # Our games with path manipulation of DL_DIR mean standard PREMIRRORS don't work # and we can't easily put 'chksum' into the url path from a url parameter with # the current fetcher url handling - ownmirror = d.getVar('SOURCE_MIRROR_URL') - if ownmirror: - localdata.appendVar("PREMIRRORS", " ${UNINATIVE_URL}${UNINATIVE_TARBALL} ${SOURCE_MIRROR_URL}/uninative/%s/${UNINATIVE_TARBALL}" % chksum) + premirrors = bb.fetch2.mirror_from_string(localdata.getVar("PREMIRRORS")) + for line in premirrors: + try: + (find, replace) = line + except ValueError: + continue + if find.startswith("http"): + localdata.appendVar("PREMIRRORS", " ${UNINATIVE_URL}${UNINATIVE_TARBALL} %s/uninative/%s/${UNINATIVE_TARBALL}" % (replace, chksum)) srcuri = d.expand("${UNINATIVE_URL}${UNINATIVE_TARBALL};sha256sum=%s" % chksum) bb.note("Fetching uninative binary shim from %s" % srcuri)
Hopefully that handles this case more generically.
Cheers,
Richard
Hi Richard,
Quoted 33 lines · Richard Purdie
Quoted 5 lines · Scott Branden
We found the source of the uninative fetch problem:
SOURCE_MIRROR_URLneeds to be set in order for uninative to be picked up from local mirror.Quoted 8 lines
The standard PREMIRRORS mechanism doesn't work for this single tarball.
So every time a clean poky build took place it always went to the internet to fetch the uninative tarball
the first time and store it in the local build download.I think this should be clearly documented somewhere in the yocto manual?
That definitely explains things. I think its assumed that people would set appropriate PREMIRRORS to cover uninative in this case but that isn't documented. I'm not even sure we want for people to have to care about this :/.
Can you try this change:
meta/classes/uninative.bbclass +1
-3@@ -56,9 +56,14 @@ python uninative_event_fetchloader() { # Our games with path manipulation of DL_DIR mean standard PREMIRRORS don't work # and we can't easily put 'chksum' into the url path from a url parameter with # the current fetcher url handling - ownmirror = d.getVar('SOURCE_MIRROR_URL') - if ownmirror: - localdata.appendVar("PREMIRRORS", " ${UNINATIVE_URL}${UNINATIVE_TARBALL} ${SOURCE_MIRROR_URL}/uninative/%s/${UNINATIVE_TARBALL}" % chksum) + premirrors = bb.fetch2.mirror_from_string(localdata.getVar("PREMIRRORS"))
meta/classes/uninative.bbclass python uninative_event_fetchloader() { · line 69
Quoted 10 lines
meta/classes/uninative.bbclass +7
-0+ for line in premirrors: + try: + (find, replace) = line + except ValueError: + continue + if find.startswith("http"): + localdata.appendVar("PREMIRRORS", " ${UNINATIVE_URL}${UNINATIVE_TARBALL} %s/uninative/%s/${UNINATIVE_TARBALL}" % (replace, chksum)) srcuri = d.expand("${UNINATIVE_URL}${UNINATIVE_TARBALL};sha256sum=%s" % chksum) bb.note("Fetching uninative binary shim from %s" % srcuri)
Note is missleading as it is going to fetch from PREMIRRORS.
Quoted 2 lines
Hopefully that handles this case more generically.
Yes, this works. uninative is now fetched from the PREMIRRORS like all the other downloads even though it prints the note about Fetching uninative binary shim from the URL.
Quoted 7 lines
Cheers,
Richard
Quoted 36 lines · Scott Branden
Quoted 28 lines
meta/classes/uninative.bbclass +0
-0@@ -56,9 +56,14 @@ python uninative_event_fetchloader() { # Our games with path manipulation of DL_DIR meanstandard PREMIRRORS don't work
# and we can't easily put 'chksum' into the url path from a url parameter with # the current fetcher url handlingmeta/classes/uninative.bbclass +0
-3- ownmirror = d.getVar('SOURCE_MIRROR_URL') - if ownmirror: - localdata.appendVar("PREMIRRORS", "${UNINATIVE_URL}${UNINATIVE_TARBALL} ${SOURCE_MIRROR_URL}/uninative/%s/${UNINATIVE_TARBALL}" % chksum)
+ premirrors =
bb.fetch2.mirror_from_string(localdata.getVar("PREMIRRORS"))meta/classes/uninative.bbclass +7
-0+ for line in premirrors: + try: + (find, replace) = line + except ValueError: + continue + if find.startswith("http"): + localdata.appendVar("PREMIRRORS", "${UNINATIVE_URL}${UNINATIVE_TARBALL} %s/uninative/%s/${UNINATIVE_TARBALL}" % (replace, chksum))
Quoted 5 lines
srcuri =
d.expand("${UNINATIVE_URL}${UNINATIVE_TARBALL};sha256sum=%s" % chksum)
bb.note("Fetching uninative binary shim from %s" %
srcuri)Note is missleading as it is going to fetch from PREMIRRORS.
Quoted 4 lines
Quoted 1 line
Hopefully that handles this case more generically.
Yes, this works. uninative is now fetched from the PREMIRRORS like all the other downloads even though it prints the note about Fetching uninative binary shim from the URL.
Thanks, I've tweaked the patch to improve the note and submitted it. Good to get to the bottom of this!
Cheers,
Richard