Skip to contentSkip to conversation

Manually fetching files (and disabling the sanity-checker)

DISCUSSION Tim Bird 5 participants 6 messages Jul 18, 2012 – Aug 3, 2012

lore
Share & tools
lore

Tim Bird

author
#0 lore raw

Hi all,

I've given up trying to work around Sony's firewall. After repeated attempts I still can't get any 'git:'-based URLS to work through the firewall.

I vaguely recall the last time I worked on this (about 5.0 timeframe) there was a mirror where tarballs could be downloaded, if the git fetch didn't work.

Is this mirror still available? If not, is there any way to disable the sanity-checker, and have bitbake tell me what repositories it's looking for, so I can manually construct a local mirror for yoctoproject to work off of?

Is there any way to extract the tarballs from a build appliance image?

Any advice or pointers would be appreciated.

-- Tim

=============================
Tim Bird
Architecture Group Chair, CE Workgroup of the Linux Foundation Senior Staff Engineer, Sony Network Entertainment
=============================

#1 lore raw
Quoted 5 lines · Tim Bird

Hi all,

I've given up trying to work around Sony's firewall. After repeated attempts I still can't get any 'git:'-based URLS to work through the firewall.

Quoted 8 lines

I vaguely recall the last time I worked on this (about 5.0 timeframe) there was a mirror where tarballs could be downloaded, if the git fetch didn't work.

Is this mirror still available? If not, is there any way to disable the sanity-checker, and have bitbake tell me what repositories it's looking for, so I can manually construct a local mirror for yoctoproject to work off of?

To disable the sanity check you can set CONNECTIVITY_CHECK_URIS to "" and that will cause it to be skipped.

Regarding the mirroring, have you tried to just set the PREMIRRORS to use only http* and ftp in your local.conf:
PREMIRRORS = "\

ftp://.*/.*      http://downloads.yoctoproject.org/mirror/sources/ \n \
http://.*/.*     http://downloads.yoctoproject.org/mirror/sources/ \n \
https://.*/.*    http://downloads.yoctoproject.org/mirror/sources/ \n"

I am not sure, but this might force protocols. You could try a "bitbake -c fetchall universe"

I am going have Michael H. get you the information about where the the Downloads are.

Let me know how it goes.

Sau!

Quoted 19 lines

Is there any way to extract the tarballs from a build appliance image?

Any advice or pointers would be appreciated.

-- Tim

=============================
Tim Bird
Architecture Group Chair, CE Workgroup of the Linux Foundation Senior Staff Engineer, Sony Network Entertainment
=============================

_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

#4 lore raw
In reply to Tim Bird
Quoted 7 lines · Tim Bird

Hi all,

I've given up trying to work around Sony's firewall. After repeated attempts I still can't get any 'git:'-based URLS to work through the firewall.

I vaguely recall the last time I worked on this (about 5.0 timeframe) there was a mirror where tarballs could be downloaded, if the git fetch didn't work.

Sources the autobuilder uses are made available via http at
http://downloads.yoctoproject.org/mirror/sources/ and can be used as a premirror.

The directions in the following post should help you with set up,
https://lists.yoctoproject.org/pipermail/yocto/2012-May/008966.html

Michael Halstead
Yocto Project / Sys Admin

Quoted 23 lines

Is this mirror still available? If not, is there any way to disable the sanity-checker, and have bitbake tell me what repositories it's looking for, so I can manually construct a local mirror for yoctoproject to work off of?

Is there any way to extract the tarballs from a build appliance image?

Any advice or pointers would be appreciated.

-- Tim

=============================
Tim Bird
Architecture Group Chair, CE Workgroup of the Linux Foundation Senior Staff Engineer, Sony Network Entertainment
=============================

_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

#5 lore raw
In reply to Tim Bird
Quoted 8 lines · Tim Bird

Hi all,

I've given up trying to work around Sony's firewall. After repeated attempts I still can't get any 'git:'-based URLS to work through the firewall.

I vaguely recall the last time I worked on this (about 5.0 timeframe) there was a mirror where tarballs could be downloaded, if the git fetch didn't work.

Quoted 8 lines

Is this mirror still available? If not, is there any way to disable the sanity-checker, and have bitbake tell me what repositories it's looking for, so I can manually construct a local mirror for yoctoproject to work off of?

Is there any way to extract the tarballs from a build appliance image?

Any advice or pointers would be appreciated.

Another less fun option is to create a siteconfig of some sort and start overriding git SRC_URI...

SRC_URI_pn-linux-yocto = "proto://some/internal/mirror/you/setup.git"

This might be useful for stuff that's not yet mirrored on yp.org

-M

author
#2 lore raw
In reply to Saul Wold
Quoted 24 lines · Saul Wold
Quoted 5 lines · Tim Bird

Hi all,

I've given up trying to work around Sony's firewall. After repeated attempts I still can't get any 'git:'-based URLS to work through the firewall.

Quoted 8 lines

I vaguely recall the last time I worked on this (about 5.0 timeframe) there was a mirror where tarballs could be downloaded, if the git fetch didn't work.

Is this mirror still available? If not, is there any way to disable the sanity-checker, and have bitbake tell me what repositories it's looking for, so I can manually construct a local mirror for yoctoproject to work off of?

To disable the sanity check you can set CONNECTIVITY_CHECK_URIS to "" and that will cause it to be skipped.

Regarding the mirroring, have you tried to just set the PREMIRRORS to use only http* and ftp in your local.conf:
PREMIRRORS = "\

ftp://.*/.*      http://downloads.yoctoproject.org/mirror/sources/ \n \
http://.*/.*     http://downloads.yoctoproject.org/mirror/sources/ \n \
https://.*/.*    http://downloads.yoctoproject.org/mirror/sources/ \n"
Quoted 7 lines

I am not sure, but this might force protocols. You could try a "bitbake -c fetchall universe"

I am going have Michael H. get you the information about where the the Downloads are.

Let me know how it goes.

Just to report back, I successfully used this and the instructions Michael Halstead provided, from https://lists.yoctoproject.org/pipermail/yocto/2012-May/008966.html
to work around my non-working Sony proxy issue.

So thank you both very much! I completed several builds in the past week, and am looking forward to doing some experimentation with my own custom distribution, for some research I'm doing.

I should note that this is independent of the work being done by Sony's internal distribution team, who have been working to migrate Sony's internal Linux distribution to the Yocto Project for a few months now.

=============================
Tim Bird
Architecture Group Chair, CE Workgroup of the Linux Foundation Senior Staff Engineer, Sony Network Entertainment
=============================

#3 lore raw
Quoted 4 lines

From: [email protected] [mailto:yocto- [email protected]] On Behalf Of Tim Bird
Sent: Friday, August 03, 2012 10:07 AM

Quoted 8 lines

Just to report back, I successfully used this and the instructions Michael Halstead provided, from https://lists.yoctoproject.org/pipermail/yocto/2012-
May/008966.html
to work around my non-working Sony proxy issue.

So thank you both very much! I completed several builds in the past week, and am looking forward to doing some experimentation with my own custom distribution, for some research I'm doing.

Tim - great to hear!

Saul/Michael - thanks for your help.