Import own-mirrors class from OE

PATCH Applied Khem Raj 6 participants 11 messages Mar 12, 2011 – Mar 15, 2011

lore

Share & tools

lore

Khem Raj

author

2011-03-12 15:06 UTC #0 lore raw

cover Import own-mirrors class from OE

Hi Richard

Pull URL: git://git.openembedded.org/openembedded-core-contrib.git
Branch: kraj/own-mirror
Browse: http://git.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=kraj/own-mirror

Thanks,
Khem Raj <[email protected]>
---

Khem Raj (1):
own-mirrors.bbclass: Bring own-mirrors.bbclass from oe commit

    0ef914b250df46a41348479446214575668943fb

 meta/classes/own-mirrors.bbclass |   12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)
 create mode 100644 meta/classes/own-mirrors.bbclass

Signature · 1 line

1.7.4.1

Khem Raj

author

2011-03-12 15:06 UTC #1 lore raw

PATCH 1/1 own-mirrors.bbclass: Bring own-mirrors.bbclass from oe commit 0ef914b250df46a41348479446214575668943fb

below is the history on this class. Documentation for newly introduced SOURCE_MIRROR_URL is already present in oe-core

commit 0ef914b250df46a41348479446214575668943fb

Author: Eric BENARD <[email protected]>
Date:   Wed Nov 3 13:28:54 2010 +0000

own-mirrors.bbclass: allow mirroring of scm fetched packages

this way, it's possible to setup a local webserver (for example using busybox httpd -p "8081" -h backuped_download_dir) serving a presiously fetched download directory and to build wihout the need for an internet access

this can also be used when connected to know to know which packages are missing from the local mirror's directory (and thus are fetched from internet as a fallback), it's possible to run the server this way :
busybox httpd -p "8081" -h backuped_download_dir -vv -f | grep -B 1 response:404 to get the name of the missing packages.

Signed-off-by Eric Bénard <[email protected]>

Acked-by Denys Dmytriyenko <[email protected]>

Acked-by Frans Meulenbroeks <[email protected]>

Signed-off-by Tom Rini <[email protected]>

commit 1b661974e3f8f844f6ec4cdb7bb42cef9595b626

Author: Marcin Juszkiewicz <[email protected]>
Date:   Sat Mar 10 14:10:06 2007 +0000

own-mirrors.bbclass: added class which can be used to set PREMIRRORS from config
- SOURCE_MIRROR_URL is new variable which point to source mirror which will be used before fetching from original SRC_URI location.

Signed-off-by Khem Raj <[email protected]>

---
 meta/classes/own-mirrors.bbclass |   12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)
 create mode 100644 meta/classes/own-mirrors.bbclass

meta/classes/own-mirrors.bbclass new file +12 -0

new file mode 100644
index 0000000..8a6feaf
@@ -0,0 +1,12 @@
+PREMIRRORS() {
+cvs://.*/.*     ${SOURCE_MIRROR_URL}
+svn://.*/.*     ${SOURCE_MIRROR_URL}
+git://.*/.*     ${SOURCE_MIRROR_URL}
+hg://.*/.*      ${SOURCE_MIRROR_URL}
+bzr://.*/.*     ${SOURCE_MIRROR_URL}
+svk://.*/.*     ${SOURCE_MIRROR_URL}
+p4://.*/.*      ${SOURCE_MIRROR_URL}
+osc://.*/.*     ${SOURCE_MIRROR_URL}
+https?$://.*/.* ${SOURCE_MIRROR_URL}
+ftp://.*/.*     ${SOURCE_MIRROR_URL}
+}

Signature · 1 line

1.7.4.1

Gary Thomas

2011-03-13 16:05 UTC #2 lore raw

In reply to Khem Raj

Quoted 13 lines · Khem Raj

Hi Richard
Pull URL: git://git.openembedded.org/openembedded-core-contrib.git
Branch: kraj/own-mirror
Browse: http://git.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=kraj/own-mirror
Thanks,
Khem Raj<[email protected]>
---

Quoted 8 lines

Khem Raj (1):
own-mirrors.bbclass: Bring own-mirrors.bbclass from oe commit
     0ef914b250df46a41348479446214575668943fb
  meta/classes/own-mirrors.bbclass |   12 ++++++++++++
  1 files changed, 12 insertions(+), 0 deletions(-)
create mode 100644 meta/classes/own-mirrors.bbclass

Poky/Yocto already has a mechanism for this by overriding PREMIRRORS I would think sticking with one extension method would be better.

Here's what I have in my [Poky] DISTRO.conf:

SOURCE_DIR ?= "/opt/poky/sources"
PREMIRRORS = "\

http://.*/.*   file://${SOURCE_DIR}/ \n \
https://.*/.*  file://${SOURCE_DIR}/ \n \
ftp://.*/.*    file://${SOURCE_DIR}/ \n \
bzr://.*/.*    file://${SOURCE_DIR}/ \n \
cvs://.*/.*    file://${SOURCE_DIR}/ \n \
git://.*/.*    file://${SOURCE_DIR}/ \n \
hg://.*/.*     file://${SOURCE_DIR}/ \n \
osc://.*/.*    file://${SOURCE_DIR}/ \n \
p4://.*/.*     file://${SOURCE_DIR}/ \n \
svk://.*/.*    file://${SOURCE_DIR}/ \n \
svn://.*/.*    file://${SOURCE_DIR}/ \n"

Users can have their own mirror by defining SOURCE_DIR in local.conf

Signature · 4 lines

------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Koen Kooi

2011-03-13 16:15 UTC #3 lore raw

Op 13 mrt 2011, om 16:56 heeft Gary Thomas het volgende geschreven:

Quoted 42 lines · Khem Raj

Quoted 13 lines
Hi Richard
Pull URL: git://git.openembedded.org/openembedded-core-contrib.git
Branch: kraj/own-mirror
Browse: http://git.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=kraj/own-mirror
Thanks,
Khem Raj<[email protected]>
---
Quoted 8 lines
Khem Raj (1):
own-mirrors.bbclass: Bring own-mirrors.bbclass from oe commit
    0ef914b250df46a41348479446214575668943fb
 meta/classes/own-mirrors.bbclass |   12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)
 create mode 100644 meta/classes/own-mirrors.bbclass
Poky/Yocto already has a mechanism for this by overriding PREMIRRORS I would think sticking with one extension method would be better.
Here's what I have in my [Poky] DISTRO.conf:
SOURCE_DIR ?= "/opt/poky/sources"
PREMIRRORS = "\
http://.*/.*   file://${SOURCE_DIR}/ \n \
https://.*/.*  file://${SOURCE_DIR}/ \n \
ftp://.*/.*    file://${SOURCE_DIR}/ \n \
bzr://.*/.*    file://${SOURCE_DIR}/ \n \
cvs://.*/.*    file://${SOURCE_DIR}/ \n \
git://.*/.*    file://${SOURCE_DIR}/ \n \
hg://.*/.*     file://${SOURCE_DIR}/ \n \
osc://.*/.*    file://${SOURCE_DIR}/ \n \
p4://.*/.*     file://${SOURCE_DIR}/ \n \
svk://.*/.*    file://${SOURCE_DIR}/ \n \
svn://.*/.*    file://${SOURCE_DIR}/ \n"

Quoted 1 line

Users can have their own mirror by defining SOURCE_DIR in local.conf

How is that easier and more maintainable than:

INHERIT += "own-mirrors"
SOURCE_MIRROR_URL = "/some/dir"

Especially when more uri methods get added to the fetcher and oe-core.

Gary Thomas

2011-03-13 17:10 UTC #4 lore raw

Quoted 47 lines · Koen Kooi

Op 13 mrt 2011, om 16:56 heeft Gary Thomas het volgende geschreven:
Quoted 42 lines · Khem Raj
Quoted 13 lines
Hi Richard
Pull URL: git://git.openembedded.org/openembedded-core-contrib.git
Branch: kraj/own-mirror
Browse: http://git.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=kraj/own-mirror
Thanks,
Khem Raj<[email protected]>
---
Quoted 8 lines
Khem Raj (1):
own-mirrors.bbclass: Bring own-mirrors.bbclass from oe commit
     0ef914b250df46a41348479446214575668943fb
  meta/classes/own-mirrors.bbclass |   12 ++++++++++++
  1 files changed, 12 insertions(+), 0 deletions(-)
create mode 100644 meta/classes/own-mirrors.bbclass
Poky/Yocto already has a mechanism for this by overriding PREMIRRORS I would think sticking with one extension method would be better.
Here's what I have in my [Poky] DISTRO.conf:
SOURCE_DIR ?= "/opt/poky/sources"
PREMIRRORS = "\
http://.*/.*   file://${SOURCE_DIR}/ \n \
https://.*/.*  file://${SOURCE_DIR}/ \n \
ftp://.*/.*    file://${SOURCE_DIR}/ \n \
bzr://.*/.*    file://${SOURCE_DIR}/ \n \
cvs://.*/.*    file://${SOURCE_DIR}/ \n \
git://.*/.*    file://${SOURCE_DIR}/ \n \
hg://.*/.*     file://${SOURCE_DIR}/ \n \
osc://.*/.*    file://${SOURCE_DIR}/ \n \
p4://.*/.*     file://${SOURCE_DIR}/ \n \
svk://.*/.*    file://${SOURCE_DIR}/ \n \
svn://.*/.*    file://${SOURCE_DIR}/ \n"
Quoted 1 line
Users can have their own mirror by defining SOURCE_DIR in local.conf

Quoted 8 lines

How is that easier and more maintainable than:
INHERIT += "own-mirrors"
SOURCE_MIRROR_URL = "/some/dir"
?
Especially when more uri methods get added to the fetcher and oe-core.

Ask Richard? I was just pointing out that Poky/Yocto already has a mechanism; perhaps the OE one is better, who knows.

Signature · 4 lines

------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Chris Larson

2011-03-13 18:56 UTC #5 lore raw

Quoted 3 lines · Gary Thomas

Ask Richard? I was just pointing out that Poky/Yocto already has a mechanism;
perhaps the OE one is better, who knows.

There seems to be some confusion here. PREMIRRORS isn't a "poky/yocto" mechanism. It started in OE, and stills works just fine there (it's implemented in bitbake). own-mirrors is simply a convenience to make premirrors manipulation easier for the user. If convenience and usability isn't a priority for poky/yocto, I'd be curious to hear the reasoning.

Signature · 5 lines

Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Gary Thomas

2011-03-13 22:19 UTC #6 lore raw

Quoted 11 lines · Chris Larson

Quoted 3 lines · Gary Thomas
Ask Richard? I was just pointing out that Poky/Yocto already has a mechanism;
perhaps the OE one is better, who knows.
There seems to be some confusion here. PREMIRRORS isn't a "poky/yocto" mechanism. It started in OE, and stills works just fine there (it's implemented in bitbake). own-mirrors is simply a convenience to make premirrors manipulation easier for the user. If convenience and usability isn't a priority for poky/yocto, I'd be curious to hear the reasoning.

Fair enough; I was merely commenting on how I had been told to handle this in Poky/Yocto. If OE has a better way, perhaps it makes sense for it to be available in both OE-CORE and Poky/Yocto.

Signature · 4 lines

------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Koen Kooi

2011-03-14 08:00 UTC #7 lore raw

Op 13 mrt 2011, om 23:18 heeft Gary Thomas het volgende geschreven:

Quoted 13 lines · Chris Larson

Quoted 11 lines · Gary Thomas
Quoted 3 lines
Ask Richard? I was just pointing out that Poky/Yocto already has a mechanism;
perhaps the OE one is better, who knows.
There seems to be some confusion here. PREMIRRORS isn't a "poky/yocto" mechanism. It started in OE, and stills works just fine there (it's implemented in bitbake). own-mirrors is simply a convenience to make premirrors manipulation easier for the user. If convenience and usability isn't a priority for poky/yocto, I'd be curious to hear the reasoning.

Quoted 3 lines

Fair enough; I was merely commenting on how I had been told to handle this in Poky/Yocto. If OE has a better way, perhaps it makes sense for it to be available in both OE-CORE and Poky/Yocto.

And next time please have a look at the actual patch posted before commenting on it. The patch does *exactly* what you are saying, namely overriding PREMIRRORS.

Richard Purdie

2011-03-15 01:13 UTC #8 lore raw

In reply to Chris Larson

Quoted 11 lines · Chris Larson

Quoted 3 lines · Gary Thomas
Ask Richard? I was just pointing out that Poky/Yocto already has a mechanism;
perhaps the OE one is better, who knows.
There seems to be some confusion here. PREMIRRORS isn't a "poky/yocto" mechanism. It started in OE, and stills works just fine there (it's implemented in bitbake). own-mirrors is simply a convenience to make premirrors manipulation easier for the user. If convenience and usability isn't a priority for poky/yocto, I'd be curious to hear the reasoning.

There were various fixes/improvements to PREMIRRORS as part of the fetch2 work which is perhaps where the confusion has come from.

This mirror enhancement is tangential to that and the patch is fine, I just merged it.

Cheers,

Richard

Tom Rini

2011-03-14 16:47 UTC #9 lore raw

In reply to Gary Thomas

Quoted 44 lines · Gary Thomas

Quoted 15 lines · Khem Raj
Hi Richard
Pull URL: git://git.openembedded.org/openembedded-core-contrib.git
Branch: kraj/own-mirror
Browse:
http://git.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=kraj/own-mirror
Thanks,
Khem Raj<[email protected]>
---
Quoted 8 lines
Khem Raj (1):
own-mirrors.bbclass: Bring own-mirrors.bbclass from oe commit 0ef914b250df46a41348479446214575668943fb
meta/classes/own-mirrors.bbclass | 12 ++++++++++++
1 files changed, 12 insertions(+), 0 deletions(-)
create mode 100644 meta/classes/own-mirrors.bbclass
Poky/Yocto already has a mechanism for this by overriding PREMIRRORS I would think sticking with one extension method would be better.
Here's what I have in my [Poky] DISTRO.conf:
SOURCE_DIR ?= "/opt/poky/sources" PREMIRRORS = "\
http://.*/.* file://${SOURCE_DIR}/ \n \
https://.*/.* file://${SOURCE_DIR}/ \n \
ftp://.*/.* file://${SOURCE_DIR}/ \n \
bzr://.*/.* file://${SOURCE_DIR}/ \n \
cvs://.*/.* file://${SOURCE_DIR}/ \n \
git://.*/.* file://${SOURCE_DIR}/ \n \
hg://.*/.* file://${SOURCE_DIR}/ \n \
osc://.*/.* file://${SOURCE_DIR}/ \n \
p4://.*/.* file://${SOURCE_DIR}/ \n \
svk://.*/.* file://${SOURCE_DIR}/ \n \
svn://.*/.* file://${SOURCE_DIR}/ \n"

Quoted 1 line

Users can have their own mirror by defining SOURCE_DIR in local.conf

I know this has been beaten to death, but I want to try answering differently. What you mention above isn't so much a solution in poky as simply using the normal PREMIRRORS mechanism. The own-mirrors patch is a wrapper around this logic to make it a 2 line local.conf change. In your example, this would become:
INHERIT += "own-mirrors"
SOURCE_MIRROR_URL = "file:///opt/poky/sources"

(Or http://localhost:8080 and do a python -m SimpleHTTPServer 8080 & in /opt/poky/sources).

Signature · 2 lines

Tom Rini
Mentor Graphics Corporation

Richard Purdie

Applied

2011-03-15 01:25 UTC #10 lore raw

In reply to Khem Raj

Quoted 13 lines · Khem Raj

Hi Richard
Pull URL: git://git.openembedded.org/openembedded-core-contrib.git
Branch: kraj/own-mirror
Browse: http://git.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=kraj/own-mirror
Thanks,
Khem Raj <[email protected]>
---

Quoted 7 lines

Khem Raj (1):
own-mirrors.bbclass: Bring own-mirrors.bbclass from oe commit

    0ef914b250df46a41348479446214575668943fb

 meta/classes/own-mirrors.bbclass |   12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)
 create mode 100644 meta/classes/own-mirrors.bbclass

Merged into OECore master, thanks.

(and the git tree commits really do help me, thanks)

Cheers,

Richard