Yocto git server via http and mirror tarballs

DISCUSSION Pascal Bach 2 participants 4 messages Jun 15, 2016

Share & tools

Pascal Bach

author

2016-06-15 11:03 UTC #0 lore raw

Because of firewall restrictions I'm not always not able to access the git.yoctoproject.org via the git protocol but I'm required to use http.

By default most of the recipes in Poky recipes use the git protocol not the http protocol. For example yocto-kernel-tools points to repositories "git://git.yoctoproject.org/yocto-kernel-tools" instead of "http://git.yoctoproject.org/git/yocto-kernel-tools".

To work around this issue I added a mirror rules for rewriting the protocol like this:

MIRRORS += "\
    git://git.yoctoproject.org/.*                               git://git.yoctoproject.org/git/PATH;protocol=http \n \
    "

This seems to work and I'm able to fetch the sources even when git access is not possible.

But when I then enable BB_GENERATE_MIRROR_TARBALLS = "1" I'm starting to get different tarball names depending on what protocol was used. For example "git2_git.yoctoproject.org.yocto-kernel-tools.git.tar.gz" vs "git2_git.yoctoproject.org.git.yocto-kernel-tools.git.tar.gz".

I assume this happens because the first tarball was create while git access was possible, while the second one was created when only http was possible:
git.yoctoproject.org/git => git.yoctoproject.org.git
git.yoctoproject.org => git.yoctoproject.org

Is there any way to prevent the tarballs from having different names? For example derive the name always from the original URL not the mirror one?

Or is there a better way to achieve what we are doing with MIRROR?

Thanks for your help.

Pascal

Woronicz, Bartosz (Nokia - PL/Wroclaw)

2016-06-15 11:23 UTC #1 lore raw

I set in .gitconfig to always use https:// (don't forget 'git' at the end of path) instead git:// for yoctoproject url

[url "https://git.yoctoproject.org/git/"]
insteadOf = git://git.yoctoproject.org/

Kind regards,
Bartosz Woronicz
Engineer, Software Configuration (SCM)
Nokia Networks - PL/Wroclaw

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Pascal Bach
Sent: Wednesday, June 15, 2016 12:51 PM
To: [email protected]
Subject: [yocto] Yocto git server via http and mirror tarballs

Because of firewall restrictions I'm not always not able to access the git.yoctoproject.org via the git protocol but I'm required to use http.

To work around this issue I added a mirror rules for rewriting the protocol like this:

MIRRORS += "\
    git://git.yoctoproject.org/.*                               git://git.yoctoproject.org/git/PATH;protocol=http \n \
    "

This seems to work and I'm able to fetch the sources even when git access is not possible.

Is there any way to prevent the tarballs from having different names? For example derive the name always from the original URL not the mirror one?

Or is there a better way to achieve what we are doing with MIRROR?

Thanks for your help.

Pascal

Signature · 4 lines

_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto

Pascal Bach

author

2016-06-15 11:33 UTC #2 lore raw

Quoted 5 lines

I set in .gitconfig to always use https:// (don't forget 'git' at the end of path) instead git:// for yoctoproject url
[url "https://git.yoctoproject.org/git/"]
insteadOf = git://git.yoctoproject.org/

I would pefer a solution that can be handled at the distro/site.conf level so that not very developer has to change their .gitconfig. However this is a good work around, thanks.

Quoted 40 lines

Kind regards,
Bartosz Woronicz
Engineer, Software Configuration (SCM)
Nokia Networks - PL/Wroclaw
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Pascal Bach
Sent: Wednesday, June 15, 2016 12:51 PM
To: [email protected]
Subject: [yocto] Yocto git server via http and mirror tarballs
Hi
Because of firewall restrictions I'm not always not able to access the git.yoctoproject.org via the git protocol but I'm required to use http.
By default most of the recipes in Poky recipes use the git protocol not the http protocol. For example yocto-kernel-tools points to repositories "git://git.yoctoproject.org/yocto-kernel-tools" instead of "http://git.yoctoproject.org/git/yocto-kernel-tools".
To work around this issue I added a mirror rules for rewriting the protocol like this:
MIRRORS += "\
    git://git.yoctoproject.org/.*                               git://git.yoctoproject.org/git/PATH;protocol=http \n \
    "
This seems to work and I'm able to fetch the sources even when git access is not possible.
But when I then enable BB_GENERATE_MIRROR_TARBALLS = "1" I'm starting to get different tarball names depending on what protocol was used. For example "git2_git.yoctoproject.org.yocto-kernel-tools.git.tar.gz" vs "git2_git.yoctoproject.org.git.yocto-kernel-tools.git.tar.gz".
I assume this happens because the first tarball was create while git access was possible, while the second one was created when only http was possible:
git.yoctoproject.org/git => git.yoctoproject.org.git
git.yoctoproject.org => git.yoctoproject.org
Is there any way to prevent the tarballs from having different names? For example derive the name always from the original URL not the mirror one?
Or is there a better way to achieve what we are doing with MIRROR?
Thanks for your help.
Pascal

Woronicz, Bartosz ( NSN - PL/Wroclaw)

2016-06-15 15:10 UTC #3 lore raw

Maybe try to prepend PREMIRRORS , then possibly everytime it will use HTTP/HTTPS protocol instead git (because it will be the very first match).

http://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#var-PREMIRRORS

Kind regards,
Bartosz Woronicz
Engineer, Software Configuration (SCM)
Nokia Networks - PL/Wroclaw

Quoted 46 lines · Pascal Bach

Quoted 4 lines
I set in .gitconfig to always use https:// (don't forget 'git' at the end of path) instead git:// for yoctoproject url
[url "https://git.yoctoproject.org/git/"]
insteadOf = git://git.yoctoproject.org/
I would pefer a solution that can be handled at the distro/site.conf level so that not very developer has to change their .gitconfig. However this is a good work around, thanks.
Quoted 39 lines
Kind regards,
Bartosz Woronicz
Engineer, Software Configuration (SCM)
Nokia Networks - PL/Wroclaw
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Pascal Bach
Sent: Wednesday, June 15, 2016 12:51 PM
To: [email protected]
Subject: [yocto] Yocto git server via http and mirror tarballs
Hi
Because of firewall restrictions I'm not always not able to access the git.yoctoproject.org via the git protocol but I'm required to use http.
By default most of the recipes in Poky recipes use the git protocol not the http protocol. For example yocto-kernel-tools points to repositories "git://git.yoctoproject.org/yocto-kernel-tools" instead of "http://git.yoctoproject.org/git/yocto-kernel-tools".
To work around this issue I added a mirror rules for rewriting the protocol like this:
MIRRORS += "\
     git://git.yoctoproject.org/.*                               git://git.yoctoproject.org/git/PATH;protocol=http \n \
     "
This seems to work and I'm able to fetch the sources even when git access is not possible.
But when I then enable BB_GENERATE_MIRROR_TARBALLS = "1" I'm starting to get different tarball names depending on what protocol was used. For example "git2_git.yoctoproject.org.yocto-kernel-tools.git.tar.gz" vs "git2_git.yoctoproject.org.git.yocto-kernel-tools.git.tar.gz".
I assume this happens because the first tarball was create while git access was possible, while the second one was created when only http was possible:
git.yoctoproject.org/git => git.yoctoproject.org.git
git.yoctoproject.org => git.yoctoproject.org
Is there any way to prevent the tarballs from having different names? For example derive the name always from the original URL not the mirror one?
Or is there a better way to achieve what we are doing with MIRROR?
Thanks for your help.
Pascal