BB_NO_NETWORK broken for git
I try to use a local source mirror, fully populated with packages, so I can deliver a complete (no network required) bundle to my customers. This has recently been broken.
I can no longer use BB_NO_NETWORK=1 for git recipes, since this revision:
commit 2354250a95eab484459f41f8715ae112295c2174
Author: Richard Purdie <[email protected]> Date: Mon Nov 18 17:17:16 2013 +0000
bitbake: git: Use merge-base instead of log for testing if a commit is present
The current use of git log to check if a given revision is present can be a little fragile.
For example if revision X was on branch A, and then later added to branch B, the update checks would not notice this since they just check for X being in the repository.
We also had some autobuilder corruption where an older packed-refs file was copied over a new repository containing newer pack files. There was no update to the refs file since the revision was present but not accessible in any branch.
The correct fix is to check that the required revisions are present on the specific branches. This patch does this using merge-base.
(Bitbake rev: 89abfbc1953e3711d6c90aff793ee622c22609b1)
To test this, just add these settings to local.conf
SOURCE_MIRROR_URL ?= "file:///work/misc/Poky/sources/" INHERIT += "own-mirrors"
BB_NO_NETWORK = "1" BB_GENERATE_MIRROR_TARBALLS ?= "1"
Make sure and put the appropriate git tarball into the SOURCE_MIRROR. I tested this using prelink, e.g.
$ bitbake prelink -c fetch the the tarball is git2_git.yoctoproject.org.prelink-cross.git.tar.gz A copy of this is at http://www.mlbassoc.com/poky/git2_git.yoctoproject.org.prelink-cross.git.tar.gz
Filed as bug #5631 (https://bugzilla.yoctoproject.org/show_bug.cgi?id=5631)
Signature · 4 lines
------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------
Quoted 8 lines · Gary Thomas
I try to use a local source mirror, fully populated with packages, so I can deliver a complete (no network required) bundle to my customers. This has recently been broken.
I can no longer use BB_NO_NETWORK=1 for git recipes, since this revision:
commit2354250a95eab484459f41f8715ae112295c2174Author: Richard Purdie <[email protected]> Date: Mon Nov 18 17:17:16 2013 +0000
Its actually the metadata that has the issue. The commit specified is not on the master branch, its on the cross_prelink branch so the fetcher is justifiably getting upset.
When the network is present it does a complete reset to upstream and then assumes the revision is present though which is something we might want to fix too.
For now we can at least correct the metadata to point at the right branch. I've sent out a patch accordingly.
Cheers,
Richard
Quoted 14 lines · Richard Purdie
Quoted 8 lines · Gary Thomas
I try to use a local source mirror, fully populated with packages, so I can deliver a complete (no network required) bundle to my customers. This has recently been broken.
I can no longer use BB_NO_NETWORK=1 for git recipes, since this revision:
commit2354250a95eab484459f41f8715ae112295c2174Author: Richard Purdie <[email protected]> Date: Mon Nov 18 17:17:16 2013 +0000Its actually the metadata that has the issue. The commit specified is not on the master branch, its on the
cross_prelinkbranch so the fetcher is justifiably getting upset.
Quoted 6 lines
When the network is present it does a complete reset to upstream and then assumes the revision is present though which is something we might want to fix too.
For now we can at least correct the metadata to point at the right branch. I've sent out a patch accordingly.
Does this work for other git based repositories? My example was for prelink, but it seems to fail for me for any SRC_URI that uses git. For example, matchbox-keyboard also fails under the same test scenario.
Signature · 4 lines
------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------
Quoted 19 lines · Gary Thomas
Quoted 18 lines · Richard Purdie
Quoted 8 lines · Gary Thomas
I try to use a local source mirror, fully populated with packages, so I can deliver a complete (no network required) bundle to my customers. This has recently been broken.
I can no longer use BB_NO_NETWORK=1 for git recipes, since this revision:
commit2354250a95eab484459f41f8715ae112295c2174Author: Richard Purdie <[email protected]> Date: Mon Nov 18 17:17:16 2013 +0000Its actually the metadata that has the issue. The commit specified is not on the master branch, its on the
cross_prelinkbranch so the fetcher is justifiably getting upset.When the network is present it does a complete reset to upstream and then assumes the revision is present though which is something we might want to fix too.
Quoted 6 lines
Quoted 2 lines
For now we can at least correct the metadata to point at the right branch. I've sent out a patch accordingly.
Does this work for other git based repositories? My example was for prelink, but it seems to fail for me for any
SRC_URIthat uses git. For example, matchbox-keyboard also fails under the same test scenario.
Its the same problem. The commit referenced is not on master but on a different branch. The URL needs fixing to point at the right branch.
Cheers,
Richard
Quoted 23 lines · Richard Purdie
Quoted 22 lines · Gary Thomas
Quoted 20 lines · Richard Purdie
Quoted 8 lines · Gary Thomas
I try to use a local source mirror, fully populated with packages, so I can deliver a complete (no network required) bundle to my customers. This has recently been broken.
I can no longer use BB_NO_NETWORK=1 for git recipes, since this revision:
commit2354250a95eab484459f41f8715ae112295c2174Author: Richard Purdie <[email protected]> Date: Mon Nov 18 17:17:16 2013 +0000Its actually the metadata that has the issue. The commit specified is not on the master branch, its on the
cross_prelinkbranch so the fetcher is justifiably getting upset.When the network is present it does a complete reset to upstream and then assumes the revision is present though which is something we might want to fix too.
For now we can at least correct the metadata to point at the right branch. I've sent out a patch accordingly.
Quoted 6 lines
Quoted 3 lines
Does this work for other git based repositories? My example was for prelink, but it seems to fail for me for any
SRC_URIthat uses git. For example, matchbox-keyboard also fails under the same test scenario.Its the same problem. The commit referenced is not on master but on a different branch. The URL needs fixing to point at the right branch.
So I just [luckily] picked two recipes that have incorrect SRC_URI?? How many others will there be? I'd wager most all since very few of them actually mention a branch, just a revision.
Signature · 4 lines
------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------
Quoted 28 lines · Gary Thomas
Quoted 27 lines · Richard Purdie
Quoted 25 lines · Gary Thomas
Quoted 14 lines · Richard Purdie
Quoted 8 lines · Gary Thomas
I try to use a local source mirror, fully populated with packages, so I can deliver a complete (no network required) bundle to my customers. This has recently been broken.
I can no longer use BB_NO_NETWORK=1 for git recipes, since this revision:
commit2354250a95eab484459f41f8715ae112295c2174Author: Richard Purdie <[email protected]> Date: Mon Nov 18 17:17:16 2013 +0000Its actually the metadata that has the issue. The commit specified is not on the master branch, its on the
cross_prelinkbranch so the fetcher is justifiably getting upset.Quoted 6 lines
When the network is present it does a complete reset to upstream and then assumes the revision is present though which is something we might want to fix too.
For now we can at least correct the metadata to point at the right branch. I've sent out a patch accordingly.
Does this work for other git based repositories? My example was for prelink, but it seems to fail for me for any
SRC_URIthat uses git. For example, matchbox-keyboard also fails under the same test scenario.
Quoted 6 lines
Quoted 2 lines
Its the same problem. The commit referenced is not on master but on a different branch. The URL needs fixing to point at the right branch.
So I just [luckily] picked two recipes that have incorrect
SRC_URI?? How many others will there be? I'd wager most all since very few of them actually mention a branch, just a revision.
Most git trees use the master branch which is the default so most urls are fine.
I just added something to the fetcher to detect this (which I will merge), then ran a world fetchall against OE-Core master.
We appear to have issues with lttng-{modules,ust,tools} and mx so its far from "all" git recipes.
Cheers,
Richard