support/download/git: add branch support
A message that is not in the archive (sent off-list or not yet received): [email protected]
PATCH 1/1 support/download/git: add branch support
CAtherine, All,
On 2019-05-08 14:22 +0000, catherine at kubos.co spake thusly:
Quoted 7 lines
From: Catherine Garabedian <[email protected]>
A prior patch added logic to verify that the given cset exists.
The command format used fails when attempting to verify a branch name, thus preventing Buildroot from downloading any packages whose source comes from a branch.
We explciitly do not want to support using a branch _by name_, because that does not work as people expect it to work, see our manual:
https://buildroot.org/downloads/manual/manual.html#generic-package-reference
Excerpt:
1. due to local caching, Buildroot will not re-fetch the repository, so people who expect to be able to follow the remote repository would be quite surprised and disappointed;
2. because two builds can never be perfectly simultaneous, and because the remote repository may get new commits on the branch anytime, two users, using the same Buildroot tree and building the same configuration, may get different source, thus rendering the build non reproducible, and people would be quite surprised and disappointed.
So, using a branch _by name_ can not owrk because of those two reasons, and as such we believe this should not be supported.
Instead, there was some effort a few months ago, to actually enforce the check that a branch was not used, and fail if that was the case.
Regards,
Yann E. MORIN.
Quoted 27 lines
This patch adds a secondary check which can validate branches (but not tags).
Signed-off-by Catherine Garabedian <[email protected]>--- support/download/git | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)support/download/git +4
-2@@ -130,8 +130,10 @@ fi # scratch won't help, so we don't want to trash the repository for a # missing commit. We just exit without going through the ERR trap. if ! _git rev-parse --quiet --verify "'${cset}^{commit}'" >/dev/null 2>&1; then - printf "Commit '%s' does not exist in this repository\n." "${cset}" - exit 1 + if ! _git rev-parse --quiet --verify "'origin/${cset}'" >/dev/null 2>&1; then + printf "'%s' does not exist in this repository\n." "${cset}" + exit 1 + fi fi # The new cset we want to checkout might have different submodules, orSignature · 1 line
2.7.4
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
A message that is not in the archive (sent off-list or not yet received): CANj1ofkg42Ly-j2v2Gh8xt-2URP+xzL3W3wMW2-qVN+0PYvDbw@mail.gmail.com
PATCH 1/1 support/download/git: add branch support
Catherine, All,
[please keep the buildroot ML in Cc on replies; if you're not subscribed, please do so, otherwise people can't follow the discussion and wonder what's going on...]
On 2019-05-08 10:47 -0500, Catherine Garabedian spake thusly:
Quoted 2 lines
Understood. I'm upgrading from an older version of Buildroot, so missed that change. I'll have to update our configurations to adjust to the new behavior.
Note that there was no change; that has always been the case. It's just that it's now documented.
Quoted 4 lines
I would like to update this code to add an explicit error message instead ("Change sets may not be referred to by branch name"). Should that be submitted as a revision to this patch, or as an entirely separate patch?
I already posted a series to that effect in the past, and it is not trivial to do.
The problem is that for some time, we allowed (and still allow) for so-called special refs, like github PR, or gerrit reviews, and so on. Due to a bug in the git download backend, when the cset to use is a sha1, this does create local branches named after that sha1.
So, if we add a check that a cset is a branch name or not, then even if the cset is a sha1, it can match a local branch (that was created by an older version of Buildroot).
As such, we first have to clean up the git cache.
I have two WIP branches to address this issue [0] and [1], but I haven't come to terms with previous reviews (which I can't seem to find again in the ML archives, and I don;t even remember why the '3' is shorter than the '2', which looks more complete, sigh...) You can get a better picture of the issue by reading the commit logs in those series.
[0] https://git.buildroot.org/~ymorin/git/buildroot/log/?h=yem/dl-git-no-branch-2
[1] https://git.buildroot.org/~ymorin/git/buildroot/log/?h=yem/dl-git-no-branch-3
I shall revive those and get them to their conclusion. Soon, I hope, soon (famous last words)...
Regards,
Yann E. MORIN.
Quoted 85 lines
Thanks,
Catherine Garabedian
On Wed, May 8, 2019 at 9:43 AM Yann E. MORIN < [1][email protected]> wrote:CAtherine, All,
On 2019-05-08 14:22 +0000, [2]catherine at kubos.co spake thusly:
Quoted 7 lines
From: Catherine Garabedian < [3][email protected]>
A prior patch added logic to verify that the given cset exists.
The command format used fails when attempting to verify a branch name, thus preventing Buildroot from downloading any packages whose source comes from a branch.
We explciitly do not want to support using a branch
_byname_, because that does not work as people expect it to work, see our manual:? ? [4]https://buildroot.org/downloads/manual/manual.html#generic-package-reference
Excerpt:
?1. due to local caching, Buildroot will not re-fetch the repository, ? ? so people who expect to be able to follow the remote repository ? ? would be quite surprised and disappointed;
?2. because two builds can never be perfectly simultaneous, and because ? ? the remote repository may get new commits on the branch anytime, two ? ? users, using the same Buildroot tree and building the same ? ? configuration, may get different source, thus rendering the build ? ? non reproducible, and people would be quite surprised and ? ? disappointed.So, using a branch
_byname_can not owrk because of those two reasons, and as such we believe this should not be supported.Instead, there was some effort a few months ago, to actually enforce the check that a branch was not used, and fail if that was the case.
Regards,
Yann E. MORIN.Quoted 18 lines
This patch adds a secondary check which can validate branches (but not tags).
Signed-off-by Catherine Garabedian < [5][email protected]>---
? support/download/git | 6 ++++--
? 1 file changed, 4 insertions(+), 2 deletions(-)support/download/git +0
-0@@ -130,8 +130,10 @@ fi? # scratch won't help, so we don't want to trash the repository for a ? # missing commit. We just exit without going through the ERR trap. ? if !
_gitrev-parse --quiet --verify "'${cset}^{commit}'" >/dev/null 2>&1; then -? ? printf "Commit '%s' does not exist in this repository\n." "${cset}" -? ? exit 1
+? ? if !_gitrev-parse --quiet --verify "'origin/${cset}'" >/dev/null 2>&1; thenQuoted 9 lines
+? ? ? ? printf "'%s' does not exist in this repository\n." "${cset}" +? ? ? ? exit 1
+? ? fi
? fi
?
? # The new cset we want to checkout might have different submodules, or --
2.7.4--
.-----------------.--------------------.------------------.--------------------.|? Yann E. MORIN? | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software? Designer | \ / CAMPAIGN? ? ?|? ___? ? ? ? ? ? ? ?| | +33 561 099 427 `------------.-------:? X? AGAINST? ? ? |? \e/? There is no? | | [6]http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL? ? |? ?v? ?conspiracy.? |'------------------------------^-------^------------------^--------------------'
Links:
1. mailto:yann.morin.1998 at free.fr
2. mailto:catherine at kubos.co
3. mailto:catherine at kubos.co
4. https://buildroot.org/downloads/manual/manual.html#generic-package-reference
5. mailto:catherine at kubos.co
6. http://ymorin.is-a-geek.org/
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/1 support/download/git: add branch support
Quoted 1 line
Quoted 1 line
Understood. I'm upgrading from an older version of Buildroot, so
missed that change.
Quoted 4 lines
Quoted 1 line
I'll have to update our configurations to adjust to the new behavior.
Note that there was no change; that has always been the case. It's just that it's now documented.
It did previously work, though with the unexpected behavior which is documented
(I had added a custom command to my personal packages which would clean the cached/downloaded repo as well as the build directory). Referencing by branch name has
been a part of my configuration/workflow for the last two years (admittedly I'm currently
running a now-outdated LTS release).
Quoted 2 lines
I shall revive those and get them to their conclusion. Soon, I hope, soon (famous last words)...
Fantastic! Best of luck getting those settled :)
Thank you very much for all your feedback,
Catherine Garabedian
Quoted 166 lines · Yann E. MORIN
Catherine, All,
[please keep the buildroot ML in Cc on replies; if you're not subscribed, please do so, otherwise people can't follow the discussion and wonder what's going on...]
On 2019-05-08 10:47 -0500, Catherine Garabedian spake thusly:
Quoted 1 line
Understood. I'm upgrading from an older version of Buildroot, so missed
that change.
Quoted 1 line
I'll have to update our configurations to adjust to the new behavior.
Note that there was no change; that has always been the case. It's just that it's now documented.
Quoted 4 lines
I would like to update this code to add an explicit error message instead ("Change sets may not be referred to by branch name"). Should that be submitted as a revision to this patch, or as an entirely separate patch?
I already posted a series to that effect in the past, and it is not trivial to do.
The problem is that for some time, we allowed (and still allow) for so-called special refs, like github PR, or gerrit reviews, and so on. Due to a bug in the git download backend, when the cset to use is a sha1, this does create local branches named after that sha1.
So, if we add a check that a cset is a branch name or not, then even if the cset is a sha1, it can match a local branch (that was created by an older version of Buildroot).
As such, we first have to clean up the git cache.
I have two WIP branches to address this issue [0] and [1], but I haven't come to terms with previous reviews (which I can't seem to find again in the ML archives, and I don;t even remember why the '3' is shorter than the '2', which looks more complete, sigh...) You can get a better picture of the issue by reading the commit logs in those series.
[0]
https://git.buildroot.org/~ymorin/git/buildroot/log/?h=yem/dl-git-no-branch-2
[1]
https://git.buildroot.org/~ymorin/git/buildroot/log/?h=yem/dl-git-no-branch-3I shall revive those and get them to their conclusion. Soon, I hope, soon (famous last words)...
Regards,
Yann E. MORIN.Quoted 3 lines
Thanks,
Catherine Garabedian
On Wed, May 8, 2019 at 9:43 AM Yann E. MORIN < [1]Quoted 9 lines · yann.morin.1998 at free.fr> wrote:
CAtherine, All,
On 2019-05-08 14:22 +0000, [2]catherine at kubos.co spake thusly:
Quoted 5 lines
From: Catherine Garabedian < [3][email protected]>
A prior patch added logic to verify that the given cset exists.
The command format used fails when attempting to verify a branch
name, thus
Quoted 1 line
Quoted 1 line
preventing Buildroot from downloading any packages whose source
comes from a
Quoted 6 lines
Quoted 1 line
branch.
We explciitly do not want to support using a branch
_byname_, because that does not work as people expect it to work, see our manual:[4]https://buildroot.org/downloads/manual/manual.html#generic-package-reference
Quoted 8 lines
Excerpt:
1. due to local caching, Buildroot will not re-fetch the repository, so people who expect to be able to follow the remote repository would be quite surprised and disappointed;
2. because two builds can never be perfectly simultaneous, and because the remote repository may get new commits on the branch anytime,two
Quoted 9 lines
users, using the same Buildroot tree and building the same configuration, may get different source, thus rendering the build non reproducible, and people would be quite surprised and disappointed.
So, using a branch
_byname_can not owrk because of those two reasons, and as such we believe this should not be supported.Instead, there was some effort a few months ago, to actually enforce
the
Quoted 6 lines
check that a branch was not used, and fail if that was the case.
Regards,
Yann E. MORIN.Quoted 1 line
This patch adds a secondary check which can validate branches (but
not tags).
Quoted 6 lines
Quoted 6 lines
Signed-off-by Catherine Garabedian < [5][email protected]>--- support/download/git | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)Quoted 10 lines
support/download/git fi · line 132
Quoted 8 lines
support/download/git +0
-0@@ -130,8 +130,10 @@ fi # scratch won't help, so we don't want to trash the repository for a # missing commit. We just exit without going through the ERR trap. if ! _git rev-parse --quiet --verify "'${cset}^{commit}'"
/dev/null2>&1; thenQuoted 1 line
support/download/git +0
-1- printf "Commit '%s' does not exist in this repository\n.""${cset}"
Quoted 9 lines
support/download/git fi · line 133
Quoted 2 lines
support/download/git +1
-1- exit 1 + if ! _git rev-parse --quiet --verify "'origin/${cset}'"
/dev/null2>&1; thenQuoted 6 lines
support/download/git +3
-0+ printf "'%s' does not exist in this repository\n." "${cset}" + exit 1 + fi fi # The new cset we want to checkout might have different submodules,or
Quoted 6 lines
Quoted 3 lines
--
2.7.4--
.-----------------.--------------------.------------------.--------------------.
Quoted 1 line
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics'
conspiracy: |
Quoted 4 lines
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___
|| +33 561 099 427 `------------.-------: X AGAINST | \e/
There is no |
Quoted 1 line
| [6]http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v
conspiracy. |
Quoted 1 line
'------------------------------^-------^------------------^--------------------'
Quoted 6 lines
Links:
1. mailto:yann.morin.1998 at free.fr
2. mailto:catherine at kubos.co
3. mailto:catherine at kubos.co
4.https://buildroot.org/downloads/manual/manual.html#generic-package-reference
Quoted 2 lines
5. mailto:catherine at kubos.co
6. http://ymorin.is-a-geek.org/--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
Quoted 2 lines
'------------------------------^-------^------------------^--------------------'
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190508/a659a88b/attachment.html>