Skip to contentSkip to conversation

Add bad-branch-first option for git-bisect

RFC New Shuang He 5 participants 14 messages Jun 15, 2016 Documentation

lore
Share & tools
lore

Shuang He

author
#0 lore raw

Hi
The default git-bisect algorithm will jump around the commit tree,
on the purpose of taking least steps to find the first culprit commit. We may find it sometime would locate a old culprit commit that we're not concerned about anymore.
In most software development, there's one or two main branch which
is maintained for release, and a bunch of feature branches are created for new feature development or bug fix. For the reason that sometime git-bisect will locate a old culprit commit would be:
1. Quality of those branches may not match the main branch,
some functionality are broken at first and fixed later on the feature branch. If git-bisect jump to there by chance, git-bisect will only find that old culprit commit which only exists on that feature branch
2. Some of those branches may not synchronized with main
branch in time. Say feature1 is broken when feature2 branch is created, and feature1 is fixed just a moment later after feature2 branch is created, and when feature2's development is done, and developer want to merge feature2 branch back to master branch, feature2 will be firstly synchronized to master branch tip, then merge into master. For the same reason addressed in issue 1, this will also lead git-bisect into wrong direction.

In all, we think we do not care about branches that we're not
currently working, unless we're sure the regression is caused by that branch.

To address those issue, we propose to add a new config option:
core.bisectbadbranchfirst::
With this algorithm, git-bisect will always try to select
commits
that on the same branch current bad commit sits. And will
fall back
to default git-bisect algorithm when bad-branch-first
algorithm does
not apply
+
This setting defaults to "false".

The draft patch will be sent out in a later email, so it could be
reviewed inline.
Any question or suggestion is welcome :-)

Thanks
--Shuang

author
#1 lore raw

PATCH 1/1 add config option core.bisectbadbranchfirst

which enable recursive bad-branch-first algorithm for git-bisect. With this algorithm, git-bisect will always try to select commits that on the same branch current bad commit sits. And will fall back to default git-bisect algorithm when bad-branch-first algorithm does not apply

---
 Documentation/config.txt |    8 ++
 bisect.c                 |  244 +++++++++++++++++++++++++++++++++++++++++-----
 bisect.h                 |    2 +-
 builtin/rev-list.c       |    2 +-
 cache.h                  |    1 +
 config.c                 |    6 +
 environment.c            |    1 +
 7 files changed, 238 insertions(+), 26 deletions(-)
Documentation/config.txt +8 -0
index ff7c225..8502859 100644
@@ -558,6 +558,14 @@ core.sparseCheckout::
 	Enable "sparse checkout" feature. See section "Sparse checkout" in
 	linkgit:git-read-tree[1] for more information.
 
+core.bisectbadbranchfirst::
+	With this algorithm, git-bisect will always try to select commits
+	that on the same branch current bad commit sits. And will fall back
+	to default git-bisect algorithm when bad-branch-first algorithm does
+	not apply
++
+This setting defaults to "false".
+
 add.ignore-errors::
 add.ignoreErrors::
 	Tells 'git add' to continue adding files when some files cannot be
bisect.c +220 -24

320 lines not on this page · raw message

bisect.h +1 -1
index 0862ce5..fad4fbe 100644
@@ -1,7 +1,7 @@
 #ifndef BISECT_H
 #define BISECT_H
 
-extern struct commit_list *find_bisection(struct commit_list *list,
+extern struct commit_list *find_bisection(struct commit_list *list, struct commit_list *list2,
 					  int *reaches, int *all,
 					  int find_all);
 
builtin/rev-list.c +1 -1
index ba27d39..ab56f6b 100644
@@ -399,7 +399,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
 	if (bisect_list) {
 		int reaches = reaches, all = all;
 
-		revs.commits = find_bisection(revs.commits, &reaches, &all,
+		revs.commits = find_bisection(revs.commits, NULL, &reaches, &all,
 					      bisect_find_all);
 
 		if (bisect_show_vars)
cache.h +1 -0
index d83d68c..bfd4448 100644
@@ -559,6 +559,7 @@ extern int read_replace_refs;
 extern int fsync_object_files;
 extern int core_preload_index;
 extern int core_apply_sparse_checkout;
+extern int core_bisect_bad_branch_first;
 
 enum safe_crlf {
 	SAFE_CRLF_FALSE = 0,
config.c +6 -0
index 625e051..b37a70c 100644
@@ -660,6 +660,12 @@ static int git_default_core_config(const char *var, const char *value)
 		return 0;
 	}
 
+	if (!strcmp(var, "core.bisectbadbranchfirst")) {
+		core_bisect_bad_branch_first = git_config_bool(var, value);
+		return 0;
+	}
+
+
 	/* Add other config variables here and to Documentation/config.txt. */
 	return 0;
 }
environment.c +1 -0
index 9564475..cf813af 100644
@@ -55,6 +55,7 @@ enum object_creation_mode object_creation_mode = OBJECT_CREATION_MODE;
 char *notes_ref_name;
 int grafts_replace_parents = 1;
 int core_apply_sparse_checkout;
+int core_bisect_bad_branch_first = 0;
 struct startup_info *startup_info;
 
 /* Parallel index stat data preload? */
Signature · 1 line
1.7.4.rc2.21.g7f7a8
#2 lore raw
In reply to Shuang He

Hi,

Quoted 5 lines · Shuang He

Hi
    The default git-bisect algorithm will jump around the commit tree, on the purpose of taking least steps to find the first culprit commit. We may find it sometime would locate a old culprit commit that we're not concerned about anymore.

Yes, it can be a problem.

Quoted 9 lines

    In most software development, there's one or two main branch which is maintained for release, and a bunch of feature branches are created for new feature development or bug fix.  For the reason that sometime git-bisect will locate a old culprit commit would be:
        1. Quality of those branches may not match the main branch, some functionality are broken at first and fixed later on the feature branch. If git-bisect jump to there by chance, git-bisect will only find that old
culprit commit which only exists on that feature branch

If the quality of these branches is too bad, I think they should not have been merged in the first place.
If they are not merged (and not marked as good), then git bisect will not look at them, since it will look only at commits that are ancestors of the bad commit it is given.

Or if one is merged but it causes too many problems, then perhaps a replacement commit could be used to unmerge the branch.

Another possibility is to have in a file a list of commits that are the last commits on these branches before the merge commits, and do a:

git bisect good $(cat good_commits_file.txt)

at the beginning of each bisection.

So I think the long term solution in this case is not what your are suggesting.

Quoted 8 lines

        2. Some of those branches may not synchronized with main branch in time.  Say feature1 is broken when feature2 branch is created, and feature1 is fixed just a moment later after feature2 branch is created, and when feature2's development is done, and developer want to merge feature2 branch back to master branch, feature2 will be firstly synchronized to master branch tip, then merge into master.  For the same reason addressed in issue 1, this will also lead git-bisect into wrong direction.

I am not sure what you mean by " feature2 will be firstly synchronized to master branch tip", and I think this should mean a rebase that would fix the bug if feature1 has already been merged into the master branch.

But anyway in this case, I think that git bisect will find that the first bad commit is the last commit in the branch, just before it was merged. And by looking at the branch graph it should be quite easy to understand what happened.

And then the obvious thing to do is to decide to just start a new bisection like it was started the first time but with an added "git bisect good <merge_commit>", where <merge_commit> is the commit that merges the branch.

Quoted 4 lines

    In all, we think we do not care about branches that we're not currently working, unless we're sure the regression is caused by that branch.

Quoted 11 lines

    To address those issue, we propose to add a new config option:
        core.bisectbadbranchfirst::
            With this algorithm, git-bisect will always try to select commits
            that on the same branch current bad commit sits. And will fall back
            to default git-bisect algorithm when bad-branch-first algorithm does
            not apply
        +
        This setting defaults to "false".

I am not opposed to an option to bisect on the first parents of the bad commit only. And after a very fast look at your patch it seems to be what it does. By the way Avery Pennarun's gitbuilder
(https://github.com/apenwarr/gitbuilder) does the same thing. So I know some people are interested in such a feature.

But here are some suggestions/comments:

- your explanations about why it could be useful should be improved,
- the name "bisectbadbranchfirst" seems wrong to me, because git branches are just some special tags; "firstparentsonly" would be a better name,
- before having a config option for it, why not have it first as an option to "git bisect start",
- if there is a config option, then there should probably be an option to "git bisect start" to use the regular algorithm,
- it seems to me that bisecting only on first parents could fail only if some "good" commits are not ancestors of the bad commit, and if the bug is fixed on a branch where such a commit is; so I don't see the point in defaulting to the usual algorithm in this case because in this case the usual algorithm just stops,
- perhaps the message given when the first bad commit is found should be changed a little bit when this option is used.

Maybe I am missing something, because as I said I didn't read your patch carefully, but in this case please try to give a concrete example.

Thanks,
Christian.

author
#3 lore raw
Quoted 9 lines · Christian Couder

Hi,

Quoted 5 lines · Shuang He

Hi
The default git-bisect algorithm will jump around the commit tree,
on the purpose of taking least steps to find the first culprit commit. We may find it sometime would locate a old culprit commit that we're not concerned about anymore.

Yes, it can be a problem.

I'm honored to be given so much comment :)
Thank you

Quoted 18 lines
Quoted 9 lines

In most software development, there's one or two main branch which
is maintained for release, and a bunch of feature branches are created for new feature development or bug fix. For the reason that sometime git-bisect will locate a old culprit commit would be:
1. Quality of those branches may not match the main branch,
some functionality are broken at first and fixed later on the feature branch. If git-bisect jump to there by chance, git-bisect will only find that old
culprit commit which only exists on that feature branch

If the quality of these branches is too bad, I think they should not have been merged in the first place.
If they are not merged (and not marked as good), then git bisect will not look at them, since it will look only at commits that are ancestors of the bad commit it is given.

Or if one is merged but it causes too many problems, then perhaps a replacement commit could be used to unmerge the branch.

Quoted 8 lines

Another possibility is to have in a file a list of commits that are the last commits on these branches before the merge commits, and do a:

git bisect good $(cat good_commits_file.txt)

at the beginning of each bisection.

So I think the long term solution in this case is not what your are suggesting.

Yeah, I agree that the issue I addressed above will not be a problem if all those branches are maintained very well. Actually we've implemented a automated bisect system for Intel Linux Graphics Driver Project, and so we'd like the system helps us to locate issue in an more automatic way when branches are not maintained as good as expected.

Quoted 18 lines
Quoted 8 lines

2. Some of those branches may not synchronized with main
branch in time. Say feature1 is broken when feature2 branch is created, and feature1 is fixed just a moment later after feature2 branch is created, and when feature2's development is done, and developer want to merge feature2 branch back to master branch, feature2 will be firstly synchronized to master branch tip, then merge into master. For the same reason addressed in issue 1, this will also lead git-bisect into wrong direction.

I am not sure what you mean by " feature2 will be firstly synchronized to master branch tip", and I think this should mean a rebase that would fix the bug if feature1 has already been merged into the master branch.

But anyway in this case, I think that git bisect will find that the first bad commit is the last commit in the branch, just before it was merged. And by looking at the branch graph it should be quite easy to understand what happened.

Quoted 4 lines

And then the obvious thing to do is to decide to just start a new bisection like it was started the first time but with an added "git bisect good<merge_commit>", where<merge_commit> is the commit that merges the branch.

For the same reason, that we're implementing automated bisect system , so we want git-bisect to be able to help with this condition also.

Quoted 21 lines
Quoted 4 lines

In all, we think we do not care about branches that we're not
currently working, unless we're sure the regression is caused by that branch.

Quoted 11 lines

To address those issue, we propose to add a new config option:
core.bisectbadbranchfirst::
With this algorithm, git-bisect will always try to select
commits
that on the same branch current bad commit sits. And will
fall back
to default git-bisect algorithm when bad-branch-first
algorithm does
not apply
+
This setting defaults to "false".

I am not opposed to an option to bisect on the first parents of the bad commit only. And after a very fast look at your patch it seems to be what it does. By the way Avery Pennarun's gitbuilder
(https://github.com/apenwarr/gitbuilder) does the same thing. So I know some people are interested in such a feature.

Quoted 3 lines

But here are some suggestions/comments:

- your explanations about why it could be useful should be improved,

I don't have much data to prove it. I could just say it could help if branches are not maintained very well.

Quoted 3 lines

- the name "bisectbadbranchfirst" seems wrong to me, because git branches are just some special tags; "firstparentsonly" would be a better name,

It's recursively applying bad branch first algorithm, not just constantly stick to first parent. Given this condition:

    A -> B -> C -> D -> E -> F -> G -> H   (master)
         \ a  -> b -> c -> d -> e /  (feature 1)
              \ x -> y -> z/      (feature 2)

start with H as bad commit, and A as good commit, if y is the target bad commit. bad-branch-first algorithm will do it like this:
1. In first round stick to master branch, so it will locate G as
first bad commit
2. In second round stick to feature1 branch, then it will locate d
as first bad commit
3. In third round stick to feature2 branch, then it will finally
locate y as first bad commit
So you could see, it's always sticking to branch where current bad commit sit

Quoted 2 lines

- before having a config option for it, why not have it first as an option to "git bisect start",

Agree, I have thought about it. Haven't done it yet

Quoted 2 lines

- if there is a config option, then there should probably be an option to "git bisect start" to use the regular algorithm,

Agree

Quoted 5 lines

- it seems to me that bisecting only on first parents could fail only if some "good" commits are not ancestors of the bad commit, and if the bug is fixed on a branch where such a commit is; so I don't see the point in defaulting to the usual algorithm in this case because in this case the usual algorithm just stops,

It should stop as default git-bisect do, when
There are a few cases that this algorithm will fall back to default algorithm:
when bisect path is specified. Since --first-parent seems not
working as expected when file path is specified
when bad branch first algorithm could not find a commit to bisect

Quoted 2 lines

- perhaps the message given when the first bad commit is found should be changed a little bit when this option is used.

Yeah, agree

Quoted 6 lines

Maybe I am missing something, because as I said I didn't read your patch carefully, but in this case please try to give a concrete example.

Thanks,
Christian.

#4 lore raw

Am 1/24/2011 11:30, schrieb Shuang He:

Quoted 6 lines

It's recursively applying bad branch first algorithm, not just constantly stick to first parent.
Given this condition:

    A -> B -> C -> D -> E -> F -> G -> H   (master)
         \ a  -> b -> c -> d -> e /  (feature 1)
              \ x -> y -> z/      (feature 2)
Quoted 9 lines

start with H as bad commit, and A as good commit, if y is the target bad commit. bad-branch-first algorithm will do it like this:
1. In first round stick to master branch, so it will locate G as first
bad commit
2. In second round stick to feature1 branch, then it will locate d as
first bad commit
3. In third round stick to feature2 branch, then it will finally
locate y as first bad commit
So you could see, it's always sticking to branch where current bad commit sit

Ok, so you explain what your algorithm does.

But you did not illustrate your problem. The history above is ordinary, somewhat branchy, has *ONE* commit that introduces a regression, and *NO* commit that fixes the regression. But in your rationale you said something about "feature1 is fixed just a moment later after feature2 branch is created". How does this fit into the picture, where is the problem, and how does your algorithm solve it?

-- Hannes

author
#5 lore raw
Quoted 18 lines · Johannes Sixt

Am 1/24/2011 11:30, schrieb Shuang He:

Quoted 6 lines

It's recursively applying bad branch first algorithm, not just constantly stick to first parent.
Given this condition:

     A ->  B ->  C ->  D ->  E ->  F ->  G ->  H   (master)
          \ a  ->  b ->  c ->  d ->  e /  (feature 1)
               \ x ->  y ->  z/      (feature 2)
Quoted 9 lines

start with H as bad commit, and A as good commit, if y is the target bad commit. bad-branch-first algorithm will do it like this:
1. In first round stick to master branch, so it will locate G as first
bad commit
2. In second round stick to feature1 branch, then it will locate d as
first bad commit
3. In third round stick to feature2 branch, then it will finally
locate y as first bad commit
So you could see, it's always sticking to branch where current bad commit sit

Ok, so you explain what your algorithm does.

Quoted 8 lines

But you did not illustrate your problem. The history above is ordinary, somewhat branchy, has *ONE* commit that introduces a regression, and *NO* commit that fixes the regression. But in your rationale you said something about "feature1 is fixed just a moment later after feature2 branch is created". How does this fit into the picture, where is the problem, and how does your algorithm solve it?

-- Hannes

If A is bad commit, and C fixed it, and then F is bad again,

A -> B -> C -> D -> E -> F -> G -> H (master)
\ \ /

    a  ->  b... c ->  d ->  e->f  (feature 1)

Start with H as bad commit, and D as good commit, it's possible git-bisect would jump to c, and it will lead to wrong direction

If bad-branch-first is used, it would be:
1. first round found F
2. end

Thanks
--Shuang

Thanks
--Shuang

#6 lore raw
Quoted 11 lines · Shuang He

If A is bad commit, and C fixed it, and then F is bad again,

A -> B -> C -> D -> E -> F -> G -> H (master)
\ \ /

    a  ->  b... c ->  d ->  e->f  (feature 1)

Start with H as bad commit, and D as good commit, it's possible git-bisect would jump to c, and it will lead to wrong direction

If bad-branch-first is used, it would be:
1. first round found F
2. end

It is unclear from the way you drew the picture if "F" is supposed to be a merge of "E" and "f", but I'd assume that it is.

So what you are saying in 1. is "skip from H until you hit a first merge (without testing any intermediate commit), find F and stop to check it, and find that it is broken".

What makes you decide "2. end"? The fact that both of its parents "E" and "f" are Ok? IOW, it won't be "2. end" if one of the parents of the merge is broken?

What if there is _no_ merge from a side branch but there were breakages in A (fixed in C) and then F in your original picture, i.e.

A---B---C---D---E---F---G---H (broken)
x o x

and you are hunting for the bug starting from H? How does your algorithm help? I grossed over the linear part by saying "skip from H until you hit a first merge", but in general, what is your plan to handle linear part of the history?

A totally unacceptable answer is "It does not help linear case, but it helps when there are merges". The a-thru-f side branch in your picture, or any "culprit side branch that was merged" your algorithm finds in general, would eventually have a linear segment, and having x-o-x in the history fundmentally breaks "bisect"---your band-aid will not help.

The whole idea behind using "bisect" to gain efficiency in isolating the issue depends on "Once you see a Good commit, you do not have to search beyond its ancestors", as it is to look for a single breakage that persists to the "Bad" commit you give, and as far as "bisect" is concerned, the breakage at A in your example is an unrelated breakage that did not persist through the history to the "Bad" commit H.

author
#7 lore raw
Quoted 8 lines · Junio C Hamano
Quoted 6 lines · Shuang He

If A is bad commit, and C fixed it, and then F is bad again,

A -> B -> C -> D -> E -> F -> G -> H (master)
\ \ /

     a  ->   b... c ->   d ->   e->f  (feature 1)
Quoted 7 lines
Quoted 5 lines

Start with H as bad commit, and D as good commit, it's possible git-bisect would jump to c, and it will lead to wrong direction

If bad-branch-first is used, it would be:
1. first round found F
2. end

It is unclear from the way you drew the picture if "F" is supposed to be a merge of "E" and "f", but I'd assume that it is.

Oh, I lost this mail
That graph is different from what I meant, when shown in different email client. It's G which is merged from e and F

Quoted 7 lines

So what you are saying in 1. is "skip from H until you hit a first merge (without testing any intermediate commit), find F and stop to check it, and find that it is broken".

What makes you decide "2. end"? The fact that both of its parents "E" and "f" are Ok? IOW, it won't be "2. end" if one of the parents of the merge is broken?

I think the correction above should have answer those two questions.

Quoted 10 lines

What if there is _no_ merge from a side branch but there were breakages in A (fixed in C) and then F in your original picture, i.e.

A---B---C---D---E---F---G---H (broken)
x o x

and you are hunting for the bug starting from H? How does your algorithm help? I grossed over the linear part by saying "skip from H until you hit a first merge", but in general, what is your plan to handle linear part of the history?

If the history is linear, the new algorithm won't help, it will just behavior like default git-bisect algorithm.

Quoted 6 lines

A totally unacceptable answer is "It does not help linear case, but it helps when there are merges". The a-thru-f side branch in your picture, or any "culprit side branch that was merged" your algorithm finds in general, would eventually have a linear segment, and having x-o-x in the history fundmentally breaks "bisect"---your band-aid will not help.

Quoted 6 lines

The whole idea behind using "bisect" to gain efficiency in isolating the issue depends on "Once you see a Good commit, you do not have to search beyond its ancestors", as it is to look for a single breakage that persists to the "Bad" commit you give, and as far as "bisect" is concerned, the breakage at A in your example is an unrelated breakage that did not persist through the history to the "Bad" commit H.

In the example above (after we know G is merged from e and F),
Those commits are old bad commit: A, B, a, b, ..., c, d (but we don't care about those old bad commits, we cared about latest bad commit that we met which is F) It's possible that default git-bisect would jump to these old bad commits, and will finally find an old first bad commit
With bad-branch-first, it could help us to get away from the trouble that old culprit commit exist on feature1 branch for a period of time not fixed

Thanks
--Shuang

#8 lore raw
In reply to Shuang He
Quoted 12 lines · Shuang He
Quoted 11 lines · Christian Couder

Hi,

Quoted 6 lines · Shuang He

Hi
    The default git-bisect algorithm will jump around the commit tree, on the purpose of taking least steps to find the first culprit commit. We may find it sometime would locate a old culprit commit that we're not concerned about anymore.

Quoted 4 lines
Quoted 1 line

Yes, it can be a problem.

I'm honored to be given so much comment :)
Thank you

I am honored by your interest in git bisect and the fact that you provided a patch :-)
Thanks!

Quoted 19 lines
Quoted 12 lines

If the quality of these branches is too bad, I think they should not have been merged in the first place.
If they are not merged (and not marked as good), then git bisect will not look at them, since it will look only at commits that are ancestors of the bad commit it is given.

Or if one is merged but it causes too many problems, then perhaps a replacement commit could be used to unmerge the branch.

Another possibility is to have in a file a list of commits that are the last commits on these branches before the merge commits, and do a:

Quoted 6 lines

git bisect good $(cat good_commits_file.txt)

at the beginning of each bisection.

So I think the long term solution in this case is not what your are suggesting.

Quoted 6 lines

Yeah, I agree that the issue I addressed above will not be a problem if all those branches are maintained very well.
Actually we've implemented a automated bisect system for Intel Linux Graphics Driver Project, and so we'd like the system
helps us to locate issue in an more automatic way when branches are not maintained as good as expected.

I think there is always a price to pay when you bisect if the branches are not well maintained.
Maybe your algorithm could help in some cases, but my opinion is that there will probably still be many problems and a human will often have to take a look.

Quoted 15 lines
Quoted 15 lines
Quoted 9 lines

        2. Some of those branches may not synchronized with main branch in time.  Say feature1 is broken when feature2 branch is created, and
feature1 is fixed just a moment later after feature2 branch is created, and when feature2's development is done, and developer want to merge feature2 branch back to master branch, feature2 will be firstly synchronized to master branch tip, then merge into master.  For the same reason addressed in issue 1, this will also lead git-bisect into wrong direction.

I am not sure what you mean by " feature2 will be firstly synchronized to master branch tip", and I think this should mean a rebase that would fix the bug if feature1 has already been merged into the master branch.

Quoted 4 lines
Quoted 4 lines

But anyway in this case, I think that git bisect will find that the first bad commit is the last commit in the branch, just before it was merged. And by looking at the branch graph it should be quite easy to understand what happened.

Now I think I was wrong here, as git bisect will probably find that the first commit in the branch (not the last one) is the first bad commit.

[...]

Quoted 4 lines
Quoted 3 lines

- the name "bisectbadbranchfirst" seems wrong to me, because git branches are just some special tags; "firstparentsonly" would be a better name,

Quoted 16 lines

It's recursively applying bad branch first algorithm, not just constantly stick to first parent.
Given this condition:
   A -> B -> C -> D -> E -> F -> G -> H   (master)
        \ a  -> b -> c -> d -> e /  (feature 1)
             \ x -> y -> z/      (feature 2)
start with H as bad commit, and A as good commit, if y is the target bad commit. bad-branch-first algorithm will do it like this:
   1. In first round stick to master branch, so it will locate G as first bad commit
   2. In second round stick to feature1 branch, then it will locate d as first bad commit
   3. In third round stick to feature2 branch, then it will finally locate y as first bad commit
So you could see, it's always sticking to branch where current bad commit sit

I see. It is interesting, but why not develop a "firstparentsonly" algorithm first?

As Avery explains in his email, it is already interesting to have a "firstparentsonly" algorithm because some people are only interested to know from which branch the bug comes from.
When they know that, they can just contact the relevant people and be done with it.

And when we have a "firstparentsonly" algorithm, then your algorithm could be just a script that repeatedly uses git bisect with the "firstparentsonly" algorithm. And this script might be integrated in the "contrib" directory if it not considered important to be integrated as an algorithm into git bisect.

Thanks,
Christian.

author
#9 lore raw
Quoted 10 lines · Christian Couder
Quoted 9 lines · Shuang He
Quoted 8 lines · Christian Couder

Hi,

Quoted 5 lines · Shuang He

Hi
The default git-bisect algorithm will jump around the commit tree,
on the purpose of taking least steps to find the first culprit commit. We may find it sometime would locate a old culprit commit that we're not concerned about anymore.

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Yes, it can be a problem.

I'm honored to be given so much comment :)
Thank you

I am honored by your interest in git bisect and the fact that you provided a patch :-)
Thanks!

I'm glad to see that git community is so hot.

Quoted 25 lines
Quoted 17 lines
Quoted 17 lines

If the quality of these branches is too bad, I think they should not have been merged in the first place.
If they are not merged (and not marked as good), then git bisect will not look at them, since it will look only at commits that are ancestors of the bad commit it is given.

Or if one is merged but it causes too many problems, then perhaps a replacement commit could be used to unmerge the branch.

Another possibility is to have in a file a list of commits that are the last commits on these branches before the merge commits, and do a:

git bisect good $(cat good_commits_file.txt)

at the beginning of each bisection.

Quoted 8 lines
Quoted 2 lines

So I think the long term solution in this case is not what your are suggesting.

Yeah, I agree that the issue I addressed above will not be a problem if all those branches are maintained very well.
Actually we've implemented a automated bisect system for Intel Linux Graphics Driver Project, and so we'd like the system
helps us to locate issue in an more automatic way when branches are not maintained as good as expected.

Quoted 6 lines

I think there is always a price to pay when you bisect if the branches are not well maintained.
Maybe your algorithm could help in some cases, but my opinion is that there will probably still be many problems and a human will often have to take a look.

Yes, I agree. What we trying to do is just make the machine to do more help for human.

Quoted 52 lines
Quoted 14 lines
Quoted 14 lines
Quoted 9 lines

2. Some of those branches may not synchronized with main
branch in time. Say feature1 is broken when feature2 branch is created, and
feature1 is fixed just a moment later after feature2 branch is created, and when feature2's development is done, and developer want to merge feature2 branch back to master branch, feature2 will be firstly synchronized to master branch tip, then merge into master. For the same reason addressed in issue 1, this will also lead git-bisect into wrong direction.

I am not sure what you mean by " feature2 will be firstly synchronized to master branch tip", and I think this should mean a rebase that would fix the bug if feature1 has already been merged into the master branch.

Quoted 4 lines
Quoted 4 lines

But anyway in this case, I think that git bisect will find that the first bad commit is the last commit in the branch, just before it was merged. And by looking at the branch graph it should be quite easy to understand what happened.

Now I think I was wrong here, as git bisect will probably find that the first commit in the branch (not the last one) is the first bad commit.

[...]

Quoted 9 lines
Quoted 3 lines

- the name "bisectbadbranchfirst" seems wrong to me, because git branches are just some special tags; "firstparentsonly" would be a better name,

It's recursively applying bad branch first algorithm, not just constantly stick to first parent.
Given this condition:

    A ->  B ->  C ->  D ->  E ->  F ->  G ->  H   (master)
         \ a  ->  b ->  c ->  d ->  e /  (feature 1)
              \ x ->  y ->  z/      (feature 2)
Quoted 10 lines

start with H as bad commit, and A as good commit, if y is the target bad commit. bad-branch-first algorithm will do it like this:
1. In first round stick to master branch, so it will locate G as first
bad commit
2. In second round stick to feature1 branch, then it will locate d as
first bad commit
3. In third round stick to feature2 branch, then it will finally locate y
as first bad commit
So you could see, it's always sticking to branch where current bad commit sit

I see. It is interesting, but why not develop a "firstparentsonly" algorithm first?

As Avery explains in his email, it is already interesting to have a "firstparentsonly" algorithm because some people are only interested to know from which branch the bug comes from.
When they know that, they can just contact the relevant people and be done with it.

Quoted 5 lines

And when we have a "firstparentsonly" algorithm, then your algorithm could be just a script that repeatedly uses git bisect with the "firstparentsonly" algorithm. And this script might be integrated in the "contrib" directory if it not considered important to be integrated as an algorithm into git bisect.

Sorry to reply so late, since I was on a long journey home for Chinese New Year vacation ;)
I agree that's also an good option.
Is it acceptable to add option to git-bisect stuff, so user could choose which algorithm to use at every step at will.
And we have tested previous attached patch with t6002-rev-list-bisect.sh and t6030-bisect-porcelain.sh, and we get:
with bad-branch-first disabled (which is the default setting):
t6002-rev-list-bisect.sh: # passed all 45 test(s)
t6030-bisect-porcelain.sh: # passed all 40 test(s)
and with bad-branch-first enabled:
t6002-rev-list-bisect.sh: # passed all 45 test(s)
t6030-bisect-porcelain.sh: # failed 5 among 40 test(s), and I
have spent some time digging into those failures ,and it seems they're all false negative since they're using hard-coded bisect path to validate specific case

Thanks
--Shuang

Quoted 2 lines

Thanks,
Christian.

#10 lore raw
Quoted 18 lines · Shuang He
Quoted 10 lines · Christian Couder
Quoted 9 lines

Yeah, I agree that the issue I addressed above will not be a problem if all
those branches are maintained very well.
Actually we've implemented a automated bisect system for Intel Linux Graphics Driver Project, and so we'd like the system
helps us to locate issue in an more automatic way when branches are not maintained as good as expected.

Quoted 6 lines

I think there is always a price to pay when you bisect if the branches are not well maintained.
Maybe your algorithm could help in some cases, but my opinion is that there will probably still be many problems and a human will often have to take a look.

Quoted 2 lines

Yes, I agree. What we trying to do is just make the machine to do more help for human.

Yeah, this is the way to go. And by the way I am happy to know that you have implemented an automated bisect system. That's great and I hope it already helps.

Quoted 31 lines
Quoted 31 lines
Quoted 13 lines
Quoted 3 lines

- the name "bisectbadbranchfirst" seems wrong to me, because git branches are just some special tags; "firstparentsonly" would be a better name,

It's recursively applying bad branch first algorithm, not just constantly stick to first parent.
Given this condition:
   A ->  B ->  C ->  D ->  E ->  F ->  G ->  H   (master)
        \ a  ->  b ->  c ->  d ->  e /  (feature 1)
             \ x ->  y ->  z/      (feature 2)
start with H as bad commit, and A as good commit, if y is the target bad commit. bad-branch-first algorithm will do it like this:
   1. In first round stick to master branch, so it will locate G as first

Quoted 8 lines

bad commit
   2. In second round stick to feature1 branch, then it will locate d as first bad commit
   3. In third round stick to feature2 branch, then it will finally locate y
as first bad commit
So you could see, it's always sticking to branch where current bad commit sit

I see. It is interesting, but why not develop a "firstparentsonly" algorithm first?

As Avery explains in his email, it is already interesting to have a "firstparentsonly" algorithm because some people are only interested to know from which branch the bug comes from.
When they know that, they can just contact the relevant people and be done with it.

Quoted 8 lines
Quoted 5 lines

And when we have a "firstparentsonly" algorithm, then your algorithm could be just a script that repeatedly uses git bisect with the "firstparentsonly" algorithm. And this script might be integrated in the "contrib" directory if it not considered important to be integrated as an algorithm into git bisect.

Sorry to reply so late, since I was on a long journey home for Chinese New Year vacation ;)

No problem. I am not in a hurry at all. In fact I don't have much time these days so I reply very late too.

Quoted 3 lines

I agree that's also an good option.
Is it acceptable to add option to git-bisect stuff, so user could choose which algorithm to use at every step at will.

Are you sure it is needed to be able to change the algorithm at every step?

This means that you would like a new "git bisect strategy <strategy>" subcommand ?

First I thought that we could just add a "--strategy <strategy>" option to "git bisect start".
But anyway, I think it should be easy to add afterward, and it can be done in a separated patch that can be discussed on its own.

Quoted 11 lines

And we have tested previous attached patch with t6002-rev-list-bisect.sh and t6030-bisect-porcelain.sh, and we get:
   with bad-branch-first disabled (which is the default setting):
       t6002-rev-list-bisect.sh: # passed all 45 test(s) t6030-bisect-porcelain.sh: # passed all 40 test(s)
   and with bad-branch-first enabled:
       t6002-rev-list-bisect.sh: # passed all 45 test(s) t6030-bisect-porcelain.sh: # failed 5 among 40 test(s), and I have spent some time digging into those failures ,and it seems they're all false negative since they're using hard-coded bisect path to validate specific case

Yes, there are some hard coded commits that depend on the algorithm. Anyway I did not look in depth at your patch yet, and as I said it would be better if you could split it into a patch series where a "firstparentsonly" algorithm is implemented first.
This way it will be easier to review, and we can start to integrate some non controversial features, and then discuss the other ones on their own merit.

Thanks in advance,
Christian.

author
#11 lore raw
Quoted 57 lines · Christian Couder
Quoted 8 lines · Shuang He
Quoted 7 lines · Christian Couder
Quoted 7 lines

Yeah, I agree that the issue I addressed above will not be a problem if all
those branches are maintained very well.
Actually we've implemented a automated bisect system for Intel Linux Graphics Driver Project, and so we'd like the system
helps us to locate issue in an more automatic way when branches are not maintained as good as expected.

Quoted 8 lines
Quoted 6 lines

I think there is always a price to pay when you bisect if the branches are not well maintained.
Maybe your algorithm could help in some cases, but my opinion is that there will probably still be many problems and a human will often have to take a look.

Yes, I agree. What we trying to do is just make the machine to do more help for human.

Yeah, this is the way to go. And by the way I am happy to know that you have implemented an automated bisect system. That's great and I hope it already helps.

Quoted 29 lines
Quoted 29 lines
Quoted 9 lines
Quoted 3 lines

- the name "bisectbadbranchfirst" seems wrong to me, because git branches are just some special tags; "firstparentsonly" would be a better name,

It's recursively applying bad branch first algorithm, not just constantly stick to first parent.
Given this condition:

    A ->    B ->    C ->    D ->    E ->    F ->    G ->    H   (master)
         \ a  ->    b ->    c ->    d ->    e /  (feature 1)
              \ x ->    y ->    z/      (feature 2)
Quoted 11 lines

start with H as bad commit, and A as good commit, if y is the target bad commit. bad-branch-first algorithm will do it like this:
1. In first round stick to master branch, so it will locate G as first
bad commit
2. In second round stick to feature1 branch, then it will locate d as
first bad commit
3. In third round stick to feature2 branch, then it will finally
locate y
as first bad commit
So you could see, it's always sticking to branch where current bad commit sit

I see. It is interesting, but why not develop a "firstparentsonly" algorithm first?

As Avery explains in his email, it is already interesting to have a "firstparentsonly" algorithm because some people are only interested to know from which branch the bug comes from.
When they know that, they can just contact the relevant people and be done with it.

Quoted 7 lines
Quoted 5 lines

And when we have a "firstparentsonly" algorithm, then your algorithm could be just a script that repeatedly uses git bisect with the "firstparentsonly" algorithm. And this script might be integrated in the "contrib" directory if it not considered important to be integrated as an algorithm into git bisect.

Sorry to reply so late, since I was on a long journey home for Chinese New Year vacation ;)

Quoted 7 lines

No problem. I am not in a hurry at all. In fact I don't have much time these days so I reply very late too.

Quoted 3 lines

I agree that's also an good option.
Is it acceptable to add option to git-bisect stuff, so user could choose which algorithm to use at every step at will.

Are you sure it is needed to be able to change the algorithm at every step?

I don't think it's needed, it would just give user more control over the algorithm.

Quoted 7 lines

This means that you would like a new "git bisect strategy<strategy>" subcommand ?

First I thought that we could just add a "--strategy<strategy>" option to "git bisect start".
But anyway, I think it should be easy to add afterward, and it can be done in a separated patch that can be discussed on its own.

Yeah, agree. We could discuss this later

Quoted 19 lines
Quoted 11 lines

And we have tested previous attached patch with t6002-rev-list-bisect.sh and t6030-bisect-porcelain.sh, and we get:
with bad-branch-first disabled (which is the default setting):
t6002-rev-list-bisect.sh: # passed all 45 test(s) t6030-bisect-porcelain.sh: # passed all 40 test(s)
and with bad-branch-first enabled:
t6002-rev-list-bisect.sh: # passed all 45 test(s) t6030-bisect-porcelain.sh: # failed 5 among 40 test(s), and I have
spent some time digging into those failures ,and it seems they're all false negative since they're using hard-coded bisect path to validate specific case

Yes, there are some hard coded commits that depend on the algorithm. Anyway I did not look in depth at your patch yet, and as I said it would be better if you could split it into a patch series where a "firstparentsonly" algorithm is implemented first.
This way it will be easier to review, and we can start to integrate some non controversial features, and then discuss the other ones on their own merit.

Quoted 2 lines

Thanks in advance,
Christian.

Thanks for the good suggestion, I'll start the work soon.

Thanks
--Shuang

#12 lore raw
In reply to Christian Couder
Quoted 5 lines · Christian Couder

I am not opposed to an option to bisect on the first parents of the bad commit only. And after a very fast look at your patch it seems to be what it does. By the way Avery Pennarun's gitbuilder
(https://github.com/apenwarr/gitbuilder) does the same thing. So I know some people are interested in such a feature.

Just some notes on gitbuilder's algorithm, since I haven't spent the time to fully understand Shuang's proposal.

I do understand at least one of his concerns, that is, that people like to do a lot of "messy" development on a branch, and when the branch is done, merge the whole messy branch into the "mainline". The messy branch would then have a lot of commits that break a lot of things before fixing them again later.

In a corporate environment, this method allows people to work all day, make frequent commits, pull from other branches at will, and never risk their lives by doing poorly-educated rebases. It works pretty well *until* you try to bisect, at which time all these messy commits start to bite you.

gitbuilder's bisection is a total hack around this situation, although it happens to work perfectly in the workflow it was designed for, thus making me feel clever.

Basically, we push/fetch *all* the branches from *everybody* into a single repo, and build all of them as frequently as we can. If you think about it, if you have all the branches that someone might have pulled/merged from, then you don't have to think of the git history as a whole complicated DAG; you can just think of it as a whole bunch of separate chunks of linear history. Moreover, as long as people are careful to only pull from a branch when that branch is passing all tests - which you can easily see by looking at the gitbuilder console
- then playing inside each of these chunks of linear history can help you figure out where particular bugs were introduced during "messy" branches.

It also allows you a nice separation of concerns. The owner of the mainline branch (the "integration manager" person) only really cares about which branch they merged that caused a problem, because that person doesn't want to fix bugs, he/she simply wants to know who owns the failing branch, so that person can fix *their* bug and their branch will merge without breaking things.

So this is why gitbuilder uses "git rev-list --first-parent" during its "fake bisection" operation: because a different person is responsible for each "linear chunk" of history.

Note that you have to use --no-ff when merging if you want this to work reliably. But the build manager person can just remember to do that. Combining --no-ff and --ff-only (which sound mutually exclusive but aren't) is a way to be extra specially sure.

Now, if you aren't using gitbuilder, what we want from "bisection" is not quite the same, but let's imagine that you at least have a similar setup, where people *only* ever merge into the mainline by using --no-ff. In that case, you'd like a bisect operation that *starts* by using --first-parent, which will tell you which merge caused the problem. After that, you might want to bisect into the branch.

(I don't actually remember if 'git bisect' understands --first-parent correctly. gitbuilder doesn't exactly bisect either, but that's another story and not relevant right now.)

I can actually imagine that there are many more projects that do what I'm talking about - "messy" branches that get broken and fixed over time, then merge into a "clean" mainline - than projects (like the kernel and git.git) that try to keep all branches clean at all times. Thus, I could see some argument that a "--first-parents first" bisection would actually help out a lot of people, and maybe even deserves to be the default.

I don't really care though, I just use gitbuilder :)

Have fun,

Avery

author
#13 lore raw
Quoted 73 lines · Avery Pennarun
Quoted 5 lines · Christian Couder

I am not opposed to an option to bisect on the first parents of the bad commit only. And after a very fast look at your patch it seems to be what it does. By the way Avery Pennarun's gitbuilder
(https://github.com/apenwarr/gitbuilder) does the same thing. So I know some people are interested in such a feature.

Just some notes on gitbuilder's algorithm, since I haven't spent the time to fully understand Shuang's proposal.

I do understand at least one of his concerns, that is, that people like to do a lot of "messy" development on a branch, and when the branch is done, merge the whole messy branch into the "mainline". The messy branch would then have a lot of commits that break a lot of things before fixing them again later.

In a corporate environment, this method allows people to work all day, make frequent commits, pull from other branches at will, and never risk their lives by doing poorly-educated rebases. It works pretty well *until* you try to bisect, at which time all these messy commits start to bite you.

gitbuilder's bisection is a total hack around this situation, although it happens to work perfectly in the workflow it was designed for, thus making me feel clever.

Basically, we push/fetch *all* the branches from *everybody* into a single repo, and build all of them as frequently as we can. If you think about it, if you have all the branches that someone might have pulled/merged from, then you don't have to think of the git history as a whole complicated DAG; you can just think of it as a whole bunch of separate chunks of linear history. Moreover, as long as people are careful to only pull from a branch when that branch is passing all tests - which you can easily see by looking at the gitbuilder console
- then playing inside each of these chunks of linear history can help you figure out where particular bugs were introduced during "messy" branches.

It also allows you a nice separation of concerns. The owner of the mainline branch (the "integration manager" person) only really cares about which branch they merged that caused a problem, because that person doesn't want to fix bugs, he/she simply wants to know who owns the failing branch, so that person can fix *their* bug and their branch will merge without breaking things.

So this is why gitbuilder uses "git rev-list --first-parent" during its "fake bisection" operation: because a different person is responsible for each "linear chunk" of history.

Note that you have to use --no-ff when merging if you want this to work reliably. But the build manager person can just remember to do that. Combining --no-ff and --ff-only (which sound mutually exclusive but aren't) is a way to be extra specially sure.

Now, if you aren't using gitbuilder, what we want from "bisection" is not quite the same, but let's imagine that you at least have a similar setup, where people *only* ever merge into the mainline by using --no-ff. In that case, you'd like a bisect operation that *starts* by using --first-parent, which will tell you which merge caused the problem. After that, you might want to bisect into the branch.

(I don't actually remember if 'git bisect' understands --first-parent correctly. gitbuilder doesn't exactly bisect either, but that's another story and not relevant right now.)

I can actually imagine that there are many more projects that do what I'm talking about - "messy" branches that get broken and fixed over time, then merge into a "clean" mainline - than projects (like the kernel and git.git) that try to keep all branches clean at all times. Thus, I could see some argument that a "--first-parents first" bisection would actually help out a lot of people, and maybe even deserves to be the default.

Quoted 5 lines

I don't really care though, I just use gitbuilder :)

Have fun,

Avery

Thanks for helping explaining those stuff, and also glad to learn more about gitbuilder :)

Thanks
--Shuang