ref-filter: solve bugs caused by enumeration

From: ZheNing Hu <[email protected]>

Johannes Schindelin seems to have introduced a bug in cc72385f(for-each-ref: let upstream/push optionally report the remote name), it use atom->u.remote_ref.option which is a member of enumeration in the judgment statement. When we use other members in the enumeration used_atom.u, and it happened to fill in remote_ref.push, this judgment may still be established and produces errors. So replace the judgment statement with starts_with(name, "push") to fix the error.

---
[GSOC] ref-filter: solve bugs caused by enumeration

This is the problem I found in the implementation of git for-each-ref --format="%(notes)".

I added a new option to the enum used_atom.u , but the program seems to output the value of "%(push)".

After research, it should be the problem of incorrect use of enumeration values here.

Thanks, may the force be with you!

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-949%2Fadlternative%2Fref-filter-enum-bug-fix-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-949/adlternative/ref-filter-enum-bug-fix-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/949

 ref-filter.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index a0adb4551d87..f467f2fbbb73 100644
@@ -1730,7 +1730,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)
 			else
 				v->s = xstrdup("");
 			continue;
-		} else if (atom->u.remote_ref.push) {
+		} else if (starts_with(name, "push")) {
 			const char *branch_name;
 			v->s = xstrdup("");
 			if (!skip_prefix(ref->refname, "refs/heads/",

base-commit: 311531c9de557d25ac087c1637818bd2aad6eb3a

gitgitgadget

Sorry but I am not sure if our readers would understand what "a member of enumeration in the judgment statement" is (I certainly do not), and even more importantly, "bugs caused by enumeration" on the title does not hint much about what problem the patch is trying to solve.

And this paragraph does not enlighten all that much, unfortunately.

Is it that a check refers to one member of a union without making sure that member is the one in effect in the union? I am most puzzled by the mention of "enumeration" when there does not appear to be any enum involved.

Sorry, I didn't make it clear. I re-describe the problem first, and then modify the commit messages.

Suppose we are dealing with "%(notes)", the name member of our used_atom item at this time is "notes", and its member union u uses a struct notes_option, we fill some values in used_atom.u.notes_option,

When we traverse in used_atom array in populate_value() and previous judgement like "if (starts_with(name, "refname"))" will failed, because we are dealing with atom "notes", but in judgement "else if (atom->u.remote_ref.push)",
The value we fill in used_atom.u.notes_option just makes used_atom.u.remote_ref.push non-zero. This leads us into the wrong case.

Is this clearer?

Thanks.
--
ZheNing Hu

This time you avoided the word enumeration, and it made it clearer. The word commonly used is "condition" where you said "judgment", I think, and wit that it would probably be even more clear.

used_atom.u is an union, and it has different members depending on what atom the auxiliary data the union part of the "struct used_atom" wants to record. At most only one of the members can be valid at any one time. Since the code checks u.remote_ref without even making sure if the atom is "push" or "push:" (which are only two cases that u.remote_ref.push becomes valid), but u.remote_ref shares the same storage for other members of the union, the check was reading from an invalid member, which was the bug.

So the fix was to see what atom it is by checking its name member? Is starts_with() a reliable test? A fictitious atom "pushe" will be different from "push" or "push:<something>", but will still pass that test, so from the point of view of future-proofing the tests, shouldn't it do the same check as the one at the beginning of remote_ref_atom_parser()?

Thanks.

OK.

Yes, that's what it means. I got it wrong before, of course used_atom.u is not a enum, it's union.

I think it's not necesssary. Before we call populate_value(), parse_ref_filter_atom() only allow user use atom which match
"valid_atom" entry fully, something like "pushe" will be rejected
and process die with "fatal: unknown field name: pushe", so it didn't pass this "starts_with()" test.

        /* Is the atom a valid one? */
        for (i = 0; i < ARRAY_SIZE(valid_atom); i++) {
                int len = strlen(valid_atom[i].name);
                if (len == atom_len && !memcmp(valid_atom[i].name, sp, len))
                        break;
        }

        if (ARRAY_SIZE(valid_atom) <= i)
                return strbuf_addf_ret(err, -1, _("unknown field name: %.*s"),
                                       (int)(ep-atom), atom);

Thanks.
--
ZheNing Hu

I do not care if it is not needed with the "current" set of atoms we accept. The example "pushe", which obviously will not be accepted by today's code, was all about future-proofing.

Well, what you said makes sense, now I think something like this will be more secure. In the future, other people may be grateful for such a choice. :)

@@ -1730,7 +1759,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)

                        else
                                v->s = xstrdup("");
                        continue;

- } else if (starts_with(name, "push")) {
+ } else if (starts_with(name, "push") &&
atom->u.remote_ref.push) {

--
ZheNing Hu

From: ZheNing Hu <[email protected]>

used_atom.u is an union, and it has different members depending on what atom the auxiliary data the union part of the "struct used_atom" wants to record. At most only one of the members can be valid at any one time. Since the code checks u.remote_ref without even making sure if the atom is "push" or "push:" (which are only two cases that u.remote_ref.push becomes valid), but u.remote_ref shares the same storage for other members of the union, the check was reading from an invalid member, which was the bug.

Modify the condition here to first check whether the atom name starts with "push", and then check u.remote_ref, to avoid reading the value of invalid member of the union.

---
[GSOC] ref-filter: fix read invalid union member bug

Change from last version: Modified the documentation description with the help of Junio. And modify the processing method of the condition:
check whether the name of the atom starts with "push" and whether u.remote_ref is non-zero.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-949%2Fadlternative%2Fref-filter-enum-bug-fix-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-949/adlternative/ref-filter-enum-bug-fix-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/949

Range-diff vs v1:

1: e51ca176f76b ! 1: 0e1923c9d722 [GSOC] ref-filter: solve bugs caused by enumeration
@@ Metadata
Author: ZheNing Hu <[email protected]>

## Commit message ##
- [GSOC] ref-filter: solve bugs caused by enumeration
+ [GSOC] ref-filter: fix read invalid union member bug

- Johannes Schindelin seems to have introduced a bug in
- cc72385f(for-each-ref: let upstream/push optionally
- report the remote name), it use atom->u.remote_ref.option
- which is a member of enumeration in the judgment statement.
- When we use other members in the enumeration used_atom.u,
- and it happened to fill in remote_ref.push, this judgment
- may still be established and produces errors. So replace the
- judgment statement with starts_with(name, "push") to fix
- the error.
+ used_atom.u is an union, and it has different members depending on
+ what atom the auxiliary data the union part of the "struct
+ used_atom" wants to record. At most only one of the members can be
+ valid at any one time. Since the code checks u.remote_ref without
+ even making sure if the atom is "push" or "push:" (which are only
+ two cases that u.remote_ref.push becomes valid), but u.remote_ref
+ shares the same storage for other members of the union, the check
+ was reading from an invalid member, which was the bug.

+ Modify the condition here to first check whether the atom name
+ starts with "push", and then check u.remote_ref, to avoid reading
+ the value of invalid member of the union.
+
+ Helped-by: Junio C Hamano <[email protected]>

## ref-filter.c ##
@@ ref-filter.c: static int populate_value(struct ref_array_item *ref, struct strbu
v->s = xstrdup("");
continue;
- } else if (atom->u.remote_ref.push) {
-+ } else if (starts_with(name, "push")) {
++ } else if (starts_with(name, "push") && atom->u.remote_ref.push) {
const char *branch_name;
v->s = xstrdup("");
if (!skip_prefix(ref->refname, "refs/heads/",

 ref-filter.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index a0adb4551d87..750b25914b82 100644
@@ -1730,7 +1730,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)
 			else
 				v->s = xstrdup("");
 			continue;
-		} else if (atom->u.remote_ref.push) {
+		} else if (starts_with(name, "push") && atom->u.remote_ref.push) {
 			const char *branch_name;
 			v->s = xstrdup("");
 			if (!skip_prefix(ref->refname, "refs/heads/",

base-commit: 311531c9de557d25ac087c1637818bd2aad6eb3a

gitgitgadget

Well, this "pushe" has atom->u.X which is different from "push" and "push:". This is indeed worth worrying about.

But I am afraid that the cost we paid for string matching here is too high, Since the string matching here is to illustrate that we use one of the members of the union, why can't we directly add a "union_type" member to used_atom? if we have this, we can just switch the "union_type" flag and also make the program correct. Perhaps this would be better than the large number of string matches have done here.

Thanks.
--
ZheNing Hu

If that is truly the concern (I do not know without measuring), perhaps we should add a member next to the union to say which one of the union members is valid, so that you can say

    if (atom->atom_type == ATOM_TYPE_REMOTE_REF &&
        atom->u.remote_ref.push)

(introduce an enum and define ATOM_TYPE_* after the member in the union).

That would help futureproofing the code even further, as a new synonym of "push" introduced laster [*] would not invalidate the check you are adding there.

[Footnote]

* remote_ref_atom_parser() in the future may begin like so:

-	if (!strcmp(atom->name, "push") || starts_with(atom->name, "push:"))
+	if (!strcmp(atom->name, "push") || starts_with(atom->name, "push:") ||
+           !strcmp(atom->name, "a-synonym-for-push"))
		atom->u.remote_ref.push = 1;

Yes, I think so. Since the content of this part needs to be modified for
the parsing of all atoms, I will put it in a separate topic to complete.

Yes, this enhances its generalization ability.

From: ZheNing Hu <[email protected]>

used_atom.u is an union, and it has different members depending on what atom the auxiliary data the union part of the "struct used_atom" wants to record. At most only one of the members can be valid at any one time. Since the code checks u.remote_ref without even making sure if the atom is "push" or "push:" (which are only two cases that u.remote_ref.push becomes valid), but u.remote_ref shares the same storage for other members of the union, the check was reading from an invalid member, which was the bug.

Modify the condition here to check whether the atom name equals to "push" or starts with "push:", to avoid reading the value of invalid member of the union.

---
[GSOC] ref-filter: fix read invalid union member bug

Change from last version:
Modify the processing method of the condition: check whether the name of the atom equals to "push" or starts with "pushs", which can enhanced security, although it may bring string match overhead.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-949%2Fadlternative%2Fref-filter-enum-bug-fix-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-949/adlternative/ref-filter-enum-bug-fix-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/949

Range-diff vs v2:

1: 0e1923c9d722 ! 1: 21cf7a44e168 [GSOC] ref-filter: fix read invalid union member bug
@@ Commit message

used_atom.u is an union, and it has different members depending on what atom the auxiliary data the union part of the "struct
- used_atom" wants to record. At most only one of the members can be
- valid at any one time. Since the code checks u.remote_ref without
+ used_atom" wants to record. At most only one of the members can be
+ valid at any one time. Since the code checks u.remote_ref without
even making sure if the atom is "push" or "push:" (which are only two cases that u.remote_ref.push becomes valid), but u.remote_ref shares the same storage for other members of the union, the check was reading from an invalid member, which was the bug.

- Modify the condition here to first check whether the atom name
- starts with "push", and then check u.remote_ref, to avoid reading
- the value of invalid member of the union.
+ Modify the condition here to check whether the atom name
+ equals to "push" or starts with "push:", to avoid reading the
+ value of invalid member of the union.

@@ ref-filter.c: static int populate_value(struct ref_array_item *ref, struct strbu
v->s = xstrdup("");
continue;
- } else if (atom->u.remote_ref.push) {
-+ } else if (starts_with(name, "push") && atom->u.remote_ref.push) {
++ } else if (!strcmp(atom->name, "push") || starts_with(atom->name, "push:")) {
const char *branch_name;
v->s = xstrdup("");
if (!skip_prefix(ref->refname, "refs/heads/",

 ref-filter.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index a0adb4551d87..213d3773ada3 100644
@@ -1730,7 +1730,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)
 			else
 				v->s = xstrdup("");
 			continue;
-		} else if (atom->u.remote_ref.push) {
+		} else if (!strcmp(atom->name, "push") || starts_with(atom->name, "push:")) {
 			const char *branch_name;
 			v->s = xstrdup("");
 			if (!skip_prefix(ref->refname, "refs/heads/",

base-commit: 311531c9de557d25ac087c1637818bd2aad6eb3a

gitgitgadget

I do not think this would have much security implication either way. What it buys us is the future-proofing.

I think it is OK to make this change without the enum thing to have it graduate early as a fix to the existing code. The enum thing can come on top.

Will queue. Thanks.

Just a final sanity check. Is this a recent breakage or was the code introduced at cc72385f (for-each-ref: let upstream/push optionally report the remote name, 2017-10-05) broken from the beginning?

I am wondering if it is easy to add a test to cover the codepath that is affected by this change.

Thanks.

Ah, truely.

Indeed. "enum atom_type" is for ref-filter performance optimization and get some other benefits like quick index. So I put it in another topic.

Thanks.
--
ZheNing Hu

Well, The trigger condition is very special, but the bug was introduced at that time. Let's see the "bug" example below.

Well, because this bug must require that the seventeenth bit of used_atom.u is not 0, it took me a long time to find this bug. in used_atom.u, only the member "color" and "contents" which size is bigger than 17 bytes, but "%(contents:trailer:only)" only fill the 16th byte of used_atom.u.

"Fortunately", I found it.

git for-each-ref --format='%(color:#aa22ac)'

I will add test for it!

Thanks!
--
ZheNing Hu

From: ZheNing Hu <[email protected]>

used_atom.u is an union, and it has different members depending on what atom the auxiliary data the union part of the "struct used_atom" wants to record. At most only one of the members can be valid at any one time. Since the code checks u.remote_ref without even making sure if the atom is "push" or "push:" (which are only two cases that u.remote_ref.push becomes valid), but u.remote_ref shares the same storage for other members of the union, the check was reading from an invalid member, which was the bug.

Modify the condition here to check whether the atom name equals to "push" or starts with "push:", to avoid reading the value of invalid member of the union.

---
[GSOC] ref-filter: fix read invalid union member bug

Change from last version:
Prove that the bug may appear when using %(color) atom. And add corresponding test for it.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-949%2Fadlternative%2Fref-filter-enum-bug-fix-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-949/adlternative/ref-filter-enum-bug-fix-v4
Pull-Request: https://github.com/gitgitgadget/git/pull/949

Range-diff vs v3:

1: 21cf7a44e168 ! 1: 8c6c0368a590 [GSOC] ref-filter: fix read invalid union member bug

     @@ ref-filter.c: static int populate_value(struct ref_array_item *ref, struct strbu
       			const char *branch_name;
       			v->s = xstrdup("");
       			if (!skip_prefix(ref->refname, "refs/heads/",
     +
     + ## t/t6302-for-each-ref-filter.sh ##
     +@@ t/t6302-for-each-ref-filter.sh: test_expect_success '%(color) must fail' '
     + 	test_must_fail git for-each-ref --format="%(color)%(refname)"
     + '
     + 
     ++test_expect_success '%(color:#aa22ac) must success' '
     ++	cat >expect <<-\EOF &&
     ++	refs/heads/main
     ++	refs/heads/side
     ++	refs/odd/spot
     ++	refs/tags/annotated-tag
     ++	refs/tags/doubly-annotated-tag
     ++	refs/tags/doubly-signed-tag
     ++	refs/tags/four
     ++	refs/tags/one
     ++	refs/tags/signed-tag
     ++	refs/tags/three
     ++	refs/tags/two
     ++	EOF
     ++	git for-each-ref --format="%(color:#aa22ac)%(refname)" >actual &&
     ++	test_cmp expect actual
     ++'
     ++
     + test_expect_success 'left alignment is default' '
     + 	cat >expect <<-\EOF &&
     + 	refname is refs/heads/main    |refs/heads/main

 ref-filter.c                   |  2 +-
 t/t6302-for-each-ref-filter.sh | 18 ++++++++++++++++++
 2 files changed, 19 insertions(+), 1 deletion(-)

index a0adb4551d87..213d3773ada3 100644
@@ -1730,7 +1730,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)
 			else
 				v->s = xstrdup("");
 			continue;
-		} else if (atom->u.remote_ref.push) {
+		} else if (!strcmp(atom->name, "push") || starts_with(atom->name, "push:")) {
 			const char *branch_name;
 			v->s = xstrdup("");
 			if (!skip_prefix(ref->refname, "refs/heads/",

index 9866b1b57368..38a7d83830aa 100755
@@ -117,6 +117,24 @@ test_expect_success '%(color) must fail' '
 	test_must_fail git for-each-ref --format="%(color)%(refname)"
 '
 
+test_expect_success '%(color:#aa22ac) must success' '
+	cat >expect <<-\EOF &&
+	refs/heads/main
+	refs/heads/side
+	refs/odd/spot
+	refs/tags/annotated-tag
+	refs/tags/doubly-annotated-tag
+	refs/tags/doubly-signed-tag
+	refs/tags/four
+	refs/tags/one
+	refs/tags/signed-tag
+	refs/tags/three
+	refs/tags/two
+	EOF
+	git for-each-ref --format="%(color:#aa22ac)%(refname)" >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'left alignment is default' '
 	cat >expect <<-\EOF &&
 	refname is refs/heads/main    |refs/heads/main

base-commit: 311531c9de557d25ac087c1637818bd2aad6eb3a

gitgitgadget

s/success/succeed/

But more importantly, I am not sure how this is supposed to demonstrate existing breakage around the %(push). Did you mean to use %(push) instead of %(refname) or something?

Ah, I don’t think this scene of damage is related to using %(push) or %(refname).

We are just in the process of using populate_value(), if the atom we specify meets the following conditions, then the condition of atom->u.remote_ref.push!=0 will be established.

1. The atom that triggers the bug , its "if" condition order must after "if (atom->u.remote_ref.push)", such as %(refname) or %(worktreepath), they can be executed correctly because their order is before "push".

2. The member size in used_atom.u corresponding to the atom must larger than 17 bytes, because the offset of "u.remote_ref.push" in "u.remote_ref" is 17, the satisfied atoms are only "%(color)" and "%(contents)", their corresponding members are u.color and u.contents.

3. We happen to be able to fill in the 17th position of these structures, which makes atom->u.remote_ref.push not equal to 0 established.

So this kind of bug is not related to %(push), an atom that satisfies the above conditions will make if (atom->u.remote_ref.push) be true. then execute the program logic related to %(push).

Now, we only have %(color) can trigger it "sometime", It is unpredictable to fill in the 17th byte of used_atom.u.color, so we cannot track all the atoms related to this bug.

git for-each-ref --format="%(color:#aa22ac)%(refname)" git for-each-ref --format="%(color:#aa22ad)%(refname)"

will trigger the bug.

git for-each-ref --format="%(color:#aa22ae)%(refname)" git for-each-ref --format="%(color:#aa22af)%(refname)"

will not trigger the bug.

In other words, we cannot use a perfect test set to cover all broken. So now %(color:#aa22ac) is enough for explain the problem of this bug.

Thanks.
--
ZheNing Hu

Well, the thing is,

    $ git checkout 49f38e2d ;# (The fifteenth batch, 2021-05-10)
    $ git am -s mbox
    $ git show --stat --oneline
    39509d100a (HEAD) ref-filter: fix read invalid union member bug
     ref-filter.c                   |  2 +-
     t/t6302-for-each-ref-filter.sh | 18 ++++++++++++++++++
     2 files changed, 19 insertions(+), 1 deletion(-)
    $ git show ref-filter.c | git apply -R ;# revert only the fix
    $ make -j32 && make -C t T=t6302-*.sh

does not seem to break anything. Perhaps there is something more than the "17th byte" thing (like structure padding that may vary depending on the compiler and architecture)?

Fine, I guess the reason for this mystery is I "push" this branch to github and you haven't done it. That may not be due to the platform. Because I can see no this bug happening when I use a new git repo without "git push", and I test in archlinux or deepin, this bug will happen in these environments.

I take back what I said before, maybe this is really related to push.

Let's see what happens:

#!/bin/sh
mkdir test
cd test
git init
echo 1>1
git add .
git branch -M main
git commit -m "test"
git remote add origin nowhere
git config branch.main.remote origin
git config branch.main.merge refs/heads/main
git for-each-ref --format="%(color:#aa22ac)%(objectname)"

These two "git config" is for simulating a push environment.

I guess you also saw this bug:

BUG: ref-filter.c:1544: unhandled RR_* enum

I will use it for the test.

Thanks.
--
ZheNing Hu

Sorry, you lost me. I was talking about what happens in the new test you added to t6302 not failing as designed, and there shouldn't be "I've pushed but you haven't pushed to GitHub" distinction. The test is running in a brand-new repository just created for the sole purpose of running the test after all.

So in short, the test script added to t6302 in the v4 patch was not testing what it was supposed to be testing, as it didn't have the configuration items related to %(push) atom necessary to trigger the error?

That I can believe. I was starting to worry if there was something more subtle going on, but I am glad that it was only an uncooked patch submitted without checking.

No, I didn't. I just tried to make sure the new test was truly checking the existing breakage by partially reverting the code fix, and saw that the new test did not fail.

Thanks.

Well, there are some expression problems here, I just want to express:
This bug is only triggered after I push and Git adds some config during the push process. And then those config effort %(push) behavior.

Truly.

Me too, I want to complain a little bit: this bug is too difficult to locate.

I understand.

Thanks.
--
ZheNing Hu

From: ZheNing Hu <[email protected]>

used_atom.u is an union, and it has different members depending on what atom the auxiliary data the union part of the "struct used_atom" wants to record. At most only one of the members can be valid at any one time. Since the code checks u.remote_ref without even making sure if the atom is "push" or "push:" (which are only two cases that u.remote_ref.push becomes valid), but u.remote_ref shares the same storage for other members of the union, the check was reading from an invalid member, which was the bug.

Modify the condition here to check whether the atom name equals to "push" or starts with "push:", to avoid reading the value of invalid member of the union.

---
[GSOC] ref-filter: fix read invalid union member bug

I and Junio discussed the situation that this bug might actually occur.

The damage that can be found currently is using %(colors:#aa22ac) or some other %(colors) atoms. But Junio found that testing %(colors:#aa22ac) alone did not show the expected bug in the commit before the repair.

So I conducted an experiment:

When we use git push, Git will add some config, these configurations will affect the result of the execution process related to atom %(push) in populate_value().

Change from last version: added a new test, which added two configurations:

git config branch.main.remote origin git config branch.main.merge refs/heads/main

used to simulate the configuration changes brought by git push.

Finally, a test on the broken atom %(colors:#aa22ac). In the commit before the repair, breakage occurs. In the commit after the repair, breakage disappeared.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-949%2Fadlternative%2Fref-filter-enum-bug-fix-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-949/adlternative/ref-filter-enum-bug-fix-v5
Pull-Request: https://github.com/gitgitgadget/git/pull/949

Range-diff vs v4:

1: 8c6c0368a590 ! 1: b546477e8c87 [GSOC] ref-filter: fix read invalid union member bug

     @@ t/t6302-for-each-ref-filter.sh: test_expect_success '%(color) must fail' '
       	test_must_fail git for-each-ref --format="%(color)%(refname)"
       '
       
     -+test_expect_success '%(color:#aa22ac) must success' '
     ++test_expect_success '%(color:#aa22ac) must successed' '
     ++	test_when_finished "cd .. && rm -rf ./test" &&
     ++	mkdir test &&
     ++	cd test &&
     ++	git init &&
      +	cat >expect <<-\EOF &&
      +	refs/heads/main
     -+	refs/heads/side
     -+	refs/odd/spot
     -+	refs/tags/annotated-tag
     -+	refs/tags/doubly-annotated-tag
     -+	refs/tags/doubly-signed-tag
     -+	refs/tags/four
     -+	refs/tags/one
     -+	refs/tags/signed-tag
     -+	refs/tags/three
     -+	refs/tags/two
      +	EOF
     ++	git add . &&
     ++	git branch -M main &&
     ++	git commit -m "test" &&
     ++	git remote add origin nowhere &&
     ++	git config branch.main.remote origin &&
     ++	git config branch.main.merge refs/heads/main &&
      +	git for-each-ref --format="%(color:#aa22ac)%(refname)" >actual &&
      +	test_cmp expect actual
      +'

 ref-filter.c                   |  2 +-
 t/t6302-for-each-ref-filter.sh | 18 ++++++++++++++++++
 2 files changed, 19 insertions(+), 1 deletion(-)

index a0adb4551d87..213d3773ada3 100644
@@ -1730,7 +1730,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)
 			else
 				v->s = xstrdup("");
 			continue;
-		} else if (atom->u.remote_ref.push) {
+		} else if (!strcmp(atom->name, "push") || starts_with(atom->name, "push:")) {
 			const char *branch_name;
 			v->s = xstrdup("");
 			if (!skip_prefix(ref->refname, "refs/heads/",

index 9866b1b57368..309cf699506f 100755
@@ -117,6 +117,24 @@ test_expect_success '%(color) must fail' '
 	test_must_fail git for-each-ref --format="%(color)%(refname)"
 '
 
+test_expect_success '%(color:#aa22ac) must successed' '
+	test_when_finished "cd .. && rm -rf ./test" &&
+	mkdir test &&
+	cd test &&
+	git init &&
+	cat >expect <<-\EOF &&
+	refs/heads/main
+	EOF
+	git add . &&
+	git branch -M main &&
+	git commit -m "test" &&
+	git remote add origin nowhere &&
+	git config branch.main.remote origin &&
+	git config branch.main.merge refs/heads/main &&
+	git for-each-ref --format="%(color:#aa22ac)%(refname)" >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'left alignment is default' '
 	cat >expect <<-\EOF &&
 	refname is refs/heads/main    |refs/heads/main

base-commit: 311531c9de557d25ac087c1637818bd2aad6eb3a

gitgitgadget

Thanks.

	git init test &&
	(
		cd test &&
		test_commit initial &&
		git branch -M main &&
		cat >expect <<-\EOF &&
		refs/heads/main
                refs/tags/initial
		EOF
		git remote add origin nowhere &&
		...

OK.

I think "git branch -M" before "git commit" also will be fine. Of course it's okay to put it after commit and before "test_cmp".

I get it now. By executing cd in the subshell, the "cd .." step can be omitted.

Hh, real knowledge comes from practice. Yesterday, I tried it on the old version of git on some classmates’ computers, and there was an error, so using "git branch -M main" can avoid this error.

Thanks!
--
ZheNing Hu

From: ZheNing Hu <[email protected]>

used_atom.u is an union, and it has different members depending on what atom the auxiliary data the union part of the "struct used_atom" wants to record. At most only one of the members can be valid at any one time. Since the code checks u.remote_ref without even making sure if the atom is "push" or "push:" (which are only two cases that u.remote_ref.push becomes valid), but u.remote_ref shares the same storage for other members of the union, the check was reading from an invalid member, which was the bug.

Modify the condition here to check whether the atom name equals to "push" or starts with "push:", to avoid reading the value of invalid member of the union.

---
[GSOC] ref-filter: fix read invalid union member bug

Change from last version: Modify the test content to make the test more correct and complete.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-949%2Fadlternative%2Fref-filter-enum-bug-fix-v6
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-949/adlternative/ref-filter-enum-bug-fix-v6
Pull-Request: https://github.com/gitgitgadget/git/pull/949

Range-diff vs v5:

1: b546477e8c87 ! 1: 518cc41a78a4 [GSOC] ref-filter: fix read invalid union member bug

     @@ t/t6302-for-each-ref-filter.sh: test_expect_success '%(color) must fail' '
       	test_must_fail git for-each-ref --format="%(color)%(refname)"
       '
       
     -+test_expect_success '%(color:#aa22ac) must successed' '
     -+	test_when_finished "cd .. && rm -rf ./test" &&
     ++test_expect_success '%(color:#aa22ac) must succeed' '
      +	mkdir test &&
     -+	cd test &&
     -+	git init &&
     -+	cat >expect <<-\EOF &&
     -+	refs/heads/main
     -+	EOF
     -+	git add . &&
     -+	git branch -M main &&
     -+	git commit -m "test" &&
     -+	git remote add origin nowhere &&
     -+	git config branch.main.remote origin &&
     -+	git config branch.main.merge refs/heads/main &&
     -+	git for-each-ref --format="%(color:#aa22ac)%(refname)" >actual &&
     -+	test_cmp expect actual
     ++	git init test &&
     ++	(
     ++		cd test &&
     ++		test_commit initial &&
     ++		git branch -M main &&
     ++		cat >expect <<-\EOF &&
     ++		refs/heads/main
     ++		refs/tags/initial
     ++		EOF
     ++		git remote add origin nowhere &&
     ++		git config branch.main.remote origin &&
     ++		git config branch.main.merge refs/heads/main &&
     ++		git for-each-ref --format="%(color:#aa22ac)%(refname)" >actual &&
     ++		test_cmp expect actual
     ++	)
      +'
      +
       test_expect_success 'left alignment is default' '

 ref-filter.c                   |  2 +-
 t/t6302-for-each-ref-filter.sh | 19 +++++++++++++++++++
 2 files changed, 20 insertions(+), 1 deletion(-)

index a0adb4551d87..213d3773ada3 100644
@@ -1730,7 +1730,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)
 			else
 				v->s = xstrdup("");
 			continue;
-		} else if (atom->u.remote_ref.push) {
+		} else if (!strcmp(atom->name, "push") || starts_with(atom->name, "push:")) {
 			const char *branch_name;
 			v->s = xstrdup("");
 			if (!skip_prefix(ref->refname, "refs/heads/",

index 9866b1b57368..b64abe4184b8 100755
@@ -117,6 +117,25 @@ test_expect_success '%(color) must fail' '
 	test_must_fail git for-each-ref --format="%(color)%(refname)"
 '
 
+test_expect_success '%(color:#aa22ac) must succeed' '
+	mkdir test &&
+	git init test &&
+	(
+		cd test &&
+		test_commit initial &&
+		git branch -M main &&
+		cat >expect <<-\EOF &&
+		refs/heads/main
+		refs/tags/initial
+		EOF
+		git remote add origin nowhere &&
+		git config branch.main.remote origin &&
+		git config branch.main.merge refs/heads/main &&
+		git for-each-ref --format="%(color:#aa22ac)%(refname)" >actual &&
+		test_cmp expect actual
+	)
+'
+
 test_expect_success 'left alignment is default' '
 	cat >expect <<-\EOF &&
 	refname is refs/heads/main    |refs/heads/main

base-commit: 311531c9de557d25ac087c1637818bd2aad6eb3a

gitgitgadget

Make sence.

Well, deleting or not deleting the directory "test" is not a big problem here. Delete it is ok.

Thanks.
--
ZheNing Hu

From: ZheNing Hu <[email protected]>

used_atom.u is an union, and it has different members depending on what atom the auxiliary data the union part of the "struct used_atom" wants to record. At most only one of the members can be valid at any one time. Since the code checks u.remote_ref without even making sure if the atom is "push" or "push:" (which are only two cases that u.remote_ref.push becomes valid), but u.remote_ref shares the same storage for other members of the union, the check was reading from an invalid member, which was the bug.

Modify the condition here to check whether the atom name equals to "push" or starts with "push:", to avoid reading the value of invalid member of the union.

---
[GSOC] ref-filter: fix read invalid union member bug

Change from last version: Modify the test content, reduce additional 'mkdir' operation and remove temp-dir "test" when finished.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-949%2Fadlternative%2Fref-filter-enum-bug-fix-v7
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-949/adlternative/ref-filter-enum-bug-fix-v7
Pull-Request: https://github.com/gitgitgadget/git/pull/949

Range-diff vs v6:

1: 518cc41a78a4 ! 1: 50bef8439a85 [GSOC] ref-filter: fix read invalid union member bug

     @@ t/t6302-for-each-ref-filter.sh: test_expect_success '%(color) must fail' '
       '
       
      +test_expect_success '%(color:#aa22ac) must succeed' '
     -+	mkdir test &&
     ++	test_when_finished rm -fr test &&
      +	git init test &&
      +	(
      +		cd test &&

 ref-filter.c                   |  2 +-
 t/t6302-for-each-ref-filter.sh | 19 +++++++++++++++++++
 2 files changed, 20 insertions(+), 1 deletion(-)

index a0adb4551d87..213d3773ada3 100644
@@ -1730,7 +1730,7 @@ static int populate_value(struct ref_array_item *ref, struct strbuf *err)
 			else
 				v->s = xstrdup("");
 			continue;
-		} else if (atom->u.remote_ref.push) {
+		} else if (!strcmp(atom->name, "push") || starts_with(atom->name, "push:")) {
 			const char *branch_name;
 			v->s = xstrdup("");
 			if (!skip_prefix(ref->refname, "refs/heads/",

index 9866b1b57368..5070d0cf1e80 100755
@@ -117,6 +117,25 @@ test_expect_success '%(color) must fail' '
 	test_must_fail git for-each-ref --format="%(color)%(refname)"
 '
 
+test_expect_success '%(color:#aa22ac) must succeed' '
+	test_when_finished rm -fr test &&
+	git init test &&
+	(
+		cd test &&
+		test_commit initial &&
+		git branch -M main &&
+		cat >expect <<-\EOF &&
+		refs/heads/main
+		refs/tags/initial
+		EOF
+		git remote add origin nowhere &&
+		git config branch.main.remote origin &&
+		git config branch.main.merge refs/heads/main &&
+		git for-each-ref --format="%(color:#aa22ac)%(refname)" >actual &&
+		test_cmp expect actual
+	)
+'
+
 test_expect_success 'left alignment is default' '
 	cat >expect <<-\EOF &&
 	refname is refs/heads/main    |refs/heads/main

base-commit: 311531c9de557d25ac087c1637818bd2aad6eb3a

gitgitgadget