Skip to contentSkip to conversation

gc: fix handling of crontab magic markers

PATCH In review Martin Ågren 4 participants 10 messages Dec 21, 2020 – Dec 23, 2020 Documentation

lore
Share & tools
lore
#0 lore raw

cover gc: fix handling of crontab magic markers

This is a fix for a new bug in the v2.30 cycle:

git maintenance [start|stop] add and/or remove entries to/from the user's cron table. When inserting entries, we add magic "BEGIN" and "END" markers. When we later removes these entries, we remove everything from "BEGIN" to ... the end of file. A realistic scenario for hitting this bug is

  $ git maintenance start
  $ crontab -e # add stuff at the end of the file
  $ git maintenance stop
  $ crontab -l # stuff from above is gone

The second patch is the actual fix. The first patch is just a very minor fix to the documentation. The third patch future-proofs the magic markers, so that we can be reasonably sure that all future versions of Git stick to these strings.

Martin Ågren (3):
git-maintenance.txt: add missing word
gc: fix handling of crontab magic markers
t7900-maintenance: test for magic markers

 Documentation/git-maintenance.txt |  2 +-
 t/t7900-maintenance.sh            | 16 ++++++++++++++++
 builtin/gc.c                      |  7 +++----
 3 files changed, 20 insertions(+), 5 deletions(-)
Signature · 1 line
2.30.0.rc1
#1 lore raw

PATCH 1/3 git-maintenance.txt: add missing word

Add a missing "a" before "bunch".

---
 Documentation/git-maintenance.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/git-maintenance.txt +1 -1
index 6fec1eb8dc..d1f9b5172d 100644
@@ -101,7 +101,7 @@ This is done to avoid disrupting the remote-tracking branches. The end users
 expect these refs to stay unmoved unless they initiate a fetch.  With prefetch
 task, however, the objects necessary to complete a later real fetch would
 already be obtained, so the real fetch would go faster.  In the ideal case,
-it will just become an update to bunch of remote-tracking branches without
+it will just become an update to a bunch of remote-tracking branches without
 any object transfer.
 
 gc::
Signature · 1 line
2.30.0.rc1
#2 lore raw

PATCH 2/3 gc: fix handling of crontab magic markers

In reply to Martin Ågren

On git maintenance start, we add a few entries to the user's cron table. We wrap our entries using two magic markers, "# BEGIN GIT MAINTENANCE SCHEDULE" and "# END GIT MAINTENANCE SCHEDULE". At a later git maintenance stop, we will go through the table and remove these lines. Or rather, we will remove the "BEGIN" marker, the "END" marker and everything between them.

Alas, we have a bug in how we detect the "END" marker: we don't. As we loop through all the lines of the crontab, if we are in the "old region", i.e., the region we're aiming to remove, we make an early continue and don't get as far as checking for the "END" marker. Thus, once we've seen our "BEGIN", we remove everything until the end of the file.

Rewrite the logic for identifying these markers. There are four cases that are mutually exclusive: The current line starts a region or it ends it, or it's firmly within the region, or it's outside of it (and should be printed).

---
 t/t7900-maintenance.sh | 7 +++++++
 builtin/gc.c           | 7 +++----
 2 files changed, 10 insertions(+), 4 deletions(-)
t/t7900-maintenance.sh +7 -0
index d1e0c8f830..4bbfce31e9 100755
@@ -446,6 +446,13 @@ test_expect_success 'start preserves existing schedule' '
 	grep "Important information!" cron.txt
 '
 
+test_expect_success 'stop preserves surrounding schedule' '
+	echo "Crucial information!" >>cron.txt &&
+	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
+	grep "Important information!" cron.txt &&
+	grep "Crucial information!" cron.txt
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
builtin/gc.c +3 -4
index b57fda4924..4c24f41852 100644
@@ -1554,11 +1554,10 @@ static int update_background_schedule(int run_maintenance)
 	while (!strbuf_getline_lf(&line, cron_list)) {
 		if (!in_old_region && !strcmp(line.buf, BEGIN_LINE))
 			in_old_region = 1;
-		if (in_old_region)
-			continue;
-		fprintf(cron_in, "%s\n", line.buf);
-		if (in_old_region && !strcmp(line.buf, END_LINE))
+		else if (in_old_region && !strcmp(line.buf, END_LINE))
 			in_old_region = 0;
+		else if (!in_old_region)
+			fprintf(cron_in, "%s\n", line.buf);
 	}
 
 	if (run_maintenance) {
Signature · 1 line
2.30.0.rc1
#8 lore raw

PATCH 3/3 t7900-maintenance: test for magic markers

In reply to Martin Ågren

When we insert our "BEGIN" and "END" markers into the cron table, it's so that a Git version from many years into the future would be able to identify this region in the cron table. Let's add a test to make sure that these markers don't ever change.

---
 t/t7900-maintenance.sh | 9 +++++++++
 1 file changed, 9 insertions(+)
t/t7900-maintenance.sh +9 -0
index 4bbfce31e9..99bf0c7582 100755
@@ -446,6 +446,15 @@ test_expect_success 'start preserves existing schedule' '
 	grep "Important information!" cron.txt
 '
 
+test_expect_success 'magic markers are correct' '
+	grep "GIT MAINTENANCE SCHEDULE" cron.txt >actual &&
+	cat >expect <<-\EOF &&
+	# BEGIN GIT MAINTENANCE SCHEDULE
+	# END GIT MAINTENANCE SCHEDULE
+	EOF
+	test_cmp actual expect
+'
+
 test_expect_success 'stop preserves surrounding schedule' '
 	echo "Crucial information!" >>cron.txt &&
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
Signature · 1 line
2.30.0.rc1
#9 lore raw
In reply to Martin Ågren
Quoted 13 lines · Martin Ågren

This is a fix for a new bug in the v2.30 cycle:

git maintenance [start|stop] add and/or remove entries to/from the user's cron table. When inserting entries, we add magic "BEGIN" and "END" markers. When we later removes these entries, we remove everything from "BEGIN" to ... the end of file. A realistic scenario for hitting this bug is

  $ git maintenance start
  $ crontab -e # add stuff at the end of the file
  $ git maintenance stop
  $ crontab -l # stuff from above is gone
Quoted 4 lines

The second patch is the actual fix. The first patch is just a very minor fix to the documentation. The third patch future-proofs the magic markers, so that we can be reasonably sure that all future versions of Git stick to these strings.

Thank you for the attention to detail here. I think this series is succinct and completely correct. Apologies for missing the obvious "test after the region, too!"

Thanks,
-Stolee

#3 lore raw
In reply to Martin Ågren
Quoted 30 lines · Martin Ågren

On git maintenance start, we add a few entries to the user's cron table. We wrap our entries using two magic markers, "# BEGIN GIT MAINTENANCE SCHEDULE" and "# END GIT MAINTENANCE SCHEDULE". At a later git maintenance stop, we will go through the table and remove these lines. Or rather, we will remove the "BEGIN" marker, the "END" marker and everything between them.

Alas, we have a bug in how we detect the "END" marker: we don't. As we loop through all the lines of the crontab, if we are in the "old region", i.e., the region we're aiming to remove, we make an early continue and don't get as far as checking for the "END" marker. Thus, once we've seen our "BEGIN", we remove everything until the end of the file.

Rewrite the logic for identifying these markers. There are four cases that are mutually exclusive: The current line starts a region or it ends it, or it's firmly within the region, or it's outside of it (and should be printed).

---
 t/t7900-maintenance.sh | 7 +++++++
 builtin/gc.c           | 7 +++----
 2 files changed, 10 insertions(+), 4 deletions(-)
t/t7900-maintenance.sh +0 -0
index d1e0c8f830..4bbfce31e9 100755

t/t7900-maintenance.sh test_expect_success 'start preserves existing schedule' ' · line 451

Quoted 6 lines
t/t7900-maintenance.sh +3 -0
@@ -446,6 +446,13 @@ test_expect_success 'start preserves existing schedule' '
 	grep "Important information!" cron.txt
 '
 
+test_expect_success 'stop preserves surrounding schedule' '
+	echo "Crucial information!" >>cron.txt &&
+	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&

31345d55 (maintenance: extract platform-specific scheduling, 2020-11-24) in ds/maintenance-part-4 needs to adjust this exported variable for the tests to pass in 'seen'

Is it just the matter of replacing it with

GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab ..."

or is there more to it?

Thanks

#4 lore raw
Quoted 8 lines · Junio C Hamano

t/t7900-maintenance.sh test_expect_success 'start preserves existing schedule' ' · line 451

Quoted 3 lines
t/t7900-maintenance.sh +3 -0
+test_expect_success 'stop preserves surrounding schedule' '
+	echo "Crucial information!" >>cron.txt &&
+	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&

31345d55 (maintenance: extract platform-specific scheduling, 2020-11-24) in ds/maintenance-part-4 needs to adjust this exported variable for the tests to pass in 'seen'

Quoted 5 lines

Is it just the matter of replacing it with

GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab ..."

or is there more to it?

We'll see soon enough; I noticed the above while viewing

    https://github.com/git/git/runs/1597371646#step:6:2093

and since then I pushed another round of 'seen' with a non-textual conflict resolution while merging the ds/maintenance-part-4 topic in.

#5 lore raw
In reply to Junio C Hamano
Quoted 5 lines · Junio C Hamano

t/t7900-maintenance.sh test_expect_success 'start preserves existing schedule' ' · line 451

Quoted 3 lines · Martin Ågren
t/t7900-maintenance.sh +3 -0
+test_expect_success 'stop preserves surrounding schedule' '
+     echo "Crucial information!" >>cron.txt &&
+     GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
Quoted 7 lines

31345d55 (maintenance: extract platform-specific scheduling, 2020-11-24) in ds/maintenance-part-4 needs to adjust this exported variable for the tests to pass in 'seen'

Is it just the matter of replacing it with
GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab ..."
or is there more to it?

Yes, renaming GIT_TEST_CRONTAB to GIT_TEST_MAINT_SCHEDULER and prepending "crontab:" to the value should be sufficient (per the proposal by [1] and realization by [2]).

[1]: https://lore.kernel.org/git/[email protected]/
[2]: https://lore.kernel.org/git/4807342b0019be29bb369ed3403a485f0ce9c15d.1605647598.git.gitgitgadget@gmail.com/

#6 lore raw
Quoted 7 lines · Eric Sunshine
Quoted 5 lines · Junio C Hamano
Quoted 3 lines · Martin Ågren
t/t7900-maintenance.sh +3 -0
+test_expect_success 'stop preserves surrounding schedule' '
+     echo "Crucial information!" >>cron.txt &&
+     GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
Quoted 7 lines
Quoted 7 lines

31345d55 (maintenance: extract platform-specific scheduling, 2020-11-24) in ds/maintenance-part-4 needs to adjust this exported variable for the tests to pass in 'seen'

Is it just the matter of replacing it with
GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab ..."
or is there more to it?

Oh, I never realized this could be a problem. My merge with seen had a textual conflict, but nothing difficult, and the tests passed, so I didn't even stop to think if there was more to it. I clearly didn't notice the environment variable changed both name and value.

Quoted 6 lines

Yes, renaming GIT_TEST_CRONTAB to GIT_TEST_MAINT_SCHEDULER and prepending "crontab:" to the value should be sufficient (per the proposal by [1] and realization by [2]).

[1]: https://lore.kernel.org/git/[email protected]/
[2]: https://lore.kernel.org/git/4807342b0019be29bb369ed3403a485f0ce9c15d.1605647598.git.gitgitgadget@gmail.com/

Thanks Junio and Eric for helping out.

Martin

#7 lore raw
Quoted 21 lines · Martin Ågren
Quoted 7 lines · Eric Sunshine
Quoted 5 lines · Junio C Hamano
Quoted 3 lines · Martin Ågren
t/t7900-maintenance.sh +3 -0
+test_expect_success 'stop preserves surrounding schedule' '
+     echo "Crucial information!" >>cron.txt &&
+     GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
Quoted 7 lines
Quoted 7 lines

31345d55 (maintenance: extract platform-specific scheduling, 2020-11-24) in ds/maintenance-part-4 needs to adjust this exported variable for the tests to pass in 'seen'

Is it just the matter of replacing it with
GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab ..."
or is there more to it?

Oh, I never realized this could be a problem. My merge with seen had a textual conflict, but nothing difficult, and the tests passed, so I didn't even stop to think if there was more to it. I clearly didn't notice the environment variable changed both name and value.

Quoted 8 lines
Quoted 6 lines

Yes, renaming GIT_TEST_CRONTAB to GIT_TEST_MAINT_SCHEDULER and prepending "crontab:" to the value should be sufficient (per the proposal by [1] and realization by [2]).

[1]: https://lore.kernel.org/git/[email protected]/
[2]: https://lore.kernel.org/git/4807342b0019be29bb369ed3403a485f0ce9c15d.1605647598.git.gitgitgadget@gmail.com/

Thanks Junio and Eric for helping out.

Thanks for coming up with the fix.