gc: fix handling of crontab magic markers
cover gc: fix handling of crontab magic markers
This is a fix for a new bug in the v2.30 cycle:
git maintenance [start|stop] add and/or remove entries to/from the user's cron table. When inserting entries, we add magic "BEGIN" and "END" markers. When we later removes these entries, we remove everything from "BEGIN" to ... the end of file. A realistic scenario for hitting this bug is
$ git maintenance start $ crontab -e # add stuff at the end of the file $ git maintenance stop $ crontab -l # stuff from above is gone
The second patch is the actual fix. The first patch is just a very minor fix to the documentation. The third patch future-proofs the magic markers, so that we can be reasonably sure that all future versions of Git stick to these strings.
Martin Ågren (3):
git-maintenance.txt: add missing word
gc: fix handling of crontab magic markers
t7900-maintenance: test for magic markers
Documentation/git-maintenance.txt | 2 +- t/t7900-maintenance.sh | 16 ++++++++++++++++ builtin/gc.c | 7 +++---- 3 files changed, 20 insertions(+), 5 deletions(-)
Signature · 1 line
2.30.0.rc1
PATCH 1/3 git-maintenance.txt: add missing word
Add a missing "a" before "bunch".
--- Documentation/git-maintenance.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/git-maintenance.txt +1 -1
@@ -101,7 +101,7 @@ This is done to avoid disrupting the remote-tracking branches. The end users expect these refs to stay unmoved unless they initiate a fetch. With prefetch task, however, the objects necessary to complete a later real fetch would already be obtained, so the real fetch would go faster. In the ideal case, -it will just become an update to bunch of remote-tracking branches without +it will just become an update to a bunch of remote-tracking branches without any object transfer. gc::
Signature · 1 line
2.30.0.rc1
PATCH 2/3 gc: fix handling of crontab magic markers
In reply to Martin ÅgrenOn git maintenance start, we add a few entries to the user's cron table. We wrap our entries using two magic markers, "# BEGIN GIT MAINTENANCE SCHEDULE" and "# END GIT MAINTENANCE SCHEDULE". At a later git maintenance stop, we will go through the table and remove these lines. Or rather, we will remove the "BEGIN" marker, the "END" marker and everything between them.
Alas, we have a bug in how we detect the "END" marker: we don't. As we loop through all the lines of the crontab, if we are in the "old region", i.e., the region we're aiming to remove, we make an early continue and don't get as far as checking for the "END" marker. Thus, once we've seen our "BEGIN", we remove everything until the end of the file.
Rewrite the logic for identifying these markers. There are four cases that are mutually exclusive: The current line starts a region or it ends it, or it's firmly within the region, or it's outside of it (and should be printed).
--- t/t7900-maintenance.sh | 7 +++++++ builtin/gc.c | 7 +++---- 2 files changed, 10 insertions(+), 4 deletions(-)
t/t7900-maintenance.sh +7 -0
@@ -446,6 +446,13 @@ test_expect_success 'start preserves existing schedule' ' grep "Important information!" cron.txt ' +test_expect_success 'stop preserves surrounding schedule' ' + echo "Crucial information!" >>cron.txt && + GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop && + grep "Important information!" cron.txt && + grep "Crucial information!" cron.txt +' + test_expect_success 'register preserves existing strategy' ' git config maintenance.strategy none && git maintenance register &&
builtin/gc.c +3 -4
@@ -1554,11 +1554,10 @@ static int update_background_schedule(int run_maintenance)while (!strbuf_getline_lf (&line, cron_list)) {if (!in_old_region && !strcmp (line.buf,BEGIN_LINE )) in_old_region =1 ; -if (in_old_region) -continue ; -fprintf (cron_in,"%s\n" , line.buf); -if (in_old_region && !strcmp (line.buf,END_LINE )) +else if (in_old_region && !strcmp (line.buf,END_LINE )) in_old_region =0 ; +else if (!in_old_region) +fprintf (cron_in,"%s\n" , line.buf); }if (run_maintenance) {
Signature · 1 line
2.30.0.rc1
Quoted 30 lines · Martin Ågren
On
git maintenance start, we add a few entries to the user's cron table. We wrap our entries using two magic markers, "# BEGIN GIT MAINTENANCE SCHEDULE" and "# END GIT MAINTENANCE SCHEDULE". At a latergit maintenance stop, we will go through the table and remove these lines. Or rather, we will remove the "BEGIN" marker, the "END" marker and everything between them.Alas, we have a bug in how we detect the "END" marker: we don't. As we loop through all the lines of the crontab, if we are in the "old region", i.e., the region we're aiming to remove, we make an early
continueand don't get as far as checking for the "END" marker. Thus, once we've seen our "BEGIN", we remove everything until the end of the file.Rewrite the logic for identifying these markers. There are four cases that are mutually exclusive: The current line starts a region or it ends it, or it's firmly within the region, or it's outside of it (and should be printed).
Signed-off-by Martin Ågren <[email protected]>--- t/t7900-maintenance.sh | 7 +++++++ builtin/gc.c | 7 +++---- 2 files changed, 10 insertions(+), 4 deletions(-)t/t7900-maintenance.sh +0
-0
t/t7900-maintenance.sh test_expect_success 'start preserves existing schedule' ' · line 451
Quoted 6 lines
t/t7900-maintenance.sh +3
-0@@ -446,6 +446,13 @@ test_expect_success 'start preserves existing schedule' ' grep "Important information!" cron.txt ' +test_expect_success 'stop preserves surrounding schedule' ' + echo "Crucial information!" >>cron.txt && + GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
31345d55 (maintenance: extract platform-specific scheduling, 2020-11-24) in ds/maintenance-part-4 needs to adjust this exported variable for the tests to pass in 'seen'
Is it just the matter of replacing it with
GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab ..."
or is there more to it?
Thanks
Quoted 8 lines · Junio C Hamano
t/t7900-maintenance.sh test_expect_success 'start preserves existing schedule' ' · line 451
Quoted 3 lines
t/t7900-maintenance.sh +3
-0+test_expect_success 'stop preserves surrounding schedule' ' + echo "Crucial information!" >>cron.txt && + GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
31345d55(maintenance: extract platform-specific scheduling, 2020-11-24) in ds/maintenance-part-4 needs to adjust this exported variable for the tests to pass in 'seen'
Quoted 5 lines
Is it just the matter of replacing it with
GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab ..."
or is there more to it?
We'll see soon enough; I noticed the above while viewing
https://github.com/git/git/runs/1597371646#step:6:2093
and since then I pushed another round of 'seen' with a non-textual conflict resolution while merging the ds/maintenance-part-4 topic in.
Quoted 5 lines · Junio C Hamano
t/t7900-maintenance.sh test_expect_success 'start preserves existing schedule' ' · line 451
Quoted 3 lines · Martin Ågren
t/t7900-maintenance.sh +3
-0+test_expect_success 'stop preserves surrounding schedule' ' + echo "Crucial information!" >>cron.txt && + GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
Quoted 7 lines
31345d55(maintenance: extract platform-specific scheduling, 2020-11-24) in ds/maintenance-part-4 needs to adjust this exported variable for the tests to pass in 'seen'Is it just the matter of replacing it with
GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab ..."
or is there more to it?
Yes, renaming GIT_TEST_CRONTAB to GIT_TEST_MAINT_SCHEDULER and prepending "crontab:" to the value should be sufficient (per the proposal by [1] and realization by [2]).
[1]: https://lore.kernel.org/git/[email protected]/
[2]: https://lore.kernel.org/git/4807342b0019be29bb369ed3403a485f0ce9c15d.1605647598.git.gitgitgadget@gmail.com/
Quoted 7 lines · Eric Sunshine
Quoted 5 lines · Junio C Hamano
Quoted 3 lines · Martin Ågren
t/t7900-maintenance.sh +3
-0+test_expect_success 'stop preserves surrounding schedule' ' + echo "Crucial information!" >>cron.txt && + GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
Quoted 7 lines
Quoted 7 lines
31345d55(maintenance: extract platform-specific scheduling, 2020-11-24) in ds/maintenance-part-4 needs to adjust this exported variable for the tests to pass in 'seen'Is it just the matter of replacing it with
GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab ..."
or is there more to it?
Oh, I never realized this could be a problem. My merge with seen had a textual conflict, but nothing difficult, and the tests passed, so I didn't even stop to think if there was more to it. I clearly didn't notice the environment variable changed both name and value.
Quoted 6 lines
Yes, renaming
GIT_TEST_CRONTABtoGIT_TEST_MAINT_SCHEDULERand prepending "crontab:" to the value should be sufficient (per the proposal by [1] and realization by [2]).[1]: https://lore.kernel.org/git/[email protected]/
[2]: https://lore.kernel.org/git/4807342b0019be29bb369ed3403a485f0ce9c15d.1605647598.git.gitgitgadget@gmail.com/
Thanks Junio and Eric for helping out.
Martin
Quoted 21 lines · Martin Ågren
Quoted 7 lines · Eric Sunshine
Quoted 5 lines · Junio C Hamano
Quoted 3 lines · Martin Ågren
t/t7900-maintenance.sh +3
-0+test_expect_success 'stop preserves surrounding schedule' ' + echo "Crucial information!" >>cron.txt && + GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&Quoted 7 lines
Quoted 7 lines
31345d55(maintenance: extract platform-specific scheduling, 2020-11-24) in ds/maintenance-part-4 needs to adjust this exported variable for the tests to pass in 'seen'Is it just the matter of replacing it with
GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab ..."
or is there more to it?Oh, I never realized this could be a problem. My merge with seen had a textual conflict, but nothing difficult, and the tests passed, so I didn't even stop to think if there was more to it. I clearly didn't notice the environment variable changed both name and value.
Quoted 8 lines
Quoted 6 lines
Yes, renaming
GIT_TEST_CRONTABtoGIT_TEST_MAINT_SCHEDULERand prepending "crontab:" to the value should be sufficient (per the proposal by [1] and realization by [2]).[1]: https://lore.kernel.org/git/[email protected]/
[2]: https://lore.kernel.org/git/4807342b0019be29bb369ed3403a485f0ce9c15d.1605647598.git.gitgitgadget@gmail.com/Thanks Junio and Eric for helping out.
Thanks for coming up with the fix.
PATCH 3/3 t7900-maintenance: test for magic markers
In reply to Martin ÅgrenWhen we insert our "BEGIN" and "END" markers into the cron table, it's so that a Git version from many years into the future would be able to identify this region in the cron table. Let's add a test to make sure that these markers don't ever change.
--- t/t7900-maintenance.sh | 9 +++++++++ 1 file changed, 9 insertions(+)
t/t7900-maintenance.sh +9 -0
@@ -446,6 +446,15 @@ test_expect_success 'start preserves existing schedule' ' grep "Important information!" cron.txt ' +test_expect_success 'magic markers are correct' ' + grep "GIT MAINTENANCE SCHEDULE" cron.txt >actual && + cat >expect <<-\EOF && + # BEGIN GIT MAINTENANCE SCHEDULE + # END GIT MAINTENANCE SCHEDULE + EOF + test_cmp actual expect +' + test_expect_success 'stop preserves surrounding schedule' ' echo "Crucial information!" >>cron.txt && GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
Signature · 1 line
2.30.0.rc1
Quoted 13 lines · Martin Ågren
This is a fix for a new bug in the v2.30 cycle:
git maintenance [start|stop]add and/or remove entries to/from the user's cron table. When inserting entries, we add magic "BEGIN" and "END" markers. When we later removes these entries, we remove everything from "BEGIN" to ... the end of file. A realistic scenario for hitting this bug is$ git maintenance start $ crontab -e # add stuff at the end of the file $ git maintenance stop $ crontab -l # stuff from above is gone
Quoted 4 lines
The second patch is the actual fix. The first patch is just a very minor fix to the documentation. The third patch future-proofs the magic markers, so that we can be reasonably sure that all future versions of Git stick to these strings.
Thank you for the attention to detail here. I think this series is succinct and completely correct. Apologies for missing the obvious "test after the region, too!"
Thanks,
-Stolee