Skip to contentSkip to conversation

t/perf/p3400: speed up setup using fast-import

RFC PATCH Superseded Tian Yuchen 4 participants 14 messages Jan 26 – Jan 30 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 t/perf/p3400: speed up setup using fast-import

The setup phase in 't/perf/p3400-rebase.sh' generates 100 commits to simulate a noisy history. It currently uses a shell loop that invokes 'git add', 'git commit', 'test_seq', and 'sort' in each iteration. This makes the whole process slow.

Optimize the setup by using 'git fast-import' to generate the commit history in a single stream. Additionally, pre-compute the forward and reversed file contents to avoid repetitive execution of 'seq' and 'sort'.

Performance enhancement:
Before: 29.045s
After: 18.081s

Measured on Lenovo Yoga 2020, Ubuntu 24.04.

---
 t/perf/p3400-rebase.sh | 53 +++++++++++++++++++++++++++++-------------
 1 file changed, 37 insertions(+), 16 deletions(-)
t/perf/p3400-rebase.sh +37 -16
index e6b0277729..9251a09b4f 100755
@@ -9,25 +9,46 @@ test_expect_success 'setup rebasing on top of a lot of changes' '
 	git checkout -f -B base &&
 	git checkout -B to-rebase &&
 	git checkout -B upstream &&
-	for i in $(test_seq 100)
-	do
-		# simulate huge diffs
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i unrelated-file$i &&
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 | sort -nr >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i-reverse unrelated-file$i ||
-		return 1
-	done &&
+
+	test_seq 1000 >content_fwd &&
+	test_seq 1000 | sort -nr >content_rev &&
+
+	(
+		for i in $(test_seq 100)
+		do
+			echo "commit refs/heads/upstream" &&
+			echo "committer WGYDY <[email protected]> $i +0000" &&
+			echo "data <<EOF" &&
+			echo "commit$i" &&
+			echo "EOF" &&
+			
+			if test "$i" = 1; then
+				echo "from refs/heads/upstream^0"
+			fi &&
+
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_fwd &&
+			echo "EOF" &&
+
+			echo "commit refs/heads/upstream" &&
+			echo "committer WGYDY <[email protected]> $i +0000" &&
+			echo "data <<EOF" &&
+			echo "commit$i-reversed" &&
+			echo "EOF" &&
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_rev &&
+			echo "EOF" || return 1
+		done
+	) | git fast-import &&
+	
+	git checkout -f upstream &&
 	git checkout to-rebase &&
 	test_commit our-patch interesting-file
 '
-
 test_perf 'rebase on top of a lot of unrelated changes' '
 	git rebase --onto upstream HEAD^ &&
 	git rebase --onto base HEAD^
Signature · 1 line
2.43.0
#1 lore raw

I noticed that while the setup phase is much faster (as shown in the commit message), the actual rebase performance test regressed slightly by about 3 seconds (from ~2s to ~5s). I don't quite understand what led to this outcome.

Additionally, I haven't delved deeply into the technical details of git fast-import,
and I'm not entirely sure if my approach is reasonable and compliant with the specifications.

I would greatly appreciate any guidance from those knowledgeable in these area!

Regards,
Yuchen

#2 lore raw

Tian Yuchen <[email protected]> writes:

Quoted 10 lines

I noticed that while the setup phase is much faster (as shown in the commit message), the actual rebase performance test regressed slightly by about 3 seconds (from ~2s to ~5s). I don't quite understand what led to this outcome.

Additionally, I haven't delved deeply into the technical details of git fast-import,
and I'm not entirely sure if my approach is reasonable and compliant with the specifications.

Quoted 5 lines

I would greatly appreciate any guidance from those knowledgeable in these area!

Regards,
Yuchen

I highly suspect that git fast-import generate the packfile that is NOT delta compressed. According to git fast-import documentation:

Quoted 3 lines

...For this reason it is strongly recommended that users repack the repository with 'git repack -a -d' after fast-import completes, allowing Git to reorganize the packfiles for faster data access...

Could this be the reason why it's even slower than handling “loose objects”? I'm working
on it currently.

Regards,
Yuchen

#3 lore raw
Quoted 3 lines

...For this reason it is strongly recommended that users repack the repository with 'git repack -a -d' after fast-import completes, allowing Git to reorganize the packfiles for faster data access...

After adding 'git repack -a -d', the time does changed:
original(no change) patch v1 NEW
real 29.845 18.081 22.231
rebase 13.34 14.94 12.78

It seems that the 'git repack -a -d' line does reduce runtime. However, the total duration
has seen a noticeable performance regression. I'm still trying to figure it out.

With regards,
Yuchen

#4 lore raw

PATCH v2 1/1 t/perf/p3400: speed up setup using fast-import

In reply to Tian Yuchen

The setup phase in 't/perf/p3400-rebase.sh' generates 100 commits to simulate a noisy history. It currently uses a shell loop that invokes 'git add', 'git commit', 'test_seq', and 'sort' in each iteration. This incurs significant overhead due to repeated process spawning.

Optimize the setup by using 'git fast-import' to generate the commit history in a single stream. Additionally, pre-compute the forward and reversed file contents to avoid repetitive execution of 'seq' and 'sort'.

To ensure the test measures rebase performance against a consistent object layout (rather than the suboptimal pack/loose objects created by the raw import), perform a full repack (git repack -a -d) at the end of the setup.

This reduces the setup time significantly while maintaining the validity of the subsequent performance tests.

Performance enhancement:

          Real         Rebase
  Before: 29.045s      13.34s
  After:  22.231s      12.78s

Measured on Lenovo Yoga 2020, Ubuntu 24.04.

---

Changes since v1:
- Added 'git repack -a -d' at the end of the setup phase.
- This fixes a performance regression observed in the subsequent rebase test, ensuring the object layout is normalized before testing.
- Updated setup timing in the commit message to reflect the repack overhead.

 t/perf/p3400-rebase.sh | 54 +++++++++++++++++++++++++++++-------------
 1 file changed, 38 insertions(+), 16 deletions(-)
t/perf/p3400-rebase.sh +38 -16
index e6b0277729..9f4251aed6 100755
@@ -9,25 +9,47 @@ test_expect_success 'setup rebasing on top of a lot of changes' '
 	git checkout -f -B base &&
 	git checkout -B to-rebase &&
 	git checkout -B upstream &&
-	for i in $(test_seq 100)
-	do
-		# simulate huge diffs
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i unrelated-file$i &&
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 | sort -nr >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i-reverse unrelated-file$i ||
-		return 1
-	done &&
+
+	test_seq 1000 >content_fwd &&
+	test_seq 1000 | sort -nr >content_rev &&
+
+	(
+		for i in $(test_seq 100)
+		do
+			echo "commit refs/heads/upstream" &&
+			echo "committer WGYDY <[email protected]> $i +0000" &&
+			echo "data <<EOF" &&
+			echo "commit$i" &&
+			echo "EOF" &&
+			
+			if test "$i" = 1; then
+				echo "from refs/heads/upstream^0"
+			fi &&
+
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_fwd &&
+			echo "EOF" &&
+
+			echo "commit refs/heads/upstream" &&
+			echo "committer WGYDY <[email protected]> $i +0000" &&
+			echo "data <<EOF" &&
+			echo "commit$i-reversed" &&
+			echo "EOF" &&
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_rev &&
+			echo "EOF" || return 1
+		done
+	) | git fast-import &&
+
+	git repack -a -d &&
+	git checkout -f upstream &&
 	git checkout to-rebase &&
 	test_commit our-patch interesting-file
 '
-
 test_perf 'rebase on top of a lot of unrelated changes' '
 	git rebase --onto upstream HEAD^ &&
 	git rebase --onto base HEAD^
Signature · 1 line
2.43.0
#5 lore raw

Am 28.01.26 um 17:07 schrieb Tian Yuchen:

t/perf/p3400-rebase.sh test_expect_success 'setup rebasing on top of a lot of changes' ' · line 14

Quoted 2 lines
t/perf/p3400-rebase.sh +2 -0
+	test_seq 1000 >content_fwd &&
+	test_seq 1000 | sort -nr >content_rev &&

Remove another pipe with

	sort -nr content_fwd >content_rev &&

t/perf/p3400-rebase.sh test_expect_success 'setup rebasing on top of a lot of changes' ' · line 44

Quoted 7 lines
t/perf/p3400-rebase.sh +7 -0
+
+	(
+		for i in $(test_seq 100)
+		do
+			...
+			cat content_rev &&
+			echo "EOF" || return 1

This is an incorrect use of return outside a function. This must be || exit 1 (or just || exit) to actually break out of the loop.

Of course, the error code is ignored, because it is in the upstream of a pipe, so in order to fail the complete command, it would be necessary that the original error left an incorrect fast-import stream such that the down-stream git fast-import fails. Otherwise, the entire command can complete successfully in an unexpected way.

Maybe instead of a pipe, dump the stream into a temporary file, so that the loop's exit code can be captured?

t/perf/p3400-rebase.sh test_expect_success 'setup rebasing on top of a lot of changes' ' · line 46

Quoted 2 lines
t/perf/p3400-rebase.sh +2 -0
+		done
+	) | git fast-import &&

-- Hannes

#6 lore raw

Am 30.01.26 um 07:41 schrieb Johannes Sixt:

Quoted 11 lines

Am 28.01.26 um 17:07 schrieb Tian Yuchen:

t/perf/p3400-rebase.sh test_expect_success 'setup rebasing on top of a lot of changes' ' · line 44

Quoted 7 lines
t/perf/p3400-rebase.sh +7 -0
+
+	(
+		for i in $(test_seq 100)
+		do
+			...
+			cat content_rev &&
+			echo "EOF" || return 1

This is an incorrect use of return outside a function. This must be || exit 1 (or just || exit) to actually break out of the loop.

I am wrong here, because my earlier tests were too narrow. This use of return actually does work as intended, because it happens inside a function. Neither the subshell nor being in the upstream of a pipe negates this fact.

The rest of the comment still stands, though.

Quoted 11 lines

Of course, the error code is ignored, because it is in the upstream of a pipe, so in order to fail the complete command, it would be necessary that the original error left an incorrect fast-import stream such that the down-stream git fast-import fails. Otherwise, the entire command can complete successfully in an unexpected way.

Maybe instead of a pipe, dump the stream into a temporary file, so that the loop's exit code can be captured?

t/perf/p3400-rebase.sh test_expect_success 'setup rebasing on top of a lot of changes' ' · line 46

Quoted 2 lines
t/perf/p3400-rebase.sh +2 -0
+		done
+	) | git fast-import &&

-- Hannes

#7 lore raw
In reply to Johannes Sixt
Quoted 7 lines · Johannes Sixt

Am 28.01.26 um 17:07 schrieb Tian Yuchen:

t/perf/p3400-rebase.sh test_expect_success 'setup rebasing on top of a lot of changes' ' · line 14

Quoted 2 lines
t/perf/p3400-rebase.sh +2 -0
+	test_seq 1000 >content_fwd &&
+	test_seq 1000 | sort -nr >content_rev &&

Remove another pipe with

	sort -nr content_fwd >content_rev &&

I agree with the outcome (i.e., we do not run test_seq twice) but not necessarily with the motivation (i.e., we do not assume test_seq may fail and care about its failure code, so no need to avoid pipes for the sake of avoiding pipes).

Quoted 9 lines

t/perf/p3400-rebase.sh test_expect_success 'setup rebasing on top of a lot of changes' ' · line 44

Quoted 6 lines
t/perf/p3400-rebase.sh +6 -0
+	(
+		for i in $(test_seq 100)
+		do
+			...
+			cat content_rev &&
+			echo "EOF" || return 1

This is an incorrect use of return outside a function. This must be || exit 1 (or just || exit) to actually break out of the loop.

Good.

Quoted 8 lines

Of course, the error code is ignored, because it is in the upstream of a pipe, so in order to fail the complete command, it would be necessary that the original error left an incorrect fast-import stream such that the down-stream git fast-import fails. Otherwise, the entire command can complete successfully in an unexpected way.

Maybe instead of a pipe, dump the stream into a temporary file, so that the loop's exit code can be captured?

I like this suggestion. A temporary file, unless it is excessively large, has an additional advantage that it allows you to inspect after "git fast-import" dies on it.

Quoted 3 lines

t/perf/p3400-rebase.sh test_expect_success 'setup rebasing on top of a lot of changes' ' · line 46

Quoted 2 lines
t/perf/p3400-rebase.sh +2 -0
+		done
+	) | git fast-import &&

-- Hannes

A message that is not in the archive (sent off-list or not yet received): [email protected]

#9 lore raw

Am 30.01.26 um 17:39 schrieb Tian Yuchen:

Quoted 1 line

I will send V4 shortly.

In this case, please also undo the unnecessary removal of a blank line at the end of the patch.

-- Hannes

#10 lore raw
In reply to Tian Yuchen
Quoted 14 lines · Tian Yuchen

The setup phase in 't/perf/p3400-rebase.sh' generates 100 commits to simulate a noisy history. It currently uses a shell loop that invokes 'git add', 'git commit', 'test_seq', and 'sort' in each iteration. This incurs significant overhead due to repeated process spawning.

Optimize the setup by using 'git fast-import' to generate the commit history in a single stream. Additionally, pre-compute the forward and reversed file contents to avoid repetitive execution of 'seq' and 'sort'.

To ensure the test measures rebase performance against a consistent object layout (rather than the suboptimal pack/loose objects created by the raw import), perform a full repack (git repack -a -d) at the end of the setup.

Quoted 7 lines

This reduces the setup time significantly while maintaining the validity of the subsequent performance tests.

Performance enhancement:

           Real         Rebase
   Before: 29.045s      13.34s
   After:  22.231s      12.78s

That's a nice speedup in the test setup

Quoted 22 lines
t/perf/p3400-rebase.sh +0 -14
index e6b0277729..9f4251aed6 100755
@@ -9,25 +9,47 @@ test_expect_success 'setup rebasing on top of a lot of changes' '
  	git checkout -f -B base &&
  	git checkout -B to-rebase &&
  	git checkout -B upstream &&
-	for i in $(test_seq 100)
-	do
-		# simulate huge diffs
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i unrelated-file$i &&
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 | sort -nr >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i-reverse unrelated-file$i ||
-		return 1

t/perf/p3400-rebase.sh test_expect_success 'setup rebasing on top of a lot of changes' ' · line 20

Quoted 10 lines
t/perf/p3400-rebase.sh +9 -1
-	done &&
+
+	test_seq 1000 >content_fwd &&
+	test_seq 1000 | sort -nr >content_rev &&
+
+	(
+		for i in $(test_seq 100)
+		do
+			echo "commit refs/heads/upstream" &&
+			echo "committer WGYDY <[email protected]> $i +0000" &&

You can keep the same author and committer as the original with

	test_tick &&
	echo "author $GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL> $GIT_AUTHOR_DATE" &&
	echo "committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 

$GIT_COMMITTER_DATE" &&

here and below

Quoted 8 lines
t/perf/p3400-rebase.sh +8 -0
+			echo "data <<EOF" &&
+			echo "commit$i" &&
+			echo "EOF" &&
+			
+			if test "$i" = 1; then
+				echo "from refs/heads/upstream^0"
+			fi &&
+

t/perf/p3400-rebase.sh test_expect_success 'setup rebasing on top of a lot of changes' ' · line 38

Quoted 10 lines
t/perf/p3400-rebase.sh +10 -0
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_fwd &&
+			echo "EOF" &&
+
+			echo "commit refs/heads/upstream" &&
+			echo "committer WGYDY <[email protected]> $i +0000" &&
+			echo "data <<EOF" &&
+			echo "commit$i-reversed" &&

The commit message in the original is "commit$i-reverse", not "reversed"

t/perf/p3400-rebase.sh test_expect_success 'setup rebasing on top of a lot of changes' ' · line 45

Quoted 7 lines
t/perf/p3400-rebase.sh +7 -0
+			echo "EOF" &&
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_rev &&
+			echo "EOF" || return 1
+		done

As Johannes pointed out we'll ignore the any failure above. We can address that by adding "echo done" here and adding "--done" to "git fast-import" below. That will cause "git fast-import" to fail because if there is an error in the loop as the last line of input to fast-import will not be "done"

Thanks

Phillip

Quoted 11 lines
t/perf/p3400-rebase.sh +4 -1
+	) | git fast-import &&
+
+	git repack -a -d &&
+	git checkout -f upstream &&
  	git checkout to-rebase &&
  	test_commit our-patch interesting-file
  '
-
  test_perf 'rebase on top of a lot of unrelated changes' '
  	git rebase --onto upstream HEAD^ &&
  	git rebase --onto base HEAD^
#11 lore raw

Hi Johannes, Hi Phillip,

Thanks a lot for the detailed review and the great suggestions!

@Johannes: I will adopt the 'sort -nr content_fwd' optimization to save a process.

@Phillip: Your point about the pipe swallowing the exit code is very insightful. I will
implement the '--done' protocol to ensure any failure in the loop
causes the test to fail.

I will also update the author/committer generation to use test_tick and the standard
environment variables, and fix the "reversed" typo.

I will prepare and send a v3 patch shortly incorporating these changes.

Thanks,
Tian Yuchen

#12 lore raw

PATCH v3 1/1 t/perf/p3400: speed up setup using fast-import

In reply to Tian Yuchen

The setup phase in 't/perf/p3400-rebase.sh' generates 100 commits to simulate a noisy history. It currently uses a shell loop that invokes 'git add', 'git commit', 'test_seq', and 'sort' in each iteration. This incurs significant overhead due to repeated process spawning.

Optimize the setup by using 'git fast-import' to generate the commit history in a single stream. Additionally, pre-compute the forward and reversed file contents to avoid repetitive execution of 'seq' and 'sort'.

To ensure the test measures rebase performance against a consistent object layout (rather than the suboptimal pack/loose objects created by the raw import), perform a full repack (git repack -a -d) at the end of the setup.

This reduces the setup time significantly while maintaining the validity of the subsequent performance tests.

Performance enhancement (Average value of 5 tests):

           Real        Rebase
  Before: 29.045s      13.34s
  After:  22.431s      12.98s

Measured on Lenovo Yoga 2020, Ubuntu 24.04.

---

Changes since v2:
- Optimized content_rev generation by sorting content_fwd directly (Johannes Sixt).
- Used test_tick and standard $GIT_COMMITTER_* variables for consistency (Phillip Wood).
- Fixed typo "reversed" -> "reverse" to match original test (Phillip Wood).
- Added "done" command and used git fast-import --done to prevent errors in the loop being ignored by the pipe (Phillip Wood).

 t/perf/p3400-rebase.sh | 56 ++++++++++++++++++++++++++++++------------
 1 file changed, 40 insertions(+), 16 deletions(-)
t/perf/p3400-rebase.sh +40 -16
index e6b0277729..6bb58282d6 100755
@@ -9,25 +9,49 @@ test_expect_success 'setup rebasing on top of a lot of changes' '
 	git checkout -f -B base &&
 	git checkout -B to-rebase &&
 	git checkout -B upstream &&
-	for i in $(test_seq 100)
-	do
-		# simulate huge diffs
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i unrelated-file$i &&
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 | sort -nr >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i-reverse unrelated-file$i ||
-		return 1
-	done &&
+
+	test_seq 1000 >content_fwd &&
+	sort -nr content_fwd >content_rev &&
+
+	(
+		for i in $(test_seq 100)
+		do
+			test_tick &&
+			echo "commit refs/heads/upstream" &&
+			echo "committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE" &&
+			echo "data <<EOF" &&
+			echo "commit$i" &&
+			echo "EOF" &&
+			
+			if test "$i" = 1; then
+				echo "from refs/heads/upstream^0"
+			fi &&
+
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_fwd &&
+			echo "EOF" &&
+
+			echo "commit refs/heads/upstream" &&
+			echo "committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE" &&
+			echo "data <<EOF" &&
+			echo "commit$i-reverse" &&
+			echo "EOF" &&
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_rev &&
+			echo "EOF" || return 1
+		done &&
+		echo "done"
+	) | git fast-import --done &&
+
+	git repack -a -d &&
+	git checkout -f upstream &&
 	git checkout to-rebase &&
 	test_commit our-patch interesting-file
 '
-
 test_perf 'rebase on top of a lot of unrelated changes' '
 	git rebase --onto upstream HEAD^ &&
 	git rebase --onto base HEAD^
Signature · 1 line
2.43.0
#13 lore raw

PATCH v4 1/1 t/perf/p3400: speed up setup using fast-import

The setup phase in 't/perf/p3400-rebase.sh' generates 100 commits to simulate a noisy history. It currently uses a shell loop that invokes 'git add', 'git commit', 'test_seq', and 'sort' in each iteration. This incurs significant overhead due to repeated process spawning.

Optimize the setup by using 'git fast-import' to generate the commit history. Additionally, pre-compute the forward and reversed file contents to avoid repetitive execution of 'seq' and 'sort'.

To ensure the test measures rebase performance against a consistent object layout (rather than the suboptimal pack/loose objects created by the raw import), perform a full repack (git repack -a -d) at the end of the setup.

This reduces the setup time significantly while maintaining the validity of the subsequent performance tests.

Performance enhancement (Average value of 5 tests):

            Real        Rebase
  Before:  29.045s      13.34s
   After:  21.989s      12.84s

Measured on Lenovo Yoga 2020, Ubuntu 24.04.

---

Changes since v3:
- Refactored the fast-import generation to write to a temporary file (fast_import_stream) instead of using a pipe. This allows for better debugging if fast-import fails, as suggested by Johannes Sixt and endorsed by Junio C Hamano.
- Switched from return 1 to exit 1 inside the subshell to correctly propagate exit codes.
- Undid the unnecessary removal of a blank line.

 t/perf/p3400-rebase.sh | 53 ++++++++++++++++++++++++++++++------------
 1 file changed, 38 insertions(+), 15 deletions(-)
t/perf/p3400-rebase.sh +38 -15
index e6b0277729..425f484681 100755
@@ -9,21 +9,44 @@ test_expect_success 'setup rebasing on top of a lot of changes' '
 	git checkout -f -B base &&
 	git checkout -B to-rebase &&
 	git checkout -B upstream &&
-	for i in $(test_seq 100)
-	do
-		# simulate huge diffs
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i unrelated-file$i &&
-		echo change$i >unrelated-file$i &&
-		test_seq 1000 | sort -nr >>unrelated-file$i &&
-		git add unrelated-file$i &&
-		test_tick &&
-		git commit -m commit$i-reverse unrelated-file$i ||
-		return 1
-	done &&
+	test_seq 1000 >content_fwd &&
+	sort -nr content_fwd >content_rev &&
+	(
+		for i in $(test_seq 100)
+		do
+			test_tick &&
+			echo "commit refs/heads/upstream" &&
+			echo "committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE" &&
+			echo "data <<EOF" &&
+			echo "commit$i" &&
+			echo "EOF" &&
+			
+			if test "$i" = 1; then
+				echo "from refs/heads/upstream^0"
+			fi &&
+
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_fwd &&
+			echo "EOF" &&
+
+			echo "commit refs/heads/upstream" &&
+			echo "committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE" &&
+			echo "data <<EOF" &&
+			echo "commit$i-reverse" &&
+			echo "EOF" &&
+			echo "M 100644 inline unrelated-file$i" &&
+			echo "data <<EOF" &&
+			echo "change$i" &&
+			cat content_rev &&
+			echo "EOF" || exit 1
+		done
+	) >fast_import_stream &&
+
+	git fast-import <fast_import_stream &&
+	git repack -a -d &&
+	git checkout -f upstream &&
 	git checkout to-rebase &&
 	test_commit our-patch interesting-file
 '
Signature · 1 line
2.43.0
#14 lore raw
In reply to Tian Yuchen
Quoted 5 lines · Tian Yuchen

Changes since v2:
- Optimized content_rev generation by sorting content_fwd directly (Johannes Sixt).
- Used test_tick and standard $GIT_COMMITTER_* variables for consistency (Phillip Wood).
- Fixed typo "reversed" -> "reverse" to match original test (Phillip Wood).
- Added "done" command and used git fast-import --done to prevent errors in the loop being ignored by the pipe (Phillip Wood).

Thanks. Will replace.