merge-ours: sparse-index integration

This short series teaches merge-ours to work with a sparse index.

Patch 1 is a preparatory cleanup that converts merge-ours away from the_repository global, using the repo parameter instead.

Patch 2 adds the actual sparse-index integration and tests. Because merge-ours is invoked as a subprocess by git merge -s ours and never previously read config, the sparse-checkout globals remained unset, causing the index to be expanded unconditionally. A repo_config() call fixes this.

Developed with AI assistance (Claude).

Sam Bostock (2):
merge-ours: drop USE_THE_REPOSITORY_VARIABLE
merge-ours: integrate with sparse-index

 builtin/merge-ours.c                     | 15 +++++++++------
 t/t1092-sparse-checkout-compatibility.sh | 14 ++++++++++++++
 2 files changed, 23 insertions(+), 6 deletions(-)

base-commit: b2826b52eb7caff9f4ed6e85ec45e338bf02ad09
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2189%2Fsambostock%2Fsb%2Fmerge-ours-sparse-index-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2189/sambostock/sb/merge-ours-sparse-index-v1
Pull-Request: https://github.com/git/git/pull/2189

gitgitgadget

From: Sam Bostock <[email protected]>

Use the repo parameter passed to cmd_merge_ours() instead of the_repository, and drop the USE_THE_REPOSITORY_VARIABLE macro that is no longer needed.

While at it, remove a stray double blank line between the #include block and the usage string.

---
 builtin/merge-ours.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

index 97b8a792c7..2312e58ab3 100644
@@ -8,20 +8,17 @@
  * Pretend we resolved the heads, but declare our tree trumps everybody else.
  */
 
-#define USE_THE_REPOSITORY_VARIABLE
-
 #include "git-compat-util.h"
 #include "builtin.h"
 #include "diff.h"
 
-
 static const char builtin_merge_ours_usage[] =
 	"git merge-ours <base>... -- HEAD <remote>...";
 
 int cmd_merge_ours(int argc,
 		   const char **argv,
 		   const char *prefix UNUSED,
-		   struct repository *repo UNUSED)
+		   struct repository *repo)
 {
 	show_usage_if_asked(argc, argv, builtin_merge_ours_usage);
 

@@ -30,9 +27,9 @@ int cmd_merge_ours(int argc,
 	 * commit.  The index must match HEAD, or this merge cannot go
 	 * through.
 	 */
-	if (repo_read_index(the_repository) < 0)
+	if (repo_read_index(repo) < 0)
 		die_errno("read_cache failed");
-	if (index_differs_from(the_repository, "HEAD", NULL, 0))
+	if (index_differs_from(repo, "HEAD", NULL, 0))
 		return 2;
 	return 0;
 }

gitgitgadget

Nice to see that the required changes are this small, only :)

Indeed. It may be worth recording that reasoning in the log message.

From: Sam Bostock <[email protected]>

The merge-ours builtin reads the index only to compare it against HEAD via index_differs_from(), whose diff machinery (run_diff_index) is already sparse-aware.

Teach merge-ours to opt out of requiring a full index by setting command_requires_full_index to 0. Because merge-ours is invoked as a subprocess by "git merge -s ours" and never previously read config, the global variables core_apply_sparse_checkout and core_sparse_checkout_cone remained unset, causing is_sparse_index_allowed() to return false and the index to be expanded anyway. Add a repo_config() call with git_default_config to populate these globals.

Add tests to t1092 verifying that "git merge -s ours" produces identical results across full-checkout, sparse-checkout, and sparse-index modes, including verifying the resulting merge commit structure, and that the sparse index is not expanded during the operation.

---
 builtin/merge-ours.c                     |  6 ++++++
 t/t1092-sparse-checkout-compatibility.sh | 14 ++++++++++++++
 2 files changed, 20 insertions(+)

index 2312e58ab3..405b2989f7 100644
@@ -10,6 +10,8 @@
 
 #include "git-compat-util.h"
 #include "builtin.h"
+#include "config.h"
+#include "environment.h"
 #include "diff.h"
 
 static const char builtin_merge_ours_usage[] =

@@ -22,6 +24,10 @@ int cmd_merge_ours(int argc,
 {
 	show_usage_if_asked(argc, argv, builtin_merge_ours_usage);
 
+	repo_config(repo, git_default_config, NULL);
+	prepare_repo_settings(repo);
+	repo->settings.command_requires_full_index = 0;
+
 	/*
 	 * The contents of the current index becomes the tree we
 	 * commit.  The index must match HEAD, or this merge cannot go

index b0f691c151..d98cb4ac11 100755
@@ -2559,4 +2559,18 @@ test_expect_success 'cat-file --batch' '
 	ensure_expanded cat-file --batch <in
 '
 
+test_expect_success 'merge -s ours' '
+	init_repos &&
+
+	test_all_match git rev-parse HEAD^{tree} &&
+	test_all_match git merge -s ours merge-right &&
+	test_all_match git rev-parse HEAD^{tree} &&
+	test_all_match git rev-parse HEAD^2
+'
+
+test_expect_success 'sparse-index is not expanded: merge-ours' '
+	init_repos &&
+	ensure_not_expanded merge -s ours merge-right
+'
+
 test_done

gitgitgadget

It may be correct but I do not see a relevance

This may be correct, but it only becomes relevant after somebody decides to do something to cause is_sparse_index_allowed() to say Yes.

In total, while individual sentences in the above may tell correct things, the order of presentation makes it hard to understand, at least to me.

The usual way to compose a log message of this project is to

- Give an observation on how the current system works in the present tense (so no need to say "Currently X is Y", or "Previously X was Y" to describe the state before your change; just "X is Y" is enough), and discuss what you perceive as a problem in it.

- Propose a solution (optional---often, problem description trivially leads to an obvious solution in reader's minds).

- Give commands to somebody editing the codebase to "make it so", instead of saying "This commit does X".

in this order.

So perhaps

The merge-ours built-in opens the index to compare it against HEAD. The machinery used to do this (i.e. run_diff_index()) is capable of working with sparse index, but because of the start up sequence of this command does not take necessary steps, we end up first expanding the index fully before doing this comparison.

In order to convince sparse-index.c:is_sparse_index_allowed() to return true, we need to:

- enable the global switch "core_apply_sparse_checkout" via the core.sparsecheckout configuration variable. merge-ours currently do not even read basic configuration, so we need to make the configuration call ourselves.

- set command_requires_full_index to 0.

With that, the command can work without expanding the index fully before doing its work.

or something.

Thanks.

This short series teaches merge-ours to work with a sparse index as a small step toward broader sparse-index support.

Patch 1 is a preparatory cleanup that converts merge-ours away from the_repository global, using the repo parameter instead.

Patch 2 adds the actual sparse-index integration and tests.

Changes since v1:

* Patch 1: note in commit message that RUN_SETUP guarantees repo is never NULL (Patrick, Junio)
* Patch 2: rewrite commit message to follow the project's standard log message structure (Junio)

Thanks Junio and Patrick for the review.

Developed with AI assistance (Claude).

Sam Bostock (2):
merge-ours: drop USE_THE_REPOSITORY_VARIABLE
merge-ours: integrate with sparse-index

 builtin/merge-ours.c                     | 15 +++++++++------
 t/t1092-sparse-checkout-compatibility.sh | 14 ++++++++++++++
 2 files changed, 23 insertions(+), 6 deletions(-)

base-commit: b2826b52eb7caff9f4ed6e85ec45e338bf02ad09
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2189%2Fsambostock%2Fsb%2Fmerge-ours-sparse-index-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2189/sambostock/sb/merge-ours-sparse-index-v2
Pull-Request: https://github.com/git/git/pull/2189

Range-diff vs v1:

1: 6cb970e512 ! 1: 775d76df69 merge-ours: drop USE_THE_REPOSITORY_VARIABLE
@@ Metadata
## Commit message ##
merge-ours: drop USE_THE_REPOSITORY_VARIABLE

- Use the repo parameter passed to cmd_merge_ours() instead of
- the_repository, and drop the USE_THE_REPOSITORY_VARIABLE macro that
- is no longer needed.
+ The merge-ours built-in uses the the_repository global to access
+ the repository. The project is moving away from this global in favor
+ of the repo parameter that is passed to each built-in command.
+ Since merge-ours is registered with RUN_SETUP, repo is guaranteed
+ to be non-NULL and can be used directly.
+
+ Drop the USE_THE_REPOSITORY_VARIABLE macro and use repo throughout.

While at it, remove a stray double blank line between the #include block and the usage string.
2: 20b9e0bf6e ! 2: 55d39ff778 merge-ours: integrate with sparse-index
@@ Metadata
## Commit message ##
merge-ours: integrate with sparse-index

- The merge-ours builtin reads the index only to compare it against HEAD
- via index_differs_from(), whose diff machinery (run_diff_index) is
- already sparse-aware.
+ The merge-ours built-in opens the index to compare it against HEAD.
+ The machinery used to do this (i.e. run_diff_index()) is capable of
+ working with a sparse index, but the start-up sequence of this
+ command does not take the necessary steps, so we end up expanding the
+ index fully before doing the comparison.

- Teach merge-ours to opt out of requiring a full index by setting
- command_requires_full_index to 0. Because merge-ours is invoked as a
- subprocess by "git merge -s ours" and never previously read config,
- the global variables core_apply_sparse_checkout and
- core_sparse_checkout_cone remained unset, causing
- is_sparse_index_allowed() to return false and the index to be expanded
- anyway. Add a repo_config() call with git_default_config to populate
- these globals.
+ In order to convince sparse-index.c:is_sparse_index_allowed() to
+ return true, we need to:

- Add tests to t1092 verifying that "git merge -s ours" produces
- identical results across full-checkout, sparse-checkout, and
- sparse-index modes, including verifying the resulting merge commit
- structure, and that the sparse index is not expanded during the
- operation.
+ - Read basic configuration with git_default_config so that global
+ variables like core_apply_sparse_checkout are populated.
+ merge-ours currently does not read configuration at all.
+
+ - Set command_requires_full_index to 0.
+
+ With that, the command can work without expanding the index fully
+ before doing its work.

gitgitgadget

From: Sam Bostock <[email protected]>

The merge-ours built-in uses the the_repository global to access the repository. The project is moving away from this global in favor of the repo parameter that is passed to each built-in command. Since merge-ours is registered with RUN_SETUP, repo is guaranteed to be non-NULL and can be used directly.

Drop the USE_THE_REPOSITORY_VARIABLE macro and use repo throughout.

While at it, remove a stray double blank line between the #include block and the usage string.

---
 builtin/merge-ours.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

index 97b8a792c7..2312e58ab3 100644
@@ -8,20 +8,17 @@
  * Pretend we resolved the heads, but declare our tree trumps everybody else.
  */
 
-#define USE_THE_REPOSITORY_VARIABLE
-
 #include "git-compat-util.h"
 #include "builtin.h"
 #include "diff.h"
 
-
 static const char builtin_merge_ours_usage[] =
 	"git merge-ours <base>... -- HEAD <remote>...";
 
 int cmd_merge_ours(int argc,
 		   const char **argv,
 		   const char *prefix UNUSED,
-		   struct repository *repo UNUSED)
+		   struct repository *repo)
 {
 	show_usage_if_asked(argc, argv, builtin_merge_ours_usage);
 

@@ -30,9 +27,9 @@ int cmd_merge_ours(int argc,
 	 * commit.  The index must match HEAD, or this merge cannot go
 	 * through.
 	 */
-	if (repo_read_index(the_repository) < 0)
+	if (repo_read_index(repo) < 0)
 		die_errno("read_cache failed");
-	if (index_differs_from(the_repository, "HEAD", NULL, 0))
+	if (index_differs_from(repo, "HEAD", NULL, 0))
 		return 2;
 	return 0;
 }

gitgitgadget

From: Sam Bostock <[email protected]>

The merge-ours built-in opens the index to compare it against HEAD. The machinery used to do this (i.e. run_diff_index()) is capable of working with a sparse index, but the start-up sequence of this command does not take the necessary steps, so we end up expanding the index fully before doing the comparison.

In order to convince sparse-index.c:is_sparse_index_allowed() to return true, we need to:

- Read basic configuration with git_default_config so that global variables like core_apply_sparse_checkout are populated. merge-ours currently does not read configuration at all.

- Set command_requires_full_index to 0.

With that, the command can work without expanding the index fully before doing its work.

---
 builtin/merge-ours.c                     |  6 ++++++
 t/t1092-sparse-checkout-compatibility.sh | 14 ++++++++++++++
 2 files changed, 20 insertions(+)

index 2312e58ab3..405b2989f7 100644
@@ -10,6 +10,8 @@
 
 #include "git-compat-util.h"
 #include "builtin.h"
+#include "config.h"
+#include "environment.h"
 #include "diff.h"
 
 static const char builtin_merge_ours_usage[] =

@@ -22,6 +24,10 @@ int cmd_merge_ours(int argc,
 {
 	show_usage_if_asked(argc, argv, builtin_merge_ours_usage);
 
+	repo_config(repo, git_default_config, NULL);
+	prepare_repo_settings(repo);
+	repo->settings.command_requires_full_index = 0;
+
 	/*
 	 * The contents of the current index becomes the tree we
 	 * commit.  The index must match HEAD, or this merge cannot go

index b0f691c151..d98cb4ac11 100755
@@ -2559,4 +2559,18 @@ test_expect_success 'cat-file --batch' '
 	ensure_expanded cat-file --batch <in
 '
 
+test_expect_success 'merge -s ours' '
+	init_repos &&
+
+	test_all_match git rev-parse HEAD^{tree} &&
+	test_all_match git merge -s ours merge-right &&
+	test_all_match git rev-parse HEAD^{tree} &&
+	test_all_match git rev-parse HEAD^2
+'
+
+test_expect_success 'sparse-index is not expanded: merge-ours' '
+	init_repos &&
+	ensure_not_expanded merge -s ours merge-right
+'
+
 test_done

gitgitgadget

I'm happy with this version of the patch series, thanks!

Patrick

Thanks for working in this area. We have a long tail of instances where the sparse-index could be integrated at its most bare-minimum case.

My experiments with such tools seem to do well when there are clear examples of how to make tests and how to make appropriate fixes. The tests added to t1092 follow the standard model for checking that the index isn't expanded. Perhaps the many cases we have for these integrations could be tackled more easily with such tools. I expect that they are _mostly_ boilerplate tests and minor fixes (but don't look too closely at 'git mv' which doesn't have clear patterns with sparse-checkout even with a full index).

The one thing I didn't see that we normally see is a case where the sparse index _is_ expanded, but that shouldn't happen with the 'ours' strategy! Excellent.

These patches LGTM.

Thanks,
-Stolee