sparse-index: expand/collapse based on 'index.sparse'

This series updates do_read_index to use the index.sparse config setting when determining whether the index should be expanded or collapsed. If the command & repo allow use of a sparse index, index.sparse is enabled, and a full index is read from disk, the index is collapsed before returning to the caller. Conversely, if index.sparse is disabled but the index read from disk is sparse, the index is expanded before returning. This allows index.sparse to control use of the sparse index in addition to its existing control over how the index is written to disk. It also introduces the ability to enable/disable the sparse index on a command-by-command basis (e.g., allowing a user to troubleshoot a sparse-aware command with '-c index.sparse=false' [1]).

While testing this change, a bug was found in 'test-tool read-cache' in which config settings for the repository were not initialized before preparing the repo settings. This caused index.sparse to always be 'false' when using the test helper in a cone-mode sparse checkout, breaking tests in t1091 and t1092. The issue is fixed by moving prepare_repo_settings after config setup.

Changes since V1
================

* Add ensure_correct_sparsity function that ensures the index is sparse if the repository settings (including index.sparse) allow it, otherwise ensuring the index is expanded to full.
* Restructure condition in do_read_index to, rather than check specifically for the index.sparse config setting, call ensure_correct_sparsity unconditionally when command_requires_full_index is false.

Changes since V2
================

* Rename can_convert_to_sparse to is_sparse_index_allowed to more accurately reflect what the function returns.
* Remove index-iterating checks from is_sparse_index_allowed, leaving only inexpensive checks on config settings & sparse checkout patterns. Checks are still part of convert_to_sparse to ensure it behaves exactly as it did before this series.
* Restructure ensure_correct_sparsity for better readability.
* Fix test_env variable scope.

Changes since V3
================

* Add a new patch to avoid unnecessary cache tree free/recreation when possible in convert_to_sparse.

Changes since V4
================

* Updated patch 4/4 commit message to better explain practical reasons for making this change.

[1]
https://lore.kernel.org/git/[email protected]/

Thanks! -Victoria

Victoria Dye (4):
test-read-cache.c: prepare_repo_settings after config init
sparse-index: avoid unnecessary cache tree clearing
sparse-index: add ensure_correct_sparsity function
sparse-index: update do_read_index to ensure correct sparsity

 read-cache.c                             |  8 ++++
 sparse-index.c                           | 58 ++++++++++++++++++------
 sparse-index.h                           |  1 +
 t/helper/test-read-cache.c               |  5 +-
 t/t1092-sparse-checkout-compatibility.sh | 31 +++++++++++++
 5 files changed, 86 insertions(+), 17 deletions(-)

base-commit: f443b226ca681d87a3a31e245a70e6bc2769123c
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1059%2Fvdye%2Fsparse%2Findex-sparse-config-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1059/vdye/sparse/index-sparse-config-v5
Pull-Request: https://github.com/gitgitgadget/git/pull/1059

Range-diff vs v4:

 1:  6974ce7e7f5 = 1:  6974ce7e7f5 test-read-cache.c: prepare_repo_settings after config init
 2:  91351ac4bde = 2:  91351ac4bde sparse-index: avoid unnecessary cache tree clearing
 3:  d2133ca1724 = 3:  d2133ca1724 sparse-index: add ensure_correct_sparsity function
 4:  b67cd9d07f8 ! 4:  3237a519c80 sparse-index: update do_read_index to ensure correct sparsity

@@ Metadata
## Commit message ##
sparse-index: update do_read_index to ensure correct sparsity

- If command_requires_full_index is false, ensure correct in-core index
- sparsity on read by calling ensure_correct_sparsity. This change is meant
- to update the how the index is read in a command after sparse index-related
- repository settings are modified. Previously, for example, if index.sparse
- were changed from true to false, the in-core index on the next command
- would be sparse. The index would only be expanded to full when it was next
- written to disk.
+ Unless command_requires_full_index forces index expansion, ensure in-core
+ index sparsity matches config settings on read by calling
+ ensure_correct_sparsity. This makes the behavior of the in-core index more
+ consistent between different methods of updating sparsity: manually changing
+ the index.sparse config setting vs. executing
+ git sparse-checkout --[no-]sparse-index init

- By adding a call to ensure_correct_sparsity, the in-core index now matches
- the sparsity dictated by the relevant repository settings as soon as it is
- read into memory, rather than when it is later written to disk.
+ Although index sparsity is normally updated with git sparse-checkout init,
+ ensuring correct sparsity after a manual index.sparse change has some
+ practical benefits:
+
+ 1. It allows for command-by-command sparsity toggling with
+ -c index.sparse=<true|false>, e.g. when troubleshooting issues with the
+ sparse index.
+ 2. It prevents users from experiencing abnormal slowness after setting
+ index.sparse to true due to use of a full index in all commands until
+ the on-disk index is updated.

gitgitgadget