Skip to contentSkip to conversation

Git v2.48.0-rc0

ANNOUNCE Junio C Hamano 3 participants 16 messages Dec 16, 2024 – Dec 21, 2024

lore
Share & tools
lore
#0 lore raw

An early preview release Git v2.48.0-rc0 is now available for testing at the usual places. It is comprised of 446 non-merge commits since v2.47.0, contributed by 66 people, 26 of which are new faces [*].

The tarballs are found at:

https://www.kernel.org/pub/software/scm/git/testing/

The following public repositories all have a copy of the 'v2.48.0-rc0' tag and the 'master' branch that the tag points at:

url = https://git.kernel.org/pub/scm/git/git
url = https://kernel.googlesource.com/pub/scm/git/git
url = git://repo.or.cz/alt-git.git
url = https://github.com/gitster/git

New contributors whose contributions weren't in v2.47.0 are as follows. Welcome to the Git development community!

Aarni Koskela, Abraham Samuel Adekunle, Bence Ferdinandy, Caleb White, Chizoba ODINAKA, Daniel Black, David Gstir, Ed Reel, Eric Mills, fox, Jaroslav Lobačevski, Josh Heinrichs, Josh Poimboeuf, Kai Koponen, Kev Kloss, Liu Zhongbo, Miguel Rincon Barahona, Monika Kairaitytė, Nicolas Guichard, Russell Stuart, Sam James, Seyi Kuforiji, Shubham Kanodia, Simon Marchi, Tobias Boesch, and Usman Akinyemi.

Returning contributors who helped this release are as follows. Thanks for your continued support.

Alejandro R. Sedeño, Andrew Kreimer, brian m. carlson, Dario Gjorgjevski, Derrick Stolee, Elijah Newren, Eli Schwartz, Eric Sesterhenn, Eric Sunshine, Jean-Noël Avila, Jeff King, Johannes Schindelin, Johannes Sixt, John Cai, Jonathan Tan, Josh Soref, Josh Steadmon, Junio C Hamano, Justin Tobler, Karthik Nayak, Koji Nakamaru, Kousik Sanagavarapu, Kristoffer Haugsbakk, Linus Arver, Oswald Buddenhagen, Patrick Steinhardt, Philippe Blain, Phillip Wood, Piotr Szlazak, Ramsay Jones, Rasmus Villemoes, René Scharfe, Rubén Justo, shejialuo, Sören Krecker, Sven Strickroth, Taylor Blau, Toon Claes, Wolfgang Müller, and Xing Xin.

[*] We are counting not just the authorship contribution but issue
reporting, mentoring, helping and reviewing that are recorded in the commit trailers.

----------------------------------------------------------------

Git v2.48 Release Notes (draft)
===============================

UI, Workflows & Features
------------------------

* A new configuration variable remote.<name>.serverOption makes the transport layer act as if the --serverOption=<value> option is given from the command line.

* "git rebase --rebase-merges" now uses branch names as labels when able.

* Describe the policy to introduce breaking changes.

* Teach 'git notes add' and 'git notes append' a new '-e' flag, instructing them to open the note in $GIT_EDITOR before saving.

* Documentation for "git bundle" saw improvements to more prominently call out the use of '--all' when creating bundles.

* Drop support for older libcURL and Perl.

* End-user experience of "git mergetool" when the command errors out has been improved.

* "git bundle --unbundle" and "git clone" running on a bundle file both learned to trigger fsck over the new objects with configurable fck check levels.

Performance, Internal Implementation, Development Support etc.
--------------------------------------------------------------

* Document "amlog" notes.

* The way AsciiDoc is used for SYNOPSIS part of the manual pages has been revamped. The sources, at least for the simple cases, got vastly pleasant to work with.

* The reftable library is now prepared to expect that the memory allocation function given to it may fail to allocate and to deal with such an error.

* An extra worktree attached to a repository points at each other to allow finding the repository from the worktree and vice versa possible. Turn this linkage to relative paths.

* Enable Windows-based CI in GitLab.

* Commands that can also work outside Git have learned to take the repository instance "repo" when we know we are in a repository, and NULL when we are not, in a parameter. The uses of the_repository variable in a few of them have been removed using the new calling convention.

* The reftable sub-system grew a new reftable-specific strbuf replacement to reduce its dependency on Git-specific data structures.

* The ref-filter machinery learns to recognize and avoid cases where sorting would be redundant.

* Various platform compatibility fixes split out of the larger effort to use Meson as the primary build tool.

* Treat ECONNABORTED the same as ECONNRESET in 'git credential-cache' to work around a possible Cygwin regression. This resolves a race condition caused by changes in Cygwin's handling of socket closures, allowing the client to exit cleanly when encountering ECONNABORTED.

* Demonstrate an assertion failure in 'git mv'.

* Documentation update to clarify that 'uploadpack.allowAnySHA1InWant' implies both 'allowTipSHA1InWant' and 'allowReachableSHA1InWant'.

* Replace various calls to atoi() with strtol_i() and strtoul_ui(), and add improved error handling.

* Documentation updates to 'git-update-ref(1)'.

* Update the project's CodingGuidelines to discourage naming functions with a "_1()" suffix.

* Updates the '.clang-format' to match project conventions.

* Centralize documentation for repository extensions into a single place.

* Buildfix and upgrade of Clar to a newer version.

* Documentation mark-up updates.

* Renaming a handful of variables and structure fields.

* Fix for clar unit tests to support CMake build.

* C23 compatibility updates.

* GCC 15 compatibility updates.

* We now ensure "index-pack" is used with the "--promisor" option only during a "git fetch".

* The migration procedure between two ref backends has been optimized.

* "git fsck" learned to issue warnings on "curiously formatted" ref contents that have always been taken valid but something Git wouldn't have written itself (e.g., missing terminating end-of-line after the full object name).

* Work around Coverity warning that would not trigger in practice.

* Built-in Git subcommands are supplied the repository object to work with; they learned to do the same when they invoke sub-subcommands.

* Drop support for ancient environments in various CI jobs.

* Isolates the reftable subsystem from the rest of Git's codebase by using fewer pieces of Git's infrastructure.

* Optimize reading random references out of the reftable backend by allowing reuse of iterator objects.

* Backport oss-fuzz tests for us to our codebase.

* Introduce a new repository extension to prevent older Git versions from mis-interpreting worktrees created with relative paths.

* Yet another "pass the repository through the callchain" topic.

* "git describe" learned to stop digging the history needlessly deeper.

* Build procedure update plus introduction of Meson based builds.

Fixes since v2.47
-----------------

* Doc update to clarify how periodical maintenance are scheduled, spread across time to avoid thundering hurds.

* Use after free and double freeing at the end in "git log -L... -p" had been identified and fixed.

* On macOS, fsmonitor can fall into a race condition that results in a client waiting forever to be notified for an event that have already happened. This problem has been corrected.

* "git maintenance start" crashed due to an uninitialized variable reference, which has been corrected.

* Fail gracefully instead of crashing when attempting to write the contents of a corrupt in-core index as a tree object.

* A "git fetch" from the superproject going down to a submodule used a wrong remote when the default remote names are set differently between them.

* Fixes compile time warnings with 64-bit MSVC.

* Teaches 'shortlog' to explicitly use SHA-1 when operating outside of a repository.

* Fix 'git grep' regression on macOS by disabling lookahead when encountering invalid UTF-8 byte sequences.

* The dumb-http code regressed when the result of re-indexing a pack yielded an *.idx file that differs in content from the *.idx file it downloaded from the remote. This has been corrected by no longer relying on: the *.idx file we got from the remote.

* When called with '--left-right' and '--use-bitmap-index', 'rev-list' will produce output without any left/right markers, which has been corrected.

* More leakfixes.

* Test modernization.

* The "--shallow-exclude=<ref>" option to various history transfer commands takes a ref, not an arbitrary revision.

* A regression where commit objects missing from a commit-graph can cause an infinite loop when doing a fetch in a partial clone has been fixed.

* The MinGW compatibility layer has been taught to support POSIX semantics for atomic renames when other process(es) have a file opened at the destination path.

* "git gc" discards any objects that are outside promisor packs that are referred to by an object in a promisor pack, and we do not refetch them from the promisor at runtime, resulting an unusable repository. Work it around by including these objects in the referring promisor pack at the receiving end of the fetch.

* Avoid build/test breakage on a system without working malloc debug support dynamic library.
(merge 72ad6dc368 jk/test-malloc-debug-check later to maint).

* Double-free fix.
(merge fe17a25905 jk/fetch-prefetch-double-free-fix later to maint).

* Use of some uninitialized variables in "git difftool" has been corrected.

* Object reuse code based on multi-pack-index sent an unwanted copy of object.
(merge e199290592 tb/multi-pack-reuse-dupfix later to maint).

* "git fast-import" can be tricked into a replace ref that maps an object to itself, which is a useless thing to do.
(merge 5e904f1a4a en/fast-import-avoid-self-replace later to maint).

* The ref-transaction hook triggered for reflog updates, which has been corrected.
(merge b886db48c6 kn/ref-transaction-hook-with-reflog later to maint).

* Give a bit of advice/hint message when "git maintenance" stops finding a lock file left by another instance that still is potentially running. (merge ba874d1dac ps/gc-stale-lock-warning later to maint).

* Use the right helper program to measure file size in performance tests. (merge 3f97f1bce6 tb/use-test-file-size-more later to maint).

* A double-free that may not trigger in practice by luck has been corrected in the reference resolution code.
(merge b6318cf23a sj/refs-symref-referent-fix later to maint).

* The sequencer failed to honor core.commentString in some places.

* Describe a case where an option value needs to be spelled as a separate argument, i.e. "--opt val", not "--opt=val". (merge 1bc1e94091 jc/doc-opt-tilde-expand later to maint).

* Loosen overly strict ownership check introduced in the recent past, to keep the promise "cloning a suspicious repository is a safe first step to inspect it".
(merge 0ffb5a6bf1 bc/allow-upload-pack-from-other-people later to maint).

* "git fast-import" learned to reject paths with ".." and "." as their components to avoid creating invalid tree objects. (merge 8cb4c6e62f en/fast-import-verify-path later to maint).

* The --ancestry-path option is designed to be given a commit that is on the path, which was not documented, which has been corrected. (merge bc1a980759 kk/doc-ancestry-path later to maint).

* "git tag" has been taught to refuse to create refs/tags/HEAD as such a tag will be confusing in the context of UI provided by the Git Porcelain commands.
(merge bbd445d5ef jc/forbid-head-as-tagname later to maint).

* The advice messages now tell the newer 'git config set' command to set the advice.token configuration variable to squelch a message. (merge 6c397d0104 bf/explicit-config-set-in-advice-messages later to maint).

* The syntax ":/<text>" to name the latest commit with the matching text was broken with a recent change, which has been corrected. (merge 0ff919e87a ps/commit-with-message-syntax-fix later to maint).

* Fix performance regression of a recent "fatten promisor pack with local objects" protection against an unwanted gc.

* Other code cleanup, docfix, build fix, etc.
(merge 77af53f56f aa/t7300-modernize later to maint). (merge dcd590a39d bf/t-readme-mention-reftable later to maint). (merge 68e3c69efa kh/trailer-in-glossary later to maint). (merge 91f88f76e6 tb/boundary-traversal-fix later to maint). (merge 168ebb7159 jc/doc-error-message-guidelines later to maint). (merge 18693d7d65 kh/doc-bundle-typofix later to maint). (merge e2f5d3b491 kh/doc-update-ref-grammofix later to maint).

----------------------------------------------------------------

Changes since v2.47.0 are as follows:

Abraham Samuel Adekunle (2):
t7300-clean.sh: use test_path_* helper functions for error logging
notes: teach the -e option to edit messages in editor

Alejandro R. Sedeño (1):
Makefile: adjust sed command for generating "clar-decls.h"

Andrew Kreimer (10):
compat: fix typos
contrib: fix typos
t/unit-tests: fix typos
t/perf: fix typos
t/helper: fix a typo
t: fix typos
t: fix typos
t1016: clean up style
t: fix typos
t1016: clean up style

Bence Ferdinandy (2):
t/README: add missing value for GIT_TEST_DEFAULT_REF_FORMAT
advice: suggest using subcommand "git config set"

Caleb White (12):
worktree: refactor infer_backlink() to use *strbuf
worktree: link worktrees with relative paths
worktree: add test for path handling in linked worktrees
doc: consolidate extensions in git-config documentation
setup: correctly reinitialize repository version
worktree: add relativeWorktrees extension
worktree: refactor infer_backlink return
worktree: add write_worktree_linking_files() function
worktree: add relative cli/config options to add command
worktree: add relative cli/config options to move command
worktree: add relative cli/config options to repair command
worktree: refactor repair_worktree_after_gitdir_move()

Chizoba ODINAKA (1):
t6050: avoid pipes with upstream Git commands

Daniel Black (1):
submodule: correct remote name with fetch

Derrick Stolee (1):
line-log: protect inner strbuf from free

Elijah Newren (5):
upload-pack: fix ambiguous error message
doc: correct misleading descriptions for --shallow-exclude
fast-import: avoid making replace refs point to themselves
fast-import: disallow "." and ".." path components
fast-import: disallow more path components

Eric Sesterhenn (3):
fuzz: port fuzz-credential-from-url-gently from OSS-Fuzz
fuzz: port fuzz-parse-attr-line from OSS-Fuzz
fuzz: port fuzz-url-decode-mem from OSS-Fuzz

Eric Sunshine (1):
worktree: repair copied repository and linked worktrees

Jean-Noël Avila (9):
doc: introduce a synopsis typesetting
doc: update the guidelines to reflect the current formatting rules
doc: apply synopsis simplification on git-clone and git-init
doc: git-add.txt: convert to new style convention
doc: git-diff: apply new documentation guidelines
doc: git-diff: apply format changes to diff-options
doc: git-diff: apply format changes to diff-format
doc: git-diff: apply format changes to diff-generate-patch
doc: git-diff: apply format changes to config part

Jeff King (36):
line-log: use diff_line_prefix() instead of custom helper
diff: drop line_prefix_length field
diff: return const char from output_prefix callback
diff: return line_prefix directly when possible
diff: store graph prefix buf in git_graph struct
simple-ipc: split async server initialization and running
fsmonitor: initialize fs event listener before accepting clients
midx: avoid duplicate packed_git entries
t5550: count fetches in "previously-fetched .idx" test
dumb-http: store downloaded pack idx as tempfile
packfile: drop has_pack_index()
packfile: drop sha1_pack_name()
packfile: drop sha1_pack_index_name()
packfile: warn people away from parse_packed_git()
http-walker: use object_id instead of bare hash
packfile: convert find_sha1_pack() to use object_id
packfile: use object_id in find_pack_entry_one()
packfile: use oidread() instead of hashcpy() to fill object_id
rev-list: skip bitmap traversal for --left-right
t6120: demonstrate weakness in disjoint-root handling t/perf: add tests for git-describe
describe: stop digging for max_candidates+1
describe: stop traversing when we run out of names
test-lib: check malloc debug LD_PRELOAD before using
fetch: adjust refspec->raw_nr when filtering prefetch refspecs
refspec: drop separate raw_nr count
refspec: store raw refspecs inside refspec_item
test-lib: move malloc-debug setup after $PATH setup
object-file: prefer array-of-bytes initializer for hash literals
object-file: drop confusing oid initializer of empty_tree struct
object-file: move empty_tree struct into find_cached_object()
object-file: drop oid field from find_cached_object() return value
object-file: treat cached_object values as const
object-file: inline empty tree and blob literals
t9300: test verification of renamed paths
describe: split "found all tags" and max_candidates logic

Johannes Schindelin (1):
docs: fix the maintain-git links in technical/platform-support

John Cai (3):
git: pass in repo to builtin based on setup_git_directory_gently
annotate: remove usage of the_repository global
archive: remove the_repository global variable

Jonathan Tan (10):
t0410: make test description clearer
t0410: use from-scratch server
t5300: move --window clamp test next to unclamped Revert "fetch-pack: add a deref_without_lazy_fetch_extended()"
fetch-pack: die if in commit graph but not obj db
index-pack: repack local links into promisor packs
index-pack: teach --promisor to forbid pack name index-pack --promisor: dedup before checking links index-pack --promisor: don't check blobs
index-pack --promisor: also check commits' trees

Josh Heinrichs (1):
git-config.1: remove value from positional args in unset usage

Josh Soref (2):
doc: update links to current pages
doc: switch links to https

Junio C Hamano (22):
doc: clarify <src> in refspec syntax
Start the 2.48 cycle
BreakingChanges: early adopter option
The eighth batch
The ninth batch
Clean up RelNotes for 2.48
Prepare for 2.47.1
The tenth batch
The eleventh batch
Git 2.47.1
doc: option value may be separate for valid reasons The twelfth batch
CodingGuidelines: a handful of error message guidelines
refs: move ref name helpers around
refs: drop strbuf_ prefix from helpers
t5604: do not expect that HEAD can be a valid tagname
tag: "git tag" refuses to use HEAD as a tagname The thirteenth batch
The fourteenth batch
The fifteenth batch
The sixteenth batch
Git 2.48-rc0

Justin Tobler (4):
bundle: add bundle verification options type
bundle: support fsck message configuration
fetch-pack: split out fsck config parsing
transport: propagate fsck configuration during bundle fetch

Kai Koponen (1):

      doc: mention rev-list --ancestry-path restrictions

Karthik Nayak (23):
loose: don't rely on repository global state
clang-format: re-adjust line break penalties
clang-format: align consecutive macro definitions
CodingGuidelines: discourage arbitrary suffixes in function names
refs: don't invoke reference-transaction hook for reflogs
builtin: pass repository to sub commands
packfile: add repository to struct packed_git
packfile: use repository from packed_git directly
packfile: pass repository to static function in the file
packfile: pass down repository to odb_pack_name
packfile: pass down repository to has_object[_kept]_pack
packfile: pass down repository to for_each_packed_object
config: make delta_base_cache_limit a non-global variable
config: make packed_git_(limit|window_size) non-global variables
midx: add repository to multi_pack_index struct
midx-write: pass down repository to static functions
midx-write: use revs->repo inside read_refs_snapshot
write-midx: add repository field to write_midx_context
midx-write: pass down repository to write_midx_file[_only]
midx: cleanup internal usage of the_repository and the_hash_algo
midx: pass repository to load_multi_pack_index
midx: pass down hash_algo to functions using global variables
midx: inline the MIDX_MIN_SIZE definition

Kousik Sanagavarapu (1):
t4205: fix typo in 'NUL termination with --stat'

Kristoffer Haugsbakk (20):
doc: merge-tree: improve example script
checkout: refer to other-worktree branch, not ref Documentation/git-update-ref.txt: drop “flag” Documentation/git-update-ref.txt: remove safety paragraphs Documentation/git-update-ref.txt: demote symlink to last section Documentation/git-update-ref.txt: remove confusing paragraph Documentation/git-update-ref.txt: discuss symbolic refs
Documentation: mutually link update-ref and symbolic-ref
SubmittingPatches: tags -> trailers
t7001: add failure test which triggers assertion Documentation/glossary: describe "trailer" Documentation/git-bundle.txt: mention full backup example Documentation/git-bundle.txt: remove old --all example Documentation/git-bundle.txt: mention --all in spec. refs Documentation/git-bundle.txt: discuss naïve backups
sequencer: comment checked-out branch properly
sequencer: comment --reference subject line properly
sequencer: comment commit messages properly Documentation/git-bundle.txt: fix word join typo Documentation/git-update-ref.txt: add missing word

Linus Arver (1):
trailer: spread usage of "trailer_block" language

Nicolas Guichard (3):
load_branch_decorations: fix memory leak with non-static filters
rebase-update-refs: extract load_branch_decorations
rebase-merges: try and use branch names as labels

Oswald Buddenhagen (2):
git-gui: strip comments and consecutive empty lines from commit messages
git-gui: strip commit messages less aggressively

Patrick Steinhardt (197):
builtin/annotate: fix leaking args vector
read-cache: fix leaking hash context in do_write_index()
scalar: fix leaking repositories
shell: fix leaking strings
wt-status: fix leaking buffer with sparse directories
submodule: fix leaking submodule entry list
builtin/stash: fix leaking pathspec_from_file builtin/pack-redundant: fix various memory leaks
builtin/clone: fix leaking repo state when cloning with bundle URIs t/helper: fix leaking repository in partial-clone helper builtin/revert: fix leaking gpg_sign and strategy config
diff: improve lifecycle management of diff queues
line-log: fix several memory leaks
pseudo-merge: fix various memory leaks
pseudo-merge: fix leaking strmap keys
pack-bitmap-write: fix leaking OID array
midx-write: fix leaking buffer
revision: fix memory leaks when rewriting parents
revision: fix leaking saved parents
pack-write: fix return parameter of write_rev_file_order() t/helper: fix leaks in proc-receive helper
remote: fix leaking push reports
builtin/send-pack: fix leaking list of push options reftable/error: introduce out-of-memory error code reftable/basics: merge "publicbasics" into "basics"
reftable: introduce reftable_strdup()
reftable/basics: handle allocation failures in reftable_calloc() reftable/basics: handle allocation failures in parse_names() reftable/record: handle allocation failures on copy reftable/record: handle allocation failures when decoding records reftable/writer: handle allocation failures in writer_index_hash() reftable/writer: handle allocation failures in reftable_new_writer() reftable/merged: handle allocation failures in merged_table_init_iter() reftable/reader: handle allocation failures for unindexed reader reftable/reader: handle allocation failures in reader_init_iter() reftable/stack: handle allocation failures on reload reftable/stack: handle allocation failures in reftable_new_stack() reftable/stack: handle allocation failures in stack_compact_range() reftable/stack: handle allocation failures in auto compaction reftable/iter: handle allocation failures when creating indexed table iter reftable/blocksource: handle allocation failures reftable/block: handle allocation failures
reftable/pq: handle allocation failures when adding entries reftable/tree: handle allocation failures
reftable: handle trivial allocation failures
reftable: fix calls to free(3P)
reftable: introduce REFTABLE_FREE_AND_NULL()
reftable/basics: ban standard allocator functions reftable/basics: fix segfault when growing names array fails
cache-tree: refactor verification to return error codes
cache-tree: detect mismatching number of index entries
unpack-trees: detect mismatching number of cache-tree/index entries
t7300: work around platform-specific behaviour with long paths on MinGW
ci: create script to set up Git for Windows SDK
ci: handle Windows-based CI jobs in GitLab CI
gitlab-ci: introduce stages and dependencies
gitlab-ci: exercise Git on Windows
builtin/gc: fix crash when running git maintenance start t/test-lib: fix quoting of TEST_RESULTS_SAN_FILE
t/test-lib: wire up NO_ICONV prerequisite
t/lib-gitweb: test against the build version of gitweb t/lib-gpg: fix setup of GNUPGHOME in MinGW
t1401: make invocation of tar(1) work with Win32-provided one
t3404: work around platform-specific behaviour on macOS 10.15 t5500, t5601: skip tests which exercise paths with '[::1]' on Cygwin
t7300: work around platform-specific behaviour with long paths on MinGW builtin/credential-cache: fix missing parameter for stub function
http: fix build error on FreeBSD
reftable: stop using strbuf_addbuf()
reftable: stop using strbuf_addf()
reftable/basics: provide new reftable_buf interface
reftable: convert from strbuf to reftable_buf reftable/blocksource: adapt interface name
t/unit-tests: check for reftable_buf allocation errors reftable/stack: adapt format_name() to handle allocation failures reftable/record: adapt reftable_record_key() to handle allocation failures reftable/stack: adapt stack_filename() to handle allocation failures
reftable: handle trivial reftable_buf errors
ref-filter: format iteratively with lexicographic refname sorting t/unit-tests: update clar to 206accb
Makefile: extract script to generate clar declarations
cmake: fix compilation of clar-based unit tests
cmake: set up proper dependencies for generated clar headers compat/mingw: share file handles created via CreateFileW() compat/mingw: allow deletion of most opened files
t6006: fix prereq handling with test_format () builtin/ls-remote: plug leaking server options
t/helper: fix leaks in "reach" test tool
grep: fix leak in grep_splice_or()
builtin/grep: fix leak with --max-count=0
revision: fix leaking bloom filters
diff-lib: fix leaking diffopts in do_diff_cache()
pretty: clear signature check
upload-pack: fix leaking URI protocols
builtin/commit: fix leaking change data contents
trailer: fix leaking trailer values
trailer: fix leaking strbufs when formatting trailers builtin/commit: fix leaking cleanup config
transport-helper: fix leaking import/export marks
builtin/tag: fix leaking key ID on failure to sign
combine-diff: fix leaking lost lines
dir: release untracked cache data
sparse-index: correctly free EWAH contents
t/helper: stop re-initialization of the_repository
t/helper: fix leaking buffer in "dump-untracked-cache"
dir: fix leak when parsing "status.showUntrackedFiles" builtin/merge: release output buffer after performing merge list-objects-filter-options: work around reported leak on error compat/mingw: support POSIX semantics for atomic renames t/unit-tests: convert "clar-generate.awk" into a shell script
cmake: use SH_EXE to execute clar scripts
cmake: use verbatim arguments when invoking clar commands
Makefile: let clar header targets depend on their scripts reftable/system: move "dir.h" to its only user
reftable: explicitly handle hash format IDs
reftable/system: stop depending on "hash.h"
reftable/stack: stop using fsync_component() directly reftable/system: provide thin wrapper for tempfile subsystem reftable/stack: drop only use of get_locked_file_path() reftable/system: provide thin wrapper for lockfile subsystem builtin/gc: provide hint when maintenance hits a stale schedule lock
refs: allow passing flags when setting up a transaction refs/files: move logic to commit initial transaction
refs: introduce "initial" transaction flag
refs/files: support symbolic and root refs in initial transaction
refs: use "initial" transaction semantics to migrate refs
refs: skip collision checks in initial transactions
refs: don't normalize log messages with REF_SKIP_CREATE_REFLOG reftable/writer: optimize allocations by using a scratch buffer reftable/block: rename block_writer::buf variable reftable/block: optimize allocations by using scratch buffer builtin/blame: fix leaking blame entries with --incremental
bisect: fix leaking good/bad terms when reading multipe times
bisect: fix leaking string in handle_bad_merge_base()
bisect: fix leaking current_bad_oid
bisect: fix multiple leaks in bisect_next_all()
bisect: fix leaking commit list items in check_merge_base()
bisect: fix various cases where we leak commit list items
line-log: fix leak when rewriting commit parents
strvec: introduce new strvec_splice() function
git: refactor alias handling to use a struct strvec
git: refactor builtin handling to use a struct strvec
split-index: fix memory leak in move_cache_to_base_index() builtin/sparse-checkout: fix leaking sanitized patterns
help: refactor to not use globals for reading config
help: fix leaking struct cmdnames
help: fix leaking return value from help_unknown_cmd() builtin/help: fix leaks in check_git_cmd()
builtin/init-db: fix leaking directory paths
builtin/branch: fix leaking sorting options
t/helper: fix leaking commit graph in "read-graph" subcommand
global: drop UNLEAK() annotation
git-compat-util: drop now-unused UNLEAK() macro
t5601: work around leak sanitizer issue
t: mark some tests as leak free
t: remove unneeded !SANITIZE_LEAK prerequisites
test-lib: unconditionally enable leak checking
t: remove TEST_PASSES_SANITIZE_LEAK annotations
t7900: fix host-dependent behaviour when testing git-maintenance(1)
refs: adapt initial_transaction flag to be unsigned
reftable: rename scratch buffer
bisect: address Coverity warning about potential double free refs/reftable: encapsulate reftable stack
refs/reftable: handle reloading stacks in the reftable backend reftable/stack: add accessor for the hash ID
refs/reftable: figure out hash via reftable_stack refs/reftable: read references via struct reftable_backend refs/reftable: refactor reading symbolic refs to use reftable backend refs/reftable: refactor reflog expiry to use reftable backend reftable/stack: add mechanism to notify callers on reload reftable/merged: drain priority queue on reseek
refs/reftable: reuse iterators when reading refs
Makefile: use common template for GIT-BUILD-OPTIONS
Makefile: consistently use @PLACEHOLDER@ to substitute
Makefile: refactor GIT-VERSION-GEN to be reusable
Makefile: propagate Git version via generated header
Makefile: generate "git.rc" via GIT-VERSION-GEN
Makefile: generate doc versions via GIT-VERSION-GEN
Makefile: consistently use PERL_PATH
Makefile: extract script to massage Perl scripts
Makefile: use "generate-perl.sh" to massage Perl library
Makefile: extract script to massage Shell scripts
Makefile: extract script to massage Python scripts
Makefile: extract script to generate gitweb.cgi
Makefile: extract script to generate gitweb.js
Makefile: refactor generators to be PWD-independent
Makefile: allow "bin-wrappers/" directory to exist
Makefile: write absolute program path into bin-wrappers
Makefile: simplify building of templates
Documentation: allow sourcing generated includes from separate dir
Documentation: teach "cmd-list.perl" about out-of-tree builds
Documentation: extract script to generate a list of mergetools
t: better support for out-of-tree builds
t: allow overriding build dir
Documentation: add comparison of build systems
Introduce support for the Meson build system
object-name: fix reversed ordering with ":/<text>" revisions

Philippe Blain (6):
Makefile: fix dependency for $(UNIT_TEST_DIR)/clar/clar.o
completion: complete '--tool-help' in 'git mergetool' git-mergetool--lib.sh: use TOOL_MODE when erroring about unknown tool git-mergetool--lib.sh: add error message if 'setup_user_tool' fails git-mergetool--lib.sh: add error message for unknown tool variant git-difftool--helper.sh: exit upon initialize_merge_tool errors

Piotr Szlazak (1):
doc: document how uploadpack.allowAnySHA1InWant impact other allow options

Ramsay Jones (1):
credential-cache: treat ECONNABORTED like ECONNRESET

René Scharfe (1):
grep: disable lookahead on error

Rubén Justo (1):
strvec: strvec_splice() to a statically initialized vector

Seyi Kuforiji (2):
t7011: ensure no whitespace after redirect
t9101: ensure no whitespace after redirect

Shubham Kanodia (1):
doc: add a note about staggering of maintenance

Simon Marchi (1):
builtin/difftool: intialize some hashmap variables

Sven Strickroth (1):
global: Fix duplicate word typos

Sören Krecker (1):
mingw.c: Fix complier warnings for a 64 bit msvc

Taylor Blau (13):
Documentation: mention the amlog in howto/maintain-git.txt The second batch
The third batch
The third batch
The fifth batch
The sixth batch
The seventh batch
t5332-multi-pack-reuse.sh: demonstrate duplicate packing failure
pack-objects: only perform verbatim reuse on the preferred pack pack-bitmap.c: typofix in find_boundary_objects() t/perf: use 'test_file_size' in more places Makefile(s): avoid recipe prefix in conditional statements packfile.c: remove unnecessary prepare_packed_git() call

Tobias Boesch (1):
git gui: add directly calling merge tool from configuration

Toon Claes (1):
bundle-uri: plug leak in unbundle_from_file()

Usman Akinyemi (5):
t3404: avoid losing exit status with focus on git show and git cat-file
t3404: replace test with test_line_count()
daemon: replace atoi() with strtoul_ui() and strtol_i()
merge: replace atoi() with strtol_i() for marker size validation
imap: replace atoi() with strtol_i() for UIDVALIDITY and UIDNEXT parsing

Wolfgang Müller (1):
builtin/shortlog: explicitly set hash algo when there is no repo

Xing Xin (6):
transport: introduce parse_transport_option() method
remote: introduce remote.<name>.serverOption configuration transport.c::handshake: make use of server options from remote
fetch: respect --server-option when fetching multiple remotes
ls-remote: leakfix for not clearing server_options Documentation/gitprotocol-v2.txt: fix a slight inconsistency in format

brian m. carlson (18):
git-curl-compat: remove check for curl 7.21.5
git-curl-compat: remove check for curl 7.25.0
git-curl-compat: remove check for curl 7.34.0
git-curl-compat: remove check for curl 7.39.0
git-curl-compat: remove check for curl 7.43.0
git-curl-compat: remove check for curl 7.44.0
git-curl-compat: remove check for curl 7.52.0
git-curl-compat: remove check for curl 7.53.0
git-curl-compat: remove check for curl 7.56.0
INSTALL: document requirement for libcurl 7.61.0 Require Perl 5.26.0
gitweb: make use of s///r
gitlab-ci: switch from Ubuntu 16.04 to 20.04
ci: remove clause for Ubuntu 16.04
Add additional CI jobs to avoid accidental breakage Allow cloning from repositories owned by another user
index-pack: rename struct thread_local
reflog: rename unreachable

shejialuo (10):
ref: initialize "fsck_ref_report" with zero
ref: check the full refname instead of basename
ref: initialize ref name outside of check functions
ref: support multiple worktrees check for refs
ref: port git-fsck(1) regular refs check for files backend
ref: add more strict checks for regular refs
ref: add basic symref content check for files backend
ref: check whether the target of the symref is a ref
ref: add symlink ref content check for files backend
ref-cache: fix invalid free operation in free_ref_entry

#1 lore raw
Quoted 8 lines · Junio C Hamano

An early preview release Git v2.48.0-rc0 is now available for testing at the usual places. It is comprised of 446 non-merge commits since v2.47.0, contributed by 66 people, 26 of which are new faces [*].

The tarballs are found at:

https://www.kernel.org/pub/software/scm/git/testing/

Quoted 7 lines

The following public repositories all have a copy of the 'v2.48.0-rc0' tag and the 'master' branch that the tag points at:

url = https://git.kernel.org/pub/scm/git/git
url = https://kernel.googlesource.com/pub/scm/git/git
url = git://repo.or.cz/alt-git.git
url = https://github.com/gitster/git

Did I miss something about GIT_TEST_DEFAULT_REF_FORMAT=reftable being removed from the test suite? That was available under 2.47.0 but if I supply it now, the tests crash at git init.

Confused a little
--Randall

--
Brief whoami: NonStop&UNIX developer since approximately UNIX(421664400)
NonStop(211288444200000000)
-- In real life, I talk too much.

#2 lore raw
Quoted 12 lines · On Mon, Dec 16, 2024 at 07:07:25PM -0500, [email protected] wrote:
Quoted 11 lines · Junio C Hamano

An early preview release Git v2.48.0-rc0 is now available for testing at the usual places. It is comprised of 446 non-merge commits since v2.47.0, contributed by 66 people, 26 of which are new faces [*].

The tarballs are found at:

https://www.kernel.org/pub/software/scm/git/testing/

The following public repositories all have a copy of the 'v2.48.0-rc0' tag and the 'master' branch that the tag points at:

Quoted 8 lines
Quoted 4 lines

url = https://git.kernel.org/pub/scm/git/git
url = https://kernel.googlesource.com/pub/scm/git/git
url = git://repo.or.cz/alt-git.git
url = https://github.com/gitster/git

Did I miss something about GIT_TEST_DEFAULT_REF_FORMAT=reftable being removed from the test suite? That was available under 2.47.0 but if I supply it now, the tests crash at git init.

Nothing should've changed here, and a local test run with that environment variable works alright on my system. Could you please provide more details?

Patrick

#3 lore raw
Quoted 6 lines

-----Original Message-----
From: Patrick Steinhardt <[email protected]>
Sent: December 16, 2024 11:51 PM
To: [email protected]
Cc: 'Junio C Hamano' <[email protected]>; [email protected]; 'Linux

Kernel'

Quoted 21 lines

<[email protected]>; [email protected]
Subject: Re: [ANNOUNCE] Git v2.48.0-rc0

Quoted 17 lines · On Mon, Dec 16, 2024 at 07:07:25PM -0500, [email protected] wrote:
Quoted 15 lines · Junio C Hamano

An early preview release Git v2.48.0-rc0 is now available for testing at the usual places. It is comprised of 446 non-merge commits since v2.47.0, contributed by 66 people, 26 of which are new faces [*].

The tarballs are found at:

https://www.kernel.org/pub/software/scm/git/testing/

The following public repositories all have a copy of the 'v2.48.0-rc0' tag and the 'master' branch that the tag points at:

url = https://git.kernel.org/pub/scm/git/git
url = https://kernel.googlesource.com/pub/scm/git/git
url = git://repo.or.cz/alt-git.git
url = https://github.com/gitster/git

Quoted 6 lines
Quoted 3 lines

Did I miss something about GIT_TEST_DEFAULT_REF_FORMAT=reftable being removed from the test suite? That was available under 2.47.0 but if I supply it now, the tests crash at git init.

Nothing should've changed here, and a local test run with that environment variable works alright on my system. Could you please provide more details?

This is on the latest OS version of NonStop x86. The test is done via the following:
With OpenSSL 3.4 The T0625 is Expat.

When run as follows:
GIT_TEST_DEFAULT_REF_FORMAT=reftable GIT_TEST_CLONE_2GB=true /usr/coreutils/bin/make -k test V=1 CFLAGS="-g -O2 -Winline -I/usr/local-ssl3.4/include -I/usr/coreutils/include -I/usr/tandem/xml/T0625L01_AAE/include" LDFLAGS="/usr/coreutils/lib/libz.a -L/usr/local-ssl3.4/lib -L/usr/coreutils/lib -L/usr/tandem/xml/T0625L01_AAE/lib" TEST_LINT= SHELL=/usr/coreutils/bin/bash

The results are:
*** t0000-basic.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0000-basic.sh' failed make[2]: *** [t0000-basic.sh] Error 1
*** t0001-init.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0001-init.sh' failed make[2]: *** [t0001-init.sh] Error 1
*** t0002-gitfile.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0002-gitfile.sh' failed make[2]: *** [t0002-gitfile.sh] Error 1
etc for all tests.

Removing GIT_TEST_DEFAULT_REF_FORMAT=reftable allows the whole test suite to pass.

There are environment size limits (about 32Kb) in bash on the box, so it is possible
a boundary was crossed internal to the test setup.

#4 lore raw
Quoted 63 lines · On Tue, Dec 17, 2024 at 08:47:47AM -0500, [email protected] wrote:
Quoted 6 lines

-----Original Message-----
From: Patrick Steinhardt <[email protected]>
Sent: December 16, 2024 11:51 PM
To: [email protected]
Cc: 'Junio C Hamano' <[email protected]>; [email protected]; 'Linux

Kernel'

Quoted 21 lines

<[email protected]>; [email protected]
Subject: Re: [ANNOUNCE] Git v2.48.0-rc0

Quoted 17 lines · On Mon, Dec 16, 2024 at 07:07:25PM -0500, [email protected] wrote:
Quoted 15 lines · Junio C Hamano

An early preview release Git v2.48.0-rc0 is now available for testing at the usual places. It is comprised of 446 non-merge commits since v2.47.0, contributed by 66 people, 26 of which are new faces [*].

The tarballs are found at:

https://www.kernel.org/pub/software/scm/git/testing/

The following public repositories all have a copy of the 'v2.48.0-rc0' tag and the 'master' branch that the tag points at:

url = https://git.kernel.org/pub/scm/git/git
url = https://kernel.googlesource.com/pub/scm/git/git
url = git://repo.or.cz/alt-git.git
url = https://github.com/gitster/git

Quoted 6 lines
Quoted 3 lines

Did I miss something about GIT_TEST_DEFAULT_REF_FORMAT=reftable being removed from the test suite? That was available under 2.47.0 but if I supply it now, the tests crash at git init.

Nothing should've changed here, and a local test run with that environment variable works alright on my system. Could you please provide more details?

This is on the latest OS version of NonStop x86. The test is done via the following:
With OpenSSL 3.4 The T0625 is Expat.

When run as follows:
GIT_TEST_DEFAULT_REF_FORMAT=reftable GIT_TEST_CLONE_2GB=true /usr/coreutils/bin/make -k test V=1 CFLAGS="-g -O2 -Winline

(quoted diff) +0 -4
-I/usr/local-ssl3.4/include -I/usr/coreutils/include
-I/usr/tandem/xml/T0625L01_AAE/include" LDFLAGS="/usr/coreutils/lib/libz.a
-L/usr/local-ssl3.4/lib -L/usr/coreutils/lib
-L/usr/tandem/xml/T0625L01_AAE/lib"  TEST_LINT=

SHELL=/usr/coreutils/bin/bash

The results are:
*** t0000-basic.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0000-basic.sh' failed make[2]: *** [t0000-basic.sh] Error 1
*** t0001-init.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0001-init.sh' failed make[2]: *** [t0001-init.sh] Error 1
*** t0002-gitfile.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0002-gitfile.sh' failed make[2]: *** [t0002-gitfile.sh] Error 1
etc for all tests.

Quoted 6 lines

Removing GIT_TEST_DEFAULT_REF_FORMAT=reftable allows the whole test suite to pass.

There are environment size limits (about 32Kb) in bash on the box, so it is possible
a boundary was crossed internal to the test setup.

Could you please provide a backtrace for git-init(1)?

Patrick

#5 lore raw
Quoted 8 lines · Patrick Steinhardt
Quoted 7 lines · On Tue, Dec 17, 2024 at 08:47:47AM -0500, [email protected] wrote:
Quoted 6 lines

-----Original Message-----
From: Patrick Steinhardt <[email protected]>
Sent: December 16, 2024 11:51 PM
To: [email protected]
Cc: 'Junio C Hamano' <[email protected]>; [email protected]; 'Linux

Kernel'

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

<[email protected]>; [email protected]
Subject: Re: [ANNOUNCE] Git v2.48.0-rc0

Quoted 4 lines · On Mon, Dec 16, 2024 at 07:07:25PM -0500, [email protected] wrote:
Quoted 3 lines · Junio C Hamano

An early preview release Git v2.48.0-rc0 is now available for testing at the usual places. It is comprised of 446 non-merge commits since v2.47.0, contributed by 66 people, 26 of which are new

faces

Quoted 20 lines

[*].

Quoted 19 lines
Quoted 19 lines
Quoted 8 lines
Quoted 8 lines

The tarballs are found at:

https://www.kernel.org/pub/software/scm/git/testing/

The following public repositories all have a copy of the 'v2.48.0-rc0' tag and the 'master' branch that the tag points at:

Quoted 8 lines
Quoted 4 lines

url = https://git.kernel.org/pub/scm/git/git
url = https://kernel.googlesource.com/pub/scm/git/git
url = git://repo.or.cz/alt-git.git url =
https://github.com/gitster/git

Did I miss something about GIT_TEST_DEFAULT_REF_FORMAT=reftable being removed from the test suite? That was available under 2.47.0 but if I supply it now, the tests crash at git init.

Nothing should've changed here, and a local test run with that environment variable works alright on my system. Could you please

provide

Quoted 31 lines

more details?

Quoted 30 lines

This is on the latest OS version of NonStop x86. The test is done via the
following:
With OpenSSL 3.4 The T0625 is Expat.

When run as follows:
GIT_TEST_DEFAULT_REF_FORMAT=reftable GIT_TEST_CLONE_2GB=true /usr/coreutils/bin/make -k test V=1 CFLAGS="-g -O2 -Winline

(quoted diff) +0 -2
-I/usr/local-ssl3.4/include -I/usr/coreutils/include
-I/usr/tandem/xml/T0625L01_AAE/include"

LDFLAGS="/usr/coreutils/lib/libz.a

(quoted diff) +0 -2
-L/usr/local-ssl3.4/lib -L/usr/coreutils/lib
-L/usr/tandem/xml/T0625L01_AAE/lib"  TEST_LINT=

SHELL=/usr/coreutils/bin/bash

The results are:
*** t0000-basic.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0000-basic.sh' failed make[2]: *** [t0000-basic.sh] Error 1
*** t0001-init.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0001-init.sh' failed make[2]: *** [t0001-init.sh] Error 1
*** t0002-gitfile.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0002-gitfile.sh' failed make[2]: *** [t0002-gitfile.sh] Error 1 etc for all tests.

Quoted 7 lines
Quoted 5 lines

Removing GIT_TEST_DEFAULT_REF_FORMAT=reftable allows the whole test suite to pass.

There are environment size limits (about 32Kb) in bash on the box, so it is possible a boundary was crossed internal to the test setup.

Could you please provide a backtrace for git-init(1)?

Where would you need the backtrace done? Any specific point in the code? I have to throw this into gdb to get the backtrace - no dump file created.

#6 lore raw
In reply to [email protected]
Quoted 5 lines · On December 17, 2024 8:48 AM, I wrote:
Quoted 5 lines · On Mon, Dec 16, 2024 at 07:07:25PM -0500, [email protected] wrote:
Quoted 4 lines · Junio C Hamano
Quoted 3 lines

An early preview release Git v2.48.0-rc0 is now available for testing at the usual places. It is comprised of 446 non-merge commits since v2.47.0, contributed by 66 people, 26 of which are new

faces

Quoted 20 lines

[*].

Quoted 19 lines
Quoted 8 lines
Quoted 8 lines

The tarballs are found at:

https://www.kernel.org/pub/software/scm/git/testing/

The following public repositories all have a copy of the 'v2.48.0-rc0' tag and the 'master' branch that the tag points at:

Quoted 8 lines
Quoted 4 lines

url = https://git.kernel.org/pub/scm/git/git
url = https://kernel.googlesource.com/pub/scm/git/git
url = git://repo.or.cz/alt-git.git url =
https://github.com/gitster/git

Did I miss something about GIT_TEST_DEFAULT_REF_FORMAT=reftable being removed from the test suite? That was available under 2.47.0 but if I supply it now, the tests crash at git init.

Nothing should've changed here, and a local test run with that environment variable works alright on my system. Could you please provide

more

Quoted 10 lines

details?

This is on the latest OS version of NonStop x86. The test is done via the following:
With OpenSSL 3.4 The T0625 is Expat.

When run as follows:
GIT_TEST_DEFAULT_REF_FORMAT=reftable GIT_TEST_CLONE_2GB=true /usr/coreutils/bin/make -k test V=1 CFLAGS="-g -O2 -Winline -I/usr/local- ssl3.4/include -I/usr/coreutils/include

-I/usr/tandem/xml/T0625L01_AAE/include"

Quoted 2 lines

LDFLAGS="/usr/coreutils/lib/libz.a
-L/usr/local-ssl3.4/lib -L/usr/coreutils/lib

-L/usr/tandem/xml/T0625L01_AAE/lib"

Quoted 17 lines

TEST_LINT= SHELL=/usr/coreutils/bin/bash

The results are:
*** t0000-basic.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0000-basic.sh' failed make[2]: *** [t0000-basic.sh] Error 1
*** t0001-init.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0001-init.sh' failed make[2]: *** [t0001-init.sh] Error 1
*** t0002-gitfile.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0002-gitfile.sh' failed make[2]: *** [t0002-gitfile.sh] Error 1
etc for all tests.

Quoted 1 line

Removing GIT_TEST_DEFAULT_REF_FORMAT=reftable allows the whole test suite

to

Quoted 3 lines

pass.

There are environment size limits (about 32Kb) in bash on the box, so it is

possible a

Quoted 1 line

boundary was crossed internal to the test setup.

I did find this, in t0000-basic:
not ok 69 - init with GIT_DEFAULT_REF_FORMAT=reftable
but have no more details. When should I break in gdb to get a backtrace?

#7 lore raw
Quoted 32 lines · On Tue, Dec 17, 2024 at 02:00:01PM -0500, [email protected] wrote:
Quoted 8 lines · On December 17, 2024 8:48 AM, I wrote:

This is on the latest OS version of NonStop x86. The test is done via the following:
With OpenSSL 3.4 The T0625 is Expat.

When run as follows:
GIT_TEST_DEFAULT_REF_FORMAT=reftable GIT_TEST_CLONE_2GB=true /usr/coreutils/bin/make -k test V=1 CFLAGS="-g -O2 -Winline -I/usr/local- ssl3.4/include -I/usr/coreutils/include

-I/usr/tandem/xml/T0625L01_AAE/include"

Quoted 2 lines

LDFLAGS="/usr/coreutils/lib/libz.a
-L/usr/local-ssl3.4/lib -L/usr/coreutils/lib

-L/usr/tandem/xml/T0625L01_AAE/lib"

Quoted 17 lines

TEST_LINT= SHELL=/usr/coreutils/bin/bash

The results are:
*** t0000-basic.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0000-basic.sh' failed make[2]: *** [t0000-basic.sh] Error 1
*** t0001-init.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0001-init.sh' failed make[2]: *** [t0001-init.sh] Error 1
*** t0002-gitfile.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0002-gitfile.sh' failed make[2]: *** [t0002-gitfile.sh] Error 1
etc for all tests.

Quoted 1 line

Removing GIT_TEST_DEFAULT_REF_FORMAT=reftable allows the whole test suite

to

Quoted 8 lines
Quoted 3 lines

pass.

There are environment size limits (about 32Kb) in bash on the box, so it is

possible a

Quoted 1 line

boundary was crossed internal to the test setup.

I did find this, in t0000-basic:
not ok 69 - init with GIT_DEFAULT_REF_FORMAT=reftable

You mean t0001-init.sh, right?

Quoted 1 line

but have no more details. When should I break in gdb to get a backtrace?

Can you please run that test with "-ix" and post the logs?

Patrick

#8 lore raw
Quoted 41 lines · Patrick Steinhardt
Quoted 40 lines · On Tue, Dec 17, 2024 at 02:00:01PM -0500, [email protected] wrote:
Quoted 9 lines · On December 17, 2024 8:48 AM, I wrote:

This is on the latest OS version of NonStop x86. The test is done via the
following:
With OpenSSL 3.4 The T0625 is Expat.

When run as follows:
GIT_TEST_DEFAULT_REF_FORMAT=reftable GIT_TEST_CLONE_2GB=true /usr/coreutils/bin/make -k test V=1 CFLAGS="-g -O2 -Winline -I/usr/local- ssl3.4/include -I/usr/coreutils/include

-I/usr/tandem/xml/T0625L01_AAE/include"

Quoted 2 lines

LDFLAGS="/usr/coreutils/lib/libz.a
-L/usr/local-ssl3.4/lib -L/usr/coreutils/lib

-L/usr/tandem/xml/T0625L01_AAE/lib"

Quoted 16 lines

TEST_LINT= SHELL=/usr/coreutils/bin/bash

The results are:
*** t0000-basic.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0000-basic.sh' failed make[2]: *** [t0000-basic.sh] Error 1
*** t0001-init.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0001-init.sh' failed make[2]: *** [t0001-init.sh] Error 1
*** t0002-gitfile.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0002-gitfile.sh' failed make[2]: *** [t0002-gitfile.sh] Error 1 etc for all tests.

Quoted 2 lines

Removing GIT_TEST_DEFAULT_REF_FORMAT=reftable allows the whole test suite

to

Quoted 4 lines

pass.

There are environment size limits (about 32Kb) in bash on the box, so it is

possible a

Quoted 1 line

boundary was crossed internal to the test setup.

Quoted 8 lines
Quoted 2 lines

I did find this, in t0000-basic:
not ok 69 - init with GIT_DEFAULT_REF_FORMAT=reftable

You mean t0001-init.sh, right?

Quoted 1 line

but have no more details. When should I break in gdb to get a backtrace?

Can you please run that test with "-ix" and post the logs?

All tests, actually.

$ GIT_TEST_DEFAULT_REF_FORMAT=reftable GIT_TEST_CLONE_2GB=true sh
t0000-basic.sh --verbose -i -x
error: reftable: transaction prepare: out of memory
error: cannot run git init
#9 lore raw
In reply to Patrick Steinhardt
Quoted 49 lines · On December 18, 2024 11:07 AM, I wrote:
Quoted 48 lines · Patrick Steinhardt
Quoted 35 lines · On Tue, Dec 17, 2024 at 02:00:01PM -0500, [email protected] wrote:
Quoted 9 lines · On December 17, 2024 8:48 AM, I wrote:

This is on the latest OS version of NonStop x86. The test is done via the
following:
With OpenSSL 3.4 The T0625 is Expat.

When run as follows:
GIT_TEST_DEFAULT_REF_FORMAT=reftable GIT_TEST_CLONE_2GB=true /usr/coreutils/bin/make -k test V=1 CFLAGS="-g -O2 -Winline -I/usr/local- ssl3.4/include -I/usr/coreutils/include

-I/usr/tandem/xml/T0625L01_AAE/include"

Quoted 2 lines

LDFLAGS="/usr/coreutils/lib/libz.a
-L/usr/local-ssl3.4/lib -L/usr/coreutils/lib

-L/usr/tandem/xml/T0625L01_AAE/lib"

Quoted 16 lines

TEST_LINT= SHELL=/usr/coreutils/bin/bash

The results are:
*** t0000-basic.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0000-basic.sh' failed make[2]: *** [t0000-basic.sh] Error 1
*** t0001-init.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0001-init.sh' failed make[2]: *** [t0001-init.sh] Error 1
*** t0002-gitfile.sh ***
error: cannot run git init
Makefile:77: recipe for target 't0002-gitfile.sh' failed make[2]: *** [t0002-gitfile.sh] Error 1 etc for all tests.

Quoted 2 lines

Removing GIT_TEST_DEFAULT_REF_FORMAT=reftable allows the whole test suite

to

Quoted 2 lines

pass.

Quoted 7 lines
Quoted 2 lines

There are environment size limits (about 32Kb) in bash on the box, so it is

possible a

Quoted 1 line

boundary was crossed internal to the test setup.

I did find this, in t0000-basic:
not ok 69 - init with GIT_DEFAULT_REF_FORMAT=reftable

You mean t0001-init.sh, right?

Quoted 1 line

but have no more details. When should I break in gdb to get a backtrace?

Quoted 8 lines
Quoted 1 line

Can you please run that test with "-ix" and post the logs?

All tests, actually.

$ GIT_TEST_DEFAULT_REF_FORMAT=reftable GIT_TEST_CLONE_2GB=true sh t0000-
basic.sh --verbose -i -x
error: reftable: transaction prepare: out of memory
error: cannot run git init

Any updates or hypothesis on this? Our test system has loads of memory - I cannot
figure out where the allocation failure takes place. There is a limit to how much
memory can be allocated, but it is very high and our virtual memory is extensive,
but this is a 32-bit build.

--Randall

#10 lore raw
Quoted 8 lines · On Thu, Dec 19, 2024 at 03:46:20PM -0500, [email protected] wrote:
Quoted 6 lines · On December 18, 2024 11:07 AM, I wrote:

All tests, actually.

$ GIT_TEST_DEFAULT_REF_FORMAT=reftable GIT_TEST_CLONE_2GB=true sh t0000-
basic.sh --verbose -i -x
error: reftable: transaction prepare: out of memory
error: cannot run git init
Quoted 4 lines

Any updates or hypothesis on this? Our test system has loads of memory
- I cannot figure out where the allocation failure takes place. There is a limit to how much memory can be allocated, but it is very high and our virtual memory is extensive, but this is a 32-bit build.

My hypothesis is that this is caused by ps/reftable-alloc-failures, but I am unable to tell where exactly the error comes from. So I'm dependent on your input.

Could you please bisect the error? Finding out where the error is raised would also be quite helpful. It has to be one of the reftable functions that returns REFTABLE_OUT_OF_MEMORY_ERROR, but other than that I do not have any more gut feeling right now.

Patrick

#11 lore raw
Quoted 9 lines · Patrick Steinhardt
Quoted 8 lines · On Thu, Dec 19, 2024 at 03:46:20PM -0500, [email protected] wrote:
Quoted 6 lines · On December 18, 2024 11:07 AM, I wrote:

All tests, actually.

$ GIT_TEST_DEFAULT_REF_FORMAT=reftable GIT_TEST_CLONE_2GB=true sh
t0000- basic.sh --verbose -i -x
error: reftable: transaction prepare: out of memory
error: cannot run git init
Quoted 6 lines
Quoted 4 lines

Any updates or hypothesis on this? Our test system has loads of memory
- I cannot figure out where the allocation failure takes place. There is a limit to how much memory can be allocated, but it is very high and our virtual memory is extensive, but this is a 32-bit build.

My hypothesis is that this is caused by ps/reftable-alloc-failures, but I

am unable to

Quoted 3 lines

tell where exactly the error comes from. So I'm dependent on your input.

Could you please bisect the error? Finding out where the error is raised

would also

Quoted 3 lines

be quite helpful. It has to be one of the reftable functions that returns REFTABLE_OUT_OF_MEMORY_ERROR, but other than that I do not have any more gut feeling right now.

This is what bisect shows:

git bisect start
# status: waiting for both good and bad commits
# good: [777489f9e09c8d0dd6b12f9d90de6376330577a2] Git 2.47 git bisect good 777489f9e09c8d0dd6b12f9d90de6376330577a2 # status: waiting for bad commit, 1 good commit known # bad: [063bcebf0c917140ca0e705cbe0fdea127e90086] Git 2.48-rc0

git bisect bad 063bcebf0c917140ca0e705cbe0fdea127e90086
# bad: [2037ca85ad93ec905b46543df6df4080f6ca258b] worktree: refactor repair_worktree_after_gitdir_move()
git bisect bad 2037ca85ad93ec905b46543df6df4080f6ca258b
# bad: [6a11438f43469f3815f2f0fc997bd45792ff04c0] The fifth batch git bisect bad 6a11438f43469f3815f2f0fc997bd45792ff04c0
# bad: [f004467b042d735a2fe8bd5706b053b04b1aec65] Merge branch 'jh/config-unset-doc-fix'
git bisect bad f004467b042d735a2fe8bd5706b053b04b1aec65
# bad: [e29296745dc92fb03f8f60111b458adc69ff84c5] Merge branch 'sk/doc-maintenance-schedule'
git bisect bad e29296745dc92fb03f8f60111b458adc69ff84c5
# bad: [5b67cc6477ce88c499caab5ebcebd492ec78932d] reftable/stack: handle allocation failures in auto compaction
git bisect bad 5b67cc6477ce88c499caab5ebcebd492ec78932d
# good: [31f5b972e0231d4211987775dd58e67815734989] reftable/record: handle allocation failures when decoding records
git bisect good 31f5b972e0231d4211987775dd58e67815734989
# bad: [18da60029319733e2d931f2758a8e47b8b25b117] reftable/reader: handle allocation failures for unindexed reader
git bisect bad 18da60029319733e2d931f2758a8e47b8b25b117
# good: [74d1c18757d1a45b95e46836adf478193a34c42c] reftable/writer: handle allocation failures in reftable_new_writer()
git bisect good 74d1c18757d1a45b95e46836adf478193a34c42c

--Randall

#12 lore raw
Quoted 50 lines · On Fri, Dec 20, 2024 at 02:23:49PM -0500, [email protected] wrote:
Quoted 9 lines · Patrick Steinhardt
Quoted 8 lines · On Thu, Dec 19, 2024 at 03:46:20PM -0500, [email protected] wrote:
Quoted 6 lines · On December 18, 2024 11:07 AM, I wrote:

All tests, actually.

$ GIT_TEST_DEFAULT_REF_FORMAT=reftable GIT_TEST_CLONE_2GB=true sh
t0000- basic.sh --verbose -i -x
error: reftable: transaction prepare: out of memory
error: cannot run git init
Quoted 6 lines
Quoted 4 lines

Any updates or hypothesis on this? Our test system has loads of memory
- I cannot figure out where the allocation failure takes place. There is a limit to how much memory can be allocated, but it is very high and our virtual memory is extensive, but this is a 32-bit build.

My hypothesis is that this is caused by ps/reftable-alloc-failures, but I

am unable to

Quoted 3 lines

tell where exactly the error comes from. So I'm dependent on your input.

Could you please bisect the error? Finding out where the error is raised

would also

Quoted 3 lines

be quite helpful. It has to be one of the reftable functions that returns REFTABLE_OUT_OF_MEMORY_ERROR, but other than that I do not have any more gut feeling right now.

This is what bisect shows:

git bisect start
# status: waiting for both good and bad commits
# good: [777489f9e09c8d0dd6b12f9d90de6376330577a2] Git 2.47 git bisect good 777489f9e09c8d0dd6b12f9d90de6376330577a2 # status: waiting for bad commit, 1 good commit known # bad: [063bcebf0c917140ca0e705cbe0fdea127e90086] Git 2.48-rc0

git bisect bad 063bcebf0c917140ca0e705cbe0fdea127e90086
# bad: [2037ca85ad93ec905b46543df6df4080f6ca258b] worktree: refactor repair_worktree_after_gitdir_move()
git bisect bad 2037ca85ad93ec905b46543df6df4080f6ca258b
# bad: [6a11438f43469f3815f2f0fc997bd45792ff04c0] The fifth batch git bisect bad 6a11438f43469f3815f2f0fc997bd45792ff04c0
# bad: [f004467b042d735a2fe8bd5706b053b04b1aec65] Merge branch 'jh/config-unset-doc-fix'
git bisect bad f004467b042d735a2fe8bd5706b053b04b1aec65
# bad: [e29296745dc92fb03f8f60111b458adc69ff84c5] Merge branch 'sk/doc-maintenance-schedule'
git bisect bad e29296745dc92fb03f8f60111b458adc69ff84c5
# bad: [5b67cc6477ce88c499caab5ebcebd492ec78932d] reftable/stack: handle allocation failures in auto compaction
git bisect bad 5b67cc6477ce88c499caab5ebcebd492ec78932d
# good: [31f5b972e0231d4211987775dd58e67815734989] reftable/record: handle

Quoted 8 lines

allocation failures when decoding records
git bisect good 31f5b972e0231d4211987775dd58e67815734989
# bad: [18da60029319733e2d931f2758a8e47b8b25b117] reftable/reader: handle allocation failures for unindexed reader
git bisect bad 18da60029319733e2d931f2758a8e47b8b25b117
# good: [74d1c18757d1a45b95e46836adf478193a34c42c] reftable/writer: handle allocation failures in reftable_new_writer()
git bisect good 74d1c18757d1a45b95e46836adf478193a34c42c

This is missing the last step for git-bisect(1). Right now it could be caused by one of these commits:

    18da600293 (reftable/reader: handle allocation failures for unindexed reader, 2024-10-02)
    802c0646ac (reftable/merged: handle allocation failures in `merged_table_init_iter()`, 2024-10-02)

The first commit seems quite unlikely to be the root cause. The second commit is rather interesting though. I wonder whether NonStop's malloc returns a NULL pointer when given a size of 0?

A quick stab into the dark, but does below patch on top of master make things work for you?

Patrick

-- >8 --

reftable/merged.c +1 -1
index bb0836e344..7ae6f78d45 100644
@@ -244,7 +244,7 @@ int merged_table_init_iter(struct reftable_merged_table *mt,
 	struct merged_iter *mi = NULL;
 	int ret;
 
-	REFTABLE_CALLOC_ARRAY(subiters, mt->readers_len);
+	REFTABLE_CALLOC_ARRAY(subiters, mt->readers_len + 1);
 	if (!subiters) {
 		ret = REFTABLE_OUT_OF_MEMORY_ERROR;
 		goto out;
reftable/stack.c +1 -1
index 59fd695a12..1b6b8cc9ea 100644
@@ -1612,7 +1612,7 @@ static uint64_t *stack_table_sizes_for_compaction(struct reftable_stack *st)
 	int overhead = header_size(version) - 1;
 	uint64_t *sizes;
 
-	REFTABLE_CALLOC_ARRAY(sizes, st->merged->readers_len);
+	REFTABLE_CALLOC_ARRAY(sizes, st->merged->readers_len + 1);
 	if (!sizes)
 		return NULL;
 
#13 lore raw
Quoted 11 lines · Patrick Steinhardt
Quoted 10 lines · On Fri, Dec 20, 2024 at 02:23:49PM -0500, [email protected] wrote:
Quoted 9 lines · Patrick Steinhardt
Quoted 8 lines · On Thu, Dec 19, 2024 at 03:46:20PM -0500, [email protected] wrote:
Quoted 6 lines · On December 18, 2024 11:07 AM, I wrote:

All tests, actually.

$ GIT_TEST_DEFAULT_REF_FORMAT=reftable GIT_TEST_CLONE_2GB=true sh
t0000- basic.sh --verbose -i -x
error: reftable: transaction prepare: out of memory
error: cannot run git init
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Any updates or hypothesis on this? Our test system has loads of memory
- I cannot figure out where the allocation failure takes place. There is a limit to how much memory can be allocated, but it is very high and our virtual memory is extensive, but this is a 32-bit

build.

Quoted 47 lines
Quoted 38 lines
Quoted 3 lines

My hypothesis is that this is caused by ps/reftable-alloc-failures, but I

am unable to

Quoted 4 lines

tell where exactly the error comes from. So I'm dependent on your input.

Could you please bisect the error? Finding out where the error is raised

would also

Quoted 3 lines

be quite helpful. It has to be one of the reftable functions that returns REFTABLE_OUT_OF_MEMORY_ERROR, but other than that I do not have any more gut feeling right now.

This is what bisect shows:

git bisect start
# status: waiting for both good and bad commits # good:
[777489f9e09c8d0dd6b12f9d90de6376330577a2] Git 2.47 git bisect good 777489f9e09c8d0dd6b12f9d90de6376330577a2
# status: waiting for bad commit, 1 good commit known # bad:
[063bcebf0c917140ca0e705cbe0fdea127e90086] Git 2.48-rc0

git bisect bad 063bcebf0c917140ca0e705cbe0fdea127e90086
# bad: [2037ca85ad93ec905b46543df6df4080f6ca258b] worktree: refactor repair_worktree_after_gitdir_move()
git bisect bad 2037ca85ad93ec905b46543df6df4080f6ca258b
# bad: [6a11438f43469f3815f2f0fc997bd45792ff04c0] The fifth batch git bisect bad 6a11438f43469f3815f2f0fc997bd45792ff04c0
# bad: [f004467b042d735a2fe8bd5706b053b04b1aec65] Merge branch 'jh/config-unset-doc-fix'
git bisect bad f004467b042d735a2fe8bd5706b053b04b1aec65
# bad: [e29296745dc92fb03f8f60111b458adc69ff84c5] Merge branch 'sk/doc-maintenance-schedule'
git bisect bad e29296745dc92fb03f8f60111b458adc69ff84c5
# bad: [5b67cc6477ce88c499caab5ebcebd492ec78932d] reftable/stack:
handle allocation failures in auto compaction git bisect bad 5b67cc6477ce88c499caab5ebcebd492ec78932d
# good: [31f5b972e0231d4211987775dd58e67815734989] reftable/record:

Quoted 8 lines

handle allocation failures when decoding records git bisect good 31f5b972e0231d4211987775dd58e67815734989
# bad: [18da60029319733e2d931f2758a8e47b8b25b117] reftable/reader:
handle allocation failures for unindexed reader git bisect bad 18da60029319733e2d931f2758a8e47b8b25b117
# good: [74d1c18757d1a45b95e46836adf478193a34c42c] reftable/writer:
handle allocation failures in reftable_new_writer() git bisect good 74d1c18757d1a45b95e46836adf478193a34c42c

Quoted 1 line

This is missing the last step for git-bisect(1). Right now it could be

caused by one of

Quoted 3 lines

these commits:

18da600293 (reftable/reader: handle allocation failures for unindexed

reader,

Quoted 3 lines

2024-10-02)
802c0646ac (reftable/merged: handle allocation failures in
merged_table_init_iter(), 2024-10-02)

They are there, it is just that Outlook wrapped the lines on me. The 802c064 is not in my repo - I bisected from 2.47.0 to 2.48.0-rc0, so may have skipped a more recent commit than rc0 has.

Quoted 1 line

The first commit seems quite unlikely to be the root cause. The second

commit is

Quoted 4 lines

rather interesting though. I wonder whether NonStop's malloc returns a NULL pointer when given a size of 0?

A quick stab into the dark, but does below patch on top of master make

things

reftable/merged.c index

Quoted 7 lines

work for you?

Patrick

-- >8 --

reftable/merged.c index renamed from reftable/merged.c +0 -0

bb0836e344..7ae6f78d45

reftable/stack.c index

Quoted 14 lines

100644

reftable/merged.c +1 -1
@@ -244,7 +244,7 @@ int merged_table_init_iter(struct reftable_merged_table
 *mt,
 	struct merged_iter *mi = NULL;
 	int ret;
 
-	REFTABLE_CALLOC_ARRAY(subiters, mt->readers_len);
+	REFTABLE_CALLOC_ARRAY(subiters, mt->readers_len + 1);
 	if (!subiters) {
 		ret = REFTABLE_OUT_OF_MEMORY_ERROR;
		goto out;
reftable/stack.c index renamed from reftable/stack.c +0 -0

59fd695a12..1b6b8cc9ea

reftable/stack.c static uint64_t · line 1618

Quoted 13 lines

100644

reftable/stack.c +1 -1
@@ -1612,7 +1612,7 @@ static uint64_t
 *stack_table_sizes_for_compaction(struct reftable_stack *st)
 	int overhead = header_size(version) - 1;
 	uint64_t *sizes;
 
-	REFTABLE_CALLOC_ARRAY(sizes, st->merged->readers_len);
+	REFTABLE_CALLOC_ARRAY(sizes, st->merged->readers_len + 1);
 	if (!sizes)
 		return NULL;

The fix above does not appear to make any difference.

#14 lore raw
Quoted 117 lines · On Fri, Dec 20, 2024 at 04:21:53PM -0500, [email protected] wrote:
Quoted 11 lines · Patrick Steinhardt
Quoted 10 lines · On Fri, Dec 20, 2024 at 02:23:49PM -0500, [email protected] wrote:
Quoted 9 lines · Patrick Steinhardt
Quoted 8 lines · On Thu, Dec 19, 2024 at 03:46:20PM -0500, [email protected] wrote:
Quoted 6 lines · On December 18, 2024 11:07 AM, I wrote:

All tests, actually.

$ GIT_TEST_DEFAULT_REF_FORMAT=reftable GIT_TEST_CLONE_2GB=true sh
t0000- basic.sh --verbose -i -x
error: reftable: transaction prepare: out of memory
error: cannot run git init
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Any updates or hypothesis on this? Our test system has loads of memory
- I cannot figure out where the allocation failure takes place. There is a limit to how much memory can be allocated, but it is very high and our virtual memory is extensive, but this is a 32-bit

build.

Quoted 47 lines
Quoted 38 lines
Quoted 3 lines

My hypothesis is that this is caused by ps/reftable-alloc-failures, but I

am unable to

Quoted 4 lines

tell where exactly the error comes from. So I'm dependent on your input.

Could you please bisect the error? Finding out where the error is raised

would also

Quoted 3 lines

be quite helpful. It has to be one of the reftable functions that returns REFTABLE_OUT_OF_MEMORY_ERROR, but other than that I do not have any more gut feeling right now.

This is what bisect shows:

git bisect start
# status: waiting for both good and bad commits # good:
[777489f9e09c8d0dd6b12f9d90de6376330577a2] Git 2.47 git bisect good 777489f9e09c8d0dd6b12f9d90de6376330577a2
# status: waiting for bad commit, 1 good commit known # bad:
[063bcebf0c917140ca0e705cbe0fdea127e90086] Git 2.48-rc0

git bisect bad 063bcebf0c917140ca0e705cbe0fdea127e90086
# bad: [2037ca85ad93ec905b46543df6df4080f6ca258b] worktree: refactor repair_worktree_after_gitdir_move()
git bisect bad 2037ca85ad93ec905b46543df6df4080f6ca258b
# bad: [6a11438f43469f3815f2f0fc997bd45792ff04c0] The fifth batch git bisect bad 6a11438f43469f3815f2f0fc997bd45792ff04c0
# bad: [f004467b042d735a2fe8bd5706b053b04b1aec65] Merge branch 'jh/config-unset-doc-fix'
git bisect bad f004467b042d735a2fe8bd5706b053b04b1aec65
# bad: [e29296745dc92fb03f8f60111b458adc69ff84c5] Merge branch 'sk/doc-maintenance-schedule'
git bisect bad e29296745dc92fb03f8f60111b458adc69ff84c5
# bad: [5b67cc6477ce88c499caab5ebcebd492ec78932d] reftable/stack:
handle allocation failures in auto compaction git bisect bad 5b67cc6477ce88c499caab5ebcebd492ec78932d
# good: [31f5b972e0231d4211987775dd58e67815734989] reftable/record:

Quoted 8 lines

handle allocation failures when decoding records git bisect good 31f5b972e0231d4211987775dd58e67815734989
# bad: [18da60029319733e2d931f2758a8e47b8b25b117] reftable/reader:
handle allocation failures for unindexed reader git bisect bad 18da60029319733e2d931f2758a8e47b8b25b117
# good: [74d1c18757d1a45b95e46836adf478193a34c42c] reftable/writer:
handle allocation failures in reftable_new_writer() git bisect good 74d1c18757d1a45b95e46836adf478193a34c42c

Quoted 1 line

This is missing the last step for git-bisect(1). Right now it could be

caused by one of

Quoted 3 lines

these commits:

18da600293 (reftable/reader: handle allocation failures for unindexed

reader,

Quoted 3 lines

2024-10-02)
802c0646ac (reftable/merged: handle allocation failures in
merged_table_init_iter(), 2024-10-02)

They are there, it is just that Outlook wrapped the lines on me. The 802c064 is not in my repo - I bisected from 2.47.0 to 2.48.0-rc0, so may have skipped a more recent commit than rc0 has.

Quoted 1 line

The first commit seems quite unlikely to be the root cause. The second

commit is

Quoted 4 lines

rather interesting though. I wonder whether NonStop's malloc returns a NULL pointer when given a size of 0?

A quick stab into the dark, but does below patch on top of master make

things

reftable/merged.c index

Quoted 7 lines

work for you?

Patrick

-- >8 --

reftable/merged.c index renamed from reftable/merged.c +0 -0

bb0836e344..7ae6f78d45

reftable/stack.c index

Quoted 14 lines

100644

reftable/merged.c +1 -1
@@ -244,7 +244,7 @@ int merged_table_init_iter(struct reftable_merged_table
 *mt,
 	struct merged_iter *mi = NULL;
 	int ret;
 
-	REFTABLE_CALLOC_ARRAY(subiters, mt->readers_len);
+	REFTABLE_CALLOC_ARRAY(subiters, mt->readers_len + 1);
 	if (!subiters) {
 		ret = REFTABLE_OUT_OF_MEMORY_ERROR;
		goto out;
reftable/stack.c index renamed from reftable/stack.c +0 -0

59fd695a12..1b6b8cc9ea

Quoted 8 lines

100644

reftable/stack.c +0 -0
@@ -1612,7 +1612,7 @@ static uint64_t
 *stack_table_sizes_for_compaction(struct reftable_stack *st)
 	int overhead = header_size(version) - 1;
 	uint64_t *sizes;
 
Quoted 7 lines

reftable/stack.c static uint64_t *stack_table_sizes_for_compaction(struct reftable_stack *st) · line 1617

Quoted 5 lines
reftable/stack.c +1 -1
-	REFTABLE_CALLOC_ARRAY(sizes, st->merged->readers_len);
+	REFTABLE_CALLOC_ARRAY(sizes, st->merged->readers_len + 1);
 	if (!sizes)
 		return NULL;

The fix above does not appear to make any difference.

Indeed, there was one more issue in reftable_stack_reload_once(). I could now reproduce those errors by adapting reftable_malloc() and reftable_realloc() to return NULL pointers when asked for a zero-sized allocation. This removes any implementation-defined behaviour of allocators and allowed me to surface the issues.

I've sent a patch series now [1] that should address your issues.

[1]: <20241221-b4-pks-reftable-oom-fix-without-readers-v1-0-12db83a3267c@pks.im>

Patrick

#15 lore raw
Quoted 5 lines · Patrick Steinhardt
Quoted 4 lines · On Fri, Dec 20, 2024 at 04:21:53PM -0500, [email protected] wrote:
Quoted 3 lines · Patrick Steinhardt
Quoted 2 lines · On Fri, Dec 20, 2024 at 02:23:49PM -0500, [email protected] wrote:
Quoted 1 line · Patrick Steinhardt

On Thu, Dec 19, 2024 at 03:46:20PM -0500, [email protected]

Quoted 15 lines · wrote:
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines · On December 18, 2024 11:07 AM, I wrote:
Quoted 7 lines

All tests, actually.

$ GIT_TEST_DEFAULT_REF_FORMAT=reftable GIT_TEST_CLONE_2GB=true
sh
t0000- basic.sh --verbose -i -x
error: reftable: transaction prepare: out of memory
error: cannot run git init

Any updates or hypothesis on this? Our test system has loads of memory
- I cannot figure out where the allocation failure takes place. There is a limit to how much memory can be allocated, but it is very high and our virtual memory is extensive, but this is a 32-bit

Quoted 6 lines
Quoted 6 lines

build.

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

My hypothesis is that this is caused by
ps/reftable-alloc-failures, but I

am unable to

Quoted 1 line

tell where exactly the error comes from. So I'm dependent on your

input.

Quoted 106 lines
Quoted 98 lines
Quoted 42 lines
Quoted 33 lines
Quoted 3 lines

Could you please bisect the error? Finding out where the error is raised

would also

Quoted 3 lines

be quite helpful. It has to be one of the reftable functions that returns REFTABLE_OUT_OF_MEMORY_ERROR, but other than that I do not have any more gut feeling right now.

This is what bisect shows:

git bisect start
# status: waiting for both good and bad commits # good:
[777489f9e09c8d0dd6b12f9d90de6376330577a2] Git 2.47 git bisect good 777489f9e09c8d0dd6b12f9d90de6376330577a2
# status: waiting for bad commit, 1 good commit known # bad:
[063bcebf0c917140ca0e705cbe0fdea127e90086] Git 2.48-rc0

git bisect bad 063bcebf0c917140ca0e705cbe0fdea127e90086 # bad: [2037ca85ad93ec905b46543df6df4080f6ca258b] worktree:
refactor repair_worktree_after_gitdir_move()
git bisect bad 2037ca85ad93ec905b46543df6df4080f6ca258b # bad: [6a11438f43469f3815f2f0fc997bd45792ff04c0] The fifth batch git bisect bad 6a11438f43469f3815f2f0fc997bd45792ff04c0 # bad: [f004467b042d735a2fe8bd5706b053b04b1aec65] Merge branch 'jh/config-unset-doc-fix'
git bisect bad f004467b042d735a2fe8bd5706b053b04b1aec65 # bad: [e29296745dc92fb03f8f60111b458adc69ff84c5] Merge branch 'sk/doc-maintenance-schedule'
git bisect bad e29296745dc92fb03f8f60111b458adc69ff84c5 # bad: [5b67cc6477ce88c499caab5ebcebd492ec78932d] reftable/stack:
handle allocation failures in auto compaction git bisect bad 5b67cc6477ce88c499caab5ebcebd492ec78932d
# good: [31f5b972e0231d4211987775dd58e67815734989] reftable/record:

Quoted 8 lines

handle allocation failures when decoding records git bisect good 31f5b972e0231d4211987775dd58e67815734989
# bad: [18da60029319733e2d931f2758a8e47b8b25b117] reftable/reader:
handle allocation failures for unindexed reader git bisect bad 18da60029319733e2d931f2758a8e47b8b25b117
# good: [74d1c18757d1a45b95e46836adf478193a34c42c] reftable/writer:
handle allocation failures in reftable_new_writer() git bisect good 74d1c18757d1a45b95e46836adf478193a34c42c

Quoted 2 lines

This is missing the last step for git-bisect(1). Right now it could be

caused by one of

Quoted 4 lines

these commits:

18da600293 (reftable/reader: handle allocation failures for
unindexed

reader,

Quoted 3 lines

2024-10-02)
802c0646ac (reftable/merged: handle allocation failures in
merged_table_init_iter(), 2024-10-02)

They are there, it is just that Outlook wrapped the lines on me. The 802c064 is not in my repo - I bisected from 2.47.0 to 2.48.0-rc0, so may have skipped a more recent commit than rc0 has.

Quoted 2 lines

The first commit seems quite unlikely to be the root cause. The second

commit is

Quoted 5 lines

rather interesting though. I wonder whether NonStop's malloc returns a NULL pointer when given a size of 0?

A quick stab into the dark, but does below patch on top of master make

things

reftable/merged.c index

Quoted 7 lines

work for you?

Patrick

-- >8 --

reftable/merged.c index renamed from reftable/merged.c +0 -0

bb0836e344..7ae6f78d45

reftable/stack.c index

Quoted 14 lines

100644

reftable/merged.c +1 -1
@@ -244,7 +244,7 @@ int merged_table_init_iter(struct
 reftable_merged_table *mt,
 	struct merged_iter *mi = NULL;
 	int ret;
 
-	REFTABLE_CALLOC_ARRAY(subiters, mt->readers_len);
+	REFTABLE_CALLOC_ARRAY(subiters, mt->readers_len + 1);
 	if (!subiters) {
 		ret = REFTABLE_OUT_OF_MEMORY_ERROR;
		goto out;
reftable/stack.c index renamed from reftable/stack.c +0 -0

59fd695a12..1b6b8cc9ea

Quoted 8 lines

100644

reftable/stack.c +0 -0
@@ -1612,7 +1612,7 @@ static uint64_t
 *stack_table_sizes_for_compaction(struct reftable_stack *st)
 	int overhead = header_size(version) - 1;
 	uint64_t *sizes;
 
Quoted 7 lines

reftable/stack.c static uint64_t *stack_table_sizes_for_compaction(struct reftable_stack *st) · line 1617

Quoted 5 lines
reftable/stack.c +1 -1
-	REFTABLE_CALLOC_ARRAY(sizes, st->merged->readers_len);
+	REFTABLE_CALLOC_ARRAY(sizes, st->merged->readers_len + 1);
 	if (!sizes)
 		return NULL;

The fix above does not appear to make any difference.

Quoted 1 line

Indeed, there was one more issue in reftable_stack_reload_once(). I could

now

Quoted 1 line

reproduce those errors by adapting reftable_malloc() and

reftable_realloc() to

Quoted 1 line

return NULL pointers when asked for a zero-sized allocation. This removes

any

Quoted 1 line

implementation-defined behaviour of allocators and allowed me to surface

the

Quoted 6 lines

issues.

I've sent a patch series now [1] that should address your issues.

[1]: <20241221-b4-pks-reftable-oom-fix-without-readers-v1-0- [email protected]>

Thank you, Patrick. This looks like a good and appropriate series to me. NonStop
indeed returns NULL when malloc(0) is invoked.
--Randall