reftable: handle allocation errors

Hi,

this is the fifth version of this patch series that converts the reftable library to start handling allocation errors. This is done such that the reftable library can truly behave like a library and let its callers handle such conditions.

Changes compared to v4:

- #undef standard allocators before banning them.

- Explain why we don't ban FREE_AND_NULL() and related as well as xmalloc() and related.

Thanks!

Patrick

Patrick Steinhardt (25):
reftable/error: introduce out-of-memory error code reftable/basics: merge "publicbasics" into "basics"
reftable: introduce reftable_strdup()
reftable/basics: handle allocation failures in reftable_calloc() reftable/basics: handle allocation failures in parse_names() reftable/record: handle allocation failures on copy reftable/record: handle allocation failures when decoding records reftable/writer: handle allocation failures in writer_index_hash() reftable/writer: handle allocation failures in reftable_new_writer() reftable/merged: handle allocation failures in

    `merged_table_init_iter()`

reftable/reader: handle allocation failures for unindexed reader reftable/reader: handle allocation failures in reader_init_iter() reftable/stack: handle allocation failures on reload reftable/stack: handle allocation failures in reftable_new_stack() reftable/stack: handle allocation failures in stack_compact_range() reftable/stack: handle allocation failures in auto compaction reftable/iter: handle allocation failures when creating indexed table
iter
reftable/blocksource: handle allocation failures reftable/block: handle allocation failures
reftable/pq: handle allocation failures when adding entries reftable/tree: handle allocation failures
reftable: handle trivial allocation failures
reftable: fix calls to free(3P)
reftable: introduce REFTABLE_FREE_AND_NULL() reftable/basics: ban standard allocator functions

 Makefile                            |   1 -
 refs/reftable-backend.c             |  39 ++++--
 reftable/basics.c                   |  93 +++++++++++++-
 reftable/basics.h                   |  28 ++++-
 reftable/block.c                    |  29 ++++-
 reftable/block.h                    |   4 +-
 reftable/blocksource.c              |  25 +++-
 reftable/error.c                    |   2 +
 reftable/iter.c                     |  22 +++-
 reftable/iter.h                     |   2 +-
 reftable/merged.c                   |  84 ++++++++-----
 reftable/merged.h                   |   6 +-
 reftable/pq.c                       |   9 +-
 reftable/pq.h                       |   2 +-
 reftable/publicbasics.c             |  66 ----------
 reftable/reader.c                   |  70 ++++++++---
 reftable/reader.h                   |   6 +-
 reftable/record.c                   | 174 +++++++++++++++++++-------
 reftable/record.h                   |   6 +-
 reftable/reftable-basics.h          |  18 +++
 reftable/reftable-error.h           |   3 +
 reftable/reftable-malloc.h          |  18 ---
 reftable/reftable-merged.h          |   8 +-
 reftable/reftable-reader.h          |   8 +-
 reftable/reftable-stack.h           |   8 +-
 reftable/reftable-writer.h          |  12 +-
 reftable/stack.c                    | 187 +++++++++++++++++++++-------
 reftable/tree.c                     |  42 +++++--
 reftable/tree.h                     |  21 +++-
 reftable/writer.c                   | 154 +++++++++++++++--------
 t/helper/test-reftable.c            |  10 +-
 t/unit-tests/lib-reftable.c         |   8 +-
 t/unit-tests/t-reftable-basics.c    |  11 +-
 t/unit-tests/t-reftable-block.c     |  24 ++--
 t/unit-tests/t-reftable-merged.c    |  16 ++-
 t/unit-tests/t-reftable-readwrite.c |  65 ++++++----
 t/unit-tests/t-reftable-stack.c     |   6 +-
 t/unit-tests/t-reftable-tree.c      |  10 +-
 38 files changed, 890 insertions(+), 407 deletions(-)
 delete mode 100644 reftable/publicbasics.c
 create mode 100644 reftable/reftable-basics.h
 delete mode 100644 reftable/reftable-malloc.h

Range-diff against v4:

 1:  94eaef3ae5 =  1:  94eaef3ae5 reftable/error: introduce out-of-memory error code
 2:  fe55051cb7 =  2:  fe55051cb7 reftable/basics: merge "publicbasics" into "basics"
 3:  b1a10d41d3 =  3:  b1a10d41d3 reftable: introduce `reftable_strdup()`
 4:  fd9141e985 =  4:  fd9141e985 reftable/basics: handle allocation failures in `reftable_calloc()`
 5:  bdfddbebce =  5:  bdfddbebce reftable/basics: handle allocation failures in `parse_names()`
 6:  756a32c285 =  6:  756a32c285 reftable/record: handle allocation failures on copy
 7:  ca64971e7b =  7:  ca64971e7b reftable/record: handle allocation failures when decoding records
 8:  fc2f113cba =  8:  fc2f113cba reftable/writer: handle allocation failures in `writer_index_hash()`
 9:  0ed99e0bdf =  9:  0ed99e0bdf reftable/writer: handle allocation failures in `reftable_new_writer()`
10:  8dfbfd9286 = 10:  8dfbfd9286 reftable/merged: handle allocation failures in `merged_table_init_iter()`
11:  7b592a6c6b = 11:  7b592a6c6b reftable/reader: handle allocation failures for unindexed reader
12:  ceed838265 = 12:  ceed838265 reftable/reader: handle allocation failures in `reader_init_iter()`
13:  1e997a5766 = 13:  1e997a5766 reftable/stack: handle allocation failures on reload
14:  40d4d81378 = 14:  40d4d81378 reftable/stack: handle allocation failures in `reftable_new_stack()`
15:  6aaae4baa6 = 15:  6aaae4baa6 reftable/stack: handle allocation failures in `stack_compact_range()`
16:  103a59ef0a = 16:  103a59ef0a reftable/stack: handle allocation failures in auto compaction
17:  6799d299fe = 17:  6799d299fe reftable/iter: handle allocation failures when creating indexed table iter
18:  c7e54d71d7 = 18:  c7e54d71d7 reftable/blocksource: handle allocation failures
19:  92d39b9021 = 19:  92d39b9021 reftable/block: handle allocation failures
20:  3416004e0d = 20:  3416004e0d reftable/pq: handle allocation failures when adding entries
21:  a363930488 = 21:  a363930488 reftable/tree: handle allocation failures
22:  28661500ff = 22:  28661500ff reftable: handle trivial allocation failures
23:  228cc81263 = 23:  228cc81263 reftable: fix calls to free(3P)
24:  1c66f6ef8d = 24:  1c66f6ef8d reftable: introduce `REFTABLE_FREE_AND_NULL()`
25:  764961e6f0 ! 25:  bc80f1e726 reftable/basics: ban standard allocator functions

@@ Commit message
Introduce a couple of macros that ban the standard allocators, similar to how we do it in "banned.h".

+ Note that we do not ban the following two classes of functions:
+
+ - Macros like FREE_AND_NULL() or REALLOC_ARRAY(). As those expand
+ to code that contains already-banned functions we'd get a compiler
+ error even without banning those macros explicitly.
+
+ - Git-specific allocators like xmalloc() and friends. The primary
+ reason is that there are simply too many of them, so we're rather
+ aiming for best effort here. Furthermore, the eventual goal is to
+ make them unavailable in the reftable library place by not pulling
+ them in via "git-compat-utils.h" anymore.
+

## reftable/basics.c ##
@@ reftable/basics.h: char *reftable_strdup(const char *str);

+#ifndef REFTABLE_ALLOW_BANNED_ALLOCATORS
+# define REFTABLE_BANNED(func) use_reftable_##func##_instead
++# undef malloc
+# define malloc(sz) REFTABLE_BANNED(malloc)
++# undef realloc
+# define realloc(ptr, sz) REFTABLE_BANNED(realloc)
++# undef free
+# define free(ptr) REFTABLE_BANNED(free)
++# undef calloc
+# define calloc(nelem, elsize) REFTABLE_BANNED(calloc)
++# undef strdup
+# define strdup(str) REFTABLE_BANNED(strdup) +#endif
+

2.47.0.rc0.dirty