Skip to contentSkip to conversation

Reftable coverity fixes

PATCH Superseded Han-Wen Nienhuys via GitGitGadget 10 participants 194 messages Dec 7, 2021 – Mar 29, 2022 1 message not in the archive Page 2 of 2

lore
Share & tools
lore
#167 lore raw

v7 cover Reftable coverity fixes

In reply to Han-Wen Nienhuys via GitGitGadget

This series was targeted to 'master'.

This addresses some complaints from Coverity that Peff reported.

v7:

* address avarab's comments.

Han-Wen Nienhuys (16):
reftable: fix OOB stack write in print functions
reftable: fix resource leak in block.c error path
reftable: fix resource leak blocksource.c
reftable: check reftable_stack_auto_compact() return value
reftable: ignore remove() return value in stack_test.c
reftable: fix resource warning
reftable: all xxx_free() functions accept NULL arguments
reftable: order unittests by complexity
reftable: drop stray printf in readwrite_test
reftable: handle null refnames in reftable_ref_record_equal
reftable: make reftable-record.h function signatures const correct
reftable: implement record equality generically
reftable: remove outdated file reftable.c
reftable: make reftable_record a tagged union
reftable: add print functions to the record types
reftable: rename typ to type

 reftable/block.c           |  35 ++--
 reftable/block_test.c      |  22 +--
 reftable/blocksource.c     |   6 +-
 reftable/generic.c         |  39 ++--
 reftable/iter.c            |   4 +-
 reftable/merged.c          |  35 ++--
 reftable/merged.h          |   2 +-
 reftable/pq.c              |   2 +-
 reftable/pq_test.c         |  29 ++-
 reftable/reader.c          | 167 +++++++++--------
 reftable/readwrite_test.c  |  67 ++++++-
 reftable/record.c          | 367 +++++++++++++++++++++++--------------
 reftable/record.h          |  49 ++---
 reftable/record_test.c     | 193 +++++++++----------
 reftable/reftable-record.h |  14 +-
 reftable/reftable.c        | 115 ------------
 reftable/stack.c           |  10 +-
 reftable/stack_test.c      |   3 +-
 reftable/writer.c          |  72 ++++----
 t/helper/test-reftable.c   |   9 +-
 20 files changed, 666 insertions(+), 574 deletions(-)
 delete mode 100644 reftable/reftable.c

base-commit: 50b2d72e110cad39ecaf2322bfdf1b60cd13dd96
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1152%2Fhanwen%2Freftable-coverity-v7
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1152/hanwen/reftable-coverity-v7
Pull-Request: https://github.com/git/git/pull/1152

Range-diff vs v6:

1: 9218bd59b25 ! 1: cc47ec7a50b reftable: fix OOB stack write in print functions

     @@ Metadata
       ## Commit message ##
          reftable: fix OOB stack write in print functions
      
     +    Remove /* BUG */ comment, whose meaning has escaped my memory.
     +
          Signed-off-by: Han-Wen Nienhuys <[email protected]>
      
       ## reftable/record.c ##
     @@ reftable/record.c: static void hex_format(char *dest, uint8_t *src, int hash_siz
       			       uint32_t hash_id)
       {
      -	char hex[2 * GIT_SHA256_RAWSZ + 1] = { 0 }; /* BUG */
     -+	char hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */
     ++	char hex[GIT_MAX_HEXSZ + 1];
       	printf("ref{%s(%" PRIu64 ") ", ref->refname, ref->update_index);
       	switch (ref->value_type) {
       	case REFTABLE_REF_SYMREF:
  2:  315ce62e710 !  2:  1c5441d2f7e reftable: fix resource leak in block.c error path
     @@ Commit message
      
       ## reftable/block.c ##
      @@ reftable/block.c: int block_reader_init(struct block_reader *br, struct reftable_block *block,
     - 	uint32_t full_block_size = table_block_size;
       	uint8_t typ = block->data[header_off];
       	uint32_t sz = get_be24(block->data + header_off + 1);
     --
     + 
      +	int err = 0;
       	uint16_t restart_count = 0;
       	uint32_t restart_start = 0;
  3:  7cf65e141cf =  3:  f11a9964383 reftable: fix resource leak blocksource.c
  4:  8740ba92981 !  4:  b0bffb6e29d reftable: check reftable_stack_auto_compact() return value

@@ Metadata
## Commit message ##
reftable: check reftable_stack_auto_compact() return value

- Fixes a problem detected by Coverity.
+ If compaction consistently errors out, we would accumulate reftables,
+ and the follow-on EXPECT would start failing, but catching errors
+ early helps debugging.
+
+ Spotted by Coverity.

  5:  08be6d90a48 !  5:  e3d575ae2b4 reftable: ignore remove() return value in stack_test.c
     @@ reftable/stack_test.c: static void test_read_file(void)
       	}
       	free_names(names);
      -	remove(fn);
     -+	(void) remove(fn);
     ++	EXPECT(unlink(fn) == 0);
       }

static void test_parse_names(void)

  6:  630f67bdb51 =  6:  a21d934f37d reftable: fix resource warning
  7:  afee67fb60d =  7:  b40d91c50a2 reftable: all xxx_free() functions accept NULL arguments
  8:  6f4f24c17ad =  8:  7a1f32a2ff0 reftable: order unittests by complexity
  9:  0abaab1d7c4 =  9:  a118cd40975 reftable: drop stray printf in readwrite_test
 10:  004b7a736bf = 10:  1d5cfbbc47a reftable: handle null refnames in reftable_ref_record_equal
 11:  951efcebcd2 ! 11:  90c72a2df0c reftable: make reftable-record.h function signatures const correct
     @@ reftable/record.c: static void hex_format(char *dest, uint8_t *src, int hash_siz
      +void reftable_ref_record_print(const struct reftable_ref_record *ref,
       			       uint32_t hash_id)
       {
     - 	char hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */
     + 	char hex[GIT_MAX_HEXSZ + 1];
      @@ reftable/record.c: static int zero_hash_eq(uint8_t *a, uint8_t *b, int sz)
       	return !memcmp(a, b, sz);
       }
 12:  6385e449ba7 ! 12:  a83a9cf59b7 reftable: implement record equality generically
     @@ reftable/record.c: static int reftable_ref_record_is_deletion_void(const void *p
       		(const struct reftable_ref_record *)p);
       }
       
     -+
      +static int reftable_ref_record_equal_void(const void *a,
      +					  const void *b, int hash_size)
      +{
     @@ reftable/record.c: static int not_a_deletion(const void *p)
      +	    || ra->offset_len != rb->offset_len)
      +		return 0;
      +
     -+	if (ra->hash_prefix_len &&
     -+	    memcmp(ra->hash_prefix, rb->hash_prefix, ra->hash_prefix_len))
     ++	if (memcmp(ra->hash_prefix, rb->hash_prefix, ra->hash_prefix_len))
      +		return 0;
     -+	if (ra->offset_len &&
     -+	    memcmp(ra->offsets, rb->offsets, ra->offset_len * sizeof(uint64_t)))
     ++	if (memcmp(ra->offsets, rb->offsets, ra->offset_len * sizeof(uint64_t)))
      +		return 0;
      +
      +	return 1;
 13:  69c4a554882 = 13:  7e23d14d1af reftable: remove outdated file reftable.c
 14:  54bcac325bd ! 14:  047bbb75fcb reftable: make reftable_record a tagged union
     @@ reftable/generic.c: https://developers.google.com/open-source/licenses/bsd
       {
      -	struct reftable_ref_record ref = {
      -		.refname = (char *)name,
     --	};
     ++	struct reftable_record rec = {
     ++		.type = BLOCK_TYPE_REF,
     ++		.u.ref = { .refname = (char *)name, }
     + 	};
      -	struct reftable_record rec = { NULL };
      -	reftable_record_from_ref(&rec, &ref);
     -+	struct reftable_record rec = { .type = BLOCK_TYPE_REF,
     -+				       .u.ref = {
     -+					       .refname = (char *)name,
     -+				       } };
       	return tab->ops->seek_record(tab->table_arg, it, &rec);
       }
       
     @@ reftable/generic.c: https://developers.google.com/open-source/licenses/bsd
      -	struct reftable_log_record log = {
      -		.refname = (char *)name,
      -		.update_index = ~((uint64_t)0),
     --	};
     ++	struct reftable_record rec = {
     ++		.type = BLOCK_TYPE_LOG,
     ++		.u.log = {
     ++			.refname = (char *)name,
     ++			.update_index = ~((uint64_t)0),
     ++		}
     + 	};
      -	struct reftable_record rec = { NULL };
      -	reftable_record_from_log(&rec, &log);
     -+	struct reftable_record rec = { .type = BLOCK_TYPE_LOG,
     -+				       .u.log = {
     -+					       .refname = (char *)name,
     -+					       .update_index = ~((uint64_t)0),
     -+				       } };
       	return tab->ops->seek_record(tab->table_arg, it, &rec);
       }
       
     @@ reftable/merged.c: int reftable_merged_table_seek_ref(struct reftable_merged_tab
      -		.refname = (char *)name,
      +	struct reftable_record rec = {
      +		.type = BLOCK_TYPE_REF,
     -+		.u.ref = {
     -+			.refname = (char *)name,
     -+		},
     ++		.u.ref = { .refname = (char *)name, },
       	};
      -	struct reftable_record rec = { NULL };
      -	reftable_record_from_ref(&rec, &ref);
     @@ reftable/merged.c: int reftable_merged_table_seek_log_at(struct reftable_merged_
      -	struct reftable_log_record log = {
      -		.refname = (char *)name,
      -		.update_index = update_index,
     --	};
     ++	struct reftable_record rec = {
     ++		.type = BLOCK_TYPE_LOG,
     ++		.u.log = {
     ++			.refname = (char *)name,
     ++			.update_index = update_index,
     ++		}
     + 	};
      -	struct reftable_record rec = { NULL };
      -	reftable_record_from_log(&rec, &log);
     -+	struct reftable_record rec = { .type = BLOCK_TYPE_LOG,
     -+				       .u.log = {
     -+					       .refname = (char *)name,
     -+					       .update_index = update_index,
     -+				       } };
       	return merged_table_seek_record(mt, it, &rec);
       }
       
      
       ## reftable/pq.c ##
     -@@ reftable/pq.c: struct pq_entry merged_iter_pqueue_remove(struct merged_iter_pqueue *pq)
     - void merged_iter_pqueue_add(struct merged_iter_pqueue *pq, struct pq_entry e)
     - {
     - 	int i = 0;
     -+
     - 	if (pq->len == pq->cap) {
     - 		pq->cap = 2 * pq->cap + 1;
     - 		pq->heap = reftable_realloc(pq->heap,
      @@ reftable/pq.c: void merged_iter_pqueue_release(struct merged_iter_pqueue *pq)
       {
       	int i = 0;
     @@ reftable/pq_test.c: static void test_pq(void)
      -
      -		reftable_record_as_ref(&rec)->refname = names[i];
      -		e.rec = rec;
     -+		struct pq_entry e = { .rec = { .type = BLOCK_TYPE_REF,
     -+					       .u.ref = {
     -+						       .refname = names[i],
     -+					       } } };
     ++		struct pq_entry e = {
     ++			.rec = {
     ++				.type = BLOCK_TYPE_REF,
     ++				.u.ref = { .refname = names[i], }
     ++			}
     ++		};
       		merged_iter_pqueue_add(&pq, e);
       		merged_iter_pqueue_check(pq);
       		i = (i * 7) % N;
     @@ reftable/pq_test.c: static void test_pq(void)
      -		last = ref->refname;
      -		ref->refname = NULL;
      -		reftable_free(ref);
     -+		// this is names[i], so don't dealloc.
     ++		/* this is names[i], so don't dealloc. */
      +		last = rec->u.ref.refname;
      +		rec->u.ref.refname = NULL;
      +		reftable_record_release(rec);
     @@ reftable/reader.c: static int reader_seek_indexed(struct reftable_reader *r,
      -	reftable_record_key(rec, &want_index.last_key);
      -	reftable_record_from_index(&want_index_rec, &want_index);
      -	reftable_record_from_index(&index_result_rec, &index_result);
     --
      +	reftable_record_key(rec, &want_index.u.idx.last_key);
     + 
       	err = reader_start(r, &index_iter, reftable_record_type(rec), 1);
       	if (err < 0)
       		goto done;
     @@ reftable/reader.c: static int reader_seek(struct reftable_reader *r, struct reft
      -		.refname = (char *)name,
      +	struct reftable_record rec = {
      +		.type = BLOCK_TYPE_REF,
     -+		.u.ref = {
     -+			.refname = (char *)name,
     -+		},
     ++		.u.ref = { .refname = (char *)name, },
       	};
      -	struct reftable_record rec = { NULL };
      -	reftable_record_from_ref(&rec, &ref);
     @@ reftable/reader.c: int reftable_reader_seek_log_at(struct reftable_reader *r,
      -	struct reftable_log_record log = {
      -		.refname = (char *)name,
      -		.update_index = update_index,
     --	};
     ++	struct reftable_record rec = {
     ++		.type = BLOCK_TYPE_LOG,
     ++		.u.log = {
     ++			.refname = (char *)name,
     ++			.update_index = update_index,
     ++		}
     + 	};
      -	struct reftable_record rec = { NULL };
      -	reftable_record_from_log(&rec, &log);
     -+	struct reftable_record rec = { .type = BLOCK_TYPE_LOG,
     -+				       .u.log = {
     -+					       .refname = (char *)name,
     -+					       .update_index = update_index,
     -+				       } };
       	return reader_seek(r, it, &rec);
       }
       
     @@ reftable/record.c: https://developers.google.com/open-source/licenses/bsd
       
      +static struct reftable_record_vtable *
      +reftable_record_vtable(struct reftable_record *rec);
     -+static void *reftable_record_data(struct reftable_record *rec);
     ++
     ++static void *reftable_record_data(struct reftable_record *rec)
     ++{
     ++	switch (rec->type) {
     ++	case BLOCK_TYPE_REF:
     ++		return &rec->u.ref;
     ++	case BLOCK_TYPE_LOG:
     ++		return &rec->u.log;
     ++	case BLOCK_TYPE_INDEX:
     ++		return &rec->u.idx;
     ++	case BLOCK_TYPE_OBJ:
     ++		return &rec->u.obj;
     ++	}
     ++	abort();
     ++}
      +
       int get_var_int(uint64_t *dest, struct string_view *in)
       {
     @@ reftable/record.c: static void reftable_obj_record_copy_from(void *rec, const vo
       	reftable_obj_record_release(obj);
      -	*obj = *src;
      -	obj->hash_prefix = reftable_malloc(obj->hash_prefix_len);
     --	memcpy(obj->hash_prefix, src->hash_prefix, obj->hash_prefix_len);
      +	obj->hash_prefix = reftable_malloc(src->hash_prefix_len);
      +	obj->hash_prefix_len = src->hash_prefix_len;
     -+	if (src->hash_prefix_len)
     -+		memcpy(obj->hash_prefix, src->hash_prefix, obj->hash_prefix_len);
     + 	memcpy(obj->hash_prefix, src->hash_prefix, obj->hash_prefix_len);
       
      -	obj->offsets = reftable_malloc(obj->offset_len * sizeof(uint64_t));
      -	COPY_ARRAY(obj->offsets, src->offsets, obj->offset_len);
     @@ reftable/record.c: static struct reftable_record_vtable reftable_log_record_vtab
       static void reftable_index_record_key(const void *r, struct strbuf *dest)
       {
       	const struct reftable_index_record *rec = r;
     +@@ reftable/record.c: static void reftable_index_record_release(void *rec)
     + {
     + 	struct reftable_index_record *idx = rec;
     + 	strbuf_release(&idx->last_key);
     ++	idx->offset = 0;
     + }
     + 
     + static uint8_t reftable_index_record_val_type(const void *rec)
      @@ reftable/record.c: static struct reftable_record_vtable reftable_index_record_vtable = {
       
       void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)
     @@ reftable/record.c: void string_view_consume(struct string_view *s, int n)
       	s->len -= n;
       }
      +
     -+static void *reftable_record_data(struct reftable_record *rec)
     -+{
     -+	switch (rec->type) {
     -+	case BLOCK_TYPE_REF:
     -+		return &rec->u.ref;
     -+	case BLOCK_TYPE_LOG:
     -+		return &rec->u.log;
     -+	case BLOCK_TYPE_INDEX:
     -+		return &rec->u.idx;
     -+	case BLOCK_TYPE_OBJ:
     -+		return &rec->u.obj;
     -+	}
     -+	abort();
     -+}
     -+
      +static struct reftable_record_vtable *
      +reftable_record_vtable(struct reftable_record *rec)
      +{
     @@ reftable/record.c: void string_view_consume(struct string_view *s, int n)
      +	abort();
      +}
      +
     -+struct reftable_record reftable_new_record(uint8_t typ)
     ++struct reftable_record reftable_new_record(uint8_t type)
      +{
      +	struct reftable_record clean = {
     -+		.type = typ,
     ++		.type = type,
      +	};
      +
      +	/* the following is involved, but the naive solution (just return
      +	 * `clean` as is, except for BLOCK_TYPE_INDEX), returns a garbage
      +	 * clean.u.obj.offsets pointer on Windows VS CI.  Go figure.
      +	 */
     -+	switch (typ) {
     ++	switch (type) {
      +	case BLOCK_TYPE_OBJ:
      +	{
      +		struct reftable_obj_record obj = { 0 };
     @@ reftable/record_test.c
      -	struct reftable_record copy =
      -		reftable_new_record(reftable_record_type(rec));
      +	struct reftable_record copy = { 0 };
     -+	uint8_t typ;
     -+
     -+	typ = reftable_record_type(rec);
     -+	copy = reftable_new_record(typ);
     ++	uint8_t type = reftable_record_type(rec);
     ++	copy = reftable_new_record(type);
       	reftable_record_copy_from(&copy, rec, GIT_SHA1_RAWSZ);
       	/* do it twice to catch memory leaks */
       	reftable_record_copy_from(&copy, rec, GIT_SHA1_RAWSZ);
     @@ reftable/record_test.c: static void test_reftable_ref_record_roundtrip(void)
       		uint8_t buffer[1024] = { 0 };
       		struct string_view dest = {
       			.buf = buffer,
     - 			.len = sizeof(buffer),
     - 		};
     --
     +@@ reftable/record_test.c: static void test_reftable_ref_record_roundtrip(void)
     + 
       		int n, m;
       
      -		in.value_type = i;
     @@ reftable/record_test.c: static void test_reftable_ref_record_roundtrip(void)
      -			set_hash(in.value.val2.value, 1);
      -			in.value.val2.target_value =
      +			in.u.ref.value.val2.value =
     - 				reftable_malloc(GIT_SHA1_RAWSZ);
     --			set_hash(in.value.val2.target_value, 2);
     ++				reftable_malloc(GIT_SHA1_RAWSZ);
      +			set_hash(in.u.ref.value.val2.value, 1);
      +			in.u.ref.value.val2.target_value =
     -+				reftable_malloc(GIT_SHA1_RAWSZ);
     + 				reftable_malloc(GIT_SHA1_RAWSZ);
     +-			set_hash(in.value.val2.target_value, 2);
      +			set_hash(in.u.ref.value.val2.target_value, 2);
       			break;
       		case REFTABLE_REF_SYMREF:
     @@ reftable/writer.c: int reftable_writer_add_refs(struct reftable_writer *w,
       	    block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {
       		int err = writer_finish_public_section(w);
      @@ reftable/writer.c: static int reftable_writer_add_log_verbatim(struct reftable_writer *w,
     - 
       	w->next -= w->pending_padding;
       	w->pending_padding = 0;
     --
     + 
      -	reftable_record_from_log(&rec, log);
       	return writer_add_record(w, &rec);
       }
     @@ reftable/writer.c: static void write_object_record(void *void_arg, void *key)
      -		.hash_prefix_len = arg->w->stats.object_id_len,
      -		.offsets = entry->offsets,
      -		.offset_len = entry->offset_len,
     --	};
     ++	struct reftable_record rec = {
     ++		.type = BLOCK_TYPE_OBJ,
     ++		.u.obj = {
     ++			.hash_prefix = (uint8_t *)entry->hash.buf,
     ++			.hash_prefix_len = arg->w->stats.object_id_len,
     ++			.offsets = entry->offsets,
     ++			.offset_len = entry->offset_len,
     ++		}
     + 	};
      -	struct reftable_record rec = { NULL };
     -+	struct reftable_record
     -+		rec = { .type = BLOCK_TYPE_OBJ,
     -+			.u.obj = {
     -+				.hash_prefix = (uint8_t *)entry->hash.buf,
     -+				.hash_prefix_len = arg->w->stats.object_id_len,
     -+				.offsets = entry->offsets,
     -+				.offset_len = entry->offset_len,
     -+			} };
       	if (arg->err < 0)
       		goto done;
       
     @@ reftable/writer.c: static void write_object_record(void *void_arg, void *key)
       	if (arg->err == 0)
       		goto done;
      -	obj_rec.offset_len = 0;
     -+
      +	rec.u.obj.offset_len = 0;
       	arg->err = block_writer_add(arg->w->block_writer, &rec);

15: 82f140cab5c ! 15: 1e8785bbc26 reftable: add print functions to the record types

     @@ reftable/record.c: static void hex_format(char *dest, uint8_t *src, int hash_siz
      +static void reftable_ref_record_print_sz(const struct reftable_ref_record *ref,
      +					 int hash_size)
       {
     - 	char hex[GIT_MAX_HEXSZ + 1] = { 0 }; /* BUG */
     + 	char hex[GIT_MAX_HEXSZ + 1];
       	printf("ref{%s(%" PRIu64 ") ", ref->refname, ref->update_index);
      @@ reftable/record.c: void reftable_ref_record_print(const struct reftable_ref_record *ref,
       		printf("=> %s", ref->value.symref);
     @@ reftable/record.c: static int reftable_ref_record_equal_void(const void *a,
      +static void reftable_ref_record_print_void(const void *rec,
      +					   int hash_size)
      +{
     -+	reftable_ref_record_print_sz((struct reftable_ref_record *) rec, hash_size);
     ++	reftable_ref_record_print_sz(
     ++		(struct reftable_ref_record *) rec, hash_size);
      +}
      +
       static struct reftable_record_vtable reftable_ref_record_vtable = {
     @@ reftable/record.c: static void reftable_obj_record_release(void *rec)
      +	int i;
      +
      +	for (i = 0; i < obj->offset_len; i++)
     -+		strbuf_addf(&offset_str, "%" PRIu64 " ", obj->offsets[i]);
     ++		strbuf_addf(&offset_str, "%"PRIu64 " ", obj->offsets[i]);
      +	hex_format(hex, obj->hash_prefix, obj->hash_prefix_len);
      +	printf("prefix %s (len %d), offsets [%s]\n",
      +	       hex, obj->hash_prefix_len, offset_str.buf);
     @@ reftable/record.c: static struct reftable_record_vtable reftable_obj_record_vtab
       	switch (log->value_type) {
       	case REFTABLE_LOG_DELETION:
      -		printf("log{%s(%" PRIu64 ") delete", log->refname,
     -+		printf("log{%s(%" PRIu64 ") delete\n", log->refname,
     ++		printf("log{%s(%"PRIu64 ") delete\n", log->refname,
       		       log->update_index);
       		break;
       	case REFTABLE_LOG_UPDATE:
     - 		printf("log{%s(%" PRIu64 ") %s <%s> %" PRIu64 " %04d\n",
     +-		printf("log{%s(%" PRIu64 ") %s <%s> %" PRIu64 " %04d\n",
      -		       log->refname, log->update_index, log->value.update.name,
      -		       log->value.update.email, log->value.update.time,
     ++		printf("log{%s(%"PRIu64 ") %s <%s> %" PRIu64 " %04d\n",
      +		       log->refname, log->update_index,
      +		       log->value.update.name ? log->value.update.name : "",
      +		       log->value.update.email ? log->value.update.email : "",
     @@ reftable/record.c: static struct reftable_record_vtable reftable_index_record_vt
       };
       
       void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)
     -@@ reftable/record.c: struct reftable_record reftable_new_record(uint8_t typ)
     +@@ reftable/record.c: struct reftable_record reftable_new_record(uint8_t type)
       	}
       	return clean;
       }
     @@ reftable/record_test.c: static void test_reftable_log_record_equal(void)
       {
       	int i;
      -	struct reftable_log_record in[2] = {
     -+
      +	struct reftable_log_record in[] = {
       		{
       			.refname = xstrdup("refs/heads/master"),
     @@ reftable/record_test.c: static void test_reftable_log_record_roundtrip(void)
       			.value_type = REFTABLE_LOG_DELETION,
      +		},
      +		{
     ++			/* make sure we handle optional fields correctly. */
      +			.refname = xstrdup("branch"),
      +			.update_index = 33,
      +			.value_type = REFTABLE_LOG_UPDATE,
     @@ reftable/record_test.c: static void test_reftable_log_record_roundtrip(void)
      +				.update = {
      +					.old_hash = reftable_malloc(GIT_SHA1_RAWSZ),
      +					.new_hash = reftable_malloc(GIT_SHA1_RAWSZ),
     -+					/* rest of fields left empty. */
      +				},
      +			},
       		}

-: ----------- > 16: 556fc11080a reftable: rename typ to type

Signature · 1 line
gitgitgadget
#168 lore raw

PATCH v7 1/16 reftable: fix OOB stack write in print functions

From: Han-Wen Nienhuys <[email protected]>

Remove /* BUG */ comment, whose meaning has escaped my memory.

---
 reftable/record.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
reftable/record.c +2 -2
index 6a5dac32dc6..1f236b47d1c 100644
@@ -254,7 +254,7 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)
 void reftable_ref_record_print(struct reftable_ref_record *ref,
 			       uint32_t hash_id)
 {
-	char hex[2 * GIT_SHA256_RAWSZ + 1] = { 0 }; /* BUG */
+	char hex[GIT_MAX_HEXSZ + 1];
 	printf("ref{%s(%" PRIu64 ") ", ref->refname, ref->update_index);
 	switch (ref->value_type) {
 	case REFTABLE_REF_SYMREF:
@@ -586,7 +586,7 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {
 void reftable_log_record_print(struct reftable_log_record *log,
 			       uint32_t hash_id)
 {
-	char hex[GIT_SHA256_RAWSZ + 1] = { 0 };
+	char hex[GIT_MAX_HEXSZ + 1] = { 0 };
 
 	switch (log->value_type) {
 	case REFTABLE_LOG_DELETION:
Signature · 1 line
gitgitgadget
#169 lore raw

Re: [PATCH v7 01/16] reftable: fix OOB stack write in print functions

Quoted 23 lines · Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>

Remove /* BUG */ comment, whose meaning has escaped my memory.

---
 reftable/record.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
reftable/record.c +1 -1
index 6a5dac32dc6..1f236b47d1c 100644
@@ -254,7 +254,7 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)
 void reftable_ref_record_print(struct reftable_ref_record *ref,
 			       uint32_t hash_id)
 {
-	char hex[2 * GIT_SHA256_RAWSZ + 1] = { 0 }; /* BUG */
+	char hex[GIT_MAX_HEXSZ + 1];
 	printf("ref{%s(%" PRIu64 ") ", ref->refname, ref->update_index);
 	switch (ref->value_type) {
 	case REFTABLE_REF_SYMREF:

reftable/record.c static struct reftable_record_vtable reftable_obj_record_vtable = { · line 592

Quoted 8 lines
reftable/record.c +1 -1
@@ -586,7 +586,7 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {
 void reftable_log_record_print(struct reftable_log_record *log,
 			       uint32_t hash_id)
 {
-	char hex[GIT_SHA256_RAWSZ + 1] = { 0 };
+	char hex[GIT_MAX_HEXSZ + 1] = { 0 };
 
 	switch (log->value_type) {
 	case REFTABLE_LOG_DELETION:

FWIW it seems to originate from
https://github.com/hanwen/reftable/commit/62bbd28 and first appeared on-list in
https://lore.kernel.org/git/[email protected]/
(i.e. any /SHA.*BUG/ in this code). Probably just in reference to having to hardcode the "2 * " and/or SHA256 which later/now is using a more generic macro.

#170 lore raw

PATCH v7 2/16 reftable: fix resource leak in block.c error path

In reply to Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>

Add test coverage for corrupt zlib data. Fix memory leaks demonstrated by unittest.

This problem was discovered by a Coverity scan.

---
 reftable/block.c          | 25 ++++++++++-----
 reftable/reader.c         | 23 ++++++++------
 reftable/readwrite_test.c | 66 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 97 insertions(+), 17 deletions(-)
reftable/block.c +17 -8
index 855e3f5c947..5832f85501d 100644
@@ -189,12 +189,16 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,
 	uint8_t typ = block->data[header_off];
 	uint32_t sz = get_be24(block->data + header_off + 1);
 
+	int err = 0;
 	uint16_t restart_count = 0;
 	uint32_t restart_start = 0;
 	uint8_t *restart_bytes = NULL;
+	uint8_t *uncompressed = NULL;
 
-	if (!reftable_is_block_type(typ))
-		return REFTABLE_FORMAT_ERROR;
+	if (!reftable_is_block_type(typ)) {
+		err =  REFTABLE_FORMAT_ERROR;
+		goto done;
+	}
 
 	if (typ == BLOCK_TYPE_LOG) {
 		int block_header_skip = 4 + header_off;
@@ -203,7 +207,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,
 		uLongf src_len = block->len - block_header_skip;
 		/* Log blocks specify the *uncompressed* size in their header.
 		 */
-		uint8_t *uncompressed = reftable_malloc(sz);
+		uncompressed = reftable_malloc(sz);
 
 		/* Copy over the block header verbatim. It's not compressed. */
 		memcpy(uncompressed, block->data, block_header_skip);
@@ -212,16 +216,19 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,
 		if (Z_OK !=
 		    uncompress2(uncompressed + block_header_skip, &dst_len,
 				block->data + block_header_skip, &src_len)) {
-			reftable_free(uncompressed);
-			return REFTABLE_ZLIB_ERROR;
+			err = REFTABLE_ZLIB_ERROR;
+			goto done;
 		}
 
-		if (dst_len + block_header_skip != sz)
-			return REFTABLE_FORMAT_ERROR;
+		if (dst_len + block_header_skip != sz) {
+			err = REFTABLE_FORMAT_ERROR;
+			goto done;
+		}
 
 		/* We're done with the input data. */
 		reftable_block_done(block);
 		block->data = uncompressed;
+		uncompressed = NULL;
 		block->len = sz;
 		block->source = malloc_block_source();
 		full_block_size = src_len + block_header_skip;
@@ -251,7 +258,9 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,
 	br->restart_count = restart_count;
 	br->restart_bytes = restart_bytes;
 
-	return 0;
+done:
+	reftable_free(uncompressed);
+	return err;
 }
 
 static uint32_t block_reader_restart_offset(struct block_reader *br, int i)
reftable/reader.c +14 -9
index 006709a645a..35781593a29 100644
@@ -290,28 +290,33 @@ int reader_init_block_reader(struct reftable_reader *r, struct block_reader *br,
 
 	err = reader_get_block(r, &block, next_off, guess_block_size);
 	if (err < 0)
-		return err;
+		goto done;
 
 	block_size = extract_block_size(block.data, &block_typ, next_off,
 					r->version);
-	if (block_size < 0)
-		return block_size;
-
+	if (block_size < 0) {
+		err = block_size;
+		goto done;
+	}
 	if (want_typ != BLOCK_TYPE_ANY && block_typ != want_typ) {
-		reftable_block_done(&block);
-		return 1;
+		err = 1;
+		goto done;
 	}
 
 	if (block_size > guess_block_size) {
 		reftable_block_done(&block);
 		err = reader_get_block(r, &block, next_off, block_size);
 		if (err < 0) {
-			return err;
+			goto done;
 		}
 	}
 
-	return block_reader_init(br, &block, header_off, r->block_size,
-				 hash_size(r->hash_id));
+	err = block_reader_init(br, &block, header_off, r->block_size,
+				hash_size(r->hash_id));
+done:
+	reftable_block_done(&block);
+
+	return err;
 }
 
 static int table_iter_next_block(struct table_iter *dest,
reftable/readwrite_test.c +66 -0

78 lines not on this page · raw message

Signature · 1 line
gitgitgadget
#171 lore raw

PATCH v7 3/16 reftable: fix resource leak blocksource.c

In reply to Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>

This would be triggered in the unlikely event of fstat() failing on an opened file.

---
 reftable/blocksource.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
reftable/blocksource.c +4 -2
index 0044eecd9aa..2605371c28d 100644
@@ -134,8 +134,10 @@ int reftable_block_source_from_file(struct reftable_block_source *bs,
 	}
 
 	err = fstat(fd, &st);
-	if (err < 0)
-		return -1;
+	if (err < 0) {
+		close(fd);
+		return REFTABLE_IO_ERROR;
+	}
 
 	p = reftable_calloc(sizeof(struct file_block_source));
 	p->size = st.st_size;
Signature · 1 line
gitgitgadget
#172 lore raw

PATCH v7 4/16 reftable: check reftable_stack_auto_compact() return value

In reply to Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>

If compaction consistently errors out, we would accumulate reftables, and the follow-on EXPECT would start failing, but catching errors early helps debugging.

Spotted by Coverity.

---
 reftable/stack_test.c | 1 +
 1 file changed, 1 insertion(+)
reftable/stack_test.c +1 -0
index f4c743db80c..e84f50d27ff 100644
@@ -839,6 +839,7 @@ static void test_reftable_stack_auto_compaction(void)
 		EXPECT_ERR(err);
 
 		err = reftable_stack_auto_compact(st);
+		EXPECT_ERR(err);
 		EXPECT(i < 3 || st->merged->stack_len < 2 * fastlog2(i));
 	}
 
Signature · 1 line
gitgitgadget
#173 lore raw

PATCH v7 5/16 reftable: ignore remove() return value in stack_test.c

In reply to Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>

If the cleanup fails, there is nothing we can do.

---
 reftable/stack_test.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
reftable/stack_test.c +1 -1
index e84f50d27ff..f0be0be9fee 100644
@@ -90,7 +90,7 @@ static void test_read_file(void)
 		EXPECT(0 == strcmp(want[i], names[i]));
 	}
 	free_names(names);
-	remove(fn);
+	EXPECT(unlink(fn) == 0);
 }
 
 static void test_parse_names(void)
Signature · 1 line
gitgitgadget
#174 lore raw

PATCH v7 6/16 reftable: fix resource warning

In reply to Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>

This would trigger in the unlikely event that we are compacting, and the next available file handle is 0.

---
 reftable/stack.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
reftable/stack.c +5 -5
index 56bf5f2d84a..ddbdf1b9c8b 100644
@@ -889,7 +889,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,
 	struct strbuf new_table_path = STRBUF_INIT;
 	int err = 0;
 	int have_lock = 0;
-	int lock_file_fd = 0;
+	int lock_file_fd = -1;
 	int compact_count = last - first + 1;
 	char **listp = NULL;
 	char **delete_on_success =
@@ -923,7 +923,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,
 	}
 	/* Don't want to write to the lock for now.  */
 	close(lock_file_fd);
-	lock_file_fd = 0;
+	lock_file_fd = -1;
 
 	have_lock = 1;
 	err = stack_uptodate(st);
@@ -1031,7 +1031,7 @@ static int stack_compact_range(struct reftable_stack *st, int first, int last,
 		goto done;
 	}
 	err = close(lock_file_fd);
-	lock_file_fd = 0;
+	lock_file_fd = -1;
 	if (err < 0) {
 		err = REFTABLE_IO_ERROR;
 		unlink(new_table_path.buf);
@@ -1068,9 +1068,9 @@ done:
 		listp++;
 	}
 	free_names(subtable_locks);
-	if (lock_file_fd > 0) {
+	if (lock_file_fd >= 0) {
 		close(lock_file_fd);
-		lock_file_fd = 0;
+		lock_file_fd = -1;
 	}
 	if (have_lock) {
 		unlink(lock_file_name.buf);
Signature · 1 line
gitgitgadget
#175 lore raw

PATCH v7 7/16 reftable: all xxx_free() functions accept NULL arguments

In reply to Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>

This fixes NULL derefs in error paths. Spotted by Coverity.

---
 reftable/reader.c | 2 ++
 reftable/writer.c | 2 ++
 2 files changed, 4 insertions(+)
reftable/reader.c +2 -0
index 35781593a29..272378ed1d5 100644
@@ -646,6 +646,8 @@ int reftable_new_reader(struct reftable_reader **p,
 
 void reftable_reader_free(struct reftable_reader *r)
 {
+	if (!r)
+		return;
 	reader_close(r);
 	reftable_free(r);
 }
reftable/writer.c +2 -0
index 35c8649c9b7..83a23daf60b 100644
@@ -150,6 +150,8 @@ void reftable_writer_set_limits(struct reftable_writer *w, uint64_t min,
 
 void reftable_writer_free(struct reftable_writer *w)
 {
+	if (!w)
+		return;
 	reftable_free(w->block);
 	reftable_free(w);
 }
Signature · 1 line
gitgitgadget
#176 lore raw

PATCH v7 8/16 reftable: order unittests by complexity

In reply to Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>

This is a more practical ordering when working on refactorings of the reftable code.

---
 t/helper/test-reftable.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
t/helper/test-reftable.c +5 -4
index 26b03d7b789..1f0a28cbb64 100644
@@ -3,15 +3,16 @@
 
 int cmd__reftable(int argc, const char **argv)
 {
+	/* test from simple to complex. */
 	basics_test_main(argc, argv);
+	record_test_main(argc, argv);
 	block_test_main(argc, argv);
-	merged_test_main(argc, argv);
+	tree_test_main(argc, argv);
 	pq_test_main(argc, argv);
-	record_test_main(argc, argv);
-	refname_test_main(argc, argv);
 	readwrite_test_main(argc, argv);
+	merged_test_main(argc, argv);
 	stack_test_main(argc, argv);
-	tree_test_main(argc, argv);
+	refname_test_main(argc, argv);
 	return 0;
 }
 
Signature · 1 line
gitgitgadget
#177 lore raw

PATCH v7 9/16 reftable: drop stray printf in readwrite_test

In reply to Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>

---
 reftable/readwrite_test.c | 1 -
 1 file changed, 1 deletion(-)
reftable/readwrite_test.c +0 -1
index edb9cfc1930..605ba0f9fd4 100644
@@ -696,7 +696,6 @@ static void test_write_key_order(void)
 	err = reftable_writer_add_ref(w, &refs[0]);
 	EXPECT_ERR(err);
 	err = reftable_writer_add_ref(w, &refs[1]);
-	printf("%d\n", err);
 	EXPECT(err == REFTABLE_API_ERROR);
 	reftable_writer_close(w);
 	reftable_writer_free(w);
Signature · 1 line
gitgitgadget
#178 lore raw

PATCH v7 10/16 reftable: handle null refnames in reftable_ref_record_equal

In reply to Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>

Spotted by Coverity.

---
 reftable/record.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
reftable/record.c +5 -3
index 1f236b47d1c..53d83e4a41a 100644
@@ -1154,9 +1154,11 @@ int reftable_ref_record_equal(struct reftable_ref_record *a,
 			      struct reftable_ref_record *b, int hash_size)
 {
 	assert(hash_size > 0);
-	if (!(0 == strcmp(a->refname, b->refname) &&
-	      a->update_index == b->update_index &&
-	      a->value_type == b->value_type))
+	if (!null_streq(a->refname, b->refname))
+		return 0;
+
+	if (a->update_index != b->update_index ||
+	    a->value_type != b->value_type)
 		return 0;
 
 	switch (a->value_type) {
Signature · 1 line
gitgitgadget
#179 lore raw

PATCH v7 11/16 reftable: make reftable-record.h function signatures const correct

In reply to Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>

---
 reftable/record.c          | 14 +++++++-------
 reftable/reftable-record.h | 14 +++++++-------
 2 files changed, 14 insertions(+), 14 deletions(-)
reftable/record.c +7 -7
index 53d83e4a41a..1ae344c745b 100644
@@ -72,7 +72,7 @@ int reftable_is_block_type(uint8_t typ)
 	return 0;
 }
 
-uint8_t *reftable_ref_record_val1(struct reftable_ref_record *rec)
+uint8_t *reftable_ref_record_val1(const struct reftable_ref_record *rec)
 {
 	switch (rec->value_type) {
 	case REFTABLE_REF_VAL1:
@@ -84,7 +84,7 @@ uint8_t *reftable_ref_record_val1(struct reftable_ref_record *rec)
 	}
 }
 
-uint8_t *reftable_ref_record_val2(struct reftable_ref_record *rec)
+uint8_t *reftable_ref_record_val2(const struct reftable_ref_record *rec)
 {
 	switch (rec->value_type) {
 	case REFTABLE_REF_VAL2:
@@ -251,7 +251,7 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)
 	}
 }
 
-void reftable_ref_record_print(struct reftable_ref_record *ref,
+void reftable_ref_record_print(const struct reftable_ref_record *ref,
 			       uint32_t hash_id)
 {
 	char hex[GIT_MAX_HEXSZ + 1];
@@ -881,8 +881,8 @@ static int zero_hash_eq(uint8_t *a, uint8_t *b, int sz)
 	return !memcmp(a, b, sz);
 }
 
-int reftable_log_record_equal(struct reftable_log_record *a,
-			      struct reftable_log_record *b, int hash_size)
+int reftable_log_record_equal(const struct reftable_log_record *a,
+			      const struct reftable_log_record *b, int hash_size)
 {
 	if (!(null_streq(a->refname, b->refname) &&
 	      a->update_index == b->update_index &&
@@ -1150,8 +1150,8 @@ static int hash_equal(uint8_t *a, uint8_t *b, int hash_size)
 	return a == b;
 }
 
-int reftable_ref_record_equal(struct reftable_ref_record *a,
-			      struct reftable_ref_record *b, int hash_size)
+int reftable_ref_record_equal(const struct reftable_ref_record *a,
+			      const struct reftable_ref_record *b, int hash_size)
 {
 	assert(hash_size > 0);
 	if (!null_streq(a->refname, b->refname))
reftable/reftable-record.h +7 -7
index 5370d2288c7..67104f8fbfe 100644
@@ -49,25 +49,25 @@ struct reftable_ref_record {
 
 /* Returns the first hash, or NULL if `rec` is not of type
  * REFTABLE_REF_VAL1 or REFTABLE_REF_VAL2. */
-uint8_t *reftable_ref_record_val1(struct reftable_ref_record *rec);
+uint8_t *reftable_ref_record_val1(const struct reftable_ref_record *rec);
 
 /* Returns the second hash, or NULL if `rec` is not of type
  * REFTABLE_REF_VAL2. */
-uint8_t *reftable_ref_record_val2(struct reftable_ref_record *rec);
+uint8_t *reftable_ref_record_val2(const struct reftable_ref_record *rec);
 
 /* returns whether 'ref' represents a deletion */
 int reftable_ref_record_is_deletion(const struct reftable_ref_record *ref);
 
 /* prints a reftable_ref_record onto stdout. Useful for debugging. */
-void reftable_ref_record_print(struct reftable_ref_record *ref,
+void reftable_ref_record_print(const struct reftable_ref_record *ref,
 			       uint32_t hash_id);
 
 /* frees and nulls all pointer values inside `ref`. */
 void reftable_ref_record_release(struct reftable_ref_record *ref);
 
 /* returns whether two reftable_ref_records are the same. Useful for testing. */
-int reftable_ref_record_equal(struct reftable_ref_record *a,
-			      struct reftable_ref_record *b, int hash_size);
+int reftable_ref_record_equal(const struct reftable_ref_record *a,
+			      const struct reftable_ref_record *b, int hash_size);
 
 /* reftable_log_record holds a reflog entry */
 struct reftable_log_record {
@@ -104,8 +104,8 @@ int reftable_log_record_is_deletion(const struct reftable_log_record *log);
 void reftable_log_record_release(struct reftable_log_record *log);
 
 /* returns whether two records are equal. Useful for testing. */
-int reftable_log_record_equal(struct reftable_log_record *a,
-			      struct reftable_log_record *b, int hash_size);
+int reftable_log_record_equal(const struct reftable_log_record *a,
+			      const struct reftable_log_record *b, int hash_size);
 
 /* dumps a reftable_log_record on stdout, for debugging/testing. */
 void reftable_log_record_print(struct reftable_log_record *log,
Signature · 1 line
gitgitgadget
#180 lore raw

PATCH v7 12/16 reftable: implement record equality generically

In reply to Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>

This simplifies unittests a little, and provides further coverage for reftable_record_copy().

---
 reftable/record.c      | 54 +++++++++++++++++++++++++++++++++++++++++-
 reftable/record.h      |  5 +++-
 reftable/record_test.c | 23 +++---------------
 3 files changed, 60 insertions(+), 22 deletions(-)
reftable/record.c +53 -1
index 1ae344c745b..b543585ad39 100644
@@ -430,6 +430,14 @@ static int reftable_ref_record_is_deletion_void(const void *p)
 		(const struct reftable_ref_record *)p);
 }
 
+static int reftable_ref_record_equal_void(const void *a,
+					  const void *b, int hash_size)
+{
+	struct reftable_ref_record *ra = (struct reftable_ref_record *) a;
+	struct reftable_ref_record *rb = (struct reftable_ref_record *) b;
+	return reftable_ref_record_equal(ra, rb, hash_size);
+}
+
 static struct reftable_record_vtable reftable_ref_record_vtable = {
 	.key = &reftable_ref_record_key,
 	.type = BLOCK_TYPE_REF,
@@ -439,6 +447,7 @@ static struct reftable_record_vtable reftable_ref_record_vtable = {
 	.decode = &reftable_ref_record_decode,
 	.release = &reftable_ref_record_release_void,
 	.is_deletion = &reftable_ref_record_is_deletion_void,
+	.equal = &reftable_ref_record_equal_void,
 };
 
 static void reftable_obj_record_key(const void *r, struct strbuf *dest)
@@ -572,6 +581,23 @@ static int not_a_deletion(const void *p)
 	return 0;
 }
 
+static int reftable_obj_record_equal_void(const void *a, const void *b, int hash_size)
+{
+	struct reftable_obj_record *ra = (struct reftable_obj_record *) a;
+	struct reftable_obj_record *rb = (struct reftable_obj_record *) b;
+
+	if (ra->hash_prefix_len != rb->hash_prefix_len
+	    || ra->offset_len != rb->offset_len)
+		return 0;
+
+	if (memcmp(ra->hash_prefix, rb->hash_prefix, ra->hash_prefix_len))
+		return 0;
+	if (memcmp(ra->offsets, rb->offsets, ra->offset_len * sizeof(uint64_t)))
+		return 0;
+
+	return 1;
+}
+
 static struct reftable_record_vtable reftable_obj_record_vtable = {
 	.key = &reftable_obj_record_key,
 	.type = BLOCK_TYPE_OBJ,
@@ -580,7 +606,8 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {
 	.encode = &reftable_obj_record_encode,
 	.decode = &reftable_obj_record_decode,
 	.release = &reftable_obj_record_release,
-	.is_deletion = not_a_deletion,
+	.is_deletion = &not_a_deletion,
+	.equal = &reftable_obj_record_equal_void,
 };
 
 void reftable_log_record_print(struct reftable_log_record *log,
@@ -881,6 +908,14 @@ static int zero_hash_eq(uint8_t *a, uint8_t *b, int sz)
 	return !memcmp(a, b, sz);
 }
 
+static int reftable_log_record_equal_void(const void *a,
+					  const void *b, int hash_size)
+{
+	return reftable_log_record_equal((struct reftable_log_record *) a,
+					 (struct reftable_log_record *) b,
+					 hash_size);
+}
+
 int reftable_log_record_equal(const struct reftable_log_record *a,
 			      const struct reftable_log_record *b, int hash_size)
 {
@@ -924,6 +959,7 @@ static struct reftable_record_vtable reftable_log_record_vtable = {
 	.decode = &reftable_log_record_decode,
 	.release = &reftable_log_record_release_void,
 	.is_deletion = &reftable_log_record_is_deletion_void,
+	.equal = &reftable_log_record_equal_void
 };
 
 struct reftable_record reftable_new_record(uint8_t typ)
@@ -1042,6 +1078,14 @@ static int reftable_index_record_decode(void *rec, struct strbuf key,
 	return start.len - in.len;
 }
 
+static int reftable_index_record_equal(const void *a, const void *b, int hash_size)
+{
+	struct reftable_index_record *ia = (struct reftable_index_record *) a;
+	struct reftable_index_record *ib = (struct reftable_index_record *) b;
+
+	return ia->offset == ib->offset && !strbuf_cmp(&ia->last_key, &ib->last_key);
+}
+
 static struct reftable_record_vtable reftable_index_record_vtable = {
 	.key = &reftable_index_record_key,
 	.type = BLOCK_TYPE_INDEX,
@@ -1051,6 +1095,7 @@ static struct reftable_record_vtable reftable_index_record_vtable = {
 	.decode = &reftable_index_record_decode,
 	.release = &reftable_index_record_release,
 	.is_deletion = &not_a_deletion,
+	.equal = &reftable_index_record_equal,
 };
 
 void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)
@@ -1098,6 +1143,13 @@ int reftable_record_is_deletion(struct reftable_record *rec)
 	return rec->ops->is_deletion(rec->data);
 }
 
+int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size)
+{
+	if (a->ops != b->ops)
+		return 0;
+	return a->ops->equal(a->data, b->data, hash_size);
+}
+
 void reftable_record_from_ref(struct reftable_record *rec,
 			      struct reftable_ref_record *ref_rec)
 {
reftable/record.h +4 -1
index 498e8c50bf4..da75d7d1f11 100644
@@ -58,6 +58,9 @@ struct reftable_record_vtable {
 
 	/* is this a tombstone? */
 	int (*is_deletion)(const void *rec);
+
+	/* Are two records equal? This assumes they have the same type. Returns 0 for non-equal. */
+	int (*equal)(const void *a, const void *b, int hash_size);
 };
 
 /* record is a generic wrapper for different types of records. */
@@ -98,7 +101,7 @@ struct reftable_obj_record {
 };
 
 /* see struct record_vtable */
-
+int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size);
 void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);
 uint8_t reftable_record_type(struct reftable_record *rec);
 void reftable_record_copy_from(struct reftable_record *rec,
reftable/record_test.c +3 -20
index f4ad7cace41..92680848156 100644
@@ -21,18 +21,7 @@ static void test_copy(struct reftable_record *rec)
 	reftable_record_copy_from(&copy, rec, GIT_SHA1_RAWSZ);
 	/* do it twice to catch memory leaks */
 	reftable_record_copy_from(&copy, rec, GIT_SHA1_RAWSZ);
-	switch (reftable_record_type(&copy)) {
-	case BLOCK_TYPE_REF:
-		EXPECT(reftable_ref_record_equal(reftable_record_as_ref(&copy),
-						 reftable_record_as_ref(rec),
-						 GIT_SHA1_RAWSZ));
-		break;
-	case BLOCK_TYPE_LOG:
-		EXPECT(reftable_log_record_equal(reftable_record_as_log(&copy),
-						 reftable_record_as_log(rec),
-						 GIT_SHA1_RAWSZ));
-		break;
-	}
+	EXPECT(reftable_record_equal(rec, &copy, GIT_SHA1_RAWSZ));
 	reftable_record_destroy(&copy);
 }
 
@@ -346,13 +335,7 @@ static void test_reftable_obj_record_roundtrip(void)
 					   GIT_SHA1_RAWSZ);
 		EXPECT(n == m);
 
-		EXPECT(in.hash_prefix_len == out.hash_prefix_len);
-		EXPECT(in.offset_len == out.offset_len);
-
-		EXPECT(!memcmp(in.hash_prefix, out.hash_prefix,
-			       in.hash_prefix_len));
-		EXPECT(0 == memcmp(in.offsets, out.offsets,
-				   sizeof(uint64_t) * in.offset_len));
+		EXPECT(reftable_record_equal(&rec, &rec_out, GIT_SHA1_RAWSZ));
 		strbuf_release(&key);
 		reftable_record_release(&rec_out);
 	}
@@ -390,7 +373,7 @@ static void test_reftable_index_record_roundtrip(void)
 	m = reftable_record_decode(&out_rec, key, extra, dest, GIT_SHA1_RAWSZ);
 	EXPECT(m == n);
 
-	EXPECT(in.offset == out.offset);
+	EXPECT(reftable_record_equal(&rec, &out_rec, GIT_SHA1_RAWSZ));
 
 	reftable_record_release(&out_rec);
 	strbuf_release(&key);
Signature · 1 line
gitgitgadget
#181 lore raw

PATCH v7 13/16 reftable: remove outdated file reftable.c

In reply to Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>

This was renamed to generic.c, but the origin was never removed

---
 reftable/reftable.c | 115 --------------------------------------------
 1 file changed, 115 deletions(-)
 delete mode 100644 reftable/reftable.c
reftable/reftable.c deleted +0 -115
deleted file mode 100644
index 0e4607a7cd6..00000000000
@@ -1,115 +0,0 @@
-/*
-Copyright 2020 Google LLC
-
-Use of this source code is governed by a BSD-style
-license that can be found in the LICENSE file or at
-https://developers.google.com/open-source/licenses/bsd
-*/
-
-#include "basics.h"
-#include "record.h"
-#include "generic.h"
-#include "reftable-iterator.h"
-#include "reftable-generic.h"
-
-int reftable_table_seek_ref(struct reftable_table *tab,
-			    struct reftable_iterator *it, const char *name)
-{
-	struct reftable_ref_record ref = {
-		.refname = (char *)name,
-	};
-	struct reftable_record rec = { NULL };
-	reftable_record_from_ref(&rec, &ref);
-	return tab->ops->seek_record(tab->table_arg, it, &rec);
-}
-
-int reftable_table_read_ref(struct reftable_table *tab, const char *name,
-			    struct reftable_ref_record *ref)
-{
-	struct reftable_iterator it = { NULL };
-	int err = reftable_table_seek_ref(tab, &it, name);
-	if (err)
-		goto done;
-
-	err = reftable_iterator_next_ref(&it, ref);
-	if (err)
-		goto done;
-
-	if (strcmp(ref->refname, name) ||
-	    reftable_ref_record_is_deletion(ref)) {
-		reftable_ref_record_release(ref);
-		err = 1;
-		goto done;
-	}
-
-done:
-	reftable_iterator_destroy(&it);
-	return err;
-}
-
-uint64_t reftable_table_max_update_index(struct reftable_table *tab)
-{
-	return tab->ops->max_update_index(tab->table_arg);
-}
-
-uint64_t reftable_table_min_update_index(struct reftable_table *tab)
-{
-	return tab->ops->min_update_index(tab->table_arg);
-}
-
-uint32_t reftable_table_hash_id(struct reftable_table *tab)
-{
-	return tab->ops->hash_id(tab->table_arg);
-}
-
-void reftable_iterator_destroy(struct reftable_iterator *it)
-{
-	if (!it->ops) {
-		return;
-	}
-	it->ops->close(it->iter_arg);
-	it->ops = NULL;
-	FREE_AND_NULL(it->iter_arg);
-}
-
-int reftable_iterator_next_ref(struct reftable_iterator *it,
-			       struct reftable_ref_record *ref)
-{
-	struct reftable_record rec = { NULL };
-	reftable_record_from_ref(&rec, ref);
-	return iterator_next(it, &rec);
-}
-
-int reftable_iterator_next_log(struct reftable_iterator *it,
-			       struct reftable_log_record *log)
-{
-	struct reftable_record rec = { NULL };
-	reftable_record_from_log(&rec, log);
-	return iterator_next(it, &rec);
-}
-
-int iterator_next(struct reftable_iterator *it, struct reftable_record *rec)
-{
-	return it->ops->next(it->iter_arg, rec);
-}
-
-static int empty_iterator_next(void *arg, struct reftable_record *rec)
-{
-	return 1;
-}
-
-static void empty_iterator_close(void *arg)
-{
-}
-
-static struct reftable_iterator_vtable empty_vtable = {
-	.next = &empty_iterator_next,
-	.close = &empty_iterator_close,
-};
-
-void iterator_set_empty(struct reftable_iterator *it)
-{
-	assert(!it->ops);
-	it->iter_arg = NULL;
-	it->ops = &empty_vtable;
-}
Signature · 1 line
gitgitgadget
#182 lore raw

PATCH v7 14/16 reftable: make reftable_record a tagged union

In reply to Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>

This reduces the amount of glue code, because we don't need a void pointer or vtable within the structure.

The only snag is that reftable_index_record contain a strbuf, so it cannot be zero-initialized. To address this, use reftable_new_record() to return fresh instance, given a record type. Since reftable_new_record() doesn't cause heap allocation anymore, it should be balanced with reftable_record_release() rather than reftable_record_destroy().

Thanks to Peff for the suggestion.

---
 reftable/block.c       |   2 +-
 reftable/block_test.c  |  22 ++---
 reftable/generic.c     |  39 +++++---
 reftable/iter.c        |   4 +-
 reftable/merged.c      |  31 +++---
 reftable/pq.c          |   2 +-
 reftable/pq_test.c     |  29 +++---
 reftable/reader.c      |  84 +++++++++--------
 reftable/record.c      | 209 +++++++++++++++++++----------------------
 reftable/record.h      |  42 ++++-----
 reftable/record_test.c | 154 +++++++++++++++---------------
 reftable/writer.c      |  38 ++++----
 12 files changed, 328 insertions(+), 328 deletions(-)
reftable/block.c +1 -1
index 5832f85501d..80ad10bb794 100644
@@ -422,7 +422,7 @@ int block_reader_seek(struct block_reader *br, struct block_iter *it,
 done:
 	strbuf_release(&key);
 	strbuf_release(&next.last_key);
-	reftable_record_destroy(&rec);
+	reftable_record_release(&rec);
 
 	return err;
 }
reftable/block_test.c +11 -11
index 4b3ea262dcb..fa2ee092ec0 100644
@@ -26,8 +26,9 @@ static void test_block_read_write(void)
 	struct block_writer bw = {
 		.last_key = STRBUF_INIT,
 	};
-	struct reftable_ref_record ref = { NULL };
-	struct reftable_record rec = { NULL };
+	struct reftable_record rec = {
+		.type = BLOCK_TYPE_REF,
+	};
 	int i = 0;
 	int n;
 	struct block_reader br = { 0 };
@@ -40,7 +41,6 @@ static void test_block_read_write(void)
 	block.source = malloc_block_source();
 	block_writer_init(&bw, BLOCK_TYPE_REF, block.data, block_size,
 			  header_off, hash_size(GIT_SHA1_FORMAT_ID));
-	reftable_record_from_ref(&rec, &ref);
 
 	for (i = 0; i < N; i++) {
 		char name[100];
@@ -48,14 +48,14 @@ static void test_block_read_write(void)
 		snprintf(name, sizeof(name), "branch%02d", i);
 		memset(hash, i, sizeof(hash));
 
-		ref.refname = name;
-		ref.value_type = REFTABLE_REF_VAL1;
-		ref.value.val1 = hash;
+		rec.u.ref.refname = name;
+		rec.u.ref.value_type = REFTABLE_REF_VAL1;
+		rec.u.ref.value.val1 = hash;
 
 		names[i] = xstrdup(name);
 		n = block_writer_add(&bw, &rec);
-		ref.refname = NULL;
-		ref.value_type = REFTABLE_REF_DELETION;
+		rec.u.ref.refname = NULL;
+		rec.u.ref.value_type = REFTABLE_REF_DELETION;
 		EXPECT(n == 0);
 	}
 
@@ -74,7 +74,7 @@ static void test_block_read_write(void)
 		if (r > 0) {
 			break;
 		}
-		EXPECT_STREQ(names[j], ref.refname);
+		EXPECT_STREQ(names[j], rec.u.ref.refname);
 		j++;
 	}
 
@@ -92,7 +92,7 @@ static void test_block_read_write(void)
 		n = block_iter_next(&it, &rec);
 		EXPECT(n == 0);
 
-		EXPECT_STREQ(names[i], ref.refname);
+		EXPECT_STREQ(names[i], rec.u.ref.refname);
 
 		want.len--;
 		n = block_reader_seek(&br, &it, &want);
@@ -100,7 +100,7 @@ static void test_block_read_write(void)
 
 		n = block_iter_next(&it, &rec);
 		EXPECT(n == 0);
-		EXPECT_STREQ(names[10 * (i / 10)], ref.refname);
+		EXPECT_STREQ(names[10 * (i / 10)], rec.u.ref.refname);
 
 		block_iter_close(&it);
 	}
reftable/generic.c +24 -15
index 7a8a738d860..b176c1cde0a 100644
@@ -7,6 +7,7 @@ https://developers.google.com/open-source/licenses/bsd
 */
 
 #include "basics.h"
+#include "constants.h"
 #include "record.h"
 #include "generic.h"
 #include "reftable-iterator.h"
@@ -15,23 +16,23 @@ https://developers.google.com/open-source/licenses/bsd
 int reftable_table_seek_ref(struct reftable_table *tab,
 			    struct reftable_iterator *it, const char *name)
 {
-	struct reftable_ref_record ref = {
-		.refname = (char *)name,
+	struct reftable_record rec = {
+		.type = BLOCK_TYPE_REF,
+		.u.ref = { .refname = (char *)name, }
 	};
-	struct reftable_record rec = { NULL };
-	reftable_record_from_ref(&rec, &ref);
 	return tab->ops->seek_record(tab->table_arg, it, &rec);
 }
 
 int reftable_table_seek_log(struct reftable_table *tab,
 			    struct reftable_iterator *it, const char *name)
 {
-	struct reftable_log_record log = {
-		.refname = (char *)name,
-		.update_index = ~((uint64_t)0),
+	struct reftable_record rec = {
+		.type = BLOCK_TYPE_LOG,
+		.u.log = {
+			.refname = (char *)name,
+			.update_index = ~((uint64_t)0),
+		}
 	};
-	struct reftable_record rec = { NULL };
-	reftable_record_from_log(&rec, &log);
 	return tab->ops->seek_record(tab->table_arg, it, &rec);
 }
 
@@ -129,17 +130,25 @@ void reftable_iterator_destroy(struct reftable_iterator *it)
 int reftable_iterator_next_ref(struct reftable_iterator *it,
 			       struct reftable_ref_record *ref)
 {
-	struct reftable_record rec = { NULL };
-	reftable_record_from_ref(&rec, ref);
-	return iterator_next(it, &rec);
+	struct reftable_record rec = {
+		.type = BLOCK_TYPE_REF,
+		.u.ref = *ref,
+	};
+	int err = iterator_next(it, &rec);
+	*ref = rec.u.ref;
+	return err;
 }
 
 int reftable_iterator_next_log(struct reftable_iterator *it,
 			       struct reftable_log_record *log)
 {
-	struct reftable_record rec = { NULL };
-	reftable_record_from_log(&rec, log);
-	return iterator_next(it, &rec);
+	struct reftable_record rec = {
+		.type = BLOCK_TYPE_LOG,
+		.u.log = *log,
+	};
+	int err = iterator_next(it, &rec);
+	*log = rec.u.log;
+	return err;
 }
 
 int iterator_next(struct reftable_iterator *it, struct reftable_record *rec)
reftable/iter.c +2 -2
index 93d04f735b8..a8d174c0406 100644
@@ -32,7 +32,7 @@ static int filtering_ref_iterator_next(void *iter_arg,
 				       struct reftable_record *rec)
 {
 	struct filtering_ref_iterator *fri = iter_arg;
-	struct reftable_ref_record *ref = rec->data;
+	struct reftable_ref_record *ref = &rec->u.ref;
 	int err = 0;
 	while (1) {
 		err = reftable_iterator_next_ref(&fri->it, ref);
@@ -127,7 +127,7 @@ static int indexed_table_ref_iter_next_block(struct indexed_table_ref_iter *it)
 static int indexed_table_ref_iter_next(void *p, struct reftable_record *rec)
 {
 	struct indexed_table_ref_iter *it = p;
-	struct reftable_ref_record *ref = rec->data;
+	struct reftable_ref_record *ref = &rec->u.ref;
 
 	while (1) {
 		int err = block_iter_next(&it->cur, rec);
reftable/merged.c +15 -16
index e5b53da6db3..82511762d2a 100644
@@ -30,7 +30,7 @@ static int merged_iter_init(struct merged_iter *mi)
 
 		if (err > 0) {
 			reftable_iterator_destroy(&mi->stack[i]);
-			reftable_record_destroy(&rec);
+			reftable_record_release(&rec);
 		} else {
 			struct pq_entry e = {
 				.rec = rec,
@@ -57,18 +57,17 @@ static void merged_iter_close(void *p)
 static int merged_iter_advance_nonnull_subiter(struct merged_iter *mi,
 					       size_t idx)
 {
-	struct reftable_record rec = reftable_new_record(mi->typ);
 	struct pq_entry e = {
-		.rec = rec,
+		.rec = reftable_new_record(mi->typ),
 		.index = idx,
 	};
-	int err = iterator_next(&mi->stack[idx], &rec);
+	int err = iterator_next(&mi->stack[idx], &e.rec);
 	if (err < 0)
 		return err;
 
 	if (err > 0) {
 		reftable_iterator_destroy(&mi->stack[idx]);
-		reftable_record_destroy(&rec);
+		reftable_record_release(&e.rec);
 		return 0;
 	}
 
@@ -126,11 +125,11 @@ static int merged_iter_next_entry(struct merged_iter *mi,
 		if (err < 0) {
 			return err;
 		}
-		reftable_record_destroy(&top.rec);
+		reftable_record_release(&top.rec);
 	}
 
 	reftable_record_copy_from(rec, &entry.rec, hash_size(mi->hash_id));
-	reftable_record_destroy(&entry.rec);
+	reftable_record_release(&entry.rec);
 	strbuf_release(&entry_key);
 	return 0;
 }
@@ -290,11 +289,10 @@ int reftable_merged_table_seek_ref(struct reftable_merged_table *mt,
 				   struct reftable_iterator *it,
 				   const char *name)
 {
-	struct reftable_ref_record ref = {
-		.refname = (char *)name,
+	struct reftable_record rec = {
+		.type = BLOCK_TYPE_REF,
+		.u.ref = { .refname = (char *)name, },
 	};
-	struct reftable_record rec = { NULL };
-	reftable_record_from_ref(&rec, &ref);
 	return merged_table_seek_record(mt, it, &rec);
 }
 
@@ -302,12 +300,13 @@ int reftable_merged_table_seek_log_at(struct reftable_merged_table *mt,
 				      struct reftable_iterator *it,
 				      const char *name, uint64_t update_index)
 {
-	struct reftable_log_record log = {
-		.refname = (char *)name,
-		.update_index = update_index,
+	struct reftable_record rec = {
+		.type = BLOCK_TYPE_LOG,
+		.u.log = {
+			.refname = (char *)name,
+			.update_index = update_index,
+		}
 	};
-	struct reftable_record rec = { NULL };
-	reftable_record_from_log(&rec, &log);
 	return merged_table_seek_record(mt, it, &rec);
 }
 
reftable/pq.c +1 -1
index efc474017a2..5ffc8380c4a 100644
@@ -98,7 +98,7 @@ void merged_iter_pqueue_release(struct merged_iter_pqueue *pq)
 {
 	int i = 0;
 	for (i = 0; i < pq->len; i++) {
-		reftable_record_destroy(&pq->heap[i].rec);
+		reftable_record_release(&pq->heap[i].rec);
 	}
 	FREE_AND_NULL(pq->heap);
 	pq->len = pq->cap = 0;
reftable/pq_test.c +14 -15
index c9bb05e37b7..5d58a4c05b6 100644
@@ -31,7 +31,7 @@ static void test_pq(void)
 	int N = ARRAY_SIZE(names) - 1;
 
 	struct merged_iter_pqueue pq = { NULL };
-	const char *last = NULL;
+	char *last = NULL;
 
 	int i = 0;
 	for (i = 0; i < N; i++) {
@@ -42,12 +42,12 @@ static void test_pq(void)
 
 	i = 1;
 	do {
-		struct reftable_record rec =
-			reftable_new_record(BLOCK_TYPE_REF);
-		struct pq_entry e = { 0 };
-
-		reftable_record_as_ref(&rec)->refname = names[i];
-		e.rec = rec;
+		struct pq_entry e = {
+			.rec = {
+				.type = BLOCK_TYPE_REF,
+				.u.ref = { .refname = names[i], }
+			}
+		};
 		merged_iter_pqueue_add(&pq, e);
 		merged_iter_pqueue_check(pq);
 		i = (i * 7) % N;
@@ -55,19 +55,18 @@ static void test_pq(void)
 
 	while (!merged_iter_pqueue_is_empty(pq)) {
 		struct pq_entry e = merged_iter_pqueue_remove(&pq);
-		struct reftable_ref_record *ref =
-			reftable_record_as_ref(&e.rec);
-
+		struct reftable_record *rec = &e.rec;
 		merged_iter_pqueue_check(pq);
 
+		EXPECT(reftable_record_type(rec) == BLOCK_TYPE_REF);
 		if (last) {
-			EXPECT(strcmp(last, ref->refname) < 0);
+			EXPECT(strcmp(last, rec->u.ref.refname) < 0);
 		}
-		last = ref->refname;
-		ref->refname = NULL;
-		reftable_free(ref);
+		/* this is names[i], so don't dealloc. */
+		last = rec->u.ref.refname;
+		rec->u.ref.refname = NULL;
+		reftable_record_release(rec);
 	}
-
 	for (i = 0; i < N; i++) {
 		reftable_free(names[i]);
 	}
reftable/reader.c +44 -40

170 lines not on this page · raw message

reftable/record.c +98 -111

281 lines not on this page · raw message

reftable/record.h +18 -24

65 lines not on this page · raw message

reftable/record_test.c +78 -76

268 lines not on this page · raw message

reftable/writer.c +22 -16
index 83a23daf60b..96a5155ea77 100644
@@ -256,8 +256,10 @@ done:
 int reftable_writer_add_ref(struct reftable_writer *w,
 			    struct reftable_ref_record *ref)
 {
-	struct reftable_record rec = { NULL };
-	struct reftable_ref_record copy = *ref;
+	struct reftable_record rec = {
+		.type = BLOCK_TYPE_REF,
+		.u.ref = *ref,
+	};
 	int err = 0;
 
 	if (ref->refname == NULL)
@@ -266,8 +268,7 @@ int reftable_writer_add_ref(struct reftable_writer *w,
 	    ref->update_index > w->max_update_index)
 		return REFTABLE_API_ERROR;
 
-	reftable_record_from_ref(&rec, &copy);
-	copy.update_index -= w->min_update_index;
+	rec.u.ref.update_index -= w->min_update_index;
 
 	err = writer_add_record(w, &rec);
 	if (err < 0)
@@ -306,7 +307,10 @@ int reftable_writer_add_refs(struct reftable_writer *w,
 static int reftable_writer_add_log_verbatim(struct reftable_writer *w,
 					    struct reftable_log_record *log)
 {
-	struct reftable_record rec = { NULL };
+	struct reftable_record rec = {
+		.type = BLOCK_TYPE_LOG,
+		.u.log = *log,
+	};
 	if (w->block_writer &&
 	    block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {
 		int err = writer_finish_public_section(w);
@@ -317,7 +321,6 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w,
 	w->next -= w->pending_padding;
 	w->pending_padding = 0;
 
-	reftable_record_from_log(&rec, log);
 	return writer_add_record(w, &rec);
 }
 
@@ -398,8 +401,10 @@ static int writer_finish_section(struct reftable_writer *w)
 		w->index_len = 0;
 		w->index_cap = 0;
 		for (i = 0; i < idx_len; i++) {
-			struct reftable_record rec = { NULL };
-			reftable_record_from_index(&rec, idx + i);
+			struct reftable_record rec = {
+				.type = BLOCK_TYPE_INDEX,
+				.u.idx = idx[i],
+			};
 			if (block_writer_add(w->block_writer, &rec) == 0) {
 				continue;
 			}
@@ -467,17 +472,18 @@ static void write_object_record(void *void_arg, void *key)
 {
 	struct write_record_arg *arg = void_arg;
 	struct obj_index_tree_node *entry = key;
-	struct reftable_obj_record obj_rec = {
-		.hash_prefix = (uint8_t *)entry->hash.buf,
-		.hash_prefix_len = arg->w->stats.object_id_len,
-		.offsets = entry->offsets,
-		.offset_len = entry->offset_len,
+	struct reftable_record rec = {
+		.type = BLOCK_TYPE_OBJ,
+		.u.obj = {
+			.hash_prefix = (uint8_t *)entry->hash.buf,
+			.hash_prefix_len = arg->w->stats.object_id_len,
+			.offsets = entry->offsets,
+			.offset_len = entry->offset_len,
+		}
 	};
-	struct reftable_record rec = { NULL };
 	if (arg->err < 0)
 		goto done;
 
-	reftable_record_from_obj(&rec, &obj_rec);
 	arg->err = block_writer_add(arg->w->block_writer, &rec);
 	if (arg->err == 0)
 		goto done;
@@ -490,7 +496,7 @@ static void write_object_record(void *void_arg, void *key)
 	arg->err = block_writer_add(arg->w->block_writer, &rec);
 	if (arg->err == 0)
 		goto done;
-	obj_rec.offset_len = 0;
+	rec.u.obj.offset_len = 0;
 	arg->err = block_writer_add(arg->w->block_writer, &rec);
 
 	/* Should be able to write into a fresh block. */
Signature · 1 line
gitgitgadget
#183 lore raw

Re: [PATCH v7 14/16] reftable: make reftable_record a tagged union

Quoted 7 lines · Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>
[...]

(quoted diff) +0 -1
@@ -129,17 +130,25 @@ void reftable_iterator_destroy(struct reftable_iterator *it)
 int reftable_iterator_next_ref(struct reftable_iterator *it,
 			       struct reftable_ref_record *ref)
 {
-	struct reftable_record rec = { NULL };

Review void reftable_iterator_destroy(struct reftable_iterator *it) · line 140

Quoted 10 lines
(quoted diff) +7 -2
-	reftable_record_from_ref(&rec, ref);
-	return iterator_next(it, &rec);
+	struct reftable_record rec = {
+		.type = BLOCK_TYPE_REF,
+		.u.ref = *ref,
+	};
+	int err = iterator_next(it, &rec);
+	*ref = rec.u.ref;
+	return err;
 }

I didn't test the v6 on xlc on AIX 7.1, but found that it refuses to compile this code (but the one on AIX 7.2 is OK with it):

"reftable/generic.c", line 135.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_ref_record" is not allowed. "reftable/generic.c", line 147.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_log_record" is not allowed. "reftable/writer.c", line 261.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_ref_record" is not allowed. "reftable/writer.c", line 312.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_log_record" is not allowed. "reftable/writer.c", line 406.45: 1506-196 (S) Initialization between types "unsigned long long" and "struct reftable_index_record" is not allowed.

I.e. you're dereferencing a struct type here to get at its first member, I'm surprised that gcc and clang don't complain about it.

I think that the below fix-up for this series as a whole (didn't check if it was just this patch) should fix it:

reftable/generic.c +2 -2
index b176c1cde0a..fe8f060a689 100644
@@ -132,7 +132,7 @@ int reftable_iterator_next_ref(struct reftable_iterator *it,
 {
 	struct reftable_record rec = {
 		.type = BLOCK_TYPE_REF,
-		.u.ref = *ref,
+		.u.ref = { ref->refname },
 	};
 	int err = iterator_next(it, &rec);
 	*ref = rec.u.ref;
@@ -144,7 +144,7 @@ int reftable_iterator_next_log(struct reftable_iterator *it,
 {
 	struct reftable_record rec = {
 		.type = BLOCK_TYPE_LOG,
-		.u.log = *log,
+		.u.log = { log->refname },
 	};
 	int err = iterator_next(it, &rec);
 	*log = rec.u.log;
reftable/record_test.c +1 -1
index 847abfa2b04..be99de5194b 100644
@@ -337,7 +337,7 @@ static void test_reftable_obj_record_roundtrip(void)
 		};
 		struct reftable_record in = {
 			.type = BLOCK_TYPE_OBJ,
-			.u.obj = recs[i],
+			.u.obj = { recs[i].hash_prefix },
 		};
 		struct strbuf key = STRBUF_INIT;
 		struct reftable_record out = { .type = BLOCK_TYPE_OBJ };
reftable/writer.c +3 -3
index 5b768438625..6cc2ef9acaf 100644
@@ -258,7 +258,7 @@ int reftable_writer_add_ref(struct reftable_writer *w,
 {
 	struct reftable_record rec = {
 		.type = BLOCK_TYPE_REF,
-		.u.ref = *ref,
+		.u.ref = { ref->refname },
 	};
 	int err = 0;
 
@@ -309,7 +309,7 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w,
 {
 	struct reftable_record rec = {
 		.type = BLOCK_TYPE_LOG,
-		.u.log = *log,
+		.u.log = { log->refname },
 	};
 	if (w->block_writer &&
 	    block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {
@@ -403,7 +403,7 @@ static int writer_finish_section(struct reftable_writer *w)
 		for (i = 0; i < idx_len; i++) {
 			struct reftable_record rec = {
 				.type = BLOCK_TYPE_INDEX,
-				.u.idx = idx[i],
+				.u.idx = { idx[i].offset },
 			};
 			if (block_writer_add(w->block_writer, &rec) == 0) {
 				continue;
#184 lore raw

Re: [PATCH v7 14/16] reftable: make reftable_record a tagged union

Quoted 11 lines · Ævar Arnfjörð Bjarmason

I didn't test the v6 on xlc on AIX 7.1, but found that it refuses to compile this code (but the one on AIX 7.2 is OK with it):

"reftable/generic.c", line 135.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_ref_record" is not allowed. "reftable/generic.c", line 147.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_log_record" is not allowed. "reftable/writer.c", line 261.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_ref_record" is not allowed. "reftable/writer.c", line 312.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_log_record" is not allowed. "reftable/writer.c", line 406.45: 1506-196 (S) Initialization between types "unsigned long long" and "struct reftable_index_record" is not allowed.

I.e. you're dereferencing a struct type here to get at its first member,

No, that's not what I'm doing.

Quoted 4 lines

Review

Quoted 4 lines
(quoted diff) +4 -0
+   struct reftable_record rec = {
+             .type = BLOCK_TYPE_REF,
+             .u.ref = *ref,
+     };

reftable_record is tagged union, and u.ref is a reftable_ref_record. I intend to shallow-copying the parameter (a reftable_record called 'ref') into 'rec.u.ref' , precisely as it is written.

Does this patch pass the unittests?

Signature · 7 lines
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--
Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
#185 lore raw

Re: [PATCH v7 14/16] reftable: make reftable_record a tagged union

Quoted 22 lines · Han-Wen Nienhuys
Quoted 11 lines · Ævar Arnfjörð Bjarmason

I didn't test the v6 on xlc on AIX 7.1, but found that it refuses to compile this code (but the one on AIX 7.2 is OK with it):

"reftable/generic.c", line 135.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_ref_record" is not allowed. "reftable/generic.c", line 147.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_log_record" is not allowed. "reftable/writer.c", line 261.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_ref_record" is not allowed. "reftable/writer.c", line 312.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_log_record" is not allowed. "reftable/writer.c", line 406.45: 1506-196 (S) Initialization between types "unsigned long long" and "struct reftable_index_record" is not allowed.

I.e. you're dereferencing a struct type here to get at its first member,

No, that's not what I'm doing.

Quoted 4 lines

Review

Quoted 4 lines
(quoted diff) +4 -0
+   struct reftable_record rec = {
+             .type = BLOCK_TYPE_REF,
+             .u.ref = *ref,
+     };
Quoted 5 lines

reftable_record is tagged union, and u.ref is a reftable_ref_record. I intend to shallow-copying the parameter (a reftable_record called 'ref') into 'rec.u.ref' , precisely as it is written.

Does this patch pass the unittests?

No, sorry about that. It fails. I didn't have time to run them earlier (and it takes a while on that platform).

I don't know how to work around it then...

#186 lore raw

master doesn't compile on xlc 21.01 anymore (old AIX compiler) (was: [PATCH v7 14/16] reftable: make reftable_record a tagged union)

Quoted 28 lines · Ævar Arnfjörð Bjarmason
Quoted 26 lines · Han-Wen Nienhuys
Quoted 11 lines · Ævar Arnfjörð Bjarmason

I didn't test the v6 on xlc on AIX 7.1, but found that it refuses to compile this code (but the one on AIX 7.2 is OK with it):

"reftable/generic.c", line 135.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_ref_record" is not allowed. "reftable/generic.c", line 147.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_log_record" is not allowed. "reftable/writer.c", line 261.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_ref_record" is not allowed. "reftable/writer.c", line 312.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_log_record" is not allowed. "reftable/writer.c", line 406.45: 1506-196 (S) Initialization between types "unsigned long long" and "struct reftable_index_record" is not allowed.

I.e. you're dereferencing a struct type here to get at its first member,

No, that's not what I'm doing.

Quoted 4 lines

Review

Quoted 4 lines
(quoted diff) +4 -0
+   struct reftable_record rec = {
+             .type = BLOCK_TYPE_REF,
+             .u.ref = *ref,
+     };

reftable_record is tagged union, and u.ref is a reftable_ref_record. I intend to shallow-copying the parameter (a reftable_record called 'ref') into 'rec.u.ref' , precisely as it is written.

Quoted 6 lines
Quoted 1 line

Does this patch pass the unittests?

No, sorry about that. It fails. I didn't have time to run them earlier (and it takes a while on that platform).

I don't know how to work around it then...

Just a reminder that now with this landed on master we've hard broken compilation on that version of xlc:

    avar@gcc111:[/home/avar]xlc -qversion
    IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72)
    Version: 12.01.0000.0000

The error is:

"reftable/generic.c", line 133.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_ref_record" is not allowed. "reftable/generic.c", line 145.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_log_record" is not allowed.

Is there really no workaround we can think of for this?

#187 lore raw

Re: master doesn't compile on xlc 21.01 anymore (old AIX compiler) (was: [PATCH v7 14/16] reftable: make reftable_record a tagged union)

Am 19.02.22 um 13:54 schrieb Ævar Arnfjörð Bjarmason:

Quoted 45 lines · Ævar Arnfjörð Bjarmason
Quoted 29 lines · Han-Wen Nienhuys
Quoted 27 lines
Quoted 11 lines · Ævar Arnfjörð Bjarmason

I didn't test the v6 on xlc on AIX 7.1, but found that it refuses to compile this code (but the one on AIX 7.2 is OK with it):

"reftable/generic.c", line 135.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_ref_record" is not allowed. "reftable/generic.c", line 147.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_log_record" is not allowed. "reftable/writer.c", line 261.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_ref_record" is not allowed. "reftable/writer.c", line 312.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_log_record" is not allowed. "reftable/writer.c", line 406.45: 1506-196 (S) Initialization between types "unsigned long long" and "struct reftable_index_record" is not allowed.

I.e. you're dereferencing a struct type here to get at its first member,

No, that's not what I'm doing.

Quoted 4 lines

Review

Quoted 4 lines
(quoted diff) +4 -0
+   struct reftable_record rec = {
+             .type = BLOCK_TYPE_REF,
+             .u.ref = *ref,
+     };

reftable_record is tagged union, and u.ref is a reftable_ref_record. I intend to shallow-copying the parameter (a reftable_record called 'ref') into 'rec.u.ref' , precisely as it is written.

Quoted 6 lines
Quoted 1 line

Does this patch pass the unittests?

No, sorry about that. It fails. I didn't have time to run them earlier (and it takes a while on that platform).

I don't know how to work around it then...

Just a reminder that now with this landed on master we've hard broken compilation on that version of xlc:

    avar@gcc111:[/home/avar]xlc -qversion
    IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72)
    Version: 12.01.0000.0000
Quoted 6 lines

The error is:

"reftable/generic.c", line 133.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_ref_record" is not allowed. "reftable/generic.c", line 145.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_log_record" is not allowed.

Is there really no workaround we can think of for this?

The code is hard to read for me in general due to its many types and function dispatch tables, but this compiler seems to have an even harder time. Where does it even get that char pointer type from?

Can it compile this?

	struct reftable_record rec = {
		.type = BLOCK_TYPE_REF,
		.u = {
			.ref = *ref
		},
	};

If it can't, what about this?

	struct reftable_record rec = {
		.type = BLOCK_TYPE_REF,
	};
	rec.u.ref = *ref;

René

#188 lore raw

PATCH 1/1 reftable: make assignments portable to AIX xlc v12.01

Change the assignment syntax introduced in 66c0dabab5e (reftable: make reftable_record a tagged union, 2022-01-20) to be portable to AIX xlc v12.1:

    avar@gcc111:[/home/avar]xlc -qversion
    IBM XL C/C++ for AIX, V12.1 (5765-J02, 5725-C72)
    Version: 12.01.0000.0000

The error emitted before this was e.g.:

"reftable/generic.c", line 133.26: 1506-196 (S) Initialization between types "char*" and "struct reftable_ref_record" is not allowed.

The syntax in the pre-image is supported by e.g. xlc 13.01 on a newer AIX version:

    avar@gcc119:[/home/avar]xlc -qversion
    IBM XL C/C++ for AIX, V13.1.3 (5725-C72, 5765-J07)
    Version: 13.01.0003.0006

But as we've otherwise supported this compiler let's not break it entirely if it's easy to work around it.

---

Quoted 5 lines · René Scharfe

Am 19.02.22 um 13:54 schrieb Ævar Arnfjörð Bjarmason:
[...]
The code is hard to read for me in general due to its many types and function dispatch tables, but this compiler seems to have an even harder time. Where does it even get that char pointer type from?

Sorry about the very late reply, yes thath first suggestion worked, d'oh in retrospect. This patch fixes it.

 reftable/generic.c     |  8 ++++++--
 reftable/record_test.c |  4 +++-
 reftable/writer.c      | 12 +++++++++---
 3 files changed, 18 insertions(+), 6 deletions(-)
reftable/generic.c +6 -2
index b27d152e89a..57f8032db94 100644
@@ -130,7 +130,9 @@ int reftable_iterator_next_ref(struct reftable_iterator *it,
 {
 	struct reftable_record rec = {
 		.type = BLOCK_TYPE_REF,
-		.u.ref = *ref,
+		.u = {
+			.ref = *ref
+		},
 	};
 	int err = iterator_next(it, &rec);
 	*ref = rec.u.ref;
@@ -142,7 +144,9 @@ int reftable_iterator_next_log(struct reftable_iterator *it,
 {
 	struct reftable_record rec = {
 		.type = BLOCK_TYPE_LOG,
-		.u.log = *log,
+		.u = {
+			.log = *log,
+		},
 	};
 	int err = iterator_next(it, &rec);
 	*log = rec.u.log;
reftable/record_test.c +3 -1
index f91ea5e8830..70ae78feca4 100644
@@ -339,7 +339,9 @@ static void test_reftable_obj_record_roundtrip(void)
 		};
 		struct reftable_record in = {
 			.type = BLOCK_TYPE_OBJ,
-			.u.obj = recs[i],
+			.u = {
+				.obj = recs[i],
+			},
 		};
 		struct strbuf key = STRBUF_INIT;
 		struct reftable_record out = { .type = BLOCK_TYPE_OBJ };
reftable/writer.c +9 -3
index 6d979e245ff..427f1317c6b 100644
@@ -257,7 +257,9 @@ int reftable_writer_add_ref(struct reftable_writer *w,
 {
 	struct reftable_record rec = {
 		.type = BLOCK_TYPE_REF,
-		.u.ref = *ref,
+		.u = {
+			.ref = *ref
+		},
 	};
 	int err = 0;
 
@@ -308,7 +310,9 @@ static int reftable_writer_add_log_verbatim(struct reftable_writer *w,
 {
 	struct reftable_record rec = {
 		.type = BLOCK_TYPE_LOG,
-		.u.log = *log,
+		.u = {
+			.log = *log,
+		},
 	};
 	if (w->block_writer &&
 	    block_writer_type(w->block_writer) == BLOCK_TYPE_REF) {
@@ -401,7 +405,9 @@ static int writer_finish_section(struct reftable_writer *w)
 		for (i = 0; i < idx_len; i++) {
 			struct reftable_record rec = {
 				.type = BLOCK_TYPE_INDEX,
-				.u.idx = idx[i],
+				.u = {
+					.idx = idx[i],
+				},
 			};
 			if (block_writer_add(w->block_writer, &rec) == 0) {
 				continue;
Signature · 1 line
2.35.1.1548.gaad2ca01774
#189 lore raw

Re: [PATCH] reftable: make assignments portable to AIX xlc v12.01

reftable/generic.c int reftable_iterator_next_ref(struct reftable_iterator *it, · line 136

Quoted 13 lines · Ævar Arnfjörð Bjarmason
reftable/generic.c +3 -1
index b27d152e89a..57f8032db94 100644
@@ -130,7 +130,9 @@ int reftable_iterator_next_ref(struct reftable_iterator *it,
 {
 	struct reftable_record rec = {
 		.type = BLOCK_TYPE_REF,
-		.u.ref = *ref,
+		.u = {
+			.ref = *ref
+		},
 	};

Having to do this does *not* look so bad.

#190 lore raw

Re: [PATCH] reftable: make assignments portable to AIX xlc v12.01

In reply to Ævar Arnfjörð Bjarmason
Quoted 4 lines · Ævar Arnfjörð Bjarmason

Change the assignment syntax introduced in 66c0dabab5e (reftable: make reftable_record a tagged union, 2022-01-20) to be portable to AIX xlc v12.1:

LGTM.

Signature · 2 lines
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
#191 lore raw

Re: [PATCH] reftable: make assignments portable to AIX xlc v12.01

Quoted 8 lines · Han-Wen Nienhuys
Quoted 4 lines · Ævar Arnfjörð Bjarmason

Change the assignment syntax introduced in 66c0dabab5e (reftable: make reftable_record a tagged union, 2022-01-20) to be portable to AIX xlc v12.1:

LGTM.

Thanks; queued.

#192 lore raw

PATCH v7 15/16 reftable: add print functions to the record types

In reply to Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>

This isn't used per se, but it is useful for debugging, especially Windows CI failures.

---
 reftable/record.c      | 88 +++++++++++++++++++++++++++++++++++-------
 reftable/record.h      |  4 ++
 reftable/record_test.c | 20 +++++++++-
 3 files changed, 96 insertions(+), 16 deletions(-)
reftable/record.c +73 -15
index 238ff882aab..c57bac0e96a 100644
@@ -269,8 +269,8 @@ static void hex_format(char *dest, uint8_t *src, int hash_size)
 	}
 }
 
-void reftable_ref_record_print(const struct reftable_ref_record *ref,
-			       uint32_t hash_id)
+static void reftable_ref_record_print_sz(const struct reftable_ref_record *ref,
+					 int hash_size)
 {
 	char hex[GIT_MAX_HEXSZ + 1];
 	printf("ref{%s(%" PRIu64 ") ", ref->refname, ref->update_index);
@@ -279,14 +279,14 @@ void reftable_ref_record_print(const struct reftable_ref_record *ref,
 		printf("=> %s", ref->value.symref);
 		break;
 	case REFTABLE_REF_VAL2:
-		hex_format(hex, ref->value.val2.value, hash_size(hash_id));
+		hex_format(hex, ref->value.val2.value, hash_size);
 		printf("val 2 %s", hex);
 		hex_format(hex, ref->value.val2.target_value,
-			   hash_size(hash_id));
+			   hash_size);
 		printf("(T %s)", hex);
 		break;
 	case REFTABLE_REF_VAL1:
-		hex_format(hex, ref->value.val1, hash_size(hash_id));
+		hex_format(hex, ref->value.val1, hash_size);
 		printf("val 1 %s", hex);
 		break;
 	case REFTABLE_REF_DELETION:
@@ -296,6 +296,11 @@ void reftable_ref_record_print(const struct reftable_ref_record *ref,
 	printf("}\n");
 }
 
+void reftable_ref_record_print(const struct reftable_ref_record *ref,
+			       uint32_t hash_id) {
+	reftable_ref_record_print_sz(ref, hash_size(hash_id));
+}
+
 static void reftable_ref_record_release_void(void *rec)
 {
 	reftable_ref_record_release(rec);
@@ -456,6 +461,13 @@ static int reftable_ref_record_equal_void(const void *a,
 	return reftable_ref_record_equal(ra, rb, hash_size);
 }
 
+static void reftable_ref_record_print_void(const void *rec,
+					   int hash_size)
+{
+	reftable_ref_record_print_sz(
+		(struct reftable_ref_record *) rec, hash_size);
+}
+
 static struct reftable_record_vtable reftable_ref_record_vtable = {
 	.key = &reftable_ref_record_key,
 	.type = BLOCK_TYPE_REF,
@@ -466,6 +478,7 @@ static struct reftable_record_vtable reftable_ref_record_vtable = {
 	.release = &reftable_ref_record_release_void,
 	.is_deletion = &reftable_ref_record_is_deletion_void,
 	.equal = &reftable_ref_record_equal_void,
+	.print = &reftable_ref_record_print_void,
 };
 
 static void reftable_obj_record_key(const void *r, struct strbuf *dest)
@@ -484,6 +497,21 @@ static void reftable_obj_record_release(void *rec)
 	memset(obj, 0, sizeof(struct reftable_obj_record));
 }
 
+static void reftable_obj_record_print(const void *rec, int hash_size)
+{
+	const struct reftable_obj_record *obj = rec;
+	char hex[GIT_MAX_HEXSZ + 1] = { 0 };
+	struct strbuf offset_str = STRBUF_INIT;
+	int i;
+
+	for (i = 0; i < obj->offset_len; i++)
+		strbuf_addf(&offset_str, "%"PRIu64 " ", obj->offsets[i]);
+	hex_format(hex, obj->hash_prefix, obj->hash_prefix_len);
+	printf("prefix %s (len %d), offsets [%s]\n",
+	       hex, obj->hash_prefix_len, offset_str.buf);
+	strbuf_release(&offset_str);
+}
+
 static void reftable_obj_record_copy_from(void *rec, const void *src_rec,
 					  int hash_size)
 {
@@ -627,31 +655,41 @@ static struct reftable_record_vtable reftable_obj_record_vtable = {
 	.release = &reftable_obj_record_release,
 	.is_deletion = &not_a_deletion,
 	.equal = &reftable_obj_record_equal_void,
+	.print = &reftable_obj_record_print,
 };
 
-void reftable_log_record_print(struct reftable_log_record *log,
-			       uint32_t hash_id)
+static void reftable_log_record_print_sz(struct reftable_log_record *log,
+					 int hash_size)
 {
 	char hex[GIT_MAX_HEXSZ + 1] = { 0 };
 
 	switch (log->value_type) {
 	case REFTABLE_LOG_DELETION:
-		printf("log{%s(%" PRIu64 ") delete", log->refname,
+		printf("log{%s(%"PRIu64 ") delete\n", log->refname,
 		       log->update_index);
 		break;
 	case REFTABLE_LOG_UPDATE:
-		printf("log{%s(%" PRIu64 ") %s <%s> %" PRIu64 " %04d\n",
-		       log->refname, log->update_index, log->value.update.name,
-		       log->value.update.email, log->value.update.time,
+		printf("log{%s(%"PRIu64 ") %s <%s> %" PRIu64 " %04d\n",
+		       log->refname, log->update_index,
+		       log->value.update.name ? log->value.update.name : "",
+		       log->value.update.email ? log->value.update.email : "",
+		       log->value.update.time,
 		       log->value.update.tz_offset);
-		hex_format(hex, log->value.update.old_hash, hash_size(hash_id));
+		hex_format(hex, log->value.update.old_hash, hash_size);
 		printf("%s => ", hex);
-		hex_format(hex, log->value.update.new_hash, hash_size(hash_id));
-		printf("%s\n\n%s\n}\n", hex, log->value.update.message);
+		hex_format(hex, log->value.update.new_hash, hash_size);
+		printf("%s\n\n%s\n}\n", hex,
+		       log->value.update.message ? log->value.update.message : "");
 		break;
 	}
 }
 
+void reftable_log_record_print(struct reftable_log_record *log,
+				      uint32_t hash_id)
+{
+	reftable_log_record_print_sz(log, hash_size(hash_id));
+}
+
 static void reftable_log_record_key(const void *r, struct strbuf *dest)
 {
 	const struct reftable_log_record *rec =
@@ -969,6 +1007,11 @@ static int reftable_log_record_is_deletion_void(const void *p)
 		(const struct reftable_log_record *)p);
 }
 
+static void reftable_log_record_print_void(const void *rec, int hash_size)
+{
+	reftable_log_record_print_sz((struct reftable_log_record*)rec, hash_size);
+}
+
 static struct reftable_record_vtable reftable_log_record_vtable = {
 	.key = &reftable_log_record_key,
 	.type = BLOCK_TYPE_LOG,
@@ -978,7 +1021,8 @@ static struct reftable_record_vtable reftable_log_record_vtable = {
 	.decode = &reftable_log_record_decode,
 	.release = &reftable_log_record_release_void,
 	.is_deletion = &reftable_log_record_is_deletion_void,
-	.equal = &reftable_log_record_equal_void
+	.equal = &reftable_log_record_equal_void,
+	.print = &reftable_log_record_print_void,
 };
 
 static void reftable_index_record_key(const void *r, struct strbuf *dest)
@@ -1054,6 +1098,13 @@ static int reftable_index_record_equal(const void *a, const void *b, int hash_si
 	return ia->offset == ib->offset && !strbuf_cmp(&ia->last_key, &ib->last_key);
 }
 
+static void reftable_index_record_print(const void *rec, int hash_size)
+{
+	const struct reftable_index_record *idx = rec;
+	/* TODO: escape null chars? */
+	printf("\"%s\" %" PRIu64 "\n", idx->last_key.buf, idx->offset);
+}
+
 static struct reftable_record_vtable reftable_index_record_vtable = {
 	.key = &reftable_index_record_key,
 	.type = BLOCK_TYPE_INDEX,
@@ -1064,6 +1115,7 @@ static struct reftable_record_vtable reftable_index_record_vtable = {
 	.release = &reftable_index_record_release,
 	.is_deletion = &not_a_deletion,
 	.equal = &reftable_index_record_equal,
+	.print = &reftable_index_record_print,
 };
 
 void reftable_record_key(struct reftable_record *rec, struct strbuf *dest)
@@ -1251,3 +1303,9 @@ struct reftable_record reftable_new_record(uint8_t type)
 	}
 	return clean;
 }
+
+void reftable_record_print(struct reftable_record *rec, int hash_size)
+{
+	printf("'%c': ", rec->type);
+	reftable_record_vtable(rec)->print(reftable_record_data(rec), hash_size);
+}
reftable/record.h +4 -0
index 010a322e901..fd80cd451d5 100644
@@ -61,6 +61,9 @@ struct reftable_record_vtable {
 
 	/* Are two records equal? This assumes they have the same type. Returns 0 for non-equal. */
 	int (*equal)(const void *a, const void *b, int hash_size);
+
+	/* Print on stdout, for debugging. */
+	void (*print)(const void *rec, int hash_size);
 };
 
 /* returns true for recognized block types. Block start with the block type. */
@@ -112,6 +115,7 @@ struct reftable_record {
 
 /* see struct record_vtable */
 int reftable_record_equal(struct reftable_record *a, struct reftable_record *b, int hash_size);
+void reftable_record_print(struct reftable_record *rec, int hash_size);
 void reftable_record_key(struct reftable_record *rec, struct strbuf *dest);
 uint8_t reftable_record_type(struct reftable_record *rec);
 void reftable_record_copy_from(struct reftable_record *rec,
reftable/record_test.c +19 -1
index 4a025c48104..847abfa2b04 100644
@@ -23,6 +23,10 @@ static void test_copy(struct reftable_record *rec)
 	/* do it twice to catch memory leaks */
 	reftable_record_copy_from(&copy, rec, GIT_SHA1_RAWSZ);
 	EXPECT(reftable_record_equal(rec, &copy, GIT_SHA1_RAWSZ));
+
+	puts("testing print coverage:\n");
+	reftable_record_print(&copy, GIT_SHA1_RAWSZ);
+
 	reftable_record_release(&copy);
 }
 
@@ -175,7 +179,7 @@ static void test_reftable_log_record_equal(void)
 static void test_reftable_log_record_roundtrip(void)
 {
 	int i;
-	struct reftable_log_record in[2] = {
+	struct reftable_log_record in[] = {
 		{
 			.refname = xstrdup("refs/heads/master"),
 			.update_index = 42,
@@ -196,10 +200,24 @@ static void test_reftable_log_record_roundtrip(void)
 			.refname = xstrdup("refs/heads/master"),
 			.update_index = 22,
 			.value_type = REFTABLE_LOG_DELETION,
+		},
+		{
+			/* make sure we handle optional fields correctly. */
+			.refname = xstrdup("branch"),
+			.update_index = 33,
+			.value_type = REFTABLE_LOG_UPDATE,
+			.value = {
+				.update = {
+					.old_hash = reftable_malloc(GIT_SHA1_RAWSZ),
+					.new_hash = reftable_malloc(GIT_SHA1_RAWSZ),
+				},
+			},
 		}
 	};
 	set_test_hash(in[0].value.update.new_hash, 1);
 	set_test_hash(in[0].value.update.old_hash, 2);
+	set_test_hash(in[2].value.update.new_hash, 3);
+	set_test_hash(in[2].value.update.old_hash, 4);
 	for (i = 0; i < ARRAY_SIZE(in); i++) {
 		struct reftable_record rec = { .type = BLOCK_TYPE_LOG };
 		struct strbuf key = STRBUF_INIT;
Signature · 1 line
gitgitgadget
#193 lore raw

PATCH v7 16/16 reftable: rename typ to type

In reply to Han-Wen Nienhuys via GitGitGadget

From: Han-Wen Nienhuys <[email protected]>

The 'typ' variable name betrays the Go origins of this code (in Go, 'type' is a reserved word). Since we now have parted ways with the Go code, simply use 'type'.

---
 reftable/block.c  | 10 ++++----
 reftable/merged.c |  6 ++---
 reftable/merged.h |  2 +-
 reftable/reader.c | 58 +++++++++++++++++++++++------------------------
 reftable/record.c |  4 ++--
 reftable/writer.c | 32 +++++++++++++-------------
 6 files changed, 56 insertions(+), 56 deletions(-)
reftable/block.c +5 -5
index 80ad10bb794..6c1f70ba563 100644
@@ -68,14 +68,14 @@ static int block_writer_register_restart(struct block_writer *w, int n,
 	return 0;
 }
 
-void block_writer_init(struct block_writer *bw, uint8_t typ, uint8_t *buf,
+void block_writer_init(struct block_writer *bw, uint8_t type, uint8_t *buf,
 		       uint32_t block_size, uint32_t header_off, int hash_size)
 {
 	bw->buf = buf;
 	bw->hash_size = hash_size;
 	bw->block_size = block_size;
 	bw->header_off = header_off;
-	bw->buf[header_off] = typ;
+	bw->buf[header_off] = type;
 	bw->next = header_off + 4;
 	bw->restart_interval = 16;
 	bw->entries = 0;
@@ -186,7 +186,7 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,
 		      int hash_size)
 {
 	uint32_t full_block_size = table_block_size;
-	uint8_t typ = block->data[header_off];
+	uint8_t type = block->data[header_off];
 	uint32_t sz = get_be24(block->data + header_off + 1);
 
 	int err = 0;
@@ -195,12 +195,12 @@ int block_reader_init(struct block_reader *br, struct reftable_block *block,
 	uint8_t *restart_bytes = NULL;
 	uint8_t *uncompressed = NULL;
 
-	if (!reftable_is_block_type(typ)) {
+	if (!reftable_is_block_type(type)) {
 		err =  REFTABLE_FORMAT_ERROR;
 		goto done;
 	}
 
-	if (typ == BLOCK_TYPE_LOG) {
+	if (type == BLOCK_TYPE_LOG) {
 		int block_header_skip = 4 + header_off;
 		uLongf dst_len = sz - block_header_skip; /* total size of dest
 							    buffer. */
reftable/merged.c +3 -3
index 82511762d2a..b5710d5f7a9 100644
@@ -22,7 +22,7 @@ static int merged_iter_init(struct merged_iter *mi)
 {
 	int i = 0;
 	for (i = 0; i < mi->stack_len; i++) {
-		struct reftable_record rec = reftable_new_record(mi->typ);
+		struct reftable_record rec = reftable_new_record(mi->type);
 		int err = iterator_next(&mi->stack[i], &rec);
 		if (err < 0) {
 			return err;
@@ -58,7 +58,7 @@ static int merged_iter_advance_nonnull_subiter(struct merged_iter *mi,
 					       size_t idx)
 {
 	struct pq_entry e = {
-		.rec = reftable_new_record(mi->typ),
+		.rec = reftable_new_record(mi->type),
 		.index = idx,
 	};
 	int err = iterator_next(&mi->stack[idx], &e.rec);
@@ -245,7 +245,7 @@ static int merged_table_seek_record(struct reftable_merged_table *mt,
 		sizeof(struct reftable_iterator) * mt->stack_len);
 	struct merged_iter merged = {
 		.stack = iters,
-		.typ = reftable_record_type(rec),
+		.type = reftable_record_type(rec),
 		.hash_id = mt->hash_id,
 		.suppress_deletions = mt->suppress_deletions,
 	};
reftable/merged.h +1 -1
index 7d9f95d27ed..63f288ff8d6 100644
@@ -28,7 +28,7 @@ struct merged_iter {
 	struct reftable_iterator *stack;
 	uint32_t hash_id;
 	size_t stack_len;
-	uint8_t typ;
+	uint8_t type;
 	int suppress_deletions;
 	struct merged_iter_pqueue pq;
 };
reftable/reader.c +29 -29

185 lines not on this page · raw message

reftable/record.c +2 -2
index c57bac0e96a..8330b7a2589 100644
@@ -78,9 +78,9 @@ int put_var_int(struct string_view *dest, uint64_t val)
 	return n;
 }
 
-int reftable_is_block_type(uint8_t typ)
+int reftable_is_block_type(uint8_t type)
 {
-	switch (typ) {
+	switch (type) {
 	case BLOCK_TYPE_REF:
 	case BLOCK_TYPE_LOG:
 	case BLOCK_TYPE_OBJ:
reftable/writer.c +16 -16
index 96a5155ea77..5b768438625 100644
@@ -26,9 +26,9 @@ static void writer_clear_index(struct reftable_writer *w);
 static int writer_finish_public_section(struct reftable_writer *w);
 
 static struct reftable_block_stats *
-writer_reftable_block_stats(struct reftable_writer *w, uint8_t typ)
+writer_reftable_block_stats(struct reftable_writer *w, uint8_t type)
 {
-	switch (typ) {
+	switch (type) {
 	case 'r':
 		return &w->stats.ref_stats;
 	case 'o':
@@ -102,7 +102,7 @@ static int writer_write_header(struct reftable_writer *w, uint8_t *dest)
 	return header_size(writer_version(w));
 }
 
-static void writer_reinit_block_writer(struct reftable_writer *w, uint8_t typ)
+static void writer_reinit_block_writer(struct reftable_writer *w, uint8_t type)
 {
 	int block_start = 0;
 	if (w->next == 0) {
@@ -110,7 +110,7 @@ static void writer_reinit_block_writer(struct reftable_writer *w, uint8_t typ)
 	}
 
 	strbuf_release(&w->last_key);
-	block_writer_init(&w->block_writer_data, typ, w->block,
+	block_writer_init(&w->block_writer_data, type, w->block,
 			  w->opts.block_size, block_start,
 			  hash_size(w->opts.hash_id));
 	w->block_writer = &w->block_writer_data;
@@ -375,7 +375,7 @@ int reftable_writer_add_logs(struct reftable_writer *w,
 
 static int writer_finish_section(struct reftable_writer *w)
 {
-	uint8_t typ = block_writer_type(w->block_writer);
+	uint8_t type = block_writer_type(w->block_writer);
 	uint64_t index_start = 0;
 	int max_level = 0;
 	int threshold = w->opts.unpadded ? 1 : 3;
@@ -434,7 +434,7 @@ static int writer_finish_section(struct reftable_writer *w)
 	if (err < 0)
 		return err;
 
-	bstats = writer_reftable_block_stats(w, typ);
+	bstats = writer_reftable_block_stats(w, type);
 	bstats->index_blocks = w->stats.idx_stats.blocks - before_blocks;
 	bstats->index_offset = index_start;
 	bstats->max_index_level = max_level;
@@ -536,17 +536,17 @@ static int writer_dump_object_index(struct reftable_writer *w)
 
 static int writer_finish_public_section(struct reftable_writer *w)
 {
-	uint8_t typ = 0;
+	uint8_t type = 0;
 	int err = 0;
 
 	if (w->block_writer == NULL)
 		return 0;
 
-	typ = block_writer_type(w->block_writer);
+	type = block_writer_type(w->block_writer);
 	err = writer_finish_section(w);
 	if (err < 0)
 		return err;
-	if (typ == BLOCK_TYPE_REF && !w->opts.skip_index_objects &&
+	if (type == BLOCK_TYPE_REF && !w->opts.skip_index_objects &&
 	    w->stats.ref_stats.index_blocks > 0) {
 		err = writer_dump_object_index(w);
 		if (err < 0)
@@ -630,10 +630,10 @@ static const int debug = 0;
 
 static int writer_flush_nonempty_block(struct reftable_writer *w)
 {
-	uint8_t typ = block_writer_type(w->block_writer);
+	uint8_t type = block_writer_type(w->block_writer);
 	struct reftable_block_stats *bstats =
-		writer_reftable_block_stats(w, typ);
-	uint64_t block_typ_off = (bstats->blocks == 0) ? w->next : 0;
+		writer_reftable_block_stats(w, type);
+	uint64_t block_type_off = (bstats->blocks == 0) ? w->next : 0;
 	int raw_bytes = block_writer_finish(w->block_writer);
 	int padding = 0;
 	int err = 0;
@@ -641,12 +641,12 @@ static int writer_flush_nonempty_block(struct reftable_writer *w)
 	if (raw_bytes < 0)
 		return raw_bytes;
 
-	if (!w->opts.unpadded && typ != BLOCK_TYPE_LOG) {
+	if (!w->opts.unpadded && type != BLOCK_TYPE_LOG) {
 		padding = w->opts.block_size - raw_bytes;
 	}
 
-	if (block_typ_off > 0) {
-		bstats->offset = block_typ_off;
+	if (block_type_off > 0) {
+		bstats->offset = block_type_off;
 	}
 
 	bstats->entries += w->block_writer->entries;
@@ -655,7 +655,7 @@ static int writer_flush_nonempty_block(struct reftable_writer *w)
 	w->stats.blocks++;
 
 	if (debug) {
-		fprintf(stderr, "block %c off %" PRIu64 " sz %d (%d)\n", typ,
+		fprintf(stderr, "block %c off %" PRIu64 " sz %d (%d)\n", type,
 			w->next, raw_bytes,
 			get_be24(w->block + w->block_writer->header_off + 1));
 	}
Signature · 1 line
gitgitgadget
#194 lore raw
In reply to Han-Wen Nienhuys via GitGitGadget
Quoted 7 lines · Han-Wen Nienhuys via GitGitGadget

This series was targeted to 'master'.

This addresses some complaints from Coverity that Peff reported.

v7:

* address avarab's comments.

Thanks. I just had time to skim it now but this overall LGTM. I noted a compilation error on xlc in another reply that I'm sorry I didn't test before, but provided a hopefully easy fix for.

Quoted 9 lines
     @@ reftable/record.c: static int reftable_ref_record_equal_void(const void *a,
      +static void reftable_ref_record_print_void(const void *rec,
      +					   int hash_size)
      +{
     -+	reftable_ref_record_print_sz((struct reftable_ref_record *) rec, hash_size);
     ++	reftable_ref_record_print_sz(
     ++		(struct reftable_ref_record *) rec, hash_size);
      +}
      +

Just a nit, and only noting this since you seemed to be adjusting your clang-format settings (or so I recall), but I think the more usual wrapping in the codebase is:

    diff --git a/reftable/record.c b/reftable/record.c
    index 8330b7a2589..ee42897460d 100644
    --- a/reftable/record.c
    +++ b/reftable/record.c
    @@ -464,8 +464,8 @@ static int reftable_ref_record_equal_void(const void *a,
     static void reftable_ref_record_print_void(const void *rec,
                                               int hash_size)
     {
    -       reftable_ref_record_print_sz(
    -               (struct reftable_ref_record *) rec, hash_size);
    +       reftable_ref_record_print_sz((struct reftable_ref_record *)rec,
    +                                    hash_size);
     }
     
     static struct reftable_record_vtable reftable_ref_record_vtable = {

I.e. to "keep going" with the initial line until we hit 79 characters, and then wrap it. Then for casts "(x *)y" rather than "(x *) y".