Skip to contentSkip to conversation

rev-list: add --missing=print-only mode

PATCH In review Siddharth Asthana 6 participants 18 messages Apr 19 – Sep 3 Documentation

lore
Share & tools
lore
#0 lore raw

cover rev-list: add --missing=print-only mode

Hi,

At GitLab, Gitaly uses rev-list --missing=print to find missing objects in partial clones. The current output mixes present and missing objects together, with missing ones prefixed by '?', so Gitaly has to post-process the output to extract just the missing OIDs. Having a dedicated mode that outputs only the missing OIDs directly would simplify this.

This patch adds --missing=print-only which suppresses all regular object output and prints only the missing OIDs, one per line, without the '?' prefix. A missing_action_prints() helper consolidates the repeated three-way checks for MA_PRINT/MA_PRINT_INFO/MA_PRINT_ONLY.

The series is based on top of 9f223ef1c0 (Git 2.54-rc2).

CI: https://gitlab.com/gitlab-org/git/-/pipelines/2463294546

Siddharth Asthana (1):
rev-list: add --missing=print-only mode

 Documentation/rev-list-options.adoc |  5 +++
 builtin/rev-list.c                  | 49 ++++++++++++++++++++++-------
 t/t6022-rev-list-missing.sh         | 26 +++++++++++++++
 3 files changed, 68 insertions(+), 12 deletions(-)

base-commit: 9f223ef1c026d91c7ac68cc0211bde255dda6199

Thanks
- Siddharth

#1 lore raw

PATCH 1/1 rev-list: add --missing=print-only mode

When working with partial clones, it's common to want just the list of missing objects. The current --missing=print mode does this but mixes present and missing objects together, with missing ones prefixed by '?'. Getting only the missing OIDs requires an extra pipe:

  git rev-list --objects --all --missing=print | perl -ne 'print if s/^[?]//'

Add --missing=print-only which outputs only the missing object OIDs, one per line, without any prefix. This makes the above one-liner unnecessary and the output directly usable by downstream tools.

---
 Documentation/rev-list-options.adoc |  5 +++
 builtin/rev-list.c                  | 49 ++++++++++++++++++++++-------
 t/t6022-rev-list-missing.sh         | 26 +++++++++++++++
 3 files changed, 68 insertions(+), 12 deletions(-)
Documentation/rev-list-options.adoc +5 -0
index 2d195a1474..5438be5975 100644
@@ -1056,6 +1056,11 @@ Unexpected missing objects will raise an error.
 The form `--missing=print` is like `allow-any`, but will also print a
 list of the missing objects.  Object IDs are prefixed with a ``?'' character.
 +
+The form `--missing=print-only` is like `print`, but will print ONLY the
+missing objects (not the present ones), and without the ``?'' prefix.  This
+is useful for scripting, as a simpler alternative to
+`--missing=print | sed -n 's/^?//p'`.
++
 The form `--missing=print-info` is like `print`, but will also print additional
 information about the missing object inferred from its containing object. The
 information is all printed on the same line with the missing object ID in the
builtin/rev-list.c +37 -12

116 lines not on this page · raw message

t/t6022-rev-list-missing.sh +26 -0
index 08e92dd002..105560ad21 100755
@@ -198,6 +198,32 @@ do
 	'
 done
 
+for obj in "HEAD~1" "HEAD~1^{tree}" "HEAD:1.t"
+do
+	test_expect_success "rev-list --missing=print-only with missing $obj" '
+		oid="$(git rev-parse $obj)" &&
+		path=".git/objects/$(test_oid_to_path $oid)" &&
+
+		# Capture present OIDs before hiding anything.
+		git rev-list --objects --no-object-names HEAD ^$obj >present.raw &&
+
+		mv "$path" "$path.hidden" &&
+		test_when_finished "mv $path.hidden $path" &&
+
+		git rev-list --missing=print-only --objects --no-object-names \
+			HEAD >actual &&
+
+		# Only the missing OID should appear, without the "?" prefix.
+		grep "^$oid$" actual &&
+
+		# Present objects must NOT appear in the output.
+		while read present_oid
+		do
+			! grep "^$present_oid$" actual || return 1
+		done <present.raw
+	'
+done
+
 test_expect_success "-z nul-delimited --missing" '
 	test_when_finished rm -rf repo &&
 
Signature · 1 line
2.53.0
#2 lore raw
Quoted 10 lines · Siddharth Asthana

When working with partial clones, it's common to want just the list of missing objects. The current --missing=print mode does this but mixes present and missing objects together, with missing ones prefixed by '?'. Getting only the missing OIDs requires an extra pipe:

   git rev-list --objects --all --missing=print | perl -ne 'print if s/^[?]//'

Add --missing=print-only which outputs only the missing object OIDs, one per line, without any prefix. This makes the above one-liner unnecessary and the output directly usable by downstream tools.

I'm a fan of this mode. It saves time dealing with the I/O.

builtin/rev-list.c static void missing_objects_map_entry_free(void *e) · line 112

Quoted 12 lines
  static struct oidmap missing_objects;
  enum missing_action {
builtin/rev-list.c +5 -4
-	MA_ERROR = 0,    /* fail if any missing objects are encountered */
-	MA_ALLOW_ANY,    /* silently allow ALL missing objects */
-	MA_PRINT,        /* print ALL missing objects in special section */
-	MA_PRINT_INFO,   /* same as MA_PRINT but also prints missing object info */
+	MA_ERROR = 0, /* fail if any missing objects are encountered */
+	MA_ALLOW_ANY, /* silently allow ALL missing objects */
+	MA_PRINT, /* print ALL missing objects in special section */
+	MA_PRINT_INFO, /* same as MA_PRINT but also prints missing object info */
+	MA_PRINT_ONLY, /* print ONLY missing objects, without the "?" prefix */
  	MA_ALLOW_PROMISOR, /* silently allow all missing PROMISOR objects */

I'm not a fan of this adjustment of spacing on the comments, though I do see that MA_ALLOW_PROMISOR had already broken the pattern. MA_PRINT_ONLY would still work with the earlier comment alignment, though.

builtin/rev-list.c static void print_missing_object(struct missing_objects_map_entry *entry, · line 169

Quoted 4 lines
builtin/rev-list.c +3 -1
-	if (line_term)
+	if (arg_missing_action == MA_PRINT_ONLY) {
+		printf("%s", oid_to_hex(&entry->entry.oid));
+		putchar(line_term);

Is there a reason you didn't use a printf("%s%c") here to put the oid and line_term together?

Quoted 85 lines
builtin/rev-list.c +21 -6
+		return;
+	} else if (line_term) {
  		printf("?%s", oid_to_hex(&entry->entry.oid));
-	else
+	} else {
  		printf("%s%cmissing=yes", oid_to_hex(&entry->entry.oid),
  		       info_term);
+	}
 
 
 
  	if (!print_missing_info) {
  		putchar(line_term);
@@ -209,6 +222,7 @@ static inline void finish_object__ma(struct object *obj, const char *name)
 
  	case MA_PRINT:
  	case MA_PRINT_INFO:
+	case MA_PRINT_ONLY:
  		add_missing_object_entry(&obj->oid, name, obj->type);
  		return;
 
@@ -246,6 +260,11 @@ static void show_commit(struct commit *commit, void *data)
  		return;
  	}
 
+	if (arg_missing_action == MA_PRINT_ONLY) {
+		finish_commit(commit);
+		return;
+	}
+
  	if (show_disk_usage)
  		total_disk_usage += get_object_disk_usage(&commit->object);
 
@@ -384,6 +403,8 @@ static void show_object(struct object *obj, const char *name, void *cb_data)
  	if (finish_object(obj, name, cb_data))
  		return;
  	display_progress(progress, ++progress_counter);
+	if (arg_missing_action == MA_PRINT_ONLY)
+		return;
  	if (show_disk_usage)
  		total_disk_usage += get_object_disk_usage(obj);
  	if (info->flags & REV_LIST_QUIET)
@@ -525,6 +546,12 @@ static inline int parse_missing_action_value(const char *value)
  		return 1;
  	}
 
+	if (!strcmp(value, "print-only")) {
+		arg_missing_action = MA_PRINT_ONLY;
+		fetch_if_missing = 0;
+		return 1;
+	}
+
  	if (!strcmp(value, "allow-promisor")) {
  		arg_missing_action = MA_ALLOW_PROMISOR;
  		fetch_if_missing = 0;
@@ -967,8 +994,7 @@ int cmd_rev_list(int argc,
 
  	if (arg_print_omitted)
  		oidset_init(&omitted_objects, DEFAULT_OIDSET_SIZE);
-	if (arg_missing_action == MA_PRINT ||
-	    arg_missing_action == MA_PRINT_INFO) {
+	if (missing_action_prints()) {
  		struct oidset_iter iter;
  		struct object_id *oid;
 
@@ -994,8 +1020,7 @@ int cmd_rev_list(int argc,
  			printf("~%s\n", oid_to_hex(oid));
  		oidset_clear(&omitted_objects);
  	}
-	if (arg_missing_action == MA_PRINT ||
-	    arg_missing_action == MA_PRINT_INFO) {
+	if (missing_action_prints()) {
  		struct missing_objects_map_entry *entry;
  		struct oidmap_iter iter;
 
@@ -1011,7 +1036,7 @@ int cmd_rev_list(int argc,
 
  	stop_progress(&progress);
 
-	if (revs.count) {
+	if (revs.count && arg_missing_action != MA_PRINT_ONLY) {
  		if (revs.left_right && revs.cherry_mark)
  			printf("%d\t%d\t%d\n", revs.count_left, revs.count_right, revs.count_same);
  		else if (revs.left_right)

builtin/rev-list.c int cmd_rev_list(int argc, · line 1051

Quoted 6 lines
builtin/rev-list.c +1 -1
@@ -1022,7 +1047,7 @@ int cmd_rev_list(int argc,
  			printf("%d\n", revs.count_left + revs.count_right);
  	}
 
-	if (show_disk_usage)
+	if (show_disk_usage && arg_missing_action != MA_PRINT_ONLY)
  		print_disk_usage(total_disk_usage);

I'm a little worried about all of these checks that need special-casing. These seem like options that are enabled by other options and could easily be grouped with the print-only setting.

If there is something wrong here where these need to be disabled, then we should update the documentation to say how this interacts with those options. And perhaps some warnings to say "these options are not compatible".

On that note: this patch is missing a document update.

Thanks,
-Stolee

#3 lore raw
Quoted 6 lines · Derrick Stolee
Quoted 5 lines · Siddharth Asthana

When working with partial clones, it's common to want just the list of missing objects. The current --missing=print mode does this but mixes present and missing objects together, with missing ones prefixed by '?'. Getting only the missing OIDs requires an extra pipe:

Quoted 7 lines
Quoted 5 lines

   git rev-list --objects --all --missing=print | perl -ne 'print if s/^[?]//'

Add --missing=print-only which outputs only the missing object OIDs, one per line, without any prefix. This makes the above one-liner unnecessary and the output directly usable by downstream tools.

I'm a fan of this mode. It saves time dealing with the I/O.

Thanks!

Quoted 17 lines
Quoted 16 lines

  static struct oidmap missing_objects;
  enum missing_action {
-    MA_ERROR = 0,    /* fail if any missing objects are encountered */ -    MA_ALLOW_ANY,    /* silently allow ALL missing objects */ -    MA_PRINT,        /* print ALL missing objects in special section */ -    MA_PRINT_INFO,   /* same as MA_PRINT but also prints missing object info */
+    MA_ERROR = 0, /* fail if any missing objects are encountered */ +    MA_ALLOW_ANY, /* silently allow ALL missing objects */
+    MA_PRINT, /* print ALL missing objects in special section */ +    MA_PRINT_INFO, /* same as MA_PRINT but also prints missing object info */
+    MA_PRINT_ONLY, /* print ONLY missing objects, without the "?" prefix */
      MA_ALLOW_PROMISOR, /* silently allow all missing PROMISOR objects */

Quoted 1 line

I'm not a fan of this adjustment of spacing on the comments,

You are right, unnecessary churn - will restore!

Quoted 11 lines

though I do see that MA_ALLOW_PROMISOR had already broken the pattern. MA_PRINT_ONLY would still work with the earlier comment alignment, though.

Quoted 4 lines

-    if (line_term)
+    if (arg_missing_action == MA_PRINT_ONLY) { +        printf("%s", oid_to_hex(&entry->entry.oid)); +        putchar(line_term);

Is there a reason you didn't use a printf("%s%c") here to put the oid and line_term together?

I just followed the existing pattern in the same function where every other path uses a separate putchar(line_term). Happy to combine if you prefer.

Quoted 80 lines
Quoted 80 lines

+        return;
+    } else if (line_term) {
          printf("?%s", oid_to_hex(&entry->entry.oid)); -    else
+    } else {
          printf("%s%cmissing=yes", oid_to_hex(&entry->entry.oid), info_term);
+    }

      if (!print_missing_info) {
          putchar(line_term);

(quoted diff) +0 -0
@@ -209,6 +222,7 @@ static inline void finish_object__ma(struct object 
 *obj, const char *name)

      case MA_PRINT:
      case MA_PRINT_INFO:
+    case MA_PRINT_ONLY:
          add_missing_object_entry(&obj->oid, name, obj->type); return;

builtin/rev-list.c +7 -0
@@ -246,6 +260,11 @@ static void show_commit(struct commit *commit, 
 void *data)
           return;
       }
+    if (arg_missing_action == MA_PRINT_ONLY) {
+        finish_commit(commit);
+        return;
+    }
+
       if (show_disk_usage)
           total_disk_usage += get_object_disk_usage(&commit->object);
@@ -384,6 +403,8 @@ static void show_object(struct object *obj, const 
 char *name, void *cb_data)
       if (finish_object(obj, name, cb_data))
           return;
       display_progress(progress, ++progress_counter);
+    if (arg_missing_action == MA_PRINT_ONLY)
+        return;
       if (show_disk_usage)
           total_disk_usage += get_object_disk_usage(obj);

      if (info->flags & REV_LIST_QUIET)

builtin/rev-list.c +9 -5
@@ -525,6 +546,12 @@ static inline int 
 parse_missing_action_value(const char *value)
           return 1;
       }
+    if (!strcmp(value, "print-only")) {
+        arg_missing_action = MA_PRINT_ONLY;
+        fetch_if_missing = 0;
+        return 1;
+    }
+
       if (!strcmp(value, "allow-promisor")) {
           arg_missing_action = MA_ALLOW_PROMISOR;
           fetch_if_missing = 0;
@@ -967,8 +994,7 @@ int cmd_rev_list(int argc,
       if (arg_print_omitted)
           oidset_init(&omitted_objects, DEFAULT_OIDSET_SIZE);
-    if (arg_missing_action == MA_PRINT ||
-        arg_missing_action == MA_PRINT_INFO) {
+    if (missing_action_prints()) {
           struct oidset_iter iter;
           struct object_id *oid;
@@ -994,8 +1020,7 @@ int cmd_rev_list(int argc,
               printf("~%s\n", oid_to_hex(oid));
           oidset_clear(&omitted_objects);
       }
-    if (arg_missing_action == MA_PRINT ||
-        arg_missing_action == MA_PRINT_INFO) {
+    if (missing_action_prints()) {
           struct missing_objects_map_entry *entry;
           struct oidmap_iter iter;
@@ -1011,7 +1036,7 @@ int cmd_rev_list(int argc,
       stop_progress(&progress);
-    if (revs.count) {
+    if (revs.count && arg_missing_action != MA_PRINT_ONLY) {
           if (revs.left_right && revs.cherry_mark)
               printf("%d\t%d\t%d\n", revs.count_left, revs.count_right, revs.count_same);
           else if (revs.left_right)
@@ -1022,7 +1047,7 @@ int cmd_rev_list(int argc,
Quoted 8 lines

builtin/rev-list.c int cmd_rev_list(int argc, · line 1051

Quoted 5 lines
builtin/rev-list.c +1 -1
               printf("%d\n", revs.count_left + revs.count_right);
       }
-    if (show_disk_usage)
+    if (show_disk_usage && arg_missing_action != MA_PRINT_ONLY)
           print_disk_usage(total_disk_usage);

I'm a little worried about all of these checks that need special-casing. These seem like options that are enabled

Phillip suggested making this a separate --missing-only flag that composes with existing --missing= modes - I think that's a better design. will work v2 around it.

Quoted 5 lines

by other options and could easily be grouped with the print-only setting.

If there is something wrong here where these need to be disabled, then we should update the documentation to say

Agreed - I will make --count and --disk-usage incompatible with --missing-only and die() with a clear message instead of silently suppressing them.

Quoted 4 lines

how this interacts with those options. And perhaps some warnings to say "these options are not compatible".

On that note: this patch is missing a document update.

There is a doc update in rev-list-options.doc, but it doesn't cover interactions with --count/--disk-usage. Will fix in v2.

Quoted 5 lines

Thanks,
-Stolee

Thanks,
Asthana

#4 lore raw
Quoted 8 lines · Siddharth Asthana
Quoted 7 lines · Derrick Stolee
Quoted 4 lines · Siddharth Asthana

-    if (line_term)
+    if (arg_missing_action == MA_PRINT_ONLY) { +        printf("%s", oid_to_hex(&entry->entry.oid)); +        putchar(line_term);

Quoted 9 lines
Quoted 2 lines

Is there a reason you didn't use a printf("%s%c") here to put the oid and line_term together?

I just followed the existing pattern in the same function where every other path uses a separate putchar(line_term). Happy to combine if you prefer.

Quoted 3 lines
Quoted 3 lines

+    } else {
          printf("%s%cmissing=yes", oid_to_hex(&entry->entry.oid), info_term);

I was just looking at this line in the patch context as comparison, but if there are other patterns that prefer "printf(); putc();" then that's enough for me.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

-    if (show_disk_usage)
+    if (show_disk_usage && arg_missing_action != MA_PRINT_ONLY) print_disk_usage(total_disk_usage);

I'm a little worried about all of these checks that need special-casing. These seem like options that are enabled

Phillip suggested making this a separate --missing-only flag that composes with existing --missing= modes - I think that's a better design. will work v2 around it.

That's a good idea. '--missing=' formats the output while your mode _filters_ the output. They are different things and worth different options.

Quoted 7 lines
Quoted 4 lines

how this interacts with those options. And perhaps some warnings to say "these options are not compatible".

On that note: this patch is missing a document update.

There is a doc update in rev-list-options.doc, but it doesn't  cover interactions with --count/--disk-usage. Will fix in v2.

You're right. I thought I had looked for it in the patch but missed it somehow.

Thanks,
-Stolee

#5 lore raw
In reply to Siddharth Asthana
Quoted 10 lines · Siddharth Asthana

When working with partial clones, it's common to want just the list of missing objects. The current --missing=print mode does this but mixes present and missing objects together, with missing ones prefixed by '?'. Getting only the missing OIDs requires an extra pipe:

  git rev-list --objects --all --missing=print | perl -ne 'print if s/^[?]//'

Add --missing=print-only which outputs only the missing object OIDs, one per line, without any prefix. This makes the above one-liner unnecessary and the output directly usable by downstream tools.

Naming is a bit tough, as "print-only" sounds as if we're only printing them without doing anything else, but it doesn't quite convey the relation to non-missing objects. I don't really have a better suggestion though -- "print-exclusively" may convey the meaning a tiny bit better, but still suffers kind of the same issue.

Quoted 7 lines
builtin/rev-list.c +0 -0
index 8f63003709..ba7e3e3919 100644
@@ -104,14 +104,22 @@ static void missing_objects_map_entry_free(void *e)
 
 static struct oidmap missing_objects;

builtin/rev-list.c static void missing_objects_map_entry_free(void *e) · line 111

Quoted 10 lines
builtin/rev-list.c +5 -4
 enum missing_action {
-	MA_ERROR = 0,    /* fail if any missing objects are encountered */
-	MA_ALLOW_ANY,    /* silently allow ALL missing objects */
-	MA_PRINT,        /* print ALL missing objects in special section */
-	MA_PRINT_INFO,   /* same as MA_PRINT but also prints missing object info */
+	MA_ERROR = 0, /* fail if any missing objects are encountered */
+	MA_ALLOW_ANY, /* silently allow ALL missing objects */
+	MA_PRINT, /* print ALL missing objects in special section */
+	MA_PRINT_INFO, /* same as MA_PRINT but also prints missing object info */
+	MA_PRINT_ONLY, /* print ONLY missing objects, without the "?" prefix */

Makes me wonder whether we'll eventually also want to have MA_PRINT_INFO_ONLY.

builtin/rev-list.c static void missing_objects_map_entry_free(void *e) · line 116

Quoted 5 lines
builtin/rev-list.c +1 -0
 	MA_ALLOW_PROMISOR, /* silently allow all missing PROMISOR objects */
 };
 static enum missing_action arg_missing_action;
 
+static inline int missing_action_prints(void)

How about naming this should_print_missing_object() instead? That gives the reader a bit more context.

builtin/rev-list.c int cmd_rev_list(int argc, · line 1042

Quoted 9 lines
builtin/rev-list.c +1 -1
@@ -1011,7 +1036,7 @@ int cmd_rev_list(int argc,
 
 	stop_progress(&progress);
 
-	if (revs.count) {
+	if (revs.count && arg_missing_action != MA_PRINT_ONLY) {
 		if (revs.left_right && revs.cherry_mark)
 			printf("%d\t%d\t%d\n", revs.count_left, revs.count_right, revs.count_same);
 		else if (revs.left_right)

Not a fault of your patch, but I really feel like git-rev-list(1) is becoming more and more tangled. The fact that we have to add this check to so many different sites doesn't inspire confidence that we have indeed catched all of them that need this check.

It would be great if this was reworked a bit to become more obvious, but that's probably outside the scope of this patch series.

Quoted 21 lines
t/t6022-rev-list-missing.sh +13 -0
index 08e92dd002..105560ad21 100755
@@ -198,6 +198,32 @@ do
 	'
 done
 
+for obj in "HEAD~1" "HEAD~1^{tree}" "HEAD:1.t"
+do
+	test_expect_success "rev-list --missing=print-only with missing $obj" '
+		oid="$(git rev-parse $obj)" &&
+		path=".git/objects/$(test_oid_to_path $oid)" &&
+
+		# Capture present OIDs before hiding anything.
+		git rev-list --objects --no-object-names HEAD ^$obj >present.raw &&
+
+		mv "$path" "$path.hidden" &&
+		test_when_finished "mv $path.hidden $path" &&
+
+		git rev-list --missing=print-only --objects --no-object-names \

t/t6022-rev-list-missing.sh do · line 223

Quoted 10 lines
t/t6022-rev-list-missing.sh +10 -0
+			HEAD >actual &&
+
+		# Only the missing OID should appear, without the "?" prefix.
+		grep "^$oid$" actual &&
+
+		# Present objects must NOT appear in the output.
+		while read present_oid
+		do
+			! grep "^$present_oid$" actual || return 1
+		done <present.raw

How many present object IDs do we have? I'm a bit worried that we now execute grep(1) hundreds of times. Can we maybe do some tricks with comm(1) instead?

Thanks!

Patrick

#6 lore raw
Quoted 16 lines · Patrick Steinhardt

builtin/rev-list.c static void missing_objects_map_entry_free(void *e) · line 111

Quoted 11 lines · Siddharth Asthana
  static struct oidmap missing_objects;
  enum missing_action {
builtin/rev-list.c +5 -4
-	MA_ERROR = 0,    /* fail if any missing objects are encountered */
-	MA_ALLOW_ANY,    /* silently allow ALL missing objects */
-	MA_PRINT,        /* print ALL missing objects in special section */
-	MA_PRINT_INFO,   /* same as MA_PRINT but also prints missing object info */
+	MA_ERROR = 0, /* fail if any missing objects are encountered */
+	MA_ALLOW_ANY, /* silently allow ALL missing objects */
+	MA_PRINT, /* print ALL missing objects in special section */
+	MA_PRINT_INFO, /* same as MA_PRINT but also prints missing object info */
+	MA_PRINT_ONLY, /* print ONLY missing objects, without the "?" prefix */

Makes me wonder whether we'll eventually also want to have MA_PRINT_INFO_ONLY.

Perhaps we'd be better to add a "--missing-only" option that limits the output to missing objects? That would avoid the problem of "--missing=print-only" not really explaining what it does as well.

Quoted 19 lines
Quoted 19 lines
t/t6022-rev-list-missing.sh +19 -0
+for obj in "HEAD~1" "HEAD~1^{tree}" "HEAD:1.t"
+do
+	test_expect_success "rev-list --missing=print-only with missing $obj" '
+		oid="$(git rev-parse $obj)" &&
+		path=".git/objects/$(test_oid_to_path $oid)" &&
+
+		# Capture present OIDs before hiding anything.
+		git rev-list --objects --no-object-names HEAD ^$obj >present.raw &&
+
+		mv "$path" "$path.hidden" &&
+		test_when_finished "mv $path.hidden $path" &&
+
+		git rev-list --missing=print-only --objects --no-object-names \
+			HEAD >actual &&
+
+		# Only the missing OID should appear, without the "?" prefix.
+		grep "^$oid$" actual &&
+
+		# Present objects must NOT appear in the output.
Quoted 8 lines

t/t6022-rev-list-missing.sh do · line 223

Quoted 4 lines
t/t6022-rev-list-missing.sh +4 -0
+		while read present_oid
+		do
+			! grep "^$present_oid$" actual || return 1
+		done <present.raw

How many present object IDs do we have? I'm a bit worried that we now execute grep(1) hundreds of times. Can we maybe do some tricks with comm(1) instead?

If we want to verify that it only prints a single oid then

	echo $oid >expect &&
	test_cmp expect actual

would be much simpler more helpful if the test fails

Thanks

Phil

#7 lore raw
Quoted 18 lines · Phillip Wood
Quoted 16 lines · Patrick Steinhardt

builtin/rev-list.c static void missing_objects_map_entry_free(void *e) · line 111

Quoted 11 lines · Siddharth Asthana
  static struct oidmap missing_objects;
  enum missing_action {
builtin/rev-list.c +5 -4
-	MA_ERROR = 0,    /* fail if any missing objects are encountered */
-	MA_ALLOW_ANY,    /* silently allow ALL missing objects */
-	MA_PRINT,        /* print ALL missing objects in special section */
-	MA_PRINT_INFO,   /* same as MA_PRINT but also prints missing object info */
+	MA_ERROR = 0, /* fail if any missing objects are encountered */
+	MA_ALLOW_ANY, /* silently allow ALL missing objects */
+	MA_PRINT, /* print ALL missing objects in special section */
+	MA_PRINT_INFO, /* same as MA_PRINT but also prints missing object info */
+	MA_PRINT_ONLY, /* print ONLY missing objects, without the "?" prefix */

Makes me wonder whether we'll eventually also want to have MA_PRINT_INFO_ONLY.

Quoted 3 lines

Perhaps we'd be better to add a "--missing-only" option that limits the output to missing objects? That would avoid the problem of "--missing=print-only" not really explaining what it does as well.

Yes, I had the same thought.

Patrick

#8 lore raw
In reply to Phillip Wood
Quoted 18 lines · Phillip Wood
Quoted 17 lines · Patrick Steinhardt
Quoted 14 lines · Siddharth Asthana

  static struct oidmap missing_objects;
  enum missing_action {
-    MA_ERROR = 0,    /* fail if any missing objects are encountered */ -    MA_ALLOW_ANY,    /* silently allow ALL missing objects */ -    MA_PRINT,        /* print ALL missing objects in special section */ -    MA_PRINT_INFO,   /* same as MA_PRINT but also prints missing object info */
+    MA_ERROR = 0, /* fail if any missing objects are encountered */ +    MA_ALLOW_ANY, /* silently allow ALL missing objects */ +    MA_PRINT, /* print ALL missing objects in special section */ +    MA_PRINT_INFO, /* same as MA_PRINT but also prints missing object info */
+    MA_PRINT_ONLY, /* print ONLY missing objects, without the "?" prefix */

Quoted 4 lines
Quoted 2 lines

Makes me wonder whether we'll eventually also want to have MA_PRINT_INFO_ONLY.

Perhaps we'd be better to add a "--missing-only" option that limits the output to missing objects? That would avoid the problem of "--

Make sense, its orthogonal to --missing= and handles print-info-only for free.

Question though: should --missing-only without --missing=print (or print-info) be an errro? I am leaning towards requiring it explicitly so the behavior is always obvious.

Will rework v2 around this.

Quoted 32 lines

missing=print-only" not really explaining what it does as well.

Quoted 21 lines
Quoted 21 lines
t/t6022-rev-list-missing.sh +3 -0
+for obj in "HEAD~1" "HEAD~1^{tree}" "HEAD:1.t"
+do
+    test_expect_success "rev-list --missing=print-only with missing 

$obj" '
+        oid="$(git rev-parse $obj)" &&
+        path=".git/objects/$(test_oid_to_path $oid)" && +
+        # Capture present OIDs before hiding anything. +        git rev-list --objects --no-object-names HEAD ^$obj

Quoted 1 line

present.raw &&

t/t6022-rev-list-missing.sh +11 -0
+
+        mv "$path" "$path.hidden" &&
+        test_when_finished "mv $path.hidden $path" &&
+
+        git rev-list --missing=print-only --objects --no-object-names \
+            HEAD >actual &&
+
+        # Only the missing OID should appear, without the "?" prefix.
+        grep "^$oid$" actual &&
+
+        # Present objects must NOT appear in the output.
Quoted 8 lines
Quoted 4 lines

+        while read present_oid
+        do
+            ! grep "^$present_oid$" actual || return 1 +        done <present.raw

How many present object IDs do we have? I'm a bit worried that we now execute grep(1) hundreds of times. Can we maybe do some tricks with comm(1) instead?

Quoted 6 lines

If we want to verify that it only prints a single oid then

    echo $oid >expect &&
    test_cmp expect actual

would be much simpler more helpful if the test fails

yeah much cleaner, will use it. Thanks

Quoted 4 lines

Thanks

Phil

Thanks,
Asthana

#9 lore raw
In reply to Phillip Wood
Quoted 18 lines · Phillip Wood
Quoted 16 lines · Patrick Steinhardt

builtin/rev-list.c static void missing_objects_map_entry_free(void *e) · line 111

Quoted 11 lines · Siddharth Asthana
  static struct oidmap missing_objects;
  enum missing_action {
builtin/rev-list.c +5 -4
-	MA_ERROR = 0,    /* fail if any missing objects are encountered */
-	MA_ALLOW_ANY,    /* silently allow ALL missing objects */
-	MA_PRINT,        /* print ALL missing objects in special section */
-	MA_PRINT_INFO,   /* same as MA_PRINT but also prints missing object info */
+	MA_ERROR = 0, /* fail if any missing objects are encountered */
+	MA_ALLOW_ANY, /* silently allow ALL missing objects */
+	MA_PRINT, /* print ALL missing objects in special section */
+	MA_PRINT_INFO, /* same as MA_PRINT but also prints missing object info */
+	MA_PRINT_ONLY, /* print ONLY missing objects, without the "?" prefix */

Makes me wonder whether we'll eventually also want to have MA_PRINT_INFO_ONLY.

Quoted 3 lines

Perhaps we'd be better to add a "--missing-only" option that limits the output to missing objects? That would avoid the problem of "--missing=print-only" not really explaining what it does as well.

The original --missing says what to do about missing objects. I find --missing=print-only to be surprising:

• Do print missing
• But also do not print present objects

Why would a --missing option dictate what to do about objects that are present (hide them)? That looks tacked-on.

--missing-only is better but has the same problem, IMO, of dictating how present objects should be treated.

From the cover letter:

CV> At GitLab, Gitaly uses rev-list --missing=print to find missing CV> objects in partial clones. The current output mixes present and missing CV> objects together, with missing ones prefixed by '?', so Gitaly has to CV> post-process the output to extract just the missing OIDs. Having a CV> dedicated mode that outputs only the missing OIDs directly would CV> simplify this.

This makes me think that I want:

1. I do want missing objects
2. I do not want present objects

Which makes me think that something like --present=no-print makes more sense.

Just speaking as a user and not looking at the code.

Quoted 1 line

[snip]

#10 lore raw
In reply to Patrick Steinhardt
Quoted 6 lines · Patrick Steinhardt
Quoted 5 lines · Siddharth Asthana

When working with partial clones, it's common to want just the list of missing objects. The current --missing=print mode does this but mixes present and missing objects together, with missing ones prefixed by '?'. Getting only the missing OIDs requires an extra pipe:

Quoted 8 lines
Quoted 5 lines
   git rev-list --objects --all --missing=print | perl -ne 'print if s/^[?]//'

Add --missing=print-only which outputs only the missing object OIDs, one per line, without any prefix. This makes the above one-liner unnecessary and the output directly usable by downstream tools.

Naming is a bit tough, as "print-only" sounds as if we're only printing them without doing anything else, but it doesn't quite convey the

The name came from Christian's original suggesttion in the issue [1], but agreed it's ambiguous. Phillip's --missing-only approach solve this.

[1] https://gitlab.com/gitlab-org/git/-/work_items/80#note_464005298

Quoted 10 lines

relation to non-missing objects. I don't really have a better suggestion though -- "print-exclusively" may convey the meaning a tiny bit better, but still suffers kind of the same issue.

Quoted 6 lines
builtin/rev-list.c +0 -0
index 8f63003709..ba7e3e3919 100644
@@ -104,14 +104,22 @@ static void missing_objects_map_entry_free(void *e)
 
Quoted 14 lines

builtin/rev-list.c static void missing_objects_map_entry_free(void *e) · line 111

Quoted 11 lines
builtin/rev-list.c +5 -4
  static struct oidmap missing_objects;
  enum missing_action {
-	MA_ERROR = 0,    /* fail if any missing objects are encountered */
-	MA_ALLOW_ANY,    /* silently allow ALL missing objects */
-	MA_PRINT,        /* print ALL missing objects in special section */
-	MA_PRINT_INFO,   /* same as MA_PRINT but also prints missing object info */
+	MA_ERROR = 0, /* fail if any missing objects are encountered */
+	MA_ALLOW_ANY, /* silently allow ALL missing objects */
+	MA_PRINT, /* print ALL missing objects in special section */
+	MA_PRINT_INFO, /* same as MA_PRINT but also prints missing object info */
+	MA_PRINT_ONLY, /* print ONLY missing objects, without the "?" prefix */

Makes me wonder whether we'll eventually also want to have MA_PRINT_INFO_ONLY.

Right - that's the strongest argument for --missing-only as a separate flag. Gets us that for free.

Quoted 9 lines

builtin/rev-list.c static void missing_objects_map_entry_free(void *e) · line 116

Quoted 6 lines
builtin/rev-list.c +1 -0
  	MA_ALLOW_PROMISOR, /* silently allow all missing PROMISOR objects */
  };
  static enum missing_action arg_missing_action;
 
+static inline int missing_action_prints(void)

How about naming this should_print_missing_object() instead? That gives the reader a bit more context.

The function is a predicate on the mode, not on a specific object, so that name would be slightly misleading. But with --missing-only this helper might change shape anyway. WDYT?

Quoted 46 lines

builtin/rev-list.c int cmd_rev_list(int argc, · line 1042

Quoted 10 lines
builtin/rev-list.c +1 -1
@@ -1011,7 +1036,7 @@ int cmd_rev_list(int argc,
 
  	stop_progress(&progress);
 
-	if (revs.count) {
+	if (revs.count && arg_missing_action != MA_PRINT_ONLY) {
  		if (revs.left_right && revs.cherry_mark)
  			printf("%d\t%d\t%d\n", revs.count_left, revs.count_right, revs.count_same);
  		else if (revs.left_right)

Not a fault of your patch, but I really feel like git-rev-list(1) is becoming more and more tangled. The fact that we have to add this check to so many different sites doesn't inspire confidence that we have indeed catched all of them that need this check.

It would be great if this was reworked a bit to become more obvious, but that's probably outside the scope of this patch series.

Quoted 27 lines
t/t6022-rev-list-missing.sh +19 -0
index 08e92dd002..105560ad21 100755
@@ -198,6 +198,32 @@ do
  	'
  done
 
+for obj in "HEAD~1" "HEAD~1^{tree}" "HEAD:1.t"
+do
+	test_expect_success "rev-list --missing=print-only with missing $obj" '
+		oid="$(git rev-parse $obj)" &&
+		path=".git/objects/$(test_oid_to_path $oid)" &&
+
+		# Capture present OIDs before hiding anything.
+		git rev-list --objects --no-object-names HEAD ^$obj >present.raw &&
+
+		mv "$path" "$path.hidden" &&
+		test_when_finished "mv $path.hidden $path" &&
+
+		git rev-list --missing=print-only --objects --no-object-names \
+			HEAD >actual &&
+
+		# Only the missing OID should appear, without the "?" prefix.
+		grep "^$oid$" actual &&
+
+		# Present objects must NOT appear in the output.
Quoted 8 lines

t/t6022-rev-list-missing.sh do · line 223

Quoted 4 lines
t/t6022-rev-list-missing.sh +4 -0
+		while read present_oid
+		do
+			! grep "^$present_oid$" actual || return 1
+		done <present.raw

How many present object IDs do we have? I'm a bit worried that we now execute grep(1) hundreds of times. Can we maybe do some tricks with comm(1) instead?

Phillip's test_cmp approach is simpler, since we hide one object, the output should be exactly that OID. Will use that.

Thanks,
Asthana

Quoted 4 lines

Thanks!

Patrick

#11 lore raw

v2 cover rev-list: add --missing-only option to filter output

In reply to Siddharth Asthana

Hi,

At GitLab, Gitaly uses rev-list --missing=print on partial clones to find missing objects. The output mixes present and missing objects and prefixes missing ones with '?', so we post-process it. --missing-only avoids that.

v1 added --missing=print-only as a --missing= mode. Review preferred a separate filter flag, so v2 adds --missing-only next to --missing=print or --missing=print-info.

Based on 1630431f32 (The 21st batch, 2026-08-31).

Changes from v1 include:
1. Separate --missing-only flag instead of print-only mode (Phillip, Patrick, Stolee).
2. Require --missing=print or --missing=print-info.
3. Die on --count / --disk-usage (Stolee).
4. No enum comment spacing churn.
5. print-info still prints path=/type=; only '?' is dropped.
6. test_cmp-based tests (Phillip).
7. Link to v1:
https://lore.kernel.org/git/[email protected]/

Thanks,
Siddharth

---
Siddharth Asthana (1):
rev-list: add --missing-only option to filter output

 Documentation/rev-list-options.adoc | 13 ++++++++
 builtin/rev-list.c                  | 42 ++++++++++++++++++++++---
 t/t6022-rev-list-missing.sh         | 49 +++++++++++++++++++++++++++++
 3 files changed, 99 insertions(+), 5 deletions(-)

Range-diff versus v1:

1: 7e8f1ad997 ! 1: 3c89e6bc38 rev-list: add --missing=print-only mode

    @@ Metadata
     Author: Siddharth Asthana <[email protected]>
     
      ## Commit message ##
    -    rev-list: add --missing=print-only mode
    +    rev-list: add --missing-only option to filter output
     
    -    When working with partial clones, it's common to want just the list of
    -    missing objects. The current --missing=print mode does this but mixes
    -    present and missing objects together, with missing ones prefixed by '?'.
    -    Getting only the missing OIDs requires an extra pipe:
    +    When working with partial clones, callers often need only the missing
    +    object IDs. Today that means post-processing --missing=print to drop
    +    present objects and strip the leading '?':
     
           git rev-list --objects --all --missing=print | perl -ne 'print if s/^[?]//'
     
    -    Add --missing=print-only which outputs only the missing object OIDs, one
    -    per line, without any prefix. This makes the above one-liner unnecessary
    -    and the output directly usable by downstream tools.
    +    Add --missing-only. Use it with --missing=print or --missing=print-info
    +    to print only missing objects. --missing= still picks the format;
    +    --missing-only only filters. The leading '?' is omitted. With
    +    print-info, path= and type= are still shown.
    +
    +    Require --missing=print or --missing=print-info. Reject --count and
    +    --disk-usage.
     
         Signed-off-by: Siddharth Asthana <[email protected]>
     
      ## Documentation/rev-list-options.adoc ##
    -@@ Documentation/rev-list-options.adoc: Unexpected missing objects will raise an error.
    - The form `--missing=print` is like `allow-any`, but will also print a
    - list of the missing objects.  Object IDs are prefixed with a ``?'' character.
    - +
    -+The form `--missing=print-only` is like `print`, but will print ONLY the
    -+missing objects (not the present ones), and without the ``?'' prefix.  This
    -+is useful for scripting, as a simpler alternative to
    -+`--missing=print | sed -n 's/^?//p'`.
    +@@ Documentation/rev-list-options.adoc: If some tips passed to the traversal are missing, they will be
    + considered as missing too, and the traversal will ignore them. In case
    + we cannot get their Object ID though, an error will be raised.
    + 
    ++`--missing-only`::
    ++	When used together with `--missing=print` or `--missing=print-info`,
    ++	suppress all output for present objects and print only the missing
    ++	ones.  The selected `--missing=` format is preserved (so
    ++	`--missing=print-info` still emits `path=` / `type=` fields), but the
    ++	leading ``?'' prefix used by the non-`-z` forms is omitted.  This is
    ++	useful for scripting, as a simpler and faster alternative to
    ++	post-processing the output of `--missing=print`.
     ++
    - The form `--missing=print-info` is like `print`, but will also print additional
    - information about the missing object inferred from its containing object. The
    - information is all printed on the same line with the missing object ID in the
    ++This option is incompatible with `--count` and `--disk-usage`.
    ++It is an error to use `--missing-only` without `--missing=print` or
    ++`--missing=print-info`.
    ++
    + `--exclude-promisor-objects`::
    + 	(For internal use only.)  Prefilter object traversal at
    + 	promisor boundary.  This is used with partial clone.  This is
     
      ## builtin/rev-list.c ##
    -@@ builtin/rev-list.c: static void missing_objects_map_entry_free(void *e)
    - 
    - static struct oidmap missing_objects;
    - enum missing_action {
    --	MA_ERROR = 0,    /* fail if any missing objects are encountered */
    --	MA_ALLOW_ANY,    /* silently allow ALL missing objects */
    --	MA_PRINT,        /* print ALL missing objects in special section */
    --	MA_PRINT_INFO,   /* same as MA_PRINT but also prints missing object info */
    -+	MA_ERROR = 0, /* fail if any missing objects are encountered */
    -+	MA_ALLOW_ANY, /* silently allow ALL missing objects */
    -+	MA_PRINT, /* print ALL missing objects in special section */
    -+	MA_PRINT_INFO, /* same as MA_PRINT but also prints missing object info */
    -+	MA_PRINT_ONLY, /* print ONLY missing objects, without the "?" prefix */
    +@@ builtin/rev-list.c: enum missing_action {
      	MA_ALLOW_PROMISOR, /* silently allow all missing PROMISOR objects */
      };
      static enum missing_action arg_missing_action;
    - 
    -+static inline int missing_action_prints(void)
    ++static int arg_missing_only;
    ++
    ++static inline int should_collect_missing(void)
     +{
     +	return arg_missing_action == MA_PRINT ||
    -+	       arg_missing_action == MA_PRINT_INFO ||
    -+	       arg_missing_action == MA_PRINT_ONLY;
    ++	       arg_missing_action == MA_PRINT_INFO;
     +}
    -+
    + 
      /* display only the oid of each object encountered */
      static int arg_show_object_names = 1;
    - 
     @@ builtin/rev-list.c: static void print_missing_object(struct missing_objects_map_entry *entry,
      {
      	struct strbuf sb = STRBUF_INIT;
      
     -	if (line_term)
    -+	if (arg_missing_action == MA_PRINT_ONLY) {
    ++	/*
    ++	 * --missing-only filters present objects out of the walk output.
    ++	 * It still uses the selected --missing= format for missing ones,
    ++	 * except the human "?" prefix is omitted (script-friendly OIDs).
    ++	 */
    ++	if (arg_missing_only && line_term)
     +		printf("%s", oid_to_hex(&entry->entry.oid));
    -+		putchar(line_term);
    -+		return;
    -+	} else if (line_term) {
    ++	else if (line_term)
      		printf("?%s", oid_to_hex(&entry->entry.oid));
    --	else
    -+	} else {
    + 	else
      		printf("%s%cmissing=yes", oid_to_hex(&entry->entry.oid),
    - 		       info_term);
    -+	}
    - 
    - 	if (!print_missing_info) {
    - 		putchar(line_term);
    -@@ builtin/rev-list.c: static inline void finish_object__ma(struct object *obj, const char *name)
    - 
    - 	case MA_PRINT:
    - 	case MA_PRINT_INFO:
    -+	case MA_PRINT_ONLY:
    - 		add_missing_object_entry(&obj->oid, name, obj->type);
    - 		return;
    - 
     @@ builtin/rev-list.c: static void show_commit(struct commit *commit, void *data)
      		return;
      	}
      
    -+	if (arg_missing_action == MA_PRINT_ONLY) {
    ++	if (arg_missing_only) {
     +		finish_commit(commit);
     +		return;
     +	}
    @@ builtin/rev-list.c: static void show_object(struct object *obj, const char *name
      	if (finish_object(obj, name, cb_data))
      		return;
      	display_progress(progress, ++progress_counter);
    -+	if (arg_missing_action == MA_PRINT_ONLY)
    ++	if (arg_missing_only)
     +		return;
      	if (show_disk_usage)
      		total_disk_usage += get_object_disk_usage(obj);
      	if (info->flags & REV_LIST_QUIET)
    -@@ builtin/rev-list.c: static inline int parse_missing_action_value(const char *value)
    - 		return 1;
    +@@ builtin/rev-list.c: int cmd_rev_list(int argc,
    + 			revs.exclude_promisor_objects = 1;
    + 		} else if (skip_prefix(arg, "--missing=", &arg)) {
    + 			parse_missing_action_value(arg);
    ++		} else if (!strcmp(arg, "--missing-only")) {
    ++			arg_missing_only = 1;
    + 		} else if (!strcmp(arg, "-z")) {
    + 			line_term = '\0';
    + 			info_term = '\0';
    + 		}
      	}
      
    -+	if (!strcmp(value, "print-only")) {
    -+		arg_missing_action = MA_PRINT_ONLY;
    -+		fetch_if_missing = 0;
    -+		return 1;
    -+	}
    ++	if (arg_missing_only && !should_collect_missing())
    ++		die(_("--missing-only requires --missing=print or --missing=print-info"));
    ++
    + 	die_for_incompatible_opt2(revs.exclude_promisor_objects,
    + 				  "--exclude_promisor_objects",
    + 				  arg_missing_action, "--missing");
    +@@ builtin/rev-list.c: int cmd_rev_list(int argc,
    + 			continue;
    + 		}
    + 
    ++		if (!strcmp(arg, "--missing-only"))
    ++			continue;
    ++
    + 		usage(rev_list_usage);
    + 
    + 	}
    +@@ builtin/rev-list.c: int cmd_rev_list(int argc,
    + 	    (revs.left_right || revs.cherry_mark))
    + 		die(_("marked counting and '%s' cannot be used together"), "--objects");
    + 
    ++	die_for_incompatible_opt2(arg_missing_only, "--missing-only",
    ++				  revs.count, "--count");
    ++	die_for_incompatible_opt2(arg_missing_only, "--missing-only",
    ++				  show_disk_usage, "--disk-usage");
     +
    - 	if (!strcmp(value, "allow-promisor")) {
    - 		arg_missing_action = MA_ALLOW_PROMISOR;
    - 		fetch_if_missing = 0;
    + 	save_commit_buffer = (revs.verbose_header ||
    + 			      revs.grep_filter.pattern_list ||
    + 			      revs.grep_filter.header_list);
     @@ builtin/rev-list.c: int cmd_rev_list(int argc,
      
      	if (arg_print_omitted)
      		oidset_init(&omitted_objects, DEFAULT_OIDSET_SIZE);
     -	if (arg_missing_action == MA_PRINT ||
     -	    arg_missing_action == MA_PRINT_INFO) {
    -+	if (missing_action_prints()) {
    ++	if (should_collect_missing()) {
      		struct oidset_iter iter;
      		struct object_id *oid;
      
    @@ builtin/rev-list.c: int cmd_rev_list(int argc,
      	}
     -	if (arg_missing_action == MA_PRINT ||
     -	    arg_missing_action == MA_PRINT_INFO) {
    -+	if (missing_action_prints()) {
    ++	if (should_collect_missing()) {
      		struct missing_objects_map_entry *entry;
      		struct oidmap_iter iter;
      
    -@@ builtin/rev-list.c: int cmd_rev_list(int argc,
    - 
    - 	stop_progress(&progress);
    - 
    --	if (revs.count) {
    -+	if (revs.count && arg_missing_action != MA_PRINT_ONLY) {
    - 		if (revs.left_right && revs.cherry_mark)
    - 			printf("%d\t%d\t%d\n", revs.count_left, revs.count_right, revs.count_same);
    - 		else if (revs.left_right)
    -@@ builtin/rev-list.c: int cmd_rev_list(int argc,
    - 			printf("%d\n", revs.count_left + revs.count_right);
    - 	}
    - 
    --	if (show_disk_usage)
    -+	if (show_disk_usage && arg_missing_action != MA_PRINT_ONLY)
    - 		print_disk_usage(total_disk_usage);
    - 
    - cleanup:
     
      ## t/t6022-rev-list-missing.sh ##
     @@ t/t6022-rev-list-missing.sh: do
    @@ t/t6022-rev-list-missing.sh: do
      
     +for obj in "HEAD~1" "HEAD~1^{tree}" "HEAD:1.t"
     +do
    -+	test_expect_success "rev-list --missing=print-only with missing $obj" '
    ++	test_expect_success "rev-list --missing-only with missing $obj" '
     +		oid="$(git rev-parse $obj)" &&
     +		path=".git/objects/$(test_oid_to_path $oid)" &&
     +
    -+		# Capture present OIDs before hiding anything.
    -+		git rev-list --objects --no-object-names HEAD ^$obj >present.raw &&
    -+
     +		mv "$path" "$path.hidden" &&
     +		test_when_finished "mv $path.hidden $path" &&
     +
    -+		git rev-list --missing=print-only --objects --no-object-names \
    -+			HEAD >actual &&
    -+
    -+		# Only the missing OID should appear, without the "?" prefix.
    -+		grep "^$oid$" actual &&
    ++		git rev-list --missing=print --missing-only --objects \
    ++			--no-object-names HEAD >actual &&
     +
    -+		# Present objects must NOT appear in the output.
    -+		while read present_oid
    -+		do
    -+			! grep "^$present_oid$" actual || return 1
    -+		done <present.raw
    ++		echo $oid >expect &&
    ++		test_cmp expect actual
     +	'
     +done
    ++
    ++test_expect_success "--missing-only requires --missing=print or --missing=print-info" '
    ++	test_must_fail git rev-list --missing-only --objects HEAD 2>err &&
    ++	test_grep "requires --missing=print" err
    ++'
    ++
    ++test_expect_success "--missing-only is incompatible with --count" '
    ++	test_must_fail git rev-list --missing=print --missing-only \
    ++		--count --objects HEAD 2>err &&
    ++	test_grep "cannot be used together" err
    ++'
    ++
    ++test_expect_success "--missing-only is incompatible with --disk-usage" '
    ++	test_must_fail git rev-list --missing=print --missing-only \
    ++		--disk-usage --objects HEAD 2>err &&
    ++	test_grep "cannot be used together" err
    ++'
    ++
    ++test_expect_success "--missing-only works with --missing=print-info" '
    ++	oid="$(git rev-parse HEAD:1.t)" &&
    ++	path=".git/objects/$(test_oid_to_path $oid)" &&
    ++
    ++	mv "$path" "$path.hidden" &&
    ++	test_when_finished "mv $path.hidden $path" &&
    ++
    ++	git rev-list --missing=print-info --missing-only --objects \
    ++		--no-object-names HEAD >actual &&
    ++
    ++	# Filter keeps print-info fields; only the "?" prefix is dropped.
    ++	echo "$oid path=1.t type=blob" >expect &&
    ++	test_cmp expect actual
    ++'
     +
      test_expect_success "-z nul-delimited --missing" '
      	test_when_finished rm -rf repo &&

base-commit: 1630431f326e15fcde608827b5ff38422528eb59

#12 lore raw

PATCH v2 1/1 rev-list: add --missing-only option to filter output

When working with partial clones, callers often need only the missing object IDs. Today that means post-processing --missing=print to drop present objects and strip the leading '?':

  git rev-list --objects --all --missing=print | perl -ne 'print if s/^[?]//'

Add --missing-only. Use it with --missing=print or --missing=print-info to print only missing objects. --missing= still picks the format; --missing-only only filters. The leading '?' is omitted. With print-info, path= and type= are still shown.

Require --missing=print or --missing=print-info. Reject --count and --disk-usage.

---
Changes from v1 include:
1. Replace --missing=print-only mode with a separate --missing-only flag (Phillip, Patrick, Stolee).
2. Require --missing=print or --missing=print-info.
3. Die when combined with --count or --disk-usage (Stolee).
4. Keep enum comment alignment (no spacing churn).
5. Keep print-info path=/type=; only drop '?'.
6. Simpler tests with test_cmp (Phillip).

 Documentation/rev-list-options.adoc | 13 ++++++++
 builtin/rev-list.c                  | 42 ++++++++++++++++++++++---
 t/t6022-rev-list-missing.sh         | 49 +++++++++++++++++++++++++++++
 3 files changed, 99 insertions(+), 5 deletions(-)
Documentation/rev-list-options.adoc +13 -0
index fd831f0ec6..bd9f345690 100644
@@ -1083,6 +1083,19 @@ If some tips passed to the traversal are missing, they will be
 considered as missing too, and the traversal will ignore them. In case
 we cannot get their Object ID though, an error will be raised.
 
+`--missing-only`::
+	When used together with `--missing=print` or `--missing=print-info`,
+	suppress all output for present objects and print only the missing
+	ones.  The selected `--missing=` format is preserved (so
+	`--missing=print-info` still emits `path=` / `type=` fields), but the
+	leading ``?'' prefix used by the non-`-z` forms is omitted.  This is
+	useful for scripting, as a simpler and faster alternative to
+	post-processing the output of `--missing=print`.
++
+This option is incompatible with `--count` and `--disk-usage`.
+It is an error to use `--missing-only` without `--missing=print` or
+`--missing=print-info`.
+
 `--exclude-promisor-objects`::
 	(For internal use only.)  Prefilter object traversal at
 	promisor boundary.  This is used with partial clone.  This is
builtin/rev-list.c +37 -5

102 lines not on this page · raw message

t/t6022-rev-list-missing.sh +49 -0

55 lines not on this page · raw message

base-commit: 1630431f326e15fcde608827b5ff38422528eb59

Signature · 1 line
2.54.0
#13 lore raw

Re: [PATCH v2 0/1] rev-list: add --missing-only option to filter output

In reply to Siddharth Asthana
Quoted 4 lines · Siddharth Asthana

At GitLab, Gitaly uses rev-list --missing=print on partial clones to find missing objects. The output mixes present and missing objects and prefixes missing ones with '?', so we post-process it. --missing-only avoids that.

Not an objection to the feature (as it can already be done with post-processing), but stepping back a bit, how is this list of missing objects used?

You know what objects you currently have. They refer to other objects, some of which you may lack. You can get a list of them, and then what? After fetching them from 'origin', you will have a new set of missing objects (e.g., after fetching a missing commit, its parents become known and missing; after fetching a missing tree, the blobs and trees contained in it become known and missing).

Fetching missing objects a batch at a time sounds like too much back-and-forth to peel the onion, if the process using '--missing-only' is trying to find out what it wants to fetch, and would be a rather inefficient way to backfill a shallow clone. So I wonder how this fits into the larger picture.

Thanks.

#14 lore raw

Re: [PATCH v2 0/1] rev-list: add --missing-only option to filter output

Quoted 10 lines · Junio C Hamano
Quoted 4 lines · Siddharth Asthana

At GitLab, Gitaly uses rev-list --missing=print on partial clones to find missing objects. The output mixes present and missing objects and prefixes missing ones with '?', so we post-process it. --missing-only avoids that.

Not an objection to the feature (as it can already be done with post-processing), but stepping back a bit, how is this list of missing objects used?

Thanks for asking

Quoted 9 lines

You know what objects you currently have. They refer to other objects, some of which you may lack. You can get a list of them, and then what? After fetching them from 'origin', you will have a new set of missing objects (e.g., after fetching a missing commit, its parents become known and missing; after fetching a missing tree, the blobs and trees contained in it become known and missing).

Quoted 5 lines

Fetching missing objects a batch at a time sounds like too much back-and-forth to peel the onion, if the process using '--missing-only' is trying to find out what it wants to fetch, and would be a rather inefficient way to backfill a shallow clone. So I wonder how this fits into the larger picture.

Agreed, we are not using it that way, and I would not want people to use it as a multi-round fetch loop either.

In Gitaly the main caller is transaction packing. When we commit a transaction we walk only the quarantine (plus the new ref tips) with something like:

	git rev-list --objects --missing=print --stdin

- Objects that exist in the quarantine get packed and logged with the transaction.

- Objects that show up as missing (the '?' lines) are recorded as the transaction's dependencies. At apply time we check those OIDs already exit in the main repo and abort if they do not. We do not fetch them; they are expected to already be outside the quarantine.

So the missing list is a one-shot dependency set from a single walk, not iterative discovery / onion peeling. Today we parse the mixed output and strip the leading '?'. --missing-only is only so that path (and the old "count missing" demo) can get bare OIDs without post-processing.

Happy to clarify this in the commit message / docs in a v3 if useful.

Thanks,
Siddharth

Quoted 2 lines

Thanks.

#15 lore raw

Re: [PATCH v2 0/1] rev-list: add --missing-only option to filter output

Quoted 3 lines · Siddharth Asthana
Quoted 1 line

... So I wonder how this fits into the larger picture.

Happy to clarify this in the commit message / docs in a v3 if useful.

Yeah, strengthing the description of the motivation in the proposed log message is always a good way to sell your ware ;-).

#16 lore raw

v3 cover rev-list: add --missing-only option to filter output

In reply to Siddharth Asthana

Hi,

v2 added --missing-only as a filter next to --missing=print / --missing=print-info. Junio asked how the missing list is used, so this v3 only updates the log message.

It is a one-shot walk. Gitaly uses --missing=print when packing a quarantine and treats '?' lines as objects that must already be in the main repo. Tests do the same (is this blob still missing). --missing-only is that list without the prefix, not a fetch loop.

Based on 1630431f32 (The 21st batch, 2026-08-31).

Changes from v2:
- Stronger motivation in the commit message (Junio).
- Link to v2:
https://lore.kernel.org/git/[email protected]/

Thanks,
Siddharth

---
Siddharth Asthana (1):
rev-list: add --missing-only option to filter output

 Documentation/rev-list-options.adoc | 13 ++++++++
 builtin/rev-list.c                  | 42 ++++++++++++++++++++++---
 t/t6022-rev-list-missing.sh         | 49 +++++++++++++++++++++++++++++
 3 files changed, 99 insertions(+), 5 deletions(-)

Range-diff versus v2:

1: 3c89e6bc38 ! 1: c751ff41a4 rev-list: add --missing-only option to filter output
@@ Commit message

git rev-list --objects --all --missing=print | perl -ne 'print if s/^[?]//'

+ This is for a one-shot walk, not a fetch loop. Callers already have
+ --missing=print and strip the leading '?'. Gitaly does that when packing
+ a quarantine: '?' lines are objects that must already exist in the main
+ repo. Tests do the same (is this blob still missing). --missing-only is
+ just that list without the prefix.
+
Add --missing-only. Use it with --missing=print or --missing=print-info to print only missing objects. --missing= still picks the format; --missing-only only filters. The leading '?' is omitted. With

base-commit: 1630431f326e15fcde608827b5ff38422528eb59

#17 lore raw

PATCH v3 1/1 rev-list: add --missing-only option to filter output

When working with partial clones, callers often need only the missing object IDs. Today that means post-processing --missing=print to drop present objects and strip the leading '?':

  git rev-list --objects --all --missing=print | perl -ne 'print if s/^[?]//'

This is for a one-shot walk, not a fetch loop. Callers already have --missing=print and strip the leading '?'. Gitaly does that when packing a quarantine: '?' lines are objects that must already exist in the main repo. Tests do the same (is this blob still missing). --missing-only is just that list without the prefix.

Add --missing-only. Use it with --missing=print or --missing=print-info to print only missing objects. --missing= still picks the format; --missing-only only filters. The leading '?' is omitted. With print-info, path= and type= are still shown.

Require --missing=print or --missing=print-info. Reject --count and --disk-usage.

---
Changes from v2:
- Strengthen the motivation in the log message (Junio). This is a one-shot walk (Gitaly quarantine deps, tests), not a multi-round fetch loop.

 Documentation/rev-list-options.adoc | 13 ++++++++
 builtin/rev-list.c                  | 42 ++++++++++++++++++++++---
 t/t6022-rev-list-missing.sh         | 49 +++++++++++++++++++++++++++++
 3 files changed, 99 insertions(+), 5 deletions(-)
Documentation/rev-list-options.adoc +13 -0
index fd831f0ec6..bd9f345690 100644
@@ -1083,6 +1083,19 @@ If some tips passed to the traversal are missing, they will be
 considered as missing too, and the traversal will ignore them. In case
 we cannot get their Object ID though, an error will be raised.
 
+`--missing-only`::
+	When used together with `--missing=print` or `--missing=print-info`,
+	suppress all output for present objects and print only the missing
+	ones.  The selected `--missing=` format is preserved (so
+	`--missing=print-info` still emits `path=` / `type=` fields), but the
+	leading ``?'' prefix used by the non-`-z` forms is omitted.  This is
+	useful for scripting, as a simpler and faster alternative to
+	post-processing the output of `--missing=print`.
++
+This option is incompatible with `--count` and `--disk-usage`.
+It is an error to use `--missing-only` without `--missing=print` or
+`--missing=print-info`.
+
 `--exclude-promisor-objects`::
 	(For internal use only.)  Prefilter object traversal at
 	promisor boundary.  This is used with partial clone.  This is
builtin/rev-list.c +37 -5

102 lines not on this page · raw message

t/t6022-rev-list-missing.sh +49 -0

55 lines not on this page · raw message

Range-diff:
1: 3c89e6bc38 ! 1: c751ff41a4 rev-list: add --missing-only option to filter output
@@ Commit message

git rev-list --objects --all --missing=print | perl -ne 'print if s/^[?]//'

+ This is for a one-shot walk, not a fetch loop. Callers already have
+ --missing=print and strip the leading '?'. Gitaly does that when packing
+ a quarantine: '?' lines are objects that must already exist in the main
+ repo. Tests do the same (is this blob still missing). --missing-only is
+ just that list without the prefix.
+
Add --missing-only. Use it with --missing=print or --missing=print-info to print only missing objects. --missing= still picks the format; --missing-only only filters. The leading '?' is omitted. With

base-commit: 1630431f326e15fcde608827b5ff38422528eb59

Signature · 1 line
2.54.0