Skip to contentSkip to conversation

bugreport: add tool to generate debugging info

PATCH Superseded Emily Shaffer 13 participants 373 messages Aug 15, 2019 – Apr 7, 2020 Page 2 of 3

Subsystems (2)

v1 of 13 Superseded Latest is v13 →

lore
Share & tools
lore
#152 lore raw

Re: [PATCH v5 10/15] bugreport: add config values from safelist

In reply to Martin Ågren
Quoted 7 lines · Martin Ågren
Quoted 6 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:
Makefile +0 -0
@@ -2465,7 +2465,7 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
        $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
Quoted 10 lines

Makefile endif · line 2470

Quoted 4 lines
Makefile +1 -1
-git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+git-bugreport$X: bugreport-config-safelist.h bugreport.o GIT-LDFLAGS $(GITLIBS)
        $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
                $(LIBS)

Haven't looked at this patch at all, except I've noticed that something is up with the dependencies. I think bugreport.o needs to depend on bugreport-config-safelist.h. As it is, the latter may or may not be available as bugreport.o is built. (Reproduces fairly well for me with something like make clean && make -j16.)

Hum, I thought that the ordering of the dependency list here was ensured. But I see where help.o relies directly on command-list.h, so I'll add a dependency for bugreport.o (and drop it from the direct dependency for git-bugreport).

- Emily

#153 lore raw

Re: [PATCH v5 10/15] bugreport: add config values from safelist

In reply to [email protected]
Quoted 4 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:

Taking the build-time generated array and putting it into a set saves us time - since git_config_bugreport() is called for every option the user has configured, performing option lookup in constant time is a useful optimization.

I'm sympathetic to your sending out what you have to obtain comments, knowing that it's not perfect. It would have saved me some time and effort if I'd known that this was the case though. An "[RFC]" tag, perhaps. Or at least tweaking the above part of this commit message to say that this might be over-engineered, with a reference to [1].

[1] https://lore.kernel.org/git/[email protected]/

Quoted 1 line

+ int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);

I was going to suggest ARRAY_SIZE, but then I realized there are some outstanding questions around whether you need this stuff in the first place. I'd be inclined to guess that the first version of this would be "for each safelisted item, obtain it and include", ignoring any "a.*.b" business. In which case you wouldn't really need this hashset stuff.

Martin

#154 lore raw

Re: [PATCH v5 10/15] bugreport: add config values from safelist

Quoted 6 lines · Martin Ågren
Quoted 4 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:

Taking the build-time generated array and putting it into a set saves us time - since git_config_bugreport() is called for every option the user has configured, performing option lookup in constant time is a useful optimization.

Quoted 7 lines

I'm sympathetic to your sending out what you have to obtain comments, knowing that it's not perfect. It would have saved me some time and effort if I'd known that this was the case though. An "[RFC]" tag, perhaps. Or at least tweaking the above part of this commit message to say that this might be over-engineered, with a reference to [1].

[1] https://lore.kernel.org/git/[email protected]/

Yeah, you are right, and thanks for the feedback. I think I had wrapped up the series before I realized there were those significant outstanding comments, but I definitely should have said so in this patch.

Quoted 8 lines
Quoted 2 lines

+ int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);

I was going to suggest ARRAY_SIZE, but then I realized there are some outstanding questions around whether you need this stuff in the first place. I'd be inclined to guess that the first version of this would be "for each safelisted item, obtain it and include", ignoring any "a.*.b" business. In which case you wouldn't really need this hashset stuff.

Regardless, I think it's worth doing nicely for now.

It seems to me that supporting wildcarding in the config safelist is a superset of supporting static strings in that safelist - that is, if I write it simply to support static strings, a later change to support wildcards would be welcome and non-breaking.

So I'm going to clean this up without making it more featureful, for now.

Sorry about the confusion!
- Emily

#155 lore raw

Re: [PATCH v5 10/15] bugreport: add config values from safelist

Quoted 29 lines · Emily Shaffer
Quoted 6 lines · Martin Ågren
Quoted 4 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:

Taking the build-time generated array and putting it into a set saves us time - since git_config_bugreport() is called for every option the user has configured, performing option lookup in constant time is a useful optimization.

Quoted 7 lines

I'm sympathetic to your sending out what you have to obtain comments, knowing that it's not perfect. It would have saved me some time and effort if I'd known that this was the case though. An "[RFC]" tag, perhaps. Or at least tweaking the above part of this commit message to say that this might be over-engineered, with a reference to [1].

[1] https://lore.kernel.org/git/[email protected]/

Yeah, you are right, and thanks for the feedback. I think I had wrapped up the series before I realized there were those significant outstanding comments, but I definitely should have said so in this patch.

Quoted 8 lines
Quoted 2 lines

+ int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);

I was going to suggest ARRAY_SIZE, but then I realized there are some outstanding questions around whether you need this stuff in the first place. I'd be inclined to guess that the first version of this would be "for each safelisted item, obtain it and include", ignoring any "a.*.b" business. In which case you wouldn't really need this hashset stuff.

Quoted 6 lines

Regardless, I think it's worth doing nicely for now.

It seems to me that supporting wildcarding in the config safelist is a superset of supporting static strings in that safelist - that is, if I write it simply to support static strings, a later change to support wildcards would be welcome and non-breaking.

Yeah, agreed. Also, whatever magic the code for gathering the config items knows, the annotations in Documentation/, i.e., the generated list in bugreport-config-safelist.h will be "synced". That is, until we know how to handle "a.*.b" in the safelist, we probably won't be marking "a.*.b" as safe. And even if we have some crazy mixed build, it's a *safe*list, so we should be fine. (Famous last words?)

On that topic though, and just so we don't mess up from the beginning, in the documentation, we typically write something like "branch.<name>.merge", i.e., not "branch.*.merge". So I guess the "a.*.b" feature would be more like "look for '.<', something, something, '>.'." Would that be sufficient? Or would we actually want more fine-grained control, i.e., something like regexes?

If so, we'd need to provide those regexes somehow, e.g., as part of the asciidoc macro notation. :-/ Do we need to prepare somehow for such a future? I don't think we do.

Quoted 5 lines

So I'm going to clean this up without making it more featureful, for now.

Sorry about the confusion!
- Emily

No worries. :)

Martin

#156 lore raw

PATCH v5 11/15 bugreport: collect list of populated hooks

In reply to [email protected]

From: Emily Shaffer <[email protected]>

Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong.

Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot. To reduce the amount of code humans need to read, we turn the list into a string_list and iterate over it (as we are calling the same find_hook operation on each string). However, since bugreport should primarily be called by the user, the performance loss from massaging the string seems acceptable.

---
 bugreport.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)
bugreport.c +53 -0
index 7a9fd36b60..4c77009f1b 100644
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "bugreport-config-safelist.h"
 #include "khash.h"
+#include "run-command.h"
 
 static void get_curl_version_info(struct strbuf *curl_info)
 {
@@ -121,6 +122,55 @@ static void get_safelisted_config(struct strbuf *config_info)
 	cfgset_clear(&safelist);
 }
 
+static void get_populated_hooks(struct strbuf *hook_info, int nongit)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hook`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+
+
+	if (nongit) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -193,6 +243,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Configured Hooks");
+	get_populated_hooks(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#157 lore raw

Re: [PATCH v5 11/15] bugreport: collect list of populated hooks

bugreport.c static void get_safelisted_config(struct strbuf *config_info) · line 129

Quoted 3 lines · [email protected] writes:
bugreport.c +3 -0
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hook`.

That's "git help hooks", if I tried my reproduction correctly. A straight-forward (in the sense of "what we want in the outcome is quite clear" and not in the sense of "anybody can design and implement it with a single 30-line patch") follow-up we can make after this series lands is to rethink how Documentation/githooks.txt is maintained and the list we have here is synchronized with it.

The design could me just the matter of running "grep" of some sort, with appropriate markups that are no-op to AsciiDoctor/AsciiDoc added to the documentation source, to produce this list.

Quoted 48 lines
bugreport.c +44 -0
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+
+
+	if (nongit) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL

bugreport.c int cmd_main(int argc, const char **argv) · line 246

Quoted 4 lines
bugreport.c +1 -0
@@ -193,6 +243,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Configured Hooks");

Phrase nit. There may be many people who just enabled hooks without configuring, so "Enabled Hooks" may be more appropriate. We do not have to inspect what is in the hook to determine if it is enabled, but we do need to if we want to tell if a hook is "configured".

Quoted 5 lines
bugreport.c +2 -0
+	get_populated_hooks(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
#158 lore raw

Re: [PATCH v5 11/15] bugreport: collect list of populated hooks

Quoted 7 lines · Junio C Hamano

bugreport.c static void get_safelisted_config(struct strbuf *config_info) · line 129

Quoted 3 lines · [email protected] writes:
bugreport.c +3 -0
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hook`.

That's "git help hooks", if I tried my reproduction correctly.

Yep. I'll fix it.

Quoted 9 lines

A straight-forward (in the sense of "what we want in the outcome is quite clear" and not in the sense of "anybody can design and implement it with a single 30-line patch") follow-up we can make after this series lands is to rethink how Documentation/githooks.txt is maintained and the list we have here is synchronized with it.

The design could me just the matter of running "grep" of some sort, with appropriate markups that are no-op to AsciiDoctor/AsciiDoc added to the documentation source, to produce this list.

Well, with Martin's suggestion[1] to use annotate:bugreport[include] instead of bugreport:include[x], this kind of change becomes a matter of course to just use "annotate:hook[x]" and write a script nearly identical to generate-bugreport-config-safelist.sh, and add it to the Makefile.

Like I said up-thread, I don't like the idea just because it's awfully temporary (pending work that's right below this on my list for the coming weeks).

It sounds like you're saying you don't mind any work of this nature being done as a follow-up - and that's fine with me, too. I'd just rather do the follow-up with enums and a library than with grep and sh. :)

Quoted 7 lines
Quoted 7 lines

static const char * const bugreport_usage[] = {

bugreport.c +0 -0
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -193,6 +243,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
Quoted 6 lines

bugreport.c int cmd_main(int argc, const char **argv) · line 246

Quoted 1 line
bugreport.c +1 -0
+	get_header(&buffer, "Configured Hooks");

Phrase nit. There may be many people who just enabled hooks without configuring, so "Enabled Hooks" may be more appropriate. We do not have to inspect what is in the hook to determine if it is enabled, but we do need to if we want to tell if a hook is "configured".

Ah, that's a good point. OK, I'll change the wording as you suggested.

- Emily

#159 lore raw

Re: [PATCH v5 11/15] bugreport: collect list of populated hooks

Quoted 5 lines · Emily Shaffer

Well, with Martin's suggestion[1] to use annotate:bugreport[include] instead of bugreport:include[x], this kind of change becomes a matter of course to just use "annotate:hook[x]" and write a script nearly identical to generate-bugreport-config-safelist.sh, and add it to the Makefile.

Bah.

[1]: https://lore.kernel.org/git/CAN0heSq_i4EitqYH-qrZyXBU+=PUNcSXOOJDHLSnJ1ufV0WtaQ@mail.gmail.com

#160 lore raw

PATCH v5 12/15 bugreport: count loose objects

In reply to [email protected]

From: Emily Shaffer <[email protected]>

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

---
The refactor removed much of the code Dscho suggested; and yet it remains true that he helped me while developing this commit (although his suggestions didn't survive). Shall I leave the Helped-by line or remove it?

- Emily

 bugreport.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)
bugreport.c +46 -0
index 4c77009f1b..bf10857183 100644
@@ -10,6 +10,7 @@
 #include "bugreport-config-safelist.h"
 #include "khash.h"
 #include "run-command.h"
+#include "object-store.h"
 
 static void get_curl_version_info(struct strbuf *curl_info)
 {
@@ -171,6 +172,48 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 }
 
+static int loose_object_cb(const struct object_id *oid, const char *path,
+			   void *data) {
+	int *loose_object_count = data;
+
+	if (loose_object_count) {
+		(*loose_object_count)++;
+		return 0;
+	}
+
+	return 1;
+}
+
+static void get_loose_object_summary(struct strbuf *obj_info) {
+
+	int local_loose_object_count = 0, total_loose_object_count = 0;
+	int local_count_questionable = 0, total_count_questionable = 0;
+
+	local_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&local_loose_object_count,
+					FOR_EACH_OBJECT_LOCAL_ONLY);
+
+	total_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&total_loose_object_count,
+					0);
+
+	strbuf_addf(obj_info, "%d local loose objects%s\n",
+		    local_loose_object_count,
+		    local_count_questionable ? " (problem during count)" : "");
+
+	strbuf_addf(obj_info, "%d alternate loose objects%s\n",
+		    total_loose_object_count - local_loose_object_count,
+		    (local_count_questionable || total_count_questionable)
+			? " (problem during count)"
+			: "");
+
+	strbuf_addf(obj_info, "%d total loose objects%s\n",
+		    total_loose_object_count,
+		    total_count_questionable ? " (problem during count)" : "");
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -246,6 +289,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Configured Hooks");
 	get_populated_hooks(&buffer, nongit_ok);
 
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#161 lore raw

Re: [PATCH v5 12/15] bugreport: count loose objects

Quoted 6 lines · [email protected] writes:

From: Emily Shaffer <[email protected]>

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

Quoted 7 lines

---
The refactor removed much of the code Dscho suggested; and yet it remains true that he helped me while developing this commit (although his suggestions didn't survive). Shall I leave the Helped-by line or remove it?

You two collectively thought about viable alternatives and decided to reject what was not wanted in the final result, and not having that rejected code was good for the project, right? If so, I would say it still is the help that deserves recognition. After all, making the result better by removing things is harder than by adding things ;-)

Quoted 3 lines
 bugreport.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

...

The patch text looked sensible.

#162 lore raw

Re: [PATCH v5 12/15] bugreport: count loose objects

Quoted 16 lines · Junio C Hamano
Quoted 6 lines · [email protected] writes:

From: Emily Shaffer <[email protected]>

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

Quoted 7 lines

---
The refactor removed much of the code Dscho suggested; and yet it remains true that he helped me while developing this commit (although his suggestions didn't survive). Shall I leave the Helped-by line or remove it?

Quoted 6 lines

You two collectively thought about viable alternatives and decided to reject what was not wanted in the final result, and not having that rejected code was good for the project, right? If so, I would say it still is the help that deserves recognition. After all, making the result better by removing things is harder than by adding things ;-)

Sounds great to me. I'd rather over-assign credit than under-assign (but I don't think this is a case of over-assigning here).

Quoted 5 lines
Quoted 3 lines
 bugreport.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

...

The patch text looked sensible.

Thanks.

- Emily

#163 lore raw

PATCH v5 13/15 bugreport: add packed object summary

In reply to [email protected]

From: Emily Shaffer <[email protected]>

Alongside the loose object counts, it can be useful to show the number of packs and packed objects. This way we can check whether the repo has an appropriate ratio of packed to loose objects to help determine whether it's behaving correctly.

---
 bugreport.c | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)
bugreport.c +38 -0
index bf10857183..45cc1764e0 100644
@@ -214,6 +214,41 @@ static void get_loose_object_summary(struct strbuf *obj_info) {
 		    total_count_questionable ? " (problem during count)" : "");
 }
 
+struct packed_object_cb_data {
+	struct packed_git *last_pack;
+	int pack_count;
+	int object_count;
+};
+
+static int packed_object_cb(const struct object_id *oid,
+			    struct packed_git *pack,
+			    uint32_t pos,
+			    void *data) {
+	struct packed_object_cb_data *cb_data = data;
+
+	if (!cb_data)
+		return 1;
+
+	if (pack && pack != cb_data->last_pack) {
+		cb_data->last_pack = pack;
+		cb_data->pack_count++;
+	}
+
+	cb_data->object_count++;
+
+	return 0;
+}
+
+static void get_packed_object_summary(struct strbuf *obj_info)
+{
+	struct packed_object_cb_data cb_data = {NULL, 0, 0};
+
+	for_each_packed_object(packed_object_cb, &cb_data, 0);
+
+	strbuf_addf(obj_info, "%d total packs (%d objects)\n",
+		    cb_data.pack_count, cb_data.object_count);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -292,6 +327,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#164 lore raw

Re: [PATCH v5 13/15] bugreport: add packed object summary

Quoted 6 lines · [email protected] writes:

From: Emily Shaffer <[email protected]>

Alongside the loose object counts, it can be useful to show the number of packs and packed objects. This way we can check whether the repo has an appropriate ratio of packed to loose objects to help determine whether it's behaving correctly.

This lets the enumeration machinery to enumerate at the individual object level, relies on the enumeration machinery to show your callback all objects from the same pack before an object from a different pack, to count both objects and packs.

But given that packfiles record how many objects there are in the pack, and that the packfile.c layer must know what are the pack files we have available, I wonder if we have an API already to allow us to enumerate each packfile, and while counting the number of times our callback was called, add the objects contained in each of them. If there isn't, it does not sound like a bad API function to add (i.e. given a repository r, iterate over r->objects->packed_git and let the API caller see each of them; the API caller can grab num_objects and add them up).

#165 lore raw

Re: [PATCH v5 13/15] bugreport: add packed object summary

Quoted 14 lines · Junio C Hamano
Quoted 6 lines · [email protected] writes:

From: Emily Shaffer <[email protected]>

Alongside the loose object counts, it can be useful to show the number of packs and packed objects. This way we can check whether the repo has an appropriate ratio of packed to loose objects to help determine whether it's behaving correctly.

This lets the enumeration machinery to enumerate at the individual object level, relies on the enumeration machinery to show your callback all objects from the same pack before an object from a different pack, to count both objects and packs.

Quoted 9 lines

But given that packfiles record how many objects there are in the pack, and that the packfile.c layer must know what are the pack files we have available, I wonder if we have an API already to allow us to enumerate each packfile, and while counting the number of times our callback was called, add the objects contained in each of them. If there isn't, it does not sound like a bad API function to add (i.e. given a repository r, iterate over r->objects->packed_git and let the API caller see each of them; the API caller can grab num_objects and add them up).

Yeah, I don't see one. I'll go ahead and add it - that sounds straightforward enough.

- Emily

#166 lore raw

Re: [PATCH v5 13/15] bugreport: add packed object summary

In reply to [email protected]
Quoted 6 lines · [email protected] writes:

From: Emily Shaffer <[email protected]>

Alongside the loose object counts, it can be useful to show the number of packs and packed objects. This way we can check whether the repo has an appropriate ratio of packed to loose objects to help determine whether it's behaving correctly.

This step makes me wonder if we want to see the midx as well. Didn't we have a bug that manifests only when midx is (or is not) in use?

Similarly, I think use (or non-use) of the commit graph may also be a useful information for diagnosing.

#167 lore raw

Re: [PATCH v5 13/15] bugreport: add packed object summary

Quoted 9 lines · Junio C Hamano
Quoted 6 lines · [email protected] writes:

From: Emily Shaffer <[email protected]>

Alongside the loose object counts, it can be useful to show the number of packs and packed objects. This way we can check whether the repo has an appropriate ratio of packed to loose objects to help determine whether it's behaving correctly.

Quoted 7 lines

This step makes me wonder if we want to see the midx as well. Didn't we have a bug that manifests only when midx is (or is not) in use?

Similarly, I think use (or non-use) of the commit graph may also be a useful information for diagnosing.

Sure. I'll look into both of these. Would you rather see them as add-ons later, or do you want me to hold off on a v6 of this set until they're ready?

- Emily

#168 lore raw

PATCH v5 14/15 bugreport: list contents of $OBJDIR/info

In reply to [email protected]

From: Emily Shaffer <[email protected]>

Miscellaneous information used about the object store can end up in .git/objects/info; this can help us understand what may be going on with the object store when the user is reporting a bug. Otherwise, it could be difficult to track down what is going wrong with an object which isn't kept locally to .git/objects/ or .git/objects/pack. Having some understanding of where the user's objects may be kept can save us some hops during the bug reporting process.

---
 bugreport.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
bugreport.c +48 -0
index 45cc1764e0..12463ffdc7 100644
@@ -249,6 +249,51 @@ static void get_packed_object_summary(struct strbuf *obj_info)
 		    cb_data.pack_count, cb_data.object_count);
 }
 
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
+{
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
+
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(obj_info, &dirpath);
+
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -330,6 +375,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Packed Object Summary");
 	get_packed_object_summary(&buffer);
 
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#169 lore raw

PATCH v5 15/15 bugreport: summarize contents of alternates file

In reply to [email protected]

From: Emily Shaffer <[email protected]>

In some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead. Since paths to alternates may be sensitive, we'll instead count how many alternates have been specified and note how many of them exist or are broken.

While object-cache.h describes a function "foreach_alt_odb()", this function does not provide information on broken alternates, which are skipped over in "link_alt_odb_entry()". Since the goal is to identify missing alternates, we can gather the contents of .git/objects/info/alternates manually.

---
 bugreport.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
bugreport.c +39 -0
index 12463ffdc7..2c540182dd 100644
@@ -294,6 +294,42 @@ static void get_object_info_summary(struct strbuf *obj_info)
 	strbuf_release(&dirpath);
 }
 
+static void get_alternates_summary(struct strbuf *alternates_info)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -378,6 +414,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#170 lore raw

Re: [PATCH v5 00/15] add git-bugreport tool

In reply to [email protected]

A sample bugreport generated from this patchset follows:

Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)

What did you expect to happen? (Expected behavior)

What happened instead? (Actual behavior)

What's different between what you expected and what actually happened?

Anything else you want to add:

Please review the rest of the bug report below.
You can delete any lines you don't wish to send.

[System Info]
git version:
git version 2.25.0.18.g682ab0d3eb
cpu: x86_64
built from commit: 682ab0d3eb8b84f8af4db1a161d24ca53d2f39fc
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
uname -a: Linux 5.2.17-1rodete3-amd64 #1 SMP Debian 5.2.17-1rodete3 (2019-10-21 > 2018) x86_64
compiler info: glibc: 2.28
$SHELL (typically, interactive shell): /bin/bash
git-remote-https --build-info:
git-http-fetch version: 2.25.0.18.g682ab0d3eb
built from commit: 682ab0d3eb8b84f8af4db1a161d24ca53d2f39fc
curl version: libcurl/7.66.0 GnuTLS/3.6.9 zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.20.2 (+libidn2/2.2.0) libssh2/1.8.0 nghttp2/1.39.2 librtmp/2.3

[Safelisted Config Info]
sendemail.from (global) : [email protected]
sendemail.from (repo) : [email protected]

[Configured Hooks]
pre-commit
prepare-commit-msg

[Loose Object Counts]
2549 local loose objects
2641 alternate loose objects
5190 total loose objects

[Packed Object Summary]
60 total packs (591308 objects)

[Object Info Summary]
.git/objects/info/foo
.git/objects/info/bar
.git/objects/info/bar/baz
.git/objects/info/alternates

[Alternates]
2 alternates found (1 working, 1 broken)

#171 lore raw

Re: [PATCH v5 00/15] add git-bugreport tool

In reply to [email protected]
Quoted 11 lines

From: Emily Shaffer <[email protected]>

This topic branch depends on the patch mailed in lore.kernel.org/git/[email protected] in order to display scopes for configs gathered during "bugreport: add config values from safelist".

I'll summarize v4-v5. Since v4 has languished for some time, I don't think an interdiff is too helpful, so I won't include one. Bonus, the code is much simplified from some suggestions from Junio on how to inspect objects, so I hope it's easy to review anyways.

To everyone in the developer community interested in this set: what is the status of this?

If this needs further review, then maybe it would be best if only patches 1-4 were put up for submission first, with a note in the bugreport documentation that more information may be added in future Git versions. For me, patches 1-4 look good and I don't have enough experience with uname (especially across libcs and OSes) to determine what should or should not be included - if this is typical of reviewers in the Git project, it might be better to submit patches 1-4 first, and then send each additional diagnostic separately, so that people who know what's going on in one area but not another can just comment on the area they know about.

Having said that, I see that a few people have already looked at the entire patchset and made comments, so if they are OK with it, we don't need to split it up.

#172 lore raw

Re: [PATCH v5 00/15] add git-bugreport tool

Quoted 26 lines · Jonathan Tan
Quoted 11 lines

From: Emily Shaffer <[email protected]>

This topic branch depends on the patch mailed in lore.kernel.org/git/[email protected] in order to display scopes for configs gathered during "bugreport: add config values from safelist".

I'll summarize v4-v5. Since v4 has languished for some time, I don't think an interdiff is too helpful, so I won't include one. Bonus, the code is much simplified from some suggestions from Junio on how to inspect objects, so I hope it's easy to review anyways.

To everyone in the developer community interested in this set: what is the status of this?

If this needs further review, then maybe it would be best if only patches 1-4 were put up for submission first, with a note in the bugreport documentation that more information may be added in future Git versions. For me, patches 1-4 look good and I don't have enough experience with uname (especially across libcs and OSes) to determine what should or should not be included - if this is typical of reviewers in the Git project, it might be better to submit patches 1-4 first, and then send each additional diagnostic separately, so that people who know what's going on in one area but not another can just comment on the area they know about.

Quoted 3 lines

Having said that, I see that a few people have already looked at the entire patchset and made comments, so if they are OK with it, we don't need to split it up.

I'd be fine with either; from now, my gut says the only ones I'm not comfortable merging today are maybe 10, as Junio had some concerns about whether to allow glob expansion, and 11, as it's prone to rot and I'm doing other work in that area which would cause 11 to be dropped/refactored anyways once it goes in. Otherwise I think they're all pretty OK to go in and let the other work continue later in a different thread.

Of course, as an author my opinion that they're good to go doesn't mean much ;)

- Emily

#173 lore raw

Re: [PATCH v5 00/15] add git-bugreport tool

In reply to [email protected]
Quoted 4 lines · On Fri, 24 Jan 2020 at 04:35, <[email protected]> wrote:

This topic branch depends on the patch mailed in lore.kernel.org/git/[email protected] in order to display scopes for configs gathered during "bugreport: add config values from safelist".

Should this use config_scope_name() which looks like it's about to graduate [1]? Disclaimer: I haven't followed that patch set too closely.

[1] https://lore.kernel.org/git/[email protected]/

Martin

#174 lore raw

Re: [PATCH v5 00/15] add git-bugreport tool

Quoted 8 lines · Martin Ågren
Quoted 4 lines · On Fri, 24 Jan 2020 at 04:35, <[email protected]> wrote:

This topic branch depends on the patch mailed in lore.kernel.org/git/[email protected] in order to display scopes for configs gathered during "bugreport: add config values from safelist".

Should this use config_scope_name() which looks like it's about to graduate [1]? Disclaimer: I haven't followed that patch set too closely.

Yeah, you're right that I can use it. config.h:config_scope_name() is a new feature on that patch since last time I looked. Thanks for the tip!

- Emily

Quoted 1 line

[1] https://lore.kernel.org/git/[email protected]/

#175 lore raw

v6 cover add git-bugreport tool

In reply to [email protected]

This patchset is based on mr/show-config-scope.

Since v5, I addressed comments from Martin and Junio.

- Reorganized the series so the ecosystem cleanup comes before the interesting new features.
- Spread out manpage updates to indicate each new feature as it's added.
- Fixed a number of patches which assumed a repo, so bugreport can now run anywhere.
- Wordsmithing in the report template.
- die() instead of overwriting an existing report.
- Added --suffix to allow user to change the name of the report to git-bugreport-<strftime format>.txt.
- General NULL-ptr, leak, and bashism cleanup.
- Changed safelist macro from bugreport:include[x] to annotate:bugreport[include].
- Use git_config_get_value_multi() instead of a config callback to gather the safelist configs.
- Added a convenience for_each_pack() macro to object-store.h and began using it in packfile.c and bugreport.c for pack summary

Emily Shaffer (15):
help: move list_config_help to builtin/help
help: add shell-path to --build-options
bugreport: add tool to generate debugging info
bugreport: gather git version and build info
bugreport: add uname info
bugreport: add compiler info
bugreport: add git-remote-https version
bugreport: include user interactive shell
bugreport: generate config safelist based on docs
bugreport: add config values from safelist
bugreport: collect list of populated hooks
bugreport: count loose objects
bugreport: add packed object summary
bugreport: list contents of $OBJDIR/info
bugreport: summarize contents of alternates file

 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   9 +
 Documentation/asciidoctor-extensions.rb |   5 +
 Documentation/config/sendemail.txt      |  56 ++--
 Documentation/git-bugreport.txt         |  55 ++++
 Makefile                                |  27 +-
 bugreport.c                             | 409 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 compat/compiler.h                       |  24 ++
 generate-bugreport-config-safelist.sh   |  17 +
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  24 ++
 help.c                                  | 132 ++------
 help.h                                  |   2 +-
 object-store.h                          |   6 +
 packfile.c                              |   3 +-
 remote-curl.c                           |   8 +
 t/t0091-bugreport.sh                    |  49 +++
 18 files changed, 778 insertions(+), 156 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100644 compat/compiler.h
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#176 lore raw

PATCH v6 1/15 help: move list_config_help to builtin/help

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o, builtin/clean.o, and builtin/config.o. This meant that help.o was unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present).

When command-list.h is included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule.

---
 .gitignore             |  1 +
 Makefile               | 13 +++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 24 ++++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 121 insertions(+), 108 deletions(-)
 create mode 100755 generate-configlist.sh
.gitignore +1 -0
index aebe7c0908..ea97de83f3 100644
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
Makefile +10 -3
index 09f98b777c..5a022367d4 100644
@@ -814,6 +814,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2128,7 +2129,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2148,6 +2149,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2781,7 +2788,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2803,7 +2810,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
builtin/help.c +86 -0

98 lines not on this page · raw message

generate-cmdlist.sh +0 -19
index 71158f7d8b..45fecf8bdf 100755
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
generate-configlist.sh new file +24 -0
new file mode 100755
index 0000000000..eca6a00c30
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
help.c +0 -85

91 lines not on this page · raw message

help.h +0 -1
index 7a455beeb7..9071894e8c 100644
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#177 lore raw

Re: [PATCH v6 01/15] help: move list_config_help to builtin/help

Quoted 46 lines · Emily Shaffer
generate-cmdlist.sh +0 -19
index 71158f7d8b..45fecf8bdf 100755
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
generate-configlist.sh new file +6 -0
new file mode 100755
index 0000000000..eca6a00c30
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {

generate-configlist.sh line 16

Quoted 10 lines
generate-configlist.sh new file +10 -0
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done

This while-read-echo was moved from generate-cmdlist.sh, which has some logic to work with read-variable.

We're moving it out, I think apply this diff on top of it will make the code easier to read.

generate-configlist.sh +1 -4
index eca6a00c30..163dbf30bb 100755
@@ -10,10 +10,7 @@ EOF
 	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
 	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
 	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
+	sed 's/^/	"/; s/$/",/'
 	cat <<EOF
 	NULL,
 };
Signature · 1 line
Danh
#178 lore raw

Re: [PATCH v6 01/15] help: move list_config_help to builtin/help

Quoted 69 lines · Danh Doan
Quoted 46 lines · Emily Shaffer
generate-cmdlist.sh +0 -19
index 71158f7d8b..45fecf8bdf 100755
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
generate-configlist.sh new file +6 -0
new file mode 100755
index 0000000000..eca6a00c30
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {

generate-configlist.sh line 16

Quoted 10 lines
generate-configlist.sh new file +10 -0
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done

This while-read-echo was moved from generate-cmdlist.sh, which has some logic to work with read-variable.

We're moving it out, I think apply this diff on top of it will make the code easier to read.

generate-configlist.sh +0 -0
index eca6a00c30..163dbf30bb 100755
@@ -10,10 +10,7 @@ EOF
 	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |

generate-configlist.sh EOF · line 16

Quoted 10 lines
generate-configlist.sh +1 -4
 	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
 	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
+	sed 's/^/	"/; s/$/",/'
 	cat <<EOF
 	NULL,
 };

Thanks for the suggestion. You're right that I didn't look into the contents of this script much, because I was just moving it; I modified your sed expression slightly so it performs just one operation:

sed 's/^.*$/ "\0",/'

- Emily

#179 lore raw

Re: [PATCH v6 01/15] help: move list_config_help to builtin/help

Quoted 7 lines · Emily Shaffer

generate-configlist.sh EOF · line 13

Quoted 5 lines · Danh Doan
generate-configlist.sh +1 -4
-     while read line
-     do
-             echo "  \"$line\","
-     done
+     sed 's/^/       "/; s/$/",/'
Quoted 5 lines

Thanks for the suggestion. You're right that I didn't look into the contents of this script much, because I was just moving it; I modified your sed expression slightly so it performs just one operation:

sed 's/^.*$/ "\0",/'

This isn't portable and won't work with 'sed' from BSD lineage, including Mac OS. More portable would be:

    sed 's/^\(.*\)$/    "\1",/'
#180 lore raw

Re: [PATCH v6 01/15] help: move list_config_help to builtin/help

Quoted 12 lines · Eric Sunshine
Quoted 11 lines · Emily Shaffer

generate-configlist.sh EOF · line 13

Quoted 5 lines · Danh Doan
generate-configlist.sh +1 -4
-     while read line
-     do
-             echo "  \"$line\","
-     done
+     sed 's/^/       "/; s/$/",/'

Thanks for the suggestion. You're right that I didn't look into the contents of this script much, because I was just moving it; I modified your sed expression slightly so it performs just one operation:

Quoted 6 lines
Quoted 1 line

sed 's/^.*$/ "\0",/'

This isn't portable and won't work with 'sed' from BSD lineage, including Mac OS. More portable would be:

    sed 's/^\(.*\)$/    "\1",/'

Or just replace Emily's \0 with &; that should be in POSIX.

#181 lore raw

Re: [PATCH v6 01/15] help: move list_config_help to builtin/help

Quoted 6 lines · Junio C Hamano
Quoted 5 lines · Eric Sunshine
Quoted 4 lines · Emily Shaffer

Thanks for the suggestion. You're right that I didn't look into the contents of this script much, because I was just moving it; I modified your sed expression slightly so it performs just one operation:

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

sed 's/^.*$/ "\0",/'

This isn't portable and won't work with 'sed' from BSD lineage, including Mac OS. More portable would be:

    sed 's/^\(.*\)$/    "\1",/'

Or just replace Emily's \0 with &; that should be in POSIX.

Yes, that's nicer.

#182 lore raw

Re: [PATCH v6 01/15] help: move list_config_help to builtin/help

Quoted 9 lines · Eric Sunshine
Quoted 8 lines · Junio C Hamano
Quoted 7 lines · Eric Sunshine
Quoted 5 lines · Emily Shaffer

Thanks for the suggestion. You're right that I didn't look into the contents of this script much, because I was just moving it; I modified your sed expression slightly so it performs just one operation:

sed 's/^.*$/ "\0",/'

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

This isn't portable and won't work with 'sed' from BSD lineage, including Mac OS. More portable would be:

    sed 's/^\(.*\)$/    "\1",/'

Or just replace Emily's \0 with &; that should be in POSIX.

Yes, that's nicer.

Sure, thanks.

#183 lore raw

PATCH v6 2/15 help: add shell-path to --build-options

In reply to Emily Shaffer

It may be useful to know which shell Git was built to try to point to, in the event that shell-based Git commands are failing. $SHELL_PATH is set during the build and used to launch the manpage viewer, as well as by git-compat-util.h, and it's used during tests. 'git version --build-options' is encouraged for use in bug reports, so it makes sense to include this information there.

---
 help.c | 1 +
 1 file changed, 1 insertion(+)
help.c +1 -0
index a21487db77..190722fb0a 100644
@@ -654,6 +654,7 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 			printf("no commit associated with this build\n");
 		printf("sizeof-long: %d\n", (int)sizeof(long));
 		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
+		printf("shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 	return 0;
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#184 lore raw

PATCH v6 3/15 bugreport: add tool to generate debugging info

In reply to Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.

Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.

---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 41 ++++++++++++++++
 Makefile                        |  5 ++
 bugreport.c                     | 85 +++++++++++++++++++++++++++++++++
 t/t0091-bugreport.sh            | 49 +++++++++++++++++++
 5 files changed, 181 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh
.gitignore +1 -0
index ea97de83f3..d89bf9e11e 100644
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
Documentation/git-bugreport.txt new file +41 -0

41 lines not on this page · raw message

Makefile +5 -0
index 5a022367d4..a01a050aa3 100644
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2457,6 +2458,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
bugreport.c new file +85 -0

85 lines not on this page · raw message

t/t0091-bugreport.sh new file +49 -0

49 lines not on this page · raw message

Signature · 1 line
2.25.0.341.g760bfbb309-goog
#185 lore raw
Quoted 21 lines · Emily Shaffer
t/t0091-bugreport.sh new file +15 -0
new file mode 100755
index 0000000000..451badff0c
@@ -0,0 +1,49 @@
+#!/bin/sh
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if test "$(grep "$HEADER_PATTERN" "$line")"; then
+			echo "$line"
+			read -r nextline

t/t0091-bugreport.sh line 25

Quoted 10 lines
t/t0091-bugreport.sh new file +10 -0
+			if test -z "$nextline"; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	REPORT="$(ls git-bugreport-*)" &&

What if the globbing were to match more than one file?

t/t0091-bugreport.sh line 27

Quoted 2 lines
t/t0091-bugreport.sh new file +2 -0
+	check_all_headers_populated <$REPORT &&
+	rm $REPORT

Please register a cleanup commands like this with 'test_when_finished'.

t/t0091-bugreport.sh line 37

Quoted 10 lines
t/t0091-bugreport.sh new file +10 -0
+'
+
+test_expect_success 'dies if file with same name as report already exists' '
+	touch git-bugreport-duplicate.txt &&
+	test_must_fail git bugreport --suffix duplicate &&
+	rm git-bugreport-duplicate.txt
+'
+
+test_expect_success '--output-directory puts the report in the provided dir' '
+	mkdir foo/ &&

Is it really necessary to create the directory in advance? Or to put it in another way: shouldn't 'git bugreport' create any missing leading directories of the path given for its '-o' option? FWIW, 'git format-patch -o dir ...' does create all necessary directories.

t/t0091-bugreport.sh line 39

Quoted 2 lines
t/t0091-bugreport.sh new file +2 -0
+	git bugreport -o foo/ &&
+	test_path_is_file foo/git-bugreport-* &&

What if the globbing were to match more than one file? :)

t/t0091-bugreport.sh line 45

Quoted 6 lines
t/t0091-bugreport.sh new file +6 -0
+	rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	grep usage output &&

This breaks the GETTEXT_POISON CI job, because "usage" is translated, so the test should not look for it with plain 'grep'. Please use 'test_i18ngrep' instead.

Quoted 7 lines
t/t0091-bugreport.sh new file +4 -0
+	test_path_is_missing git-bugreport-*
+'
+
+test_done
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#186 lore raw
Quoted 26 lines · SZEDER Gábor
Quoted 25 lines · Emily Shaffer
t/t0091-bugreport.sh new file +19 -0
new file mode 100755
index 0000000000..451badff0c
@@ -0,0 +1,49 @@
+#!/bin/sh
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if test "$(grep "$HEADER_PATTERN" "$line")"; then
+			echo "$line"
+			read -r nextline
+			if test -z "$nextline"; then
+				return 1;
+			fi
+		fi
Quoted 8 lines

t/t0091-bugreport.sh line 25

Quoted 6 lines
t/t0091-bugreport.sh new file +6 -0
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	REPORT="$(ls git-bugreport-*)" &&

What if the globbing were to match more than one file?

An often-useful pattern is to make the command report the output filename, i.e.

	REPORT=$(git butreport) &&

if the design insists that "git bugreport" should allocate a filename in order to make it easy to guarantee uniqueness.

Of course, we can make the invoker supply filename, e.g.

	REPORT=$(generate-output-filename) &&
	git bugreport -o "$REPORT" ;# or git bugreport >"$REPORT"
#187 lore raw
Quoted 42 lines · Junio C Hamano
Quoted 26 lines · SZEDER Gábor
Quoted 25 lines · Emily Shaffer
t/t0091-bugreport.sh new file +19 -0
new file mode 100755
index 0000000000..451badff0c
@@ -0,0 +1,49 @@
+#!/bin/sh
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if test "$(grep "$HEADER_PATTERN" "$line")"; then
+			echo "$line"
+			read -r nextline
+			if test -z "$nextline"; then
+				return 1;
+			fi
+		fi
Quoted 8 lines

t/t0091-bugreport.sh line 25

Quoted 6 lines
t/t0091-bugreport.sh new file +6 -0
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	REPORT="$(ls git-bugreport-*)" &&

What if the globbing were to match more than one file?

An often-useful pattern is to make the command report the output filename, i.e.

	REPORT=$(git butreport) &&
Quoted 7 lines

if the design insists that "git bugreport" should allocate a filename in order to make it easy to guarantee uniqueness.

Of course, we can make the invoker supply filename, e.g.

	REPORT=$(generate-output-filename) &&
	git bugreport -o "$REPORT" ;# or git bugreport >"$REPORT"

Yeah, this is an option and I've changed the offending test - which captures a variable based on globbing - to use --suffix and force a filename.

#188 lore raw
In reply to Emily Shaffer
Quoted 14 lines · Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.

Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.

Quoted 10 lines
---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 41 ++++++++++++++++
 Makefile                        |  5 ++
 bugreport.c                     | 85 +++++++++++++++++++++++++++++++++
 t/t0091-bugreport.sh            | 49 +++++++++++++++++++
 5 files changed, 181 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh

Please add a corresponding entry to 'command-list.txt' as well. According to the comment at the beginning of that file, all, even external commands should be included there.

#189 lore raw
In reply to Emily Shaffer

As this topic seems to break under GIT_TEST_GETTEXT_POISON=yes, I'll apply the following band-aid on top before merging it to 'pu'.

Most of them are style fixes, but quoting $REPORT, not just makes the redirection honor the coding guidelines, will ensure that an error is caught if git-bugreport-* glob matches no paths or more than on paths.

 t/t0091-bugreport.sh | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)
t/t0091-bugreport.sh +10 -7
index 451badff0c..f2186941ce 100755
@@ -8,9 +8,12 @@ test_description='git bugreport'
 # information there; we can make sure all our headers were followed by some
 # information to check if the command was successful.
 HEADER_PATTERN="^\[.*\]$"
-check_all_headers_populated() {
-	while read -r line; do
-		if test "$(grep "$HEADER_PATTERN" "$line")"; then
+
+check_all_headers_populated () {
+	while read -r line
+	do
+		if test "$(grep "$HEADER_PATTERN" "$line")"
+		then
 			echo "$line"
 			read -r nextline
 			if test -z "$nextline"; then
@@ -23,12 +26,12 @@ check_all_headers_populated() {
 test_expect_success 'creates a report with content in the right places' '
 	git bugreport &&
 	REPORT="$(ls git-bugreport-*)" &&
-	check_all_headers_populated <$REPORT &&
-	rm $REPORT
+	check_all_headers_populated <"$REPORT" &&
+	rm "$REPORT"
 '
 
 test_expect_success 'dies if file with same name as report already exists' '
-	touch git-bugreport-duplicate.txt &&
+	>>git-bugreport-duplicate.txt &&
 	test_must_fail git bugreport --suffix duplicate &&
 	rm git-bugreport-duplicate.txt
 '
@@ -42,7 +45,7 @@ test_expect_success '--output-directory puts the report in the provided dir' '
 
 test_expect_success 'incorrect arguments abort with usage' '
 	test_must_fail git bugreport --false 2>output &&
-	grep usage output &&
+	test_i18ngrep usage output &&
 	test_path_is_missing git-bugreport-*
 '
 
Signature · 1 line
2.25.0-455-gcf3c3a5ab4
#190 lore raw
Quoted 7 lines · Junio C Hamano

As this topic seems to break under GIT_TEST_GETTEXT_POISON=yes, I'll apply the following band-aid on top before merging it to 'pu'.

Most of them are style fixes, but quoting $REPORT, not just makes the redirection honor the coding guidelines, will ensure that an error is caught if git-bugreport-* glob matches no paths or more than on paths.

Thanks. Most of these I have locally, but not all. I'll apply this as a fixup on patch 3/n (the one that introduces the tests) and add a Helped-by line.

I'm hoping to get a reroll out today; it was the first thing on my list this week, and then a high-priority internal issue shows up (so it goes). Sorry for the inconvenience with the CI failures.

- Emily

Quoted 51 lines
 t/t0091-bugreport.sh | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)
t/t0091-bugreport.sh +10 -7
index 451badff0c..f2186941ce 100755
@@ -8,9 +8,12 @@ test_description='git bugreport'
 # information there; we can make sure all our headers were followed by some
 # information to check if the command was successful.
 HEADER_PATTERN="^\[.*\]$"
-check_all_headers_populated() {
-	while read -r line; do
-		if test "$(grep "$HEADER_PATTERN" "$line")"; then
+
+check_all_headers_populated () {
+	while read -r line
+	do
+		if test "$(grep "$HEADER_PATTERN" "$line")"
+		then
 			echo "$line"
 			read -r nextline
 			if test -z "$nextline"; then
@@ -23,12 +26,12 @@ check_all_headers_populated() {
 test_expect_success 'creates a report with content in the right places' '
 	git bugreport &&
 	REPORT="$(ls git-bugreport-*)" &&
-	check_all_headers_populated <$REPORT &&
-	rm $REPORT
+	check_all_headers_populated <"$REPORT" &&
+	rm "$REPORT"
 '
 
 test_expect_success 'dies if file with same name as report already exists' '
-	touch git-bugreport-duplicate.txt &&
+	>>git-bugreport-duplicate.txt &&
 	test_must_fail git bugreport --suffix duplicate &&
 	rm git-bugreport-duplicate.txt
 '
@@ -42,7 +45,7 @@ test_expect_success '--output-directory puts the report in the provided dir' '
 
 test_expect_success 'incorrect arguments abort with usage' '
 	test_must_fail git bugreport --false 2>output &&
-	grep usage output &&
+	test_i18ngrep usage output &&
 	test_path_is_missing git-bugreport-*
 '
 
Signature · 1 line
2.25.0-455-gcf3c3a5ab4
#191 lore raw

PATCH v6 4/15 bugreport: gather git version and build info

In reply to Emily Shaffer

Knowing which version of Git a user has and how it was built allows us to more precisely pin down the circumstances when a certain issue occurs, so teach bugreport how to tell us the same output as 'git version --build-options'.

It's not ideal to directly call 'git version --build-options' because that output goes to stdout. Instead, wrap the version string in a helper within help.[ch] library, and call that helper from within the bugreport library.

---
 Documentation/git-bugreport.txt |  4 +++
 bugreport.c                     | 19 ++++++++++++-
 help.c                          | 48 ++++++++++++++++++++-------------
 help.h                          |  1 +
 4 files changed, 52 insertions(+), 20 deletions(-)
Documentation/git-bugreport.txt +4 -0
index 52d49ed7aa..6ce2ae2eff 100644
@@ -23,6 +23,10 @@ The following information is requested from the user:
  - Expected behavior
  - Actual behavior
 
+The following information is captured automatically:
+
+ - 'git version --build-options'
+
 OPTIONS
 -------
 -o <path>::
bugreport.c +18 -1
index db46fb88be..dd21be9d68 100644
@@ -1,8 +1,17 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(sys_info, 1);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
@@ -32,6 +41,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -67,6 +81,9 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
help.c +29 -19

63 lines not on this page · raw message

help.h +1 -0
index 9071894e8c..500521b908 100644
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void get_version_info(struct strbuf *buf, int show_build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#192 lore raw

PATCH v6 5/15 bugreport: add uname info

In reply to Emily Shaffer

The contents of uname() can give us some insight into what sort of system the user is running on, and help us replicate their setup if need be. The domainname field is not guaranteed to be available, so don't collect it.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 13 +++++++++++++
 2 files changed, 14 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 6ce2ae2eff..4dd72c60f5 100644
@@ -26,6 +26,7 @@ The following information is requested from the user:
 The following information is captured automatically:
 
  - 'git version --build-options'
+ - uname sysname, release, version, and machine strings
 
 OPTIONS
 -------
bugreport.c +13 -0
index dd21be9d68..4c4b877ea1 100644
@@ -7,10 +7,23 @@
 
 static void get_system_info(struct strbuf *sys_info)
 {
+	struct utsname uname_info;
+
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	get_version_info(sys_info, 1);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#193 lore raw

PATCH v6 6/15 bugreport: add compiler info

In reply to Emily Shaffer

To help pinpoint the source of a regression, it is useful to know some info about the compiler which the user's Git client was built with. By adding a generic get_compiler_info() in 'compat/' we can choose which relevant information to share per compiler; to get started, let's demonstrate the version of glibc if the user built with 'gcc'.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     |  5 +++++
 compat/compiler.h               | 24 ++++++++++++++++++++++++
 3 files changed, 30 insertions(+)
 create mode 100644 compat/compiler.h
Documentation/git-bugreport.txt +1 -0
index 4dd72c60f5..8bbc4c960c 100644
@@ -27,6 +27,7 @@ The following information is captured automatically:
 
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
+ - Compiler-specific info string
 
 OPTIONS
 -------
bugreport.c +5 -0
index 4c4b877ea1..aaeb94c10c 100644
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include "compat/compiler.h"
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -24,6 +25,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
compat/compiler.h new file +24 -0
new file mode 100644
index 0000000000..bda5098e1b
@@ -0,0 +1,24 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
+}
+
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addstr(info, "get_compiler_info() not implemented");
+}
+
+#endif
+
+#endif /* COMPILER_H */
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#194 lore raw

PATCH v6 7/15 bugreport: add git-remote-https version

In reply to Emily Shaffer

It's possible for git-remote-curl to be built separately from git; in that case we want to know what version of cURL is used by git-remote-curl, not necessarily which version was present at git-bugreport's build time. So instead, ask git-remote-curl for the version information it knows about.

Today, "git-remote-http" and "git-remote-https" are aliased to "git-remote-curl"; but in case we rely on a different library than cURL in the future, let's not explicitly reference cURL from bugreport.

For longevity purposes, invoke the alias "git-remote-https" instead of "git-remote-http".

Since it could have been built at a different time, also report the version and built-from commit of git-remote-curl alongside the cURL info.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 16 ++++++++++++++++
 remote-curl.c                   |  8 ++++++++
 3 files changed, 25 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 8bbc4c960c..33df4dec7f 100644
@@ -28,6 +28,7 @@ The following information is captured automatically:
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
+ - 'git remote-https --build-info'
 
 OPTIONS
 -------
bugreport.c +16 -0
index aaeb94c10c..da5859c131 100644
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include "compat/compiler.h"
+#include "run-command.h"
+
+static void get_git_remote_https_version_info(struct strbuf *version_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, version_info, 0))
+	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -29,6 +41,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "compiler info: ");
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+	get_git_remote_https_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
remote-curl.c +8 -0
index 350d92a074..c590fbfae3 100644
@@ -17,6 +17,7 @@
 #include "protocol.h"
 #include "quote.h"
 #include "transport.h"
+#include "version.h"
 
 static struct remote *remote;
 /* always ends with a trailing slash */
@@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {
+		printf("git-http-fetch version: %s\n", git_version_string);
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#195 lore raw

PATCH v6 8/15 bugreport: include user interactive shell

In reply to Emily Shaffer

It's possible a user may complain about the way that Git interacts with their interactive shell, e.g. autocompletion or shell prompt. In that case, it's useful for us to know which shell they're using interactively.

---
 Documentation/git-bugreport.txt | 1 +
 bugreport.c                     | 5 +++++
 2 files changed, 6 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 33df4dec7f..23265b0d74 100644
@@ -29,6 +29,7 @@ The following information is captured automatically:
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
  - 'git remote-https --build-info'
+ - $SHELL
 
 OPTIONS
 -------
bugreport.c +5 -0
index da5859c131..9b51250155 100644
@@ -21,6 +21,7 @@ static void get_git_remote_https_version_info(struct strbuf *version_info)
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct utsname uname_info;
+	char *shell = NULL;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
@@ -42,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
 
+	shell = getenv("SHELL");
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    shell ? shell : "<unset>");
+
 	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
 	get_git_remote_https_version_info(sys_info);
 	strbuf_complete_line(sys_info);
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#196 lore raw

PATCH v6 9/15 bugreport: generate config safelist based on docs

In reply to Emily Shaffer

Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "annotate" macro, which is a no-op in AsciiDoc and AsciiDoctor.

Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.

However, it's possible that in some situations, an organization may be less concerned with privacy of things like remote URLs and branch names, and more concerned with ease of diagnosis for their support staff. In those cases, it may make more sense for that organization to modify the code to use a blocklist. To that end, we should try to mark configs which are definitely safe, and configs which are definitely unsafe, and leave blank configs which are somewhere in between. To mark a config as safe, add "annotate:bugreport[include]" to the corresponding line in the config documentation; to mark it as unsafe, add "annotate:bugreport[exclude]" instead.

Generating bugreport-config-safelist.h at build time by grepping the documentation for this new macro helps us prevent staleness. The macro itself is a no-op and should not alter the appearance of the documentation in either AsciiDoc or AsciiDoctor, confirmable by running:

  cd Documentation
  ./doc-diff --asciidoctor HEAD^ HEAD
  ./doc-diff --asciidoc HEAD^ HEAD

Diffing the rendered HTML shows that only inline comments were added, which shouldn't be a problem.

Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.

---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  9 ++++
 Documentation/asciidoctor-extensions.rb |  5 +++
 Documentation/config/sendemail.txt      | 56 ++++++++++++-------------
 Makefile                                |  7 ++++
 generate-bugreport-config-safelist.sh   | 17 ++++++++
 6 files changed, 67 insertions(+), 28 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh
.gitignore +1 -0
index d89bf9e11e..bd2f49b996 100644
@@ -192,6 +192,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
Documentation/asciidoc.conf +9 -0
index 8fc4b67081..663e06481f 100644
@@ -6,9 +6,14 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The annotate macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources to populate
+# things like the bugreport safelist.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>annotate):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +33,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[annotate-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +101,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[annotate-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
Documentation/asciidoctor-extensions.rb +5 -0
index d906a00803..382bd8f6f4 100644
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, target, attrs)
+        ""
       end
     end
   end
@@ -45,4 +49,5 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  inline_macro Git::Documentation::AnnotateProcessor, :annotate
 end
Documentation/config/sendemail.txt +28 -28

83 lines not on this page · raw message

Makefile +7 -0
index a01a050aa3..2bc9f112ea 100644
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2163,6 +2164,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
generate-bugreport-config-safelist.sh new file +17 -0
new file mode 100755
index 0000000000..17d92a91c5
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the annotate-bugreport macro
+sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
+
+cat <<EOF
+};
+EOF
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#197 lore raw

Re: [PATCH v6 09/15] bugreport: generate config safelist based on docs

Quoted 105 lines · Emily Shaffer

Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "annotate" macro, which is a no-op in AsciiDoc and AsciiDoctor.

Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.

However, it's possible that in some situations, an organization may be less concerned with privacy of things like remote URLs and branch names, and more concerned with ease of diagnosis for their support staff. In those cases, it may make more sense for that organization to modify the code to use a blocklist. To that end, we should try to mark configs which are definitely safe, and configs which are definitely unsafe, and leave blank configs which are somewhere in between. To mark a config as safe, add "annotate:bugreport[include]" to the corresponding line in the config documentation; to mark it as unsafe, add "annotate:bugreport[exclude]" instead.

Generating bugreport-config-safelist.h at build time by grepping the documentation for this new macro helps us prevent staleness. The macro itself is a no-op and should not alter the appearance of the documentation in either AsciiDoc or AsciiDoctor, confirmable by running:

  cd Documentation
  ./doc-diff --asciidoctor HEAD^ HEAD
  ./doc-diff --asciidoc HEAD^ HEAD

Diffing the rendered HTML shows that only inline comments were added, which shouldn't be a problem.

Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.

---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  9 ++++
 Documentation/asciidoctor-extensions.rb |  5 +++
 Documentation/config/sendemail.txt      | 56 ++++++++++++-------------
 Makefile                                |  7 ++++
 generate-bugreport-config-safelist.sh   | 17 ++++++++
 6 files changed, 67 insertions(+), 28 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh
.gitignore +1 -0
index d89bf9e11e..bd2f49b996 100644
@@ -192,6 +192,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
Documentation/asciidoc.conf +9 -0
index 8fc4b67081..663e06481f 100644
@@ -6,9 +6,14 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The annotate macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources to populate
+# things like the bugreport safelist.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>annotate):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +33,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[annotate-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +101,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[annotate-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
Documentation/asciidoctor-extensions.rb +0 -0
index d906a00803..382bd8f6f4 100644

Documentation/asciidoctor-extensions.rb module Git · line 43

Quoted 7 lines
Documentation/asciidoctor-extensions.rb +4 -0
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, target, attrs)
+        ""

This change breaks building the documentation with Asciidoctor v1.5.8:

/home/travis/.rvm/rubies/ruby-2.5.3/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': /home/travis/build/git/git/Documentation/asciidoctor-extensions.rb:41: class definition in method body (SyntaxError)

...xtensions::InlineMacroProcessor

... ^

/home/travis/build/git/git/Documentation/asciidoctor-extensions.rb:53: syntax error, unexpected end-of-input, expecting keyword_end

https://travis-ci.org/git/git/jobs/647093871#L1127

Quoted 9 lines
Documentation/asciidoctor-extensions.rb +1 -0
       end
     end
   end
@@ -45,4 +49,5 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  inline_macro Git::Documentation::AnnotateProcessor, :annotate
 end
#198 lore raw

Re: [PATCH v6 09/15] bugreport: generate config safelist based on docs

Quoted 18 lines · SZEDER Gábor

Documentation/asciidoctor-extensions.rb module Git · line 43

Quoted 12 lines · Emily Shaffer
Documentation/asciidoctor-extensions.rb +4 -0
index d906a00803..382bd8f6f4 100644
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, target, attrs)
+        ""

This change breaks building the documentation with Asciidoctor v1.5.8:

/home/travis/.rvm/rubies/ruby-2.5.3/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': /home/travis/build/git/git/Documentation/asciidoctor-extensions.rb:41: class definition in method body (SyntaxError)

Quoted 8 lines

...xtensions::InlineMacroProcessor

... ^

/home/travis/build/git/git/Documentation/asciidoctor-extensions.rb:53: syntax error, unexpected end-of-input, expecting keyword_end

https://travis-ci.org/git/git/jobs/647093871#L1127

Done. It passes locally for me now if I run:

USE_ASCIIDOCTOR=1 make doc

Thanks for the heads up.

- Emily

#199 lore raw

PATCH v6 10/15 bugreport: add config values from safelist

In reply to Emily Shaffer

Teach bugreport to gather the values of config options which are present in 'bugreport-config-safelist.h', and show their origin scope.

Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.

Rather than including the path to someone's config, which can reveal filesystem layout and project names, just name the scope (e.g. system, global, local) of the config source.

---
 Documentation/git-bugreport.txt |  1 +
 Makefile                        |  2 ++
 bugreport.c                     | 34 +++++++++++++++++++++++++++++++++
 3 files changed, 37 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 23265b0d74..4e9171d1bd 100644
@@ -30,6 +30,7 @@ The following information is captured automatically:
  - Compiler-specific info string
  - 'git remote-https --build-info'
  - $SHELL
+ - Selected config values
 
 OPTIONS
 -------
Makefile +2 -0
index 2bc9f112ea..7b139230ae 100644
@@ -2131,6 +2131,8 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
+bugreport.sp bugreport.s bugreport.o: bugreport-config-safelist.h
+
 builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
bugreport.c +34 -0
index 9b51250155..16216bff8e 100644
@@ -6,6 +6,9 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -52,6 +55,29 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	size_t idx;
+	struct string_list_item *it = NULL;
+	struct key_value_info *kv_info = NULL;
+
+	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
+		const struct string_list *list =
+			git_config_get_value_multi(bugreport_config_safelist[idx]);
+
+		if (!list)
+			continue;
+
+		strbuf_addf(config_info, "%s:\n", bugreport_config_safelist[idx]);
+		for_each_string_list_item(it, list) {
+			kv_info = it->util;
+			strbuf_addf(config_info, "  %s (%s)\n", it->string,
+				    kv_info ? config_scope_name(kv_info->scope)
+					    : "source unknown");
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -94,6 +120,7 @@ int cmd_main(int argc, const char **argv)
 	char *option_output = NULL;
 	char *option_suffix = "%F-%H%M";
 	struct stat statbuf;
+	int nongit_ok = 0;
 
 	const struct option bugreport_options[] = {
 		OPT_STRING('o', "output-directory", &option_output, N_("path"),
@@ -102,6 +129,10 @@ int cmd_main(int argc, const char **argv)
 			   N_("specify a strftime format suffix for the filename")),
 		OPT_END()
 	};
+
+	/* Prerequisite for hooks and config checks */
+	setup_git_directory_gently(&nongit_ok);
+
 	argc = parse_options(argc, argv, "", bugreport_options,
 			     bugreport_usage, 0);
 
@@ -123,6 +154,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#200 lore raw

Re: [PATCH v6 10/15] bugreport: add config values from safelist

bugreport.c static void get_system_info(struct strbuf *sys_info) · line 64

Quoted 7 lines · Emily Shaffer
bugreport.c +7 -0
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	size_t idx;
+	struct string_list_item *it = NULL;
+	struct key_value_info *kv_info = NULL;
+
+	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {

GCC 9 complains about this loop condition:

CC bugreport.o

bugreport.c: In function 'get_safelisted_config':

bugreport.c:66:20: error: comparison of unsigned expression < 0 is always false [-Werror=type-limits]

     66 |  for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
  
        |                    ^

I don't understand this error, that autogenerated 'bugreport_config_safelist' array clearly has a a non-zero size. What am I missing?

Quoted 16 lines
bugreport.c +16 -0
+		const struct string_list *list =
+			git_config_get_value_multi(bugreport_config_safelist[idx]);
+
+		if (!list)
+			continue;
+
+		strbuf_addf(config_info, "%s:\n", bugreport_config_safelist[idx]);
+		for_each_string_list_item(it, list) {
+			kv_info = it->util;
+			strbuf_addf(config_info, "  %s (%s)\n", it->string,
+				    kv_info ? config_scope_name(kv_info->scope)
+					    : "source unknown");
+		}
+	}
+}
+
#201 lore raw

Re: [PATCH v6 10/15] bugreport: add config values from safelist

Quoted 17 lines · SZEDER Gábor

bugreport.c static void get_system_info(struct strbuf *sys_info) · line 64

Quoted 7 lines · Emily Shaffer
bugreport.c +7 -0
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	size_t idx;
+	struct string_list_item *it = NULL;
+	struct key_value_info *kv_info = NULL;
+
+	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {

GCC 9 complains about this loop condition:

CC bugreport.o

bugreport.c: In function 'get_safelisted_config':

bugreport.c:66:20: error: comparison of unsigned expression < 0 is always false [-Werror=type-limits]

Quoted 7 lines
     66 |  for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
  
        |                    ^

I don't understand this error, that autogenerated 'bugreport_config_safelist' array clearly has a a non-zero size. What am I missing?

macOS 'sed', that's what I was missing :)

So that array is in fact empty on macOS, because the entries of that array are generated with:

  # cat all regular files in Documentation/config
  find Documentation/config -type f -exec cat {} \; |
  # print the command name which matches the annotate-bugreport macro
  sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort

and the 'sed' included in macOS apparently interprets that '\+' differently than GNU 'sed', and as a result won't match anything.

FWIW, that '\+' doesn't seem to be necessary, though, and after removing it the resulting generated array looked good to me (and to the compiler) both on Linux and macOS.

#202 lore raw

Re: [PATCH v6 10/15] bugreport: add config values from safelist

Quoted 6 lines · SZEDER Gábor

macOS 'sed', that's what I was missing :)

sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/ "\1",/p' | sort

and the 'sed' included in macOS apparently interprets that '\+' differently than GNU 'sed', and as a result won't match anything.

More generally, this would be a problem with any 'sed' of BSD lineage.

Quoted 2 lines

FWIW, that '\+' doesn't seem to be necessary, though, and after removing it the resulting generated array looked good to me [...]

A reasonable replacement for "<SP>\+" would be "<SP><SP>*" (where <SP> represents 'space').

#203 lore raw

Re: [PATCH v6 10/15] bugreport: add config values from safelist

Quoted 8 lines · Eric Sunshine
Quoted 6 lines · SZEDER Gábor

macOS 'sed', that's what I was missing :)

sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/ "\1",/p' | sort

and the 'sed' included in macOS apparently interprets that '\+' differently than GNU 'sed', and as a result won't match anything.

Quoted 7 lines

More generally, this would be a problem with any 'sed' of BSD lineage.

Quoted 2 lines

FWIW, that '\+' doesn't seem to be necessary, though, and after removing it the resulting generated array looked good to me [...]

A reasonable replacement for "<SP>\+" would be "<SP><SP>*" (where <SP> represents 'space').

Another problem with that regexp is that it contains two adjacent repetitions matching the same character. When there are two or more spaces before "annotate:" all but the last of them can be matched by either '\(.*\)' or ' \+'. To fix that '\(.*\)' needs to be modified to not match a trailing space.

Andreas.

Signature · 3 lines
Andreas Schwab, [email protected]
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."
#204 lore raw

Re: [PATCH v6 10/15] bugreport: add config values from safelist

Quoted 15 lines · Andreas Schwab
Quoted 13 lines · Eric Sunshine
Quoted 6 lines · SZEDER Gábor

macOS 'sed', that's what I was missing :)

sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/ "\1",/p' | sort

and the 'sed' included in macOS apparently interprets that '\+' differently than GNU 'sed', and as a result won't match anything.

More generally, this would be a problem with any 'sed' of BSD lineage.

Quoted 2 lines

FWIW, that '\+' doesn't seem to be necessary, though, and after removing it the resulting generated array looked good to me [...]

Quoted 8 lines
Quoted 2 lines

A reasonable replacement for "<SP>\+" would be "<SP><SP>*" (where <SP> represents 'space').

Another problem with that regexp is that it contains two adjacent repetitions matching the same character. When there are two or more spaces before "annotate:" all but the last of them can be matched by either '\(.*\)' or ' \+'. To fix that '\(.*\)' needs to be modified to not match a trailing space.

Hum. I had assumed since the capture group was not greedy, it would not capture any trailing spaces that could be captured by ' \+'. I don't see a problem in making it non-explicit, though.

I'll hack on this some more. I do find myself pretty annoyed at the matcher difference between 'sed' and 'also sed', though, and don't see in the manpage a way to guarantee which matcher 'sed' should use (a la 'grep -[EFGP]'). :)

- Emily

#205 lore raw

PATCH v6 11/15 bugreport: collect list of populated hooks

In reply to Emily Shaffer

Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong.

Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot. To reduce the amount of code humans need to read, we turn the list into a string_list and iterate over it (as we are calling the same find_hook operation on each string). However, since bugreport should primarily be called by the user, the performance loss from massaging the string seems acceptable.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 53 +++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 4e9171d1bd..779c9c7fd9 100644
@@ -31,6 +31,7 @@ The following information is captured automatically:
  - 'git remote-https --build-info'
  - $SHELL
  - Selected config values
+ - A list of enabled hooks
 
 OPTIONS
 -------
bugreport.c +53 -0

71 lines not on this page · raw message

Signature · 1 line
2.25.0.341.g760bfbb309-goog
#206 lore raw

PATCH v6 12/15 bugreport: count loose objects

In reply to Emily Shaffer

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 52 +++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 779c9c7fd9..7e1ed44cb4 100644
@@ -32,6 +32,7 @@ The following information is captured automatically:
  - $SHELL
  - Selected config values
  - A list of enabled hooks
+ - The number of loose objects in the repository
 
 OPTIONS
 -------
bugreport.c +52 -0

70 lines not on this page · raw message

Signature · 1 line
2.25.0.341.g760bfbb309-goog
#207 lore raw

PATCH v6 13/15 bugreport: add packed object summary

In reply to Emily Shaffer

Alongside the loose object counts, it can be useful to show the number of packs and packed objects. This way we can check whether the repo has an appropriate ratio of packed to loose objects to help determine whether it's behaving correctly.

Add a utility to easily traverse all packfiles in a given repository. Use it in packfile.c and remove a redundant call to prepare_packed_git(), which is already called in get_all_packs().

---
Perhaps the macro should be in another patch? I didn't spend a lot of time looking for anybody who could conceivably be iterating packs; if I were to do so, I'd break this change out separately.

I considered following the pattern in the rest of the file, where the foreach takes a callback and data pointer, but found it would be cumbersome to use with for_each_packed_object() - I'd have to add a new callback just to handle the pack, and bundle that function's callback data along with the callback pointer. It seemed like overkill.

- Emily

 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 30 ++++++++++++++++++++++++++++++
 object-store.h                  |  6 ++++++
 packfile.c                      |  3 +--
 4 files changed, 38 insertions(+), 2 deletions(-)
Documentation/git-bugreport.txt +1 -0
index 7e1ed44cb4..309b34fc42 100644
@@ -33,6 +33,7 @@ The following information is captured automatically:
  - Selected config values
  - A list of enabled hooks
  - The number of loose objects in the repository
+ - The number of packs and packed objects in the repository
 
 OPTIONS
 -------
bugreport.c +30 -0
index 9552e2922e..d554488f3e 100644
@@ -177,6 +177,33 @@ static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
 		    total_count_questionable ? " (problem during count)" : "");
 }
 
+static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
+{
+	struct packed_git *pack = NULL;
+	int pack_count = 0;
+	int object_count = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	for_each_pack(the_repository, pack) {
+		pack_count++;
+		/*
+		 * To accurately count how many objects are packed, look inside
+		 * the packfile's index.
+		 */
+		open_pack_index(pack);
+		object_count += pack->num_objects;
+	}
+
+	strbuf_addf(obj_info, "%d total packs (%d objects)\n", pack_count,
+		    object_count);
+
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -262,6 +289,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
object-store.h +6 -0
index 55ee639350..518c73469d 100644
@@ -6,6 +6,7 @@
 #include "list.h"
 #include "sha1-array.h"
 #include "strbuf.h"
+#include "packfile.h"
 
 struct object_directory {
 	struct object_directory *next;
@@ -403,4 +404,9 @@ int for_each_object_in_pack(struct packed_git *p,
 int for_each_packed_object(each_packed_object_fn, void *,
 			   enum for_each_object_flags flags);
 
+#define for_each_pack(repo, pack) 		\
+		for (pack = get_all_packs(repo);\
+		     pack;			\
+		     pack = pack->next)
+
 #endif /* OBJECT_STORE_H */
packfile.c +1 -2
index 7e7c04e4d8..6873827fb5 100644
@@ -2061,8 +2061,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data,
 	int r = 0;
 	int pack_errors = 0;
 
-	prepare_packed_git(the_repository);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for_each_pack(the_repository, p) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#208 lore raw

PATCH v6 14/15 bugreport: list contents of $OBJDIR/info

In reply to Emily Shaffer

Miscellaneous information used about the object store can end up in .git/objects/info; this can help us understand what may be going on with the object store when the user is reporting a bug. Otherwise, it could be difficult to track down what is going wrong with an object which isn't kept locally to .git/objects/ or .git/objects/pack. Having some understanding of where the user's objects may be kept can save us some hops during the bug reporting process.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 54 +++++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 309b34fc42..a21d081616 100644
@@ -34,6 +34,7 @@ The following information is captured automatically:
  - A list of enabled hooks
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
+ - A list of the contents of .git/objects/info (or equivalent)
 
 OPTIONS
 -------
bugreport.c +54 -0

66 lines not on this page · raw message

Signature · 1 line
2.25.0.341.g760bfbb309-goog
#209 lore raw

PATCH v6 15/15 bugreport: summarize contents of alternates file

In reply to Emily Shaffer

In some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead. Since paths to alternates may be sensitive, we'll instead count how many alternates have been specified and note how many of them exist or are broken.

While object-cache.h describes a function "foreach_alt_odb()", this function does not provide information on broken alternates, which are skipped over in "link_alt_odb_entry()". Since the goal is to identify missing alternates, we can gather the contents of .git/objects/info/alternates manually.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 45 +++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)
Documentation/git-bugreport.txt +1 -0
index a21d081616..e870900c80 100644
@@ -35,6 +35,7 @@ The following information is captured automatically:
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
  - A list of the contents of .git/objects/info (or equivalent)
+ - The number of valid and invalid alternates
 
 OPTIONS
 -------
bugreport.c +45 -0

57 lines not on this page · raw message

Signature · 1 line
2.25.0.341.g760bfbb309-goog
#210 lore raw

v7 cover add git-bugreport tool

In reply to Emily Shaffer

This patchset is based on mr/show-config-scope, which looks close to ready for 'next'.

Since v6:

- Applied Danh's suggestion[1] (with modifications recommended by Eric and Junio downthread)
- Fixed a missed end-of-macro in the Asciidoctor config which was pointed out most recently by SZEDER[2]
- Fixed generate-bugreport-safelist-config.sh to be more friendly to BSD-derived 'sed' and to capture trailing ' ' when extra whitespace is present(patch 9/15). This now uses POSIX character classes and {} notation instead of + and has been tested on OSX; I'd love to hear if it's still not working correctly.
- Applied Junio's patch[3] to make GIT_TEST_GETTEXT_POISON happy and to make the tests more idiomatic.
- Stop globbing into variables, start making the output directory if it doesn't exist before[4]
- Add an entry in 'command-list.txt'[5]. I sure wish someone had explained how to do this somewhere in a tutorial about writing new commands ;)
- Added a test to ensure that git-bugreport runs gracefully in a non-repo directory.
- Added nongit_ok flags to relevant pieces of info to ensure the above test passes.

[1] https://lore.kernel.org/git/[email protected]
[2] https://lore.kernel.org/git/[email protected]
[3] https://lore.kernel.org/git/[email protected]
[4] https://lore.kernel.org/git/[email protected]
[5] https://lore.kernel.org/git/[email protected]

Emily Shaffer (15):
help: move list_config_help to builtin/help
help: add shell-path to --build-options
bugreport: add tool to generate debugging info
bugreport: gather git version and build info
bugreport: add uname info
bugreport: add compiler info
bugreport: add git-remote-https version
bugreport: include user interactive shell
bugreport: generate config safelist based on docs
bugreport: add config values from safelist
bugreport: collect list of populated hooks
bugreport: count loose objects
bugreport: add packed object summary
bugreport: list contents of $OBJDIR/info
bugreport: summarize contents of alternates file

 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   9 +
 Documentation/asciidoctor-extensions.rb |   7 +
 Documentation/config/sendemail.txt      |  56 ++--
 Documentation/git-bugreport.txt         |  55 ++++
 Makefile                                |  27 +-
 bugreport.c                             | 418 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 command-list.txt                        |   1 +
 compat/compiler.h                       |  24 ++
 generate-bugreport-config-safelist.sh   |  18 +
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  21 ++
 help.c                                  | 132 ++------
 help.h                                  |   2 +-
 object-store.h                          |   6 +
 packfile.c                              |   3 +-
 remote-curl.c                           |   8 +
 t/t0091-bugreport.sh                    |  55 ++++
 19 files changed, 794 insertions(+), 156 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100644 compat/compiler.h
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh

v6 to v7 range-diff:

Documentation/asciidoctor-extensions.rb +2 -0
index 382bd8f6f4..03c80af0e5 100644
@@ -37,6 +37,8 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+      end
+    end
 
     class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
       def process(parent, target, attrs)
bugreport.c +9 -0
index 9d4d5c8e6f..3bc8cb3579 100644
@@ -368,6 +368,15 @@ int cmd_main(int argc, const char **argv)
 	if (!stat(report_path.buf, &statbuf))
 		die("'%s' already exists", report_path.buf);
 
+	switch (safe_create_leading_directories(report_path.buf)) {
+	case SCLD_OK:
+	case SCLD_EXISTS:
+		break;
+	default:
+		die(_("could not create leading directories for '%s'"),
+		    report_path.buf);
+	}
+
 	get_bug_template(&buffer);
 
 	get_header(&buffer, "System Info");
command-list.txt +1 -0
index 2087894655..185e5e3f05 100644
@@ -54,6 +54,7 @@ git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
 git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
+git-bugreport                           ancillaryinterrogators
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
 git-check-attr                          purehelpers
generate-bugreport-config-safelist.sh +2 -1
index 17d92a91c5..2794a8ae77 100755
@@ -10,7 +10,8 @@ EOF
 # cat all regular files in Documentation/config
 find Documentation/config -type f -exec cat {} \; |
 # print the command name which matches the annotate-bugreport macro
-sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
+sed -n 's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/  "\1",/p' \
+	| sort
 
 cat <<EOF
 };
generate-configlist.sh +1 -4
index eca6a00c30..8692fe5cf4 100755
@@ -10,10 +10,7 @@ EOF
 	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
 	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
 	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
+	sed 's/^.*$/	"&",/'
 	cat <<EOF
 	NULL,
 };
t/t0091-bugreport.sh +18 -12
index 451badff0c..6585a2d144 100755
@@ -8,9 +8,12 @@ test_description='git bugreport'
 # information there; we can make sure all our headers were followed by some
 # information to check if the command was successful.
 HEADER_PATTERN="^\[.*\]$"
-check_all_headers_populated() {
-	while read -r line; do
-		if test "$(grep "$HEADER_PATTERN" "$line")"; then
+
+check_all_headers_populated () {
+	while read -r line
+	do
+		if test "$(grep "$HEADER_PATTERN" "$line")"
+		then
 			echo "$line"
 			read -r nextline
 			if test -z "$nextline"; then
@@ -21,29 +24,32 @@ check_all_headers_populated() {
 }
 
 test_expect_success 'creates a report with content in the right places' '
-	git bugreport &&
-	REPORT="$(ls git-bugreport-*)" &&
-	check_all_headers_populated <$REPORT &&
-	rm $REPORT
+	git bugreport -s check-headers &&
+	check_all_headers_populated <git-bugreport-check-headers.txt &&
+	test_when_finished rm git-bugreport-check-headers.txt
 '
 
 test_expect_success 'dies if file with same name as report already exists' '
-	touch git-bugreport-duplicate.txt &&
+	>>git-bugreport-duplicate.txt &&
 	test_must_fail git bugreport --suffix duplicate &&
-	rm git-bugreport-duplicate.txt
+	test_when_finished rm git-bugreport-duplicate.txt
 '
 
 test_expect_success '--output-directory puts the report in the provided dir' '
-	mkdir foo/ &&
 	git bugreport -o foo/ &&
 	test_path_is_file foo/git-bugreport-* &&
-	rm -fr foo/
+	test_when_finished rm -fr foo/
 '
 
 test_expect_success 'incorrect arguments abort with usage' '
 	test_must_fail git bugreport --false 2>output &&
-	grep usage output &&
+	test_i18ngrep usage output &&
 	test_path_is_missing git-bugreport-*
 '
 
+test_expect_success 'runs outside of a git dir' '
+	nongit git bugreport &&
+	test_when_finished rm non-repo/git-bugreport-*
+'
+
 test_done
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#211 lore raw

PATCH v7 1/15 help: move list_config_help to builtin/help

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o, builtin/clean.o, and builtin/config.o. This meant that help.o was unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present).

When command-list.h is included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule.

---
 .gitignore             |  1 +
 Makefile               | 13 +++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 21 +++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 118 insertions(+), 108 deletions(-)
 create mode 100755 generate-configlist.sh
.gitignore +1 -0
index aebe7c0908..ea97de83f3 100644
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
Makefile +10 -3
index 09f98b777c..5a022367d4 100644
@@ -814,6 +814,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2128,7 +2129,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2148,6 +2149,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2781,7 +2788,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2803,7 +2810,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
builtin/help.c +86 -0

98 lines not on this page · raw message

generate-cmdlist.sh +0 -19
index 71158f7d8b..45fecf8bdf 100755
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
generate-configlist.sh new file +21 -0
new file mode 100755
index 0000000000..8692fe5cf4
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	sed 's/^.*$/	"&",/'
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
help.c +0 -85

91 lines not on this page · raw message

help.h +0 -1
index 7a455beeb7..9071894e8c 100644
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#212 lore raw

PATCH v7 2/15 help: add shell-path to --build-options

In reply to Emily Shaffer

It may be useful to know which shell Git was built to try to point to, in the event that shell-based Git commands are failing. $SHELL_PATH is set during the build and used to launch the manpage viewer, as well as by git-compat-util.h, and it's used during tests. 'git version --build-options' is encouraged for use in bug reports, so it makes sense to include this information there.

---
 help.c | 1 +
 1 file changed, 1 insertion(+)
help.c +1 -0
index a21487db77..190722fb0a 100644
@@ -654,6 +654,7 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 			printf("no commit associated with this build\n");
 		printf("sizeof-long: %d\n", (int)sizeof(long));
 		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
+		printf("shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 	return 0;
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#213 lore raw

PATCH v7 3/15 bugreport: add tool to generate debugging info

In reply to Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.

Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.

---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 41 ++++++++++++++
 Makefile                        |  5 ++
 bugreport.c                     | 94 +++++++++++++++++++++++++++++++++
 command-list.txt                |  1 +
 t/t0091-bugreport.sh            | 55 +++++++++++++++++++
 6 files changed, 197 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh
.gitignore +1 -0
index ea97de83f3..d89bf9e11e 100644
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
Documentation/git-bugreport.txt new file +41 -0

41 lines not on this page · raw message

Makefile +5 -0
index 5a022367d4..a01a050aa3 100644
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2457,6 +2458,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
bugreport.c new file +94 -0

94 lines not on this page · raw message

command-list.txt +1 -0
index 2087894655..185e5e3f05 100644
@@ -54,6 +54,7 @@ git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
 git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
+git-bugreport                           ancillaryinterrogators
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
 git-check-attr                          purehelpers
t/t0091-bugreport.sh new file +55 -0

55 lines not on this page · raw message

Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#214 lore raw

bugreport.c line 68

Quoted 1 line · Emily Shaffer
bugreport.c new file +1 -0
+	switch (safe_create_leading_directories(report_path.buf)) {

This helper is about creating paths in the working tree and Git repository, hence it has a call to adjust_shared_perm() which in turn calls get_shared_repo(), i.e. requiring a repository.

I thought I read somewhere that this tool is meant to be usable outside a repository? If that is not the case, then the use of this helper is OK. If not, we may want to make sure that it will stay to be safe to use the helper (I think it happens to be OK right now, but if the reason why the user is trying to run the tool is because the user broke Git by writing garbage into .git/config, we may die("your configuration file is broken") before this helper returns).

Thanks.

#215 lore raw
Quoted 6 lines · Junio C Hamano

bugreport.c line 68

Quoted 1 line · Emily Shaffer
bugreport.c new file +1 -0
+	switch (safe_create_leading_directories(report_path.buf)) {

This helper is about creating paths in the working tree and Git repository,

It's also used by cmd_format_patch() with --output-directory specified, which is how I found it. My usual workflow is to run format-patch with -o ~/mailed-patches/topic/ specified so I don't clutter my repo, so I'm surprised to hear the intent of safe_create_leading_directores() is for paths in the working tree or repo.

Quoted 2 lines

hence it has a call to adjust_shared_perm() which in turn calls get_shared_repo(), i.e. requiring a repository.

Hmmm. I was able to run it pretty happily from a nongit dir:

  emilyshaffer@podkayne:~$ tg bugreport -o other/very/deep/path/
  Created new report at 'other/very/deep/path/git-bugreport-2020-02-14-1721.txt'.
  emilyshaffer@podkayne:~$ cat other/very/deep/path/git-bugreport-2020-02-14-1721.txt
  Thank you for filling out a Git bug report!
Quoted 7 lines

I thought I read somewhere that this tool is meant to be usable outside a repository? If that is not the case, then the use of this helper is OK. If not, we may want to make sure that it will stay to be safe to use the helper (I think it happens to be OK right now, but if the reason why the user is trying to run the tool is because the user broke Git by writing garbage into .git/config, we may die("your configuration file is broken") before this helper returns).

Can you explain a little more about what you mean? A broken local .git/config seems to be preventing git.c from dispatching the 'bugreport' subcommand (lots of other Git commands are broken) at all - breakpoints in cmd_main() right after the variable declarations are not being hit.

With junk in the .git/config, I can't run bugreport from within that repo. With junk in ~/.gitconfig, I can't run bugreport anywhere.

(To make the configs invalid, I leaned on my keyboard and added a line 'garbaghe~*~$%)%)(@' to the bottom of the config file. Most commands terminate early with "fatal: bad config line 37 in file .git/config".)

Do you mean there's some specific config that could be misconfigured and prevent that utility from working?

- Emily

#216 lore raw
Quoted 10 lines · Emily Shaffer
Quoted 6 lines · Junio C Hamano

bugreport.c line 68

Quoted 1 line · Emily Shaffer
bugreport.c new file +1 -0
+	switch (safe_create_leading_directories(report_path.buf)) {

This helper is about creating paths in the working tree and Git repository,

It's also used by cmd_format_patch() with --output-directory specified, which is how I found it.

And that is an example of a good use of this helper. What will be written out there should be visible by the same people as those who have access to the repository; get_shared_repo() and adjust_perm() based on what the repository you are taking patches from is perfectly sensible. And as it is format-patch, we know we have get_shared_repo() working, and we know which repository we are working on.

Output directory for bugreport is on the same boat when we know the users are producing a report on their use of Git within a repository. It is not clear if the tool is started without any repository---it happens to do a random safe thing (i.e. I think get_shared_repo() gives PERM_UMASK, which tells adjust_perm() not to do anything especial) right now, but there is no guarantee that we will keep it working like that. Somebody may come and demaind get_shared_perm() to die() when outside a repository, for example, and that would break the nice property that lets bugreport working outside a repository.

I just wanted to make sure that somebody will be keeping an eye to remind those who propose such a change in the future. A comment near where get_shared_repo() happens may be something that can be done with a minimum effort when code that depends on that property is introduced at the same time.

Quoted 4 lines
Quoted 4 lines

I thought I read somewhere that this tool is meant to be usable outside a repository? If that is not the case, then the use of this helper is OK. If not, we may want to make sure that it will stay to be safe to use the helper (I think it happens to be OK right now,

I am actually OK if we limit the use of this tool to "use with a repository that is not corrupt", as coping with these kinds of breakages that in the main Git executable we deem "needs manual intervention" inside a single process is too painful (it would have been easier to cope with these too if we stuck with a script that invokes many discrete commands and acts on their errors, but that is optimizing for rare case and not recommended). But we should tell users about the limitation and encourage them to ask for help in non automatable means.

Thanks.

#217 lore raw
Quoted 21 lines · Junio C Hamano
Quoted 10 lines · Emily Shaffer
Quoted 6 lines · Junio C Hamano

bugreport.c line 68

Quoted 1 line · Emily Shaffer
bugreport.c new file +1 -0
+	switch (safe_create_leading_directories(report_path.buf)) {

This helper is about creating paths in the working tree and Git repository,

It's also used by cmd_format_patch() with --output-directory specified, which is how I found it.

And that is an example of a good use of this helper. What will be written out there should be visible by the same people as those who have access to the repository; get_shared_repo() and adjust_perm() based on what the repository you are taking patches from is perfectly sensible. And as it is format-patch, we know we have get_shared_repo() working, and we know which repository we are working on.

Quoted 10 lines

Output directory for bugreport is on the same boat when we know the users are producing a report on their use of Git within a repository. It is not clear if the tool is started without any repository---it happens to do a random safe thing (i.e. I think get_shared_repo() gives PERM_UMASK, which tells adjust_perm() not to do anything especial) right now, but there is no guarantee that we will keep it working like that. Somebody may come and demaind get_shared_perm() to die() when outside a repository, for example, and that would break the nice property that lets bugreport working outside a repository.

Hm, this would break the convention of the name of safe_create_leading_directories() too though right? As I understood it, "safe_foo" in this codebase means "this function will not die()"?

Quoted 6 lines

I just wanted to make sure that somebody will be keeping an eye to remind those who propose such a change in the future. A comment near where get_shared_repo() happens may be something that can be done with a minimum effort when code that depends on that property is introduced at the same time.

Ok. I want to make sure I understand you right. I think you're saying, "This is OK, except if someone changes get_shared_repo() it could break, so we need a way to warn someone that it could break." It sounds like you suggested leaving a comment in the safe_create_leading_directories() helper. My own preference is to write a test so that it's explicit that we depend on that behavior, instead - it's easy to gloss over a comment or read a different part of the codebase, but it's hard to ignore a breaking test. It'd be trivial to add one, so I will in v8 - unless I misunderstood you.

Quoted 6 lines
Quoted 5 lines
Quoted 5 lines

I thought I read somewhere that this tool is meant to be usable outside a repository? If that is not the case, then the use of this helper is OK. If not, we may want to make sure that it will stay to be safe to use the helper (I think it happens to be OK right now,

Quoted 9 lines

I am actually OK if we limit the use of this tool to "use with a repository that is not corrupt", as coping with these kinds of breakages that in the main Git executable we deem "needs manual intervention" inside a single process is too painful (it would have been easier to cope with these too if we stuck with a script that invokes many discrete commands and acts on their errors, but that is optimizing for rare case and not recommended). But we should tell users about the limitation and encourage them to ask for help in non automatable means.

I think you're saying, "Mention this drawback in the manpage for git-bugreport." Sounds like a good idea to me, so I'll add it for v8.

- Emily

#218 lore raw
Quoted 11 lines · Emily Shaffer
Quoted 9 lines · Junio C Hamano

I am actually OK if we limit the use of this tool to "use with a repository that is not corrupt", as coping with these kinds of breakages that in the main Git executable we deem "needs manual intervention" inside a single process is too painful (it would have been easier to cope with these too if we stuck with a script that invokes many discrete commands and acts on their errors, but that is optimizing for rare case and not recommended). But we should tell users about the limitation and encourage them to ask for help in non automatable means.

Quoted 2 lines

I think you're saying, "Mention this drawback in the manpage for git-bugreport." Sounds like a good idea to me, so I'll add it for v8.

I'm pretty unsure about how you wanted this to sound; rather than sending a v8 only to revise it a lot, I'll send the paragraph for wordsmithing beforehand. Is this what you meant?

This tool is invoked via the typical Git setup process, which means that in some cases, it might not be able to launch - for example, if a relevant config file is unreadable. In this kind of scenario, it may be helpful to manually gather the kind of information listed above when manually asking for help.

- Emily

#219 lore raw
Quoted 15 lines · Emily Shaffer
Quoted 11 lines · Emily Shaffer
Quoted 9 lines · Junio C Hamano

I am actually OK if we limit the use of this tool to "use with a repository that is not corrupt", as coping with these kinds of breakages that in the main Git executable we deem "needs manual intervention" inside a single process is too painful (it would have been easier to cope with these too if we stuck with a script that invokes many discrete commands and acts on their errors, but that is optimizing for rare case and not recommended). But we should tell users about the limitation and encourage them to ask for help in non automatable means.

Quoted 2 lines

I think you're saying, "Mention this drawback in the manpage for git-bugreport." Sounds like a good idea to me, so I'll add it for v8.

Quoted 8 lines

I'm pretty unsure about how you wanted this to sound; rather than sending a v8 only to revise it a lot, I'll send the paragraph for wordsmithing beforehand. Is this what you meant?

This tool is invoked via the typical Git setup process, which means that in some cases, it might not be able to launch - for example, if a relevant config file is unreadable. In this kind of scenario, it may be helpful to manually gather the kind of information listed above when manually asking for help.

Since I saw lots of replies from Junio elsewhere in this thread, I'll take the silence here as assent. v8 on its way momentarily.

- Emily

#220 lore raw
Quoted 20 lines · Emily Shaffer
Quoted 19 lines · Emily Shaffer
Quoted 11 lines · Emily Shaffer
Quoted 9 lines · Junio C Hamano

I am actually OK if we limit the use of this tool to "use with a repository that is not corrupt", as coping with these kinds of breakages that in the main Git executable we deem "needs manual intervention" inside a single process is too painful (it would have been easier to cope with these too if we stuck with a script that invokes many discrete commands and acts on their errors, but that is optimizing for rare case and not recommended). But we should tell users about the limitation and encourage them to ask for help in non automatable means.

Quoted 2 lines

I think you're saying, "Mention this drawback in the manpage for git-bugreport." Sounds like a good idea to me, so I'll add it for v8.

I'm pretty unsure about how you wanted this to sound; rather than sending a v8 only to revise it a lot, I'll send the paragraph for wordsmithing beforehand. Is this what you meant?

Quoted 7 lines
Quoted 4 lines

This tool is invoked via the typical Git setup process, which means that in some cases, it might not be able to launch - for example, if a relevant config file is unreadable. In this kind of scenario, it may be helpful to manually gather the kind of information listed above when manually asking for help.

Since I saw lots of replies from Junio elsewhere in this thread, I'll take the silence here as assent. v8 on its way momentarily.

Heh, it's just I have too little time to allocate on this single topic X-<. Don't take silence as anything else.

#221 lore raw
In reply to Emily Shaffer

Hi Emily,

Quoted 89 lines · Emily Shaffer
bugreport.c new file +83 -0
new file mode 100644
index 0000000000..a9398e6a2a
@@ -0,0 +1,94 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to share.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+	char *option_suffix = "%F-%H%M";
+	struct stat statbuf;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output-directory", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_STRING('s', "suffix", &option_suffix, N_("format"),
+			   N_("specify a strftime format suffix for the filename")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+	if (!stat(report_path.buf, &statbuf))
+		die("'%s' already exists", report_path.buf);
+
+	switch (safe_create_leading_directories(report_path.buf)) {
+	case SCLD_OK:
+	case SCLD_EXISTS:
+		break;
+	default:
+		die(_("could not create leading directories for '%s'"),
+		    report_path.buf);
+	}
+
+	get_bug_template(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+
+	if (report == NULL) {
+		strbuf_release(&report_path);
+		die("couldn't open '%s' for writing", report_path.buf);

bugreport.c line 93

Quoted 10 lines
bugreport.c new file +10 -0
+	}
+
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
+
+	UNLEAK(buffer);
+	UNLEAK(report_path);
+	return -launch_editor(report_path.buf, NULL, NULL);

This would be the first time (at least that _I_ know of) that we use - in this way. We seem to use !! a lot more often. And now I wonder whether there is a reason for that - that I missed?

Ciao,
Dscho

#222 lore raw
Quoted 12 lines · Johannes Schindelin
Quoted 12 lines
bugreport.c new file +12 -0
+int cmd_main(int argc, const char **argv)
+{
+...
+	if (report == NULL) {
+		strbuf_release(&report_path);
+		die("couldn't open '%s' for writing", report_path.buf);
+	}
+
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
Quoted 8 lines

bugreport.c line 93

Quoted 4 lines
bugreport.c new file +4 -0
+
+	UNLEAK(buffer);
+	UNLEAK(report_path);
+	return -launch_editor(report_path.buf, NULL, NULL);

This would be the first time (at least that _I_ know of) that we use - in this way. We seem to use !! a lot more often. And now I wonder whether there is a reason for that - that I missed?

In general, our preferred way to report an error from API functions is to return a negative number and a success is reported by returning zero.

The argument to exit(3), which the return value from the main() function essentially is, on the other hand, is expected to be a small non-negative integer. As long as we are in tight control of the range of the returned value from launch_editor() (i.e. it must return a small non-positive integer whose negation is suitable to be fed to exit(3)), the above is fine.

The idiom "return !!fn();" is to canonicalize the value to 0 or 1 for the caller who asked "so, did fn() give us 0 or non-zero?", which can also be used as the value given to exit(3), and could be more appropriate under some conditions:

- we MUST know launch_editor() returns zero if and only if it is successful.

- we do not have to be confident to be in tight control of the range of the returned value from launch_editor() (e.g. it could return a positive upon an error).

- we MUST NOT care to differenciate different error codes returned from launch_editor(). IOW, we must be fine to give the invoker of the program only 0 (success) or 1 (unspecified failure).

Use of "return !!fn();" that is not to give to exit(3) is probably more common in our codebase. See apply.c::try_create_file() and the comment before the function about its possible return values for example.

#223 lore raw
Quoted 34 lines · Junio C Hamano
Quoted 12 lines · Johannes Schindelin
Quoted 12 lines
bugreport.c new file +12 -0
+int cmd_main(int argc, const char **argv)
+{
+...
+	if (report == NULL) {
+		strbuf_release(&report_path);
+		die("couldn't open '%s' for writing", report_path.buf);
+	}
+
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
Quoted 8 lines

bugreport.c line 93

Quoted 4 lines
bugreport.c new file +4 -0
+
+	UNLEAK(buffer);
+	UNLEAK(report_path);
+	return -launch_editor(report_path.buf, NULL, NULL);

This would be the first time (at least that _I_ know of) that we use - in this way. We seem to use !! a lot more often. And now I wonder whether there is a reason for that - that I missed?

In general, our preferred way to report an error from API functions is to return a negative number and a success is reported by returning zero.

The argument to exit(3), which the return value from the main() function essentially is, on the other hand, is expected to be a small non-negative integer. As long as we are in tight control of the range of the returned value from launch_editor() (i.e. it must return a small non-positive integer whose negation is suitable to be fed to exit(3)), the above is fine.

Quoted 4 lines

The idiom "return !!fn();" is to canonicalize the value to 0 or 1 for the caller who asked "so, did fn() give us 0 or non-zero?", which can also be used as the value given to exit(3), and could be more appropriate under some conditions:

In editor.c, launch_editor() returns launch_specified_editor() without altering the return code.

Quoted 3 lines

- we MUST know launch_editor() returns zero if and only if it is successful.

launch_specified_editor() has a handful of exit points, of three kinds:
1. return error(something)
2. raise(sigsomething)
3. return 0
a. when the editor process closed happily, but the user supplied
NULL instead of a buffer. That is, the user didn't want the contents of the editor given back to them in a strbuf.
b. when the editor process closed happily and the user's supplied
buffer was filled with the file's contents with no issue.

So I think we can check "yes" here.

Quoted 4 lines

- we do not have to be confident to be in tight control of the range of the returned value from launch_editor() (e.g. it could return a positive upon an error).

According to 'man 3 raise' (POSIX), raise() exits the current thread. If it can't exit itself(?) it seems it returns "nonzero for failure". We've also got a mingw_raise() in compat/mingw.h which could be used instead; this one seems to return -1 or the result of raise(), presumably the one from the MSC runtime[1], which also returns "a nonzero value" if not successful.

So it's true that we aren't confident that launch_editor() returns a positive or negative value.

Quoted 3 lines

- we MUST NOT care to differenciate different error codes returned from launch_editor(). IOW, we must be fine to give the invoker of the program only 0 (success) or 1 (unspecified failure).

This part is a little sticking point for me; I think I'd rather give the user some hint of what's broken than no hint at all, especially in this scenario, where it's conceivable someone could say "I tried to run 'git-bugreport' and it crashed, here is the return code and I have manually collected some relevant info too".

The uncertainty coming from raise() (POSIX and MSCR both) leads me to believe if I do want to return the error on exit, I should at least check the sign before I flip it. Anybody else have opinions?

- Emily

[1] https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/raise?view=vs-2019

#224 lore raw
Quoted 11 lines · Emily Shaffer

launch_specified_editor() has a handful of exit points, of three kinds:
1. return error(something)
2. raise(sigsomething)
3. return 0
a. when the editor process closed happily, but the user supplied
NULL instead of a buffer. That is, the user didn't want the contents of the editor given back to them in a strbuf.
b. when the editor process closed happily and the user's supplied
buffer was filled with the file's contents with no issue.

So I think we can check "yes" here.

Heh. If we raised a signal to kill ourselves, then we won't be returning a value from launch_editor() anyway. That case won't affect the "between returning negation or !!, which is more appropriate?" discussion, I think.

Quoted 3 lines
Quoted 3 lines

- we MUST NOT care to differenciate different error codes returned from launch_editor(). IOW, we must be fine to give the invoker of the program only 0 (success) or 1 (unspecified failure).

I actually think this holds for the codepath. A failure from start_command() returns error(), and finish_command() that waits for the spawned editor process to complete yields the exit status from the editor, but unless we re-raise the signal that killed the editor process to ourselves, we just turn any non-zero exit into "return error()", so it is safe to say launch_editor() can return either 0 or -1 and nothing else. Would we later want to tell callers of launch_editor() how/why the editor session failed by returning something else? I do not offhand think of any---we do not even differenciate between failure to start (e.g. misspelt command name for the editor) and other failures WITH the return value and consider it sufficient to tell the user with different error message right now.

So in practice returning -launch_editor() and !!launch_editor() would not make any difference, I would think.

#225 lore raw
Quoted 37 lines · Junio C Hamano
Quoted 11 lines · Emily Shaffer

launch_specified_editor() has a handful of exit points, of three kinds:
1. return error(something)
2. raise(sigsomething)
3. return 0
a. when the editor process closed happily, but the user supplied
NULL instead of a buffer. That is, the user didn't want the contents of the editor given back to them in a strbuf.
b. when the editor process closed happily and the user's supplied
buffer was filled with the file's contents with no issue.

So I think we can check "yes" here.

Heh. If we raised a signal to kill ourselves, then we won't be returning a value from launch_editor() anyway. That case won't affect the "between returning negation or !!, which is more appropriate?" discussion, I think.

Quoted 3 lines
Quoted 3 lines

- we MUST NOT care to differenciate different error codes returned from launch_editor(). IOW, we must be fine to give the invoker of the program only 0 (success) or 1 (unspecified failure).

I actually think this holds for the codepath. A failure from start_command() returns error(), and finish_command() that waits for the spawned editor process to complete yields the exit status from the editor, but unless we re-raise the signal that killed the editor process to ourselves, we just turn any non-zero exit into "return error()", so it is safe to say launch_editor() can return either 0 or -1 and nothing else. Would we later want to tell callers of launch_editor() how/why the editor session failed by returning something else? I do not offhand think of any---we do not even differenciate between failure to start (e.g. misspelt command name for the editor) and other failures WITH the return value and consider it sufficient to tell the user with different error message right now.

Quoted 2 lines

So in practice returning -launch_editor() and !!launch_editor() would not make any difference, I would think.

Then, let's do the least surprising thing. I'll switch it to !! for the next reroll.

#226 lore raw

PATCH v7 4/15 bugreport: gather git version and build info

In reply to Emily Shaffer

Knowing which version of Git a user has and how it was built allows us to more precisely pin down the circumstances when a certain issue occurs, so teach bugreport how to tell us the same output as 'git version --build-options'.

It's not ideal to directly call 'git version --build-options' because that output goes to stdout. Instead, wrap the version string in a helper within help.[ch] library, and call that helper from within the bugreport library.

---
 Documentation/git-bugreport.txt |  4 +++
 bugreport.c                     | 19 ++++++++++++-
 help.c                          | 48 ++++++++++++++++++++-------------
 help.h                          |  1 +
 4 files changed, 52 insertions(+), 20 deletions(-)
Documentation/git-bugreport.txt +4 -0
index 52d49ed7aa..6ce2ae2eff 100644
@@ -23,6 +23,10 @@ The following information is requested from the user:
  - Expected behavior
  - Actual behavior
 
+The following information is captured automatically:
+
+ - 'git version --build-options'
+
 OPTIONS
 -------
 -o <path>::
bugreport.c +18 -1
index a9398e6a2a..b7504fe322 100644
@@ -1,8 +1,17 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(sys_info, 1);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
@@ -32,6 +41,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -76,6 +90,9 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
help.c +29 -19

63 lines not on this page · raw message

help.h +1 -0
index 9071894e8c..500521b908 100644
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void get_version_info(struct strbuf *buf, int show_build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#227 lore raw

PATCH v7 5/15 bugreport: add uname info

In reply to Emily Shaffer

The contents of uname() can give us some insight into what sort of system the user is running on, and help us replicate their setup if need be. The domainname field is not guaranteed to be available, so don't collect it.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 13 +++++++++++++
 2 files changed, 14 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 6ce2ae2eff..4dd72c60f5 100644
@@ -26,6 +26,7 @@ The following information is requested from the user:
 The following information is captured automatically:
 
  - 'git version --build-options'
+ - uname sysname, release, version, and machine strings
 
 OPTIONS
 -------
bugreport.c +13 -0
index b7504fe322..b76a1dfb2a 100644
@@ -7,10 +7,23 @@
 
 static void get_system_info(struct strbuf *sys_info)
 {
+	struct utsname uname_info;
+
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	get_version_info(sys_info, 1);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#228 lore raw

PATCH v7 6/15 bugreport: add compiler info

In reply to Emily Shaffer

To help pinpoint the source of a regression, it is useful to know some info about the compiler which the user's Git client was built with. By adding a generic get_compiler_info() in 'compat/' we can choose which relevant information to share per compiler; to get started, let's demonstrate the version of glibc if the user built with 'gcc'.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     |  5 +++++
 compat/compiler.h               | 24 ++++++++++++++++++++++++
 3 files changed, 30 insertions(+)
 create mode 100644 compat/compiler.h
Documentation/git-bugreport.txt +1 -0
index 4dd72c60f5..8bbc4c960c 100644
@@ -27,6 +27,7 @@ The following information is captured automatically:
 
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
+ - Compiler-specific info string
 
 OPTIONS
 -------
bugreport.c +5 -0
index b76a1dfb2a..4f9101caeb 100644
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include "compat/compiler.h"
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -24,6 +25,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
compat/compiler.h new file +24 -0
new file mode 100644
index 0000000000..bda5098e1b
@@ -0,0 +1,24 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
+}
+
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addstr(info, "get_compiler_info() not implemented");
+}
+
+#endif
+
+#endif /* COMPILER_H */
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#229 lore raw

Re: [PATCH v7 06/15] bugreport: add compiler info

Hi Emily,

Quoted 5 lines · Emily Shaffer

To help pinpoint the source of a regression, it is useful to know some info about the compiler which the user's Git client was built with. By adding a generic get_compiler_info() in 'compat/' we can choose which relevant information to share per compiler; to get started, let's demonstrate the version of glibc if the user built with 'gcc'.

I agree with the need for the compiler information, but in the patch I only see information about glibc being printed out. Shouldn't we use __GNUC__ and __GNUC_MINOR__ here?

Don't get me wrong, the glibc version is good and all, but the compiler information might be even more crucial. Git for Windows had to hold back compiling with GCC v8.x for a while, for example, because the stacksmasher was broken. Similar issues are not unheard of, and could help pinpoint compiler-related problems a lot quicker.

Thanks,
Dscho

Quoted 78 lines
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     |  5 +++++
 compat/compiler.h               | 24 ++++++++++++++++++++++++
 3 files changed, 30 insertions(+)
 create mode 100644 compat/compiler.h
Documentation/git-bugreport.txt +1 -0
index 4dd72c60f5..8bbc4c960c 100644
@@ -27,6 +27,7 @@ The following information is captured automatically:
 
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
+ - Compiler-specific info string
 
 OPTIONS
 -------
bugreport.c +5 -0
index b76a1dfb2a..4f9101caeb 100644
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include "compat/compiler.h"
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -24,6 +25,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
compat/compiler.h new file +24 -1
new file mode 100644
index 0000000000..bda5098e1b
@@ -0,0 +1,24 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
+}
+
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addstr(info, "get_compiler_info() not implemented");
+}
+
+#endif
+
+#endif /* COMPILER_H */
--

2.25.0.265.gbab2e86ba0-goog

#230 lore raw

Re: [PATCH v7 06/15] bugreport: add compiler info

Quoted 14 lines · Johannes Schindelin

Hi Emily,

Quoted 5 lines · Emily Shaffer

To help pinpoint the source of a regression, it is useful to know some info about the compiler which the user's Git client was built with. By adding a generic get_compiler_info() in 'compat/' we can choose which relevant information to share per compiler; to get started, let's demonstrate the version of glibc if the user built with 'gcc'.

I agree with the need for the compiler information, but in the patch I only see information about glibc being printed out. Shouldn't we use __GNUC__ and __GNUC_MINOR__ here?

Quoted 5 lines

Don't get me wrong, the glibc version is good and all, but the compiler information might be even more crucial. Git for Windows had to hold back compiling with GCC v8.x for a while, for example, because the stacksmasher was broken. Similar issues are not unheard of, and could help pinpoint compiler-related problems a lot quicker.

Hm, sure. Good point - thanks.

This does make me start to wonder, though - does it really make sense to have ifdef-gated redefinitions of the whole get_compiler_info() method like I do now? I wonder if it makes more sense to have only one definition, so we can write down everything we know regardless of which pieces are put together. My thinking is something like this - what if I am using glibc, but not a GNU compiler? (The GNU docs on __GCC__ indicate this is a situation that might occur - "a non-GCC compiler that claims to accept the GNU C dialects") Is there some striking reason not to implement this compat command thusly instead:

  #ifdef __GLIBC__
  #include <gnu/libc-version.h>
  #endif

static inline void get_compiler_info(struct strbuf *info) {

  	#ifdef __GLIBC__
	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
	#endif

	#ifdef __GNUC__
	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
	#endif

	#ifdef _MSC_VER
	strbuf_addf(info, "msc runtime: %s\n", some_msc_info());
	#endif

}

The thinking being - this way if I decide to use, say, LLVM + glibc, then I don't need to reimplement this command with all the glibc diagnostics again. Or, if someone else already wrote diagnostics for LLVM with some other libc, then it even Just Works for me and my new combination.

That said, I'm reasoning about these combinations of compilers and libcs and whatever else from an inexperienced viewpoint, so maybe this isn't necessary?

- Emily

#231 lore raw

Re: [PATCH v7 06/15] bugreport: add compiler info

Hi Emily,

Quoted 6 lines · Emily Shaffer
  #ifdef __GLIBC__
  #include <gnu/libc-version.h>
  #endif

static inline void get_compiler_info(struct strbuf *info) {

Quoted 10 lines
  	#ifdef __GLIBC__
	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
	#endif

	#ifdef __GNUC__
	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
	#endif

	#ifdef _MSC_VER
	strbuf_addf(info, "msc runtime: %s\n", some_msc_info());

You could do it this way right away:

	strbuf_addf(info, "MSVC version: %d.%d\n",
		    _MSC_VER / 100, _MSC_VER % 100);

Note: this is _not_ the MSVC _runtime_ version, but really the compiler version.

You could also use _MSC_FULL_VER, which is a bit more complete.

Quoted 9 lines
	#endif

}

The thinking being - this way if I decide to use, say, LLVM + glibc, then I don't need to reimplement this command with all the glibc diagnostics again. Or, if someone else already wrote diagnostics for LLVM with some other libc, then it even Just Works for me and my new combination.

Quoted 3 lines

That said, I'm reasoning about these combinations of compilers and libcs and whatever else from an inexperienced viewpoint, so maybe this isn't necessary?

I would have hoped that the argument I made earlier about the broken GCC version would have convinced you that the answer to this question is: Yes, this is necessary.

Ciao,
Dscho

Quoted 3 lines

- Emily

#232 lore raw

Re: [PATCH v7 06/15] bugreport: add compiler info

Quoted 23 lines · Johannes Schindelin

Hi Emily,

Quoted 6 lines · Emily Shaffer
  #ifdef __GLIBC__
  #include <gnu/libc-version.h>
  #endif

static inline void get_compiler_info(struct strbuf *info) {

Quoted 10 lines
  	#ifdef __GLIBC__
	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
	#endif

	#ifdef __GNUC__
	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
	#endif

	#ifdef _MSC_VER
	strbuf_addf(info, "msc runtime: %s\n", some_msc_info());

You could do it this way right away:

Quoted 7 lines
	strbuf_addf(info, "MSVC version: %d.%d\n",
		    _MSC_VER / 100, _MSC_VER % 100);

Note: this is _not_ the MSVC _runtime_ version, but really the compiler version.

You could also use _MSC_FULL_VER, which is a bit more complete.

Sorry, but I'm not comfortable sending code I can't check for myself (and already muscle-memoried into my format-patch/send-email workflow). If you send a scissors I can roll it into the series with your SOB.

- Emily

#233 lore raw

Re: [PATCH v7 06/15] bugreport: add compiler info

Hi Emily,

Quoted 27 lines · Emily Shaffer
Quoted 26 lines · Johannes Schindelin

Hi Emily,

Quoted 6 lines · Emily Shaffer
  #ifdef __GLIBC__
  #include <gnu/libc-version.h>
  #endif

static inline void get_compiler_info(struct strbuf *info) {

Quoted 10 lines
  	#ifdef __GLIBC__
	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
	#endif

	#ifdef __GNUC__
	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
	#endif

	#ifdef _MSC_VER
	strbuf_addf(info, "msc runtime: %s\n", some_msc_info());

You could do it this way right away:

	strbuf_addf(info, "MSVC version: %d.%d\n",
		    _MSC_VER / 100, _MSC_VER % 100);
Quoted 8 lines
Quoted 4 lines

Note: this is _not_ the MSVC _runtime_ version, but really the compiler version.

You could also use _MSC_FULL_VER, which is a bit more complete.

Sorry, but I'm not comfortable sending code I can't check for myself (and already muscle-memoried into my format-patch/send-email workflow). If you send a scissors I can roll it into the series with your SOB.

But you can check it yourself! I worked _really_ hard on that Azure Pipeline backing the PR builds at https://github.com/git/git. _REALLY_ hard. You might just as well reap the benefits so that I did not spend all of that time and sweat and stress in vain...

Ciao,
Dscho

#234 lore raw

Re: [PATCH v7 06/15] bugreport: add compiler info

Quoted 36 lines · Johannes Schindelin

Hi Emily,

Quoted 32 lines · Emily Shaffer
Quoted 30 lines · Johannes Schindelin

Hi Emily,

Quoted 6 lines · Emily Shaffer
  #ifdef __GLIBC__
  #include <gnu/libc-version.h>
  #endif

static inline void get_compiler_info(struct strbuf *info) {

Quoted 10 lines
  	#ifdef __GLIBC__
	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
	#endif

	#ifdef __GNUC__
	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
	#endif

	#ifdef _MSC_VER
	strbuf_addf(info, "msc runtime: %s\n", some_msc_info());

You could do it this way right away:

	strbuf_addf(info, "MSVC version: %d.%d\n",
		    _MSC_VER / 100, _MSC_VER % 100);

Note: this is _not_ the MSVC _runtime_ version, but really the compiler version.

You could also use _MSC_FULL_VER, which is a bit more complete.

Quoted 8 lines
Quoted 3 lines

Sorry, but I'm not comfortable sending code I can't check for myself (and already muscle-memoried into my format-patch/send-email workflow). If you send a scissors I can roll it into the series with your SOB.

But you can check it yourself! I worked _really_ hard on that Azure Pipeline backing the PR builds at https://github.com/git/git. _REALLY_ hard. You might just as well reap the benefits so that I did not spend all of that time and sweat and stress in vain...

I thought a bit about this. From your Github-using point of view, "just check my Pipeline" sounds like "just look at one more thing". From my format-patch using point of view, "just check my Pipeline" sounds like "ugh, I have to add this remote again... I don't have a fork already? How do I make that? Or is my fork just 6 months behind? How do I open a PR again? Yeesh."

So I did what any good developer who's supposed to be writing semi-annual performance reviews would, and wrote a first pass at script with this brand new fancy 'gh' thing (which is, by the way, clearly not intended for scripting): https://github.com/nasamuffin/quimby. Maybe someone will find it useful.

I'll try and add your MSC snippet for the next rollup.

- Emily

P.S. You can thank Jonathan Nieder for the name, as I asked for ideas and he said "Wikipedia tells me Chief Quimby gives Inspector Gadget something to read that later self-destructs." :P

#235 lore raw

Re: [PATCH v7 06/15] bugreport: add compiler info

Quoted 9 lines · Emily Shaffer
Quoted 8 lines
Quoted 3 lines

Sorry, but I'm not comfortable sending code I can't check for myself (and already muscle-memoried into my format-patch/send-email workflow). If you send a scissors I can roll it into the series with your SOB.

But you can check it yourself! I worked _really_ hard on that Azure Pipeline backing the PR builds at https://github.com/git/git. _REALLY_ hard. You might just as well reap the benefits so that I did not spend all of that time and sweat and stress in vain...

Quoted 6 lines

I thought a bit about this. From your Github-using point of view, "just check my Pipeline" sounds like "just look at one more thing". From my format-patch using point of view, "just check my Pipeline" sounds like "ugh, I have to add this remote again... I don't have a fork already? How do I make that? Or is my fork just 6 months behind? How do I open a PR again? Yeesh."

Sorry, but this is how a typical conversation between two techies who are more intelligent than good for their social skills ;-) Each side is not extending its hand enough to reach the other side, expecting that what s/he knows should be obvious to the others.

Dscho may be frustrated for you not being aware of what he worked on, but "I worked really hard" is much less useful thing for others, who may benefit from what he worked hard on, to hear, than "here is one pager that tells you how to use it". Quite honestly, reading the above exchange from the sidelines, it is not clear to me if it is a part of, or a completely separate from, the service offered by GGG from the end-users' point of view.

Can some of you come up with a one-pager to cover the following (and related but not listed) areas, that can be added as an appendix to Documentation/SubmittingPatches? It does not have to cover the common stuff like style guidelines, writing good log messages, need for test coverage, choice of fork point, etc. "Now you have a patch or a series of patches, but you obviously do not have direct and personal access to all the platforms Git supports. But there are servies to help you test them on the more common ones."

- How to cause TravisCI to run our test suite with your modification.

- How to trigger the same on Azure Pipeline PR builds.

There may be others.

Thanks.

#236 lore raw

PATCH v7 7/15 bugreport: add git-remote-https version

In reply to Emily Shaffer

It's possible for git-remote-curl to be built separately from git; in that case we want to know what version of cURL is used by git-remote-curl, not necessarily which version was present at git-bugreport's build time. So instead, ask git-remote-curl for the version information it knows about.

Today, "git-remote-http" and "git-remote-https" are aliased to "git-remote-curl"; but in case we rely on a different library than cURL in the future, let's not explicitly reference cURL from bugreport.

For longevity purposes, invoke the alias "git-remote-https" instead of "git-remote-http".

Since it could have been built at a different time, also report the version and built-from commit of git-remote-curl alongside the cURL info.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 16 ++++++++++++++++
 remote-curl.c                   |  8 ++++++++
 3 files changed, 25 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 8bbc4c960c..33df4dec7f 100644
@@ -28,6 +28,7 @@ The following information is captured automatically:
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
+ - 'git remote-https --build-info'
 
 OPTIONS
 -------
bugreport.c +16 -0
index 4f9101caeb..bfdff33368 100644
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include "compat/compiler.h"
+#include "run-command.h"
+
+static void get_git_remote_https_version_info(struct strbuf *version_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, version_info, 0))
+	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -29,6 +41,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "compiler info: ");
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+	get_git_remote_https_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
remote-curl.c +8 -0
index 350d92a074..c590fbfae3 100644
@@ -17,6 +17,7 @@
 #include "protocol.h"
 #include "quote.h"
 #include "transport.h"
+#include "version.h"
 
 static struct remote *remote;
 /* always ends with a trailing slash */
@@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {
+		printf("git-http-fetch version: %s\n", git_version_string);
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#237 lore raw

Re: [PATCH v7 07/15] bugreport: add git-remote-https version

Hi Emily,

Quoted 8 lines · Emily Shaffer
bugreport.c +0 -0
index 4f9101caeb..bfdff33368 100644
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include "compat/compiler.h"

bugreport.c line 17

Quoted 10 lines
bugreport.c +10 -0
+#include "run-command.h"
+
+static void get_git_remote_https_version_info(struct strbuf *version_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, version_info, 0))

Let's use RUN_GIT_CMD instead of adding "git" explicitly; It documents that we're interested in a Git command, and if we ever build a single-binary version of Git (as some Git for Windows users already asked for), it will make things easier.

Quoted 10 lines
bugreport.c +2 -0
+	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
remote-curl.c +0 -0
index 350d92a074..c590fbfae3 100644

remote-curl.c int cmd_main(int argc, const char **argv) · line 1378

Quoted 4 lines
remote-curl.c +1 -0
@@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {

The context does not say this, but at this point, we already verified that argc is larger than 1. Good.

Also, in keeping with the existing code, we would need to use --build-options here (this is what git version calls the equivalent mode).

_However_.

I like your --build-info a lot more than --build-options (because the latter is very misleading: the commit and the date of the build are not "options" at all).

Thanks,
Dscho

Quoted 13 lines
remote-curl.c +6 -1
+		printf("git-http-fetch version: %s\n", git_version_string);
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they
--

2.25.0.265.gbab2e86ba0-goog

#238 lore raw

Re: [PATCH v7 07/15] bugreport: add git-remote-https version

Quoted 19 lines · Johannes Schindelin

Hi Emily,

Quoted 15 lines · Emily Shaffer
bugreport.c +7 -0
index 4f9101caeb..bfdff33368 100644
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include "compat/compiler.h"
+#include "run-command.h"
+
+static void get_git_remote_https_version_info(struct strbuf *version_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
Quoted 8 lines

bugreport.c line 17

Quoted 3 lines
bugreport.c +3 -0
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, version_info, 0))

Let's use RUN_GIT_CMD instead of adding "git" explicitly; It documents that we're interested in a Git command, and if we ever build a single-binary version of Git (as some Git for Windows users already asked for), it will make things easier.

Hm. RUN_GIT_CMD is an argument used for the run_command_v_opt* family of calls, but it seems that setting child_process.git_cmd has the same effect. Done.

Quoted 23 lines
Quoted 11 lines
bugreport.c +2 -0
+	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
remote-curl.c +0 -0
index 350d92a074..c590fbfae3 100644

remote-curl.c int cmd_main(int argc, const char **argv) · line 1378

Quoted 4 lines
remote-curl.c +1 -0
@@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {

The context does not say this, but at this point, we already verified that argc is larger than 1. Good.

Also, in keeping with the existing code, we would need to use --build-options here (this is what git version calls the equivalent mode).

Quoted 5 lines

_However_.

I like your --build-info a lot more than --build-options (because the latter is very misleading: the commit and the date of the build are not "options" at all).

Sure. Thanks for saying so.

- Emily

#239 lore raw

Re: [PATCH v7 07/15] bugreport: add git-remote-https version

Quoted 11 lines · Emily Shaffer
Quoted 9 lines

Also, in keeping with the existing code, we would need to use --build-options here (this is what git version calls the equivalent mode).

_However_.

I like your --build-info a lot more than --build-options (because the latter is very misleading: the commit and the date of the build are not "options" at all).

Sure. Thanks for saying so.

I don't think anybody would mind introducing --build-info to "git version" as a synonym and deprecate --build-options from it ;-)

#240 lore raw

Re: [PATCH v7 07/15] bugreport: add git-remote-https version

Hi Junio,

Quoted 8 lines · Junio C Hamano
Quoted 6 lines · Emily Shaffer
Quoted 6 lines

Also, in keeping with the existing code, we would need to use --build-options here (this is what git version calls the equivalent mode).

_However_.

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

I like your --build-info a lot more than --build-options (because the latter is very misleading: the commit and the date of the build are not "options" at all).

Sure. Thanks for saying so.

I don't think anybody would mind introducing --build-info to "git version" as a synonym and deprecate --build-options from it ;-)

Heh. I almost suggested this. Almost.

Ciao,
Dscho

#241 lore raw

PATCH v7 8/15 bugreport: include user interactive shell

In reply to Emily Shaffer

It's possible a user may complain about the way that Git interacts with their interactive shell, e.g. autocompletion or shell prompt. In that case, it's useful for us to know which shell they're using interactively.

---
 Documentation/git-bugreport.txt | 1 +
 bugreport.c                     | 5 +++++
 2 files changed, 6 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 33df4dec7f..23265b0d74 100644
@@ -29,6 +29,7 @@ The following information is captured automatically:
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
  - 'git remote-https --build-info'
+ - $SHELL
 
 OPTIONS
 -------
bugreport.c +5 -0
index bfdff33368..c1475601bf 100644
@@ -21,6 +21,7 @@ static void get_git_remote_https_version_info(struct strbuf *version_info)
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct utsname uname_info;
+	char *shell = NULL;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
@@ -42,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
 
+	shell = getenv("SHELL");
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    shell ? shell : "<unset>");
+
 	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
 	get_git_remote_https_version_info(sys_info);
 	strbuf_complete_line(sys_info);
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#242 lore raw

PATCH v7 9/15 bugreport: generate config safelist based on docs

In reply to Emily Shaffer

Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "annotate" macro, which is a no-op in AsciiDoc and AsciiDoctor.

Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.

However, it's possible that in some situations, an organization may be less concerned with privacy of things like remote URLs and branch names, and more concerned with ease of diagnosis for their support staff. In those cases, it may make more sense for that organization to modify the code to use a blocklist. To that end, we should try to mark configs which are definitely safe, and configs which are definitely unsafe, and leave blank configs which are somewhere in between. To mark a config as safe, add "annotate:bugreport[include]" to the corresponding line in the config documentation; to mark it as unsafe, add "annotate:bugreport[exclude]" instead.

Generating bugreport-config-safelist.h at build time by grepping the documentation for this new macro helps us prevent staleness. The macro itself is a no-op and should not alter the appearance of the documentation in either AsciiDoc or AsciiDoctor, confirmable by running:

  cd Documentation
  ./doc-diff --asciidoctor HEAD^ HEAD
  ./doc-diff --asciidoc HEAD^ HEAD

Diffing the rendered HTML shows that only inline comments were added, which shouldn't be a problem.

Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.

---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  9 ++++
 Documentation/asciidoctor-extensions.rb |  7 ++++
 Documentation/config/sendemail.txt      | 56 ++++++++++++-------------
 Makefile                                |  7 ++++
 generate-bugreport-config-safelist.sh   | 18 ++++++++
 6 files changed, 70 insertions(+), 28 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh
.gitignore +1 -0
index d89bf9e11e..bd2f49b996 100644
@@ -192,6 +192,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
Documentation/asciidoc.conf +9 -0
index 8fc4b67081..663e06481f 100644
@@ -6,9 +6,14 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The annotate macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources to populate
+# things like the bugreport safelist.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>annotate):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +33,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[annotate-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +101,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[annotate-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
Documentation/asciidoctor-extensions.rb +7 -0
index d906a00803..03c80af0e5 100644
@@ -39,10 +39,17 @@ module Git
         output
       end
     end
+
+    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, target, attrs)
+        ""
+      end
+    end
   end
 end
 
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  inline_macro Git::Documentation::AnnotateProcessor, :annotate
 end
Documentation/config/sendemail.txt +28 -28

83 lines not on this page · raw message

Makefile +7 -0
index a01a050aa3..2bc9f112ea 100644
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2163,6 +2164,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
generate-bugreport-config-safelist.sh new file +18 -0
new file mode 100755
index 0000000000..2794a8ae77
@@ -0,0 +1,18 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the annotate-bugreport macro
+sed -n 's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/  "\1",/p' \
+	| sort
+
+cat <<EOF
+};
+EOF
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#243 lore raw

PATCH v7 10/15 bugreport: add config values from safelist

In reply to Emily Shaffer

Teach bugreport to gather the values of config options which are present in 'bugreport-config-safelist.h', and show their origin scope.

Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.

Rather than including the path to someone's config, which can reveal filesystem layout and project names, just name the scope (e.g. system, global, local) of the config source.

---
 Documentation/git-bugreport.txt |  1 +
 Makefile                        |  2 ++
 bugreport.c                     | 34 +++++++++++++++++++++++++++++++++
 3 files changed, 37 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 23265b0d74..4e9171d1bd 100644
@@ -30,6 +30,7 @@ The following information is captured automatically:
  - Compiler-specific info string
  - 'git remote-https --build-info'
  - $SHELL
+ - Selected config values
 
 OPTIONS
 -------
Makefile +2 -0
index 2bc9f112ea..7b139230ae 100644
@@ -2131,6 +2131,8 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
+bugreport.sp bugreport.s bugreport.o: bugreport-config-safelist.h
+
 builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
bugreport.c +34 -0
index c1475601bf..1a9430a857 100644
@@ -6,6 +6,9 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -52,6 +55,29 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	size_t idx;
+	struct string_list_item *it = NULL;
+	struct key_value_info *kv_info = NULL;
+
+	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
+		const struct string_list *list =
+			git_config_get_value_multi(bugreport_config_safelist[idx]);
+
+		if (!list)
+			continue;
+
+		strbuf_addf(config_info, "%s:\n", bugreport_config_safelist[idx]);
+		for_each_string_list_item(it, list) {
+			kv_info = it->util;
+			strbuf_addf(config_info, "  %s (%s)\n", it->string,
+				    kv_info ? config_scope_name(kv_info->scope)
+					    : "source unknown");
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -94,6 +120,7 @@ int cmd_main(int argc, const char **argv)
 	char *option_output = NULL;
 	char *option_suffix = "%F-%H%M";
 	struct stat statbuf;
+	int nongit_ok = 0;
 
 	const struct option bugreport_options[] = {
 		OPT_STRING('o', "output-directory", &option_output, N_("path"),
@@ -102,6 +129,10 @@ int cmd_main(int argc, const char **argv)
 			   N_("specify a strftime format suffix for the filename")),
 		OPT_END()
 	};
+
+	/* Prerequisite for hooks and config checks */
+	setup_git_directory_gently(&nongit_ok);
+
 	argc = parse_options(argc, argv, "", bugreport_options,
 			     bugreport_usage, 0);
 
@@ -132,6 +163,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#244 lore raw

PATCH v7 11/15 bugreport: collect list of populated hooks

In reply to Emily Shaffer

Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong.

Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot. To reduce the amount of code humans need to read, we turn the list into a string_list and iterate over it (as we are calling the same find_hook operation on each string). However, since bugreport should primarily be called by the user, the performance loss from massaging the string seems acceptable.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 53 +++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 4e9171d1bd..779c9c7fd9 100644
@@ -31,6 +31,7 @@ The following information is captured automatically:
  - 'git remote-https --build-info'
  - $SHELL
  - Selected config values
+ - A list of enabled hooks
 
 OPTIONS
 -------
bugreport.c +53 -0

71 lines not on this page · raw message

Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#245 lore raw

PATCH v7 12/15 bugreport: count loose objects

In reply to Emily Shaffer

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 52 +++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 779c9c7fd9..7e1ed44cb4 100644
@@ -32,6 +32,7 @@ The following information is captured automatically:
  - $SHELL
  - Selected config values
  - A list of enabled hooks
+ - The number of loose objects in the repository
 
 OPTIONS
 -------
bugreport.c +52 -0

70 lines not on this page · raw message

Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#246 lore raw

PATCH v7 13/15 bugreport: add packed object summary

In reply to Emily Shaffer

Alongside the loose object counts, it can be useful to show the number of packs and packed objects. This way we can check whether the repo has an appropriate ratio of packed to loose objects to help determine whether it's behaving correctly.

Add a utility to easily traverse all packfiles in a given repository. Use it in packfile.c and remove a redundant call to prepare_packed_git(), which is already called in get_all_packs().

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 30 ++++++++++++++++++++++++++++++
 object-store.h                  |  6 ++++++
 packfile.c                      |  3 +--
 4 files changed, 38 insertions(+), 2 deletions(-)
Documentation/git-bugreport.txt +1 -0
index 7e1ed44cb4..309b34fc42 100644
@@ -33,6 +33,7 @@ The following information is captured automatically:
  - Selected config values
  - A list of enabled hooks
  - The number of loose objects in the repository
+ - The number of packs and packed objects in the repository
 
 OPTIONS
 -------
bugreport.c +30 -0
index f5201ce416..580de13ddc 100644
@@ -177,6 +177,33 @@ static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
 		    total_count_questionable ? " (problem during count)" : "");
 }
 
+static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
+{
+	struct packed_git *pack = NULL;
+	int pack_count = 0;
+	int object_count = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	for_each_pack(the_repository, pack) {
+		pack_count++;
+		/*
+		 * To accurately count how many objects are packed, look inside
+		 * the packfile's index.
+		 */
+		open_pack_index(pack);
+		object_count += pack->num_objects;
+	}
+
+	strbuf_addf(obj_info, "%d total packs (%d objects)\n", pack_count,
+		    object_count);
+
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -271,6 +298,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
object-store.h +6 -0
index 55ee639350..518c73469d 100644
@@ -6,6 +6,7 @@
 #include "list.h"
 #include "sha1-array.h"
 #include "strbuf.h"
+#include "packfile.h"
 
 struct object_directory {
 	struct object_directory *next;
@@ -403,4 +404,9 @@ int for_each_object_in_pack(struct packed_git *p,
 int for_each_packed_object(each_packed_object_fn, void *,
 			   enum for_each_object_flags flags);
 
+#define for_each_pack(repo, pack) 		\
+		for (pack = get_all_packs(repo);\
+		     pack;			\
+		     pack = pack->next)
+
 #endif /* OBJECT_STORE_H */
packfile.c +1 -2
index 7e7c04e4d8..6873827fb5 100644
@@ -2061,8 +2061,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data,
 	int r = 0;
 	int pack_errors = 0;
 
-	prepare_packed_git(the_repository);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for_each_pack(the_repository, p) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#247 lore raw

PATCH v7 14/15 bugreport: list contents of $OBJDIR/info

In reply to Emily Shaffer

Miscellaneous information used about the object store can end up in .git/objects/info; this can help us understand what may be going on with the object store when the user is reporting a bug. Otherwise, it could be difficult to track down what is going wrong with an object which isn't kept locally to .git/objects/ or .git/objects/pack. Having some understanding of where the user's objects may be kept can save us some hops during the bug reporting process.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 54 +++++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 309b34fc42..a21d081616 100644
@@ -34,6 +34,7 @@ The following information is captured automatically:
  - A list of enabled hooks
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
+ - A list of the contents of .git/objects/info (or equivalent)
 
 OPTIONS
 -------
bugreport.c +54 -0

66 lines not on this page · raw message

Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#248 lore raw

Re: [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info

Quoted 7 lines · Emily Shaffer

Miscellaneous information used about the object store can end up in .git/objects/info; this can help us understand what may be going on with the object store when the user is reporting a bug. Otherwise, it could be difficult to track down what is going wrong with an object which isn't kept locally to .git/objects/ or .git/objects/pack. Having some understanding of where the user's objects may be kept can save us some hops during the bug reporting process.

This step seems to have a new whitespace breakage that did not exist in the previous round.

#249 lore raw

Re: [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info

Quoted 10 lines · Junio C Hamano
Quoted 7 lines · Emily Shaffer

Miscellaneous information used about the object store can end up in .git/objects/info; this can help us understand what may be going on with the object store when the user is reporting a bug. Otherwise, it could be difficult to track down what is going wrong with an object which isn't kept locally to .git/objects/ or .git/objects/pack. Having some understanding of where the user's objects may be kept can save us some hops during the bug reporting process.

Quoted 2 lines

This step seems to have a new whitespace breakage that did not exist in the previous round.

Hmph. I saw it warned at some point during the rebase-and-format-patch workflow for v7 but thought I had quashed it. Very sorry.

- Emily

#250 lore raw

PATCH v7 15/15 bugreport: summarize contents of alternates file

In reply to Emily Shaffer

In some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead. Since paths to alternates may be sensitive, we'll instead count how many alternates have been specified and note how many of them exist or are broken.

While object-cache.h describes a function "foreach_alt_odb()", this function does not provide information on broken alternates, which are skipped over in "link_alt_odb_entry()". Since the goal is to identify missing alternates, we can gather the contents of .git/objects/info/alternates manually.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 45 +++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)
Documentation/git-bugreport.txt +1 -0
index a21d081616..e870900c80 100644
@@ -35,6 +35,7 @@ The following information is captured automatically:
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
  - A list of the contents of .git/objects/info (or equivalent)
+ - The number of valid and invalid alternates
 
 OPTIONS
 -------
bugreport.c +45 -0

57 lines not on this page · raw message

Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#251 lore raw

Re: [PATCH v7 00/15] add git-bugreport tool

In reply to Emily Shaffer
Quoted 2 lines · Emily Shaffer

present(patch 9/15). This now uses POSIX character classes and {} notation instead of + and has been tested on OSX; I'd love to hear if

I'd rather not to see unnecessary uses of POSIX character classes.

The interdiff of this step between the previous and this round looks to me more like "I used it, just because POSIX says I *can* use it", not "I did so because I needed to do, and it should be OK on POSIX platforms."

Instead of overly long

's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/ "\1",/p'

just limiting ourselves to SP and saying

's/^\([^ ]*\) *annotate:bugreport\[include\].* ::$/ "\1",/p'

would keep the result much easier to read, I would think.

#252 lore raw

Re: [PATCH v7 00/15] add git-bugreport tool

Quoted 14 lines · Junio C Hamano
Quoted 2 lines · Emily Shaffer

present(patch 9/15). This now uses POSIX character classes and {} notation instead of + and has been tested on OSX; I'd love to hear if

I'd rather not to see unnecessary uses of POSIX character classes.

The interdiff of this step between the previous and this round looks to me more like "I used it, just because POSIX says I *can* use it", not "I did so because I needed to do, and it should be OK on POSIX platforms."

Instead of overly long

Quoted 7 lines

's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/ "\1",/p'

just limiting ourselves to SP and saying

's/^\([^ ]*\) *annotate:bugreport\[include\].* ::$/ "\1",/p'

would keep the result much easier to read, I would think.

That's fine by me. I find the [[:syntax:]] extremely ugly, but I was worried about whether that was more portable somehow. Your proposal seems fine to me.

- Emily

#253 lore raw

Re: [PATCH v7 00/15] add git-bugreport tool

Quoted 1 line · Emily Shaffer

That's fine by me. I find the [[:syntax:]] extremely ugly,...

FWIW, I find it so too.

#254 lore raw

v8 cover add git-bugreport tool

In reply to Emily Shaffer

No longer relies on any topic branches, since mr/show-config-scope is merged. Range-diff at the bottom of the cover letter.

Since v7:
- Dropped some POSIX regex magic in generate-bugreport-config-safelist.sh.
- Added some advice in doc for scenarios when git-bugreport may not run nicely
- Quit using unusual "return -foo(bar)" and start using "return !!foo(bar)"
- Add a test to ensure path creation works outside of a git dir
- Add some GNUC and GNUC_MINOR reporting
- Use git_cmd instead of hardcoding "git" as the first arg when invoking another process
- Fix some pesky whitespace

Thanks.
- Emily

Emily Shaffer (15):
help: move list_config_help to builtin/help
help: add shell-path to --build-options
bugreport: add tool to generate debugging info
bugreport: gather git version and build info
bugreport: add uname info
bugreport: add compiler info
bugreport: add git-remote-https version
bugreport: include user interactive shell
bugreport: generate config safelist based on docs
bugreport: add config values from safelist
bugreport: collect list of populated hooks
bugreport: count loose objects
bugreport: add packed object summary
bugreport: list contents of $OBJDIR/info
bugreport: summarize contents of alternates file

 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   9 +
 Documentation/asciidoctor-extensions.rb |   7 +
 Documentation/config/sendemail.txt      |  56 ++--
 Documentation/git-bugreport.txt         |  60 ++++
 Makefile                                |  27 +-
 bugreport.c                             | 418 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 command-list.txt                        |   1 +
 compat/compiler.h                       |  27 ++
 generate-bugreport-config-safelist.sh   |  18 +
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  21 ++
 help.c                                  | 132 ++------
 help.h                                  |   2 +-
 object-store.h                          |   6 +
 packfile.c                              |   3 +-
 remote-curl.c                           |   8 +
 t/t0091-bugreport.sh                    |  61 ++++
 19 files changed, 808 insertions(+), 156 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100644 compat/compiler.h
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh
 1:  429c7e093b =  1:  69f54afc6b help: move list_config_help to builtin/help
 2:  0cfb24f878 =  2:  9ea1c3ab34 help: add shell-path to --build-options
 3:  02bb151176 !  3:  8a290b5445 bugreport: add tool to generate debugging info
    @@ Documentation/git-bugreport.txt (new)
     + - Expected behavior
     + - Actual behavior
     +
    ++This tool is invoked via the typical Git setup process, which means that in some
    ++cases, it might not be able to launch - for example, if a relevant config file
    ++is unreadable. In this kind of scenario, it may be helpful to manually gather
    ++the kind of information listed above when manually asking for help.
    ++
     +OPTIONS
     +-------
     +-o <path>::
    @@ bugreport.c (new)
     +
     +	UNLEAK(buffer);
     +	UNLEAK(report_path);
    -+	return -launch_editor(report_path.buf, NULL, NULL);
    ++	return !!launch_editor(report_path.buf, NULL, NULL);
     +}
     
      ## command-list.txt ##
    @@ t/t0091-bugreport.sh (new)
     +	test_when_finished rm non-repo/git-bugreport-*
     +'
     +
    ++test_expect_success 'can create leading directories outside of a git dir' '
    ++	nongit git bugreport -o foo/bar/baz &&
    ++	test_when_finished rm -fr foo/bar/baz
    ++'
    ++
    ++
     +test_done

4: 463612e409 ! 4: 4eb4a7aef3 bugreport: gather git version and build info
@@ Documentation/git-bugreport.txt: The following information is requested from the
+
+ - 'git version --build-options'
+
- OPTIONS
- -------
- -o <path>::
+ This tool is invoked via the typical Git setup process, which means that in some
+ cases, it might not be able to launch - for example, if a relevant config file
+ is unreadable. In this kind of scenario, it may be helpful to manually gather

## bugreport.c ##
@@
5: 85e11ea125 ! 5: 4e18b4678e bugreport: add uname info
@@ Documentation/git-bugreport.txt: The following information is requested from the
- 'git version --build-options'
+ - uname sysname, release, version, and machine strings

- OPTIONS
- -------
+ This tool is invoked via the typical Git setup process, which means that in some
+ cases, it might not be able to launch - for example, if a relevant config file

## bugreport.c ##
@@
6: d9eafbaa48 ! 6: c18d17443d bugreport: add compiler info

    @@ Documentation/git-bugreport.txt: The following information is captured automatic
       - uname sysname, release, version, and machine strings
     + - Compiler-specific info string
      
    - OPTIONS
    - -------
    + This tool is invoked via the typical Git setup process, which means that in some
    + cases, it might not be able to launch - for example, if a relevant config file
     
      ## bugreport.c ##
     @@
    @@ compat/compiler.h (new)
     +
     +static inline void get_compiler_info(struct strbuf *info)
     +{
    -+	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
    ++	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
    ++#ifdef __GNUC__
    ++	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
    ++#endif
     +}
     +
     +#else
 7:  f5b6dd4174 !  7:  6d807cf857 bugreport: add git-remote-https version
    @@ Documentation/git-bugreport.txt: The following information is captured automatic
       - Compiler-specific info string
     + - 'git remote-https --build-info'
      
    - OPTIONS
    - -------
    + This tool is invoked via the typical Git setup process, which means that in some
    + cases, it might not be able to launch - for example, if a relevant config file
     
      ## bugreport.c ##
     @@
    @@ bugreport.c
     +{
     +	struct child_process cp = CHILD_PROCESS_INIT;
     +
    -+	argv_array_push(&cp.args, "git");
    ++	cp.git_cmd = 1;
     +	argv_array_push(&cp.args, "remote-https");
     +	argv_array_push(&cp.args, "--build-info");
     +	if (capture_command(&cp, version_info, 0))

8: 3119c9c1e3 ! 8: f85e1b69d0 bugreport: include user interactive shell
@@ Documentation/git-bugreport.txt: The following information is captured automatic
- 'git remote-https --build-info'
+ - $SHELL

- OPTIONS
- -------
+ This tool is invoked via the typical Git setup process, which means that in some
+ cases, it might not be able to launch - for example, if a relevant config file

## bugreport.c ##
@@ bugreport.c: static void get_git_remote_https_version_info(struct strbuf *version_info)
9: c35bab2735 ! 9: 9980728360 bugreport: generate config safelist based on docs

    @@ generate-bugreport-config-safelist.sh (new)
     +# cat all regular files in Documentation/config
     +find Documentation/config -type f -exec cat {} \; |
     +# print the command name which matches the annotate-bugreport macro
    -+sed -n 's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/  "\1",/p' \
    ++sed -n 's/^\([^ ]*\)  *annotate:bugreport\[include\].* ::$/  "\1",/p' \
     +	| sort
     +
     +cat <<EOF

10: 8a54ebdbbb ! 10: e9cd8ffec9 bugreport: add config values from safelist
@@ Documentation/git-bugreport.txt: The following information is captured automatic
- $SHELL
+ - Selected config values

- OPTIONS
- -------
+ This tool is invoked via the typical Git setup process, which means that in some
+ cases, it might not be able to launch - for example, if a relevant config file

## Makefile ##
@@ Makefile: git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
11: 8bb5bcd8b8 ! 11: 881b62369d bugreport: collect list of populated hooks
@@ Documentation/git-bugreport.txt: The following information is captured automatic
- Selected config values
+ - A list of enabled hooks

- OPTIONS
- -------
+ This tool is invoked via the typical Git setup process, which means that in some
+ cases, it might not be able to launch - for example, if a relevant config file

## bugreport.c ##
@@
12: 744dd6aeee ! 12: 715e48e9a7 bugreport: count loose objects
@@ Documentation/git-bugreport.txt: The following information is captured automatic
- A list of enabled hooks
+ - The number of loose objects in the repository

- OPTIONS
- -------
+ This tool is invoked via the typical Git setup process, which means that in some
+ cases, it might not be able to launch - for example, if a relevant config file

## bugreport.c ##
@@
13: 2f23e272bf ! 13: c9e2f1f1d8 bugreport: add packed object summary
@@ Documentation/git-bugreport.txt: The following information is captured automatic
- The number of loose objects in the repository
+ - The number of packs and packed objects in the repository

- OPTIONS
- -------
+ This tool is invoked via the typical Git setup process, which means that in some
+ cases, it might not be able to launch - for example, if a relevant config file

## bugreport.c ##
@@ bugreport.c: static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
@@ bugreport.c: int cmd_main(int argc, const char **argv)

## object-store.h ##
@@
- #include "list.h"
#include "sha1-array.h"
#include "strbuf.h"
+ #include "thread-utils.h"
+#include "packfile.h"

struct object_directory {
14: 7b5ae1aca1 ! 14: 89762a0de1 bugreport: list contents of $OBJDIR/info
@@ Documentation/git-bugreport.txt: The following information is captured automatic
- The number of packs and packed objects in the repository
+ - A list of the contents of .git/objects/info (or equivalent)

- OPTIONS
- -------
+ This tool is invoked via the typical Git setup process, which means that in some
+ cases, it might not be able to launch - for example, if a relevant config file

## bugreport.c ##
@@ bugreport.c: static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
@@ bugreport.c: static void get_packed_object_summary(struct strbuf *obj_info, int
}

+static void list_contents_of_dir_recursively(struct strbuf *contents,
-+ struct strbuf *dirpath)
++ struct strbuf *dirpath)
+{
+ struct dirent *d;
+ DIR *dir;
15: 315b00ed54 ! 15: aa81e197aa bugreport: summarize contents of alternates file
@@ Documentation/git-bugreport.txt: The following information is captured automatic
- A list of the contents of .git/objects/info (or equivalent)
+ - The number of valid and invalid alternates

- OPTIONS
- -------
+ This tool is invoked via the typical Git setup process, which means that in some
+ cases, it might not be able to launch - for example, if a relevant config file

## bugreport.c ##
@@ bugreport.c: static void get_object_info_summary(struct strbuf *obj_info, int nongit)

Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#255 lore raw

PATCH v8 1/15 help: move list_config_help to builtin/help

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o, builtin/clean.o, and builtin/config.o. This meant that help.o was unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present).

When command-list.h is included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule.

---
 .gitignore             |  1 +
 Makefile               | 13 +++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 21 +++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 118 insertions(+), 108 deletions(-)
 create mode 100755 generate-configlist.sh
.gitignore +1 -0
index aebe7c0908..ea97de83f3 100644
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
Makefile +10 -3
index 6134104ae6..c552312d3f 100644
@@ -815,6 +815,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2132,7 +2133,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2152,6 +2153,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2785,7 +2792,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2807,7 +2814,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
builtin/help.c +86 -0

98 lines not on this page · raw message

generate-cmdlist.sh +0 -19
index 71158f7d8b..45fecf8bdf 100755
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
generate-configlist.sh new file +21 -0
new file mode 100755
index 0000000000..8692fe5cf4
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	sed 's/^.*$/	"&",/'
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
help.c +0 -85

91 lines not on this page · raw message

help.h +0 -1
index 7a455beeb7..9071894e8c 100644
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#256 lore raw

PATCH v8 2/15 help: add shell-path to --build-options

In reply to Emily Shaffer

It may be useful to know which shell Git was built to try to point to, in the event that shell-based Git commands are failing. $SHELL_PATH is set during the build and used to launch the manpage viewer, as well as by git-compat-util.h, and it's used during tests. 'git version --build-options' is encouraged for use in bug reports, so it makes sense to include this information there.

---
 help.c | 1 +
 1 file changed, 1 insertion(+)
help.c +1 -0
index a21487db77..190722fb0a 100644
@@ -654,6 +654,7 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 			printf("no commit associated with this build\n");
 		printf("sizeof-long: %d\n", (int)sizeof(long));
 		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
+		printf("shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 	return 0;
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#257 lore raw

Re: [PATCH v8 02/15] help: add shell-path to --build-options

Quoted 6 lines · Emily Shaffer

It may be useful to know which shell Git was built to try to point to, in the event that shell-based Git commands are failing. $SHELL_PATH is set during the build and used to launch the manpage viewer, as well as by git-compat-util.h, and it's used during tests. 'git version --build-options' is encouraged for use in bug reports, so it makes sense to include this information there.

It probably makes sense to capture PERL_PATH, PYTHON_PATH, TCLTK_PATH, and TCL_PATH, as they fall into exactly the same category as this one.

Quoted 17 lines
---
 help.c | 1 +
 1 file changed, 1 insertion(+)
help.c +1 -0
index a21487db77..190722fb0a 100644
@@ -654,6 +654,7 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 			printf("no commit associated with this build\n");
 		printf("sizeof-long: %d\n", (int)sizeof(long));
 		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
+		printf("shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 	return 0;
#258 lore raw

Re: [PATCH v8 02/15] help: add shell-path to --build-options

Quoted 9 lines · Junio C Hamano
Quoted 6 lines · Emily Shaffer

It may be useful to know which shell Git was built to try to point to, in the event that shell-based Git commands are failing. $SHELL_PATH is set during the build and used to launch the manpage viewer, as well as by git-compat-util.h, and it's used during tests. 'git version --build-options' is encouraged for use in bug reports, so it makes sense to include this information there.

Quoted 3 lines

It probably makes sense to capture PERL_PATH, PYTHON_PATH, TCLTK_PATH, and TCL_PATH, as they fall into exactly the same category as this one.

SHELL_PATH was already made sure to exist in git-compat-util.h before this change; these other utility paths are not. Do you want me to add them now?

Quoted 18 lines
Quoted 18 lines
---
 help.c | 1 +
 1 file changed, 1 insertion(+)
help.c +1 -0
index a21487db77..190722fb0a 100644
@@ -654,6 +654,7 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 			printf("no commit associated with this build\n");
 		printf("sizeof-long: %d\n", (int)sizeof(long));
 		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
+		printf("shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 	return 0;
#259 lore raw

PATCH v8 3/15 bugreport: add tool to generate debugging info

In reply to Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.

Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.

---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 46 ++++++++++++++++
 Makefile                        |  5 ++
 bugreport.c                     | 94 +++++++++++++++++++++++++++++++++
 command-list.txt                |  1 +
 t/t0091-bugreport.sh            | 61 +++++++++++++++++++++
 6 files changed, 208 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh
.gitignore +1 -0
index ea97de83f3..d89bf9e11e 100644
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
Documentation/git-bugreport.txt new file +46 -0

46 lines not on this page · raw message

Makefile +5 -0
index c552312d3f..9e6705061d 100644
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2461,6 +2462,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
bugreport.c new file +94 -0

94 lines not on this page · raw message

command-list.txt +1 -0
index 2087894655..185e5e3f05 100644
@@ -54,6 +54,7 @@ git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
 git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
+git-bugreport                           ancillaryinterrogators
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
 git-check-attr                          purehelpers
t/t0091-bugreport.sh new file +61 -0

61 lines not on this page · raw message

Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#260 lore raw

bugreport.c line 53

Quoted 2 lines · Emily Shaffer
bugreport.c new file +2 -0
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);

Which one between an empty string and NULL is more appropriate to be passed as "prefix" here? I assume that this is *not* really a git program, and no repository/working-tree discovery is involved, and you won't be using OPT_FILENAME, so it would probably be OK.

bugreport.c line 66

Quoted 13 lines
bugreport.c new file +13 -0
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+	if (!stat(report_path.buf, &statbuf))
+		die("'%s' already exists", report_path.buf);

Missed i18n/l10n, but I do not think it is a useful thing for this check to be here in the first place.

bugreport.c line 79

Quoted 12 lines
bugreport.c new file +12 -0
+	switch (safe_create_leading_directories(report_path.buf)) {
+	case SCLD_OK:
+	case SCLD_EXISTS:
+		break;
+	default:
+		die(_("could not create leading directories for '%s'"),
+		    report_path.buf);
+	}
+
+	get_bug_template(&buffer);
+
+	report = fopen_for_writing(report_path.buf);

fopen_for_writing() does not give good semantics for what you seem to want to do here (i.e. to make sure you do not overwrite an existing one). It even has "if we cannot open it, retry after unlinking" logic in it, which screams "this function is designed for those who want to overwrite the file", and if you look at its callers, they are _all_ about opening a file for writing with a well known name like FETCH_HEAD, COMMIT_EDITMSG, etc.

Besides, a stat() that dies when a file exists would not help---since that check, you've spent non-zero time, creating leading directories and preparing boilerplate in the buffer, and there is no guarantee somebody else used the same filename in the meantime.

If you want to avoid overwriting an existing file (which I think is a good idea---I just do not think the earlier "if (!stat()) die()" is a good way to do so), the only sensible way to do so is to ask your open/creat to fail if there already is a file---that is how you'd avoid racing with another process that may be creating such a file. Grep for O_EXCL to find where the flag is used with O_CREAT to see how it is done.

bugreport.c line 81

Quoted 1 line
bugreport.c new file +1 -0
+	if (report == NULL) {

Style. "if (!report)"

bugreport.c line 83

Quoted 2 lines
bugreport.c new file +2 -0
+		strbuf_release(&report_path);
+		die("couldn't open '%s' for writing", report_path.buf);

Do we see a use-after-free here? Also missed i18n/l10n.

It is embarrassing on the reviewers' side that this (which is not new in this round at all and hasn't changed since v6) hasn't been caught for a few rounds X-<.

Thanks.

#261 lore raw
Quoted 48 lines · Junio C Hamano

bugreport.c line 53

Quoted 2 lines · Emily Shaffer
bugreport.c new file +2 -0
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);

Which one between an empty string and NULL is more appropriate to be passed as "prefix" here? I assume that this is *not* really a git program, and no repository/working-tree discovery is involved, and you won't be using OPT_FILENAME, so it would probably be OK.

bugreport.c line 66

Quoted 13 lines
bugreport.c new file +13 -0
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+	if (!stat(report_path.buf, &statbuf))
+		die("'%s' already exists", report_path.buf);

Missed i18n/l10n, but I do not think it is a useful thing for this check to be here in the first place.

bugreport.c line 79

Quoted 12 lines
bugreport.c new file +12 -0
+	switch (safe_create_leading_directories(report_path.buf)) {
+	case SCLD_OK:
+	case SCLD_EXISTS:
+		break;
+	default:
+		die(_("could not create leading directories for '%s'"),
+		    report_path.buf);
+	}
+
+	get_bug_template(&buffer);
+
+	report = fopen_for_writing(report_path.buf);

fopen_for_writing() does not give good semantics for what you seem to want to do here (i.e. to make sure you do not overwrite an existing one). It even has "if we cannot open it, retry after unlinking" logic in it, which screams "this function is designed for those who want to overwrite the file", and if you look at its callers, they are _all_ about opening a file for writing with a well known name like FETCH_HEAD, COMMIT_EDITMSG, etc.

Quoted 5 lines

Besides, a stat() that dies when a file exists would not help---since that check, you've spent non-zero time, creating leading directories and preparing boilerplate in the buffer, and there is no guarantee somebody else used the same filename in the meantime.

Good point. Ouch.

Quoted 8 lines

If you want to avoid overwriting an existing file (which I think is a good idea---I just do not think the earlier "if (!stat()) die()" is a good way to do so), the only sensible way to do so is to ask your open/creat to fail if there already is a file---that is how you'd avoid racing with another process that may be creating such a file. Grep for O_EXCL to find where the flag is used with O_CREAT to see how it is done.

Sure. Thanks, I reworked it to use open().

By the way, I noticed reading the GNU manual that file descriptors may not be supported outside of GNU environments; but I also noticed that 1) Git uses open() lots of places to use O_EXCL, and 2) fopen() doesn't support exclusive opens (except in glibc, and nobody is using that particular option in the Git codebase now). So I guess it's safe to use open()...

Quoted 4 lines

bugreport.c line 81

Quoted 2 lines
bugreport.c new file +1 -0
+	if (report == NULL) {

Style. "if (!report)"

The type of 'report' changes using open(), so this check changes too. Now it says "if (report < 0)" - per the open() doc, it returns a positive fd or -1.

Quoted 5 lines

bugreport.c line 83

Quoted 3 lines
bugreport.c new file +2 -0
+		strbuf_release(&report_path);
+		die("couldn't open '%s' for writing", report_path.buf);

Do we see a use-after-free here? Also missed i18n/l10n.

Hm. I suppose it's OK to UNLEAK() like we do at the successful exit since the die() will end the process. Added i18n too.

Quoted 3 lines

It is embarrassing on the reviewers' side that this (which is not new in this round at all and hasn't changed since v6) hasn't been caught for a few rounds X-<.

Well, I'm embarrassed to have written it in the first place.

- Emily

#262 lore raw
In reply to Emily Shaffer

Hi Emily,

Quoted 38 lines · Emily Shaffer
bugreport.c new file +32 -0
new file mode 100644
index 0000000000..8d4a76fdac
@@ -0,0 +1,94 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to share.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;

bugreport.c line 42

Quoted 10 lines
bugreport.c new file +10 -0
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+	char *option_suffix = "%F-%H%M";

This is not a portable strftime() format. Let's squash this in?

-- snipsnap --
Subject: [PATCH] fixup??? bugreport: add tool to generate debugging info

The %F format is an optional extension to the ISO C standard, see
https://pubs.opengroup.org/onlinepubs/9699919799/functions/strftime.html

And sure enough, in Git for Windows, this leads to a test failure because that format is not supported in the version of the MSVC runtime that we're stuck with.

Let's just use the long-hand instead.

---
 bugreport.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
bugreport.c +1 -1
index a3528a2e2b8..3f78db182e6 100644
@@ -338,7 +338,7 @@ int cmd_main(int argc, const char **argv)
 	FILE *report;
 	time_t now = time(NULL);
 	char *option_output = NULL;
-	char *option_suffix = "%F-%H%M";
+	char *option_suffix = "%Y-%m-%d-%H%M";
 	struct stat statbuf;
 	int nongit_ok = 0;
 

--
2.25.1.windows.1

#263 lore raw

PATCH v8 4/15 bugreport: gather git version and build info

In reply to Emily Shaffer

Knowing which version of Git a user has and how it was built allows us to more precisely pin down the circumstances when a certain issue occurs, so teach bugreport how to tell us the same output as 'git version --build-options'.

It's not ideal to directly call 'git version --build-options' because that output goes to stdout. Instead, wrap the version string in a helper within help.[ch] library, and call that helper from within the bugreport library.

---
 Documentation/git-bugreport.txt |  4 +++
 bugreport.c                     | 19 ++++++++++++-
 help.c                          | 48 ++++++++++++++++++++-------------
 help.h                          |  1 +
 4 files changed, 52 insertions(+), 20 deletions(-)
Documentation/git-bugreport.txt +4 -0
index 1f9fde5cde..f44ae8cbe7 100644
@@ -23,6 +23,10 @@ The following information is requested from the user:
  - Expected behavior
  - Actual behavior
 
+The following information is captured automatically:
+
+ - 'git version --build-options'
+
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
 is unreadable. In this kind of scenario, it may be helpful to manually gather
bugreport.c +18 -1
index 8d4a76fdac..27f813643d 100644
@@ -1,8 +1,17 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(sys_info, 1);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
@@ -32,6 +41,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -76,6 +90,9 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
help.c +29 -19

63 lines not on this page · raw message

help.h +1 -0
index 9071894e8c..500521b908 100644
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void get_version_info(struct strbuf *buf, int show_build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#264 lore raw

Re: [PATCH v8 04/15] bugreport: gather git version and build info

bugreport.c line 13

Quoted 6 lines · Emily Shaffer
bugreport.c +6 -0
+static void get_system_info(struct strbuf *sys_info)
+{
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(sys_info, 1);
+	strbuf_complete_line(sys_info);

It is a bit curious use of "don't do anything if sys_info ends with a complete line, but complete it if it ends with an imcomplete line". That tells the readers that we do not know what get_version_info() will do (now or in the future) to its output buffer.

Quoted 9 lines
(quoted diff) +1 -0
+}
...
help.c +0 -0
index 190722fb0a..44cee69c11 100644
@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }

help.c const char *help_unknown_cmd(const char *cmd) · line 633

Quoted 10 lines
help.c +9 -0
 
+void get_version_info(struct strbuf *buf, int show_build_options)
+{
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);

This ends the output with a complete line when !show_build_options ...

help.c const char *help_unknown_cmd(const char *cmd) · line 645

Quoted 11 lines
help.c +11 -0
+	if (show_build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addstr(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */

... and the pattern indicates the output will end with a complete line when !!show_build_options, too.

help.c const char *help_unknown_cmd(const char *cmd) · line 647

Quoted 2 lines
help.c +2 -0
+	}
+}

So, was the strbuf_complete_line() merely defensive programming? It may deserve a comment if it will stay there.

Quoted 25 lines
help.c +5 -10
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -637,26 +662,11 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	get_version_info(&buf, build_options);
+	printf("%s", buf.buf);
+
+	strbuf_release(&buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])

help.c int cmd_version(int argc, const char **argv, const char *prefix) · line 670

Quoted 10 lines
help.c +0 -9
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		printf("shell-path: %s\n", SHELL_PATH);
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;

Makes sense.

#265 lore raw

Re: [PATCH v8 04/15] bugreport: gather git version and build info

Quoted 52 lines · Junio C Hamano

bugreport.c line 13

Quoted 6 lines · Emily Shaffer
bugreport.c +6 -0
+static void get_system_info(struct strbuf *sys_info)
+{
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(sys_info, 1);
+	strbuf_complete_line(sys_info);

It is a bit curious use of "don't do anything if sys_info ends with a complete line, but complete it if it ends with an imcomplete line". That tells the readers that we do not know what get_version_info() will do (now or in the future) to its output buffer.

Quoted 9 lines
(quoted diff) +1 -0
+}
...
help.c +0 -0
index 190722fb0a..44cee69c11 100644
@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }

help.c const char *help_unknown_cmd(const char *cmd) · line 633

Quoted 10 lines
help.c +9 -0
 
+void get_version_info(struct strbuf *buf, int show_build_options)
+{
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);

This ends the output with a complete line when !show_build_options ...

help.c const char *help_unknown_cmd(const char *cmd) · line 645

Quoted 11 lines
help.c +11 -0
+	if (show_build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addstr(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */

... and the pattern indicates the output will end with a complete line when !!show_build_options, too.

Quoted 5 lines

help.c const char *help_unknown_cmd(const char *cmd) · line 647

Quoted 2 lines
help.c +2 -0
+	}
+}

So, was the strbuf_complete_line() merely defensive programming? It may deserve a comment if it will stay there.

It was meant defensively, here and elsewhere in the series. I figured that for something like this, which is mostly bounded by human writing in an editor and then by file IO, spurious string-checking was not such a big deal.

Are you suggesting to comment around the strbuf_complete_line() calls, or to comment around get_version_info() that it should end in newline?

- Emily

#266 lore raw

Re: [PATCH v8 04/15] bugreport: gather git version and build info

Quoted 19 lines · Emily Shaffer
Quoted 17 lines · Junio C Hamano

bugreport.c line 13

Quoted 6 lines · Emily Shaffer
bugreport.c +6 -0
+static void get_system_info(struct strbuf *sys_info)
+{
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(sys_info, 1);
+	strbuf_complete_line(sys_info);

It is a bit curious use of "don't do anything if sys_info ends with a complete line, but complete it if it ends with an imcomplete line". That tells the readers that we do not know what get_version_info() will do (now or in the future) to its output buffer.
...
So, was the strbuf_complete_line() merely defensive programming? It may deserve a comment if it will stay there.

Quoted 7 lines

It was meant defensively, here and elsewhere in the series. I figured that for something like this, which is mostly bounded by human writing in an editor and then by file IO, spurious string-checking was not such a big deal.

Are you suggesting to comment around the strbuf_complete_line() calls, or to comment around get_version_info() that it should end in newline?

I meant a comment in get_system_info() next to the use of this particular use of strbuf_complete_line(), if the use stays there.

But after reading the whole series through, I saw no need to use strbuf_complete_line() in the first place, as there is no source of input that is not under our control (if we do not count sloppy programming, that is).

#267 lore raw

PATCH v8 5/15 bugreport: add uname info

In reply to Emily Shaffer

The contents of uname() can give us some insight into what sort of system the user is running on, and help us replicate their setup if need be. The domainname field is not guaranteed to be available, so don't collect it.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 13 +++++++++++++
 2 files changed, 14 insertions(+)
Documentation/git-bugreport.txt +1 -0
index f44ae8cbe7..17b0d14e8d 100644
@@ -26,6 +26,7 @@ The following information is requested from the user:
 The following information is captured automatically:
 
  - 'git version --build-options'
+ - uname sysname, release, version, and machine strings
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
bugreport.c +13 -0
index 27f813643d..06a63cc283 100644
@@ -7,10 +7,23 @@
 
 static void get_system_info(struct strbuf *sys_info)
 {
+	struct utsname uname_info;
+
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	get_version_info(sys_info, 1);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#268 lore raw

Re: [PATCH v8 05/15] bugreport: add uname info

bugreport.c line 18

Quoted 2 lines · Emily Shaffer
bugreport.c +2 -0
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");

Is that "-a" portable across systems? That is, when given "-a", would "uname" on all platforms show sysname, release, version and machine in that order and nothing else?

If we are merely saying "in this section, we are showing a selected subset from what we learned about the system with uname(2)", perhaps just stop at saying "uname: "?

bugreport.c line 20

Quoted 2 lines
bugreport.c +2 -0
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);

i18n/l10n? Don't we want to use strerror() here?

Quoted 9 lines
bugreport.c +6 -0
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
#269 lore raw

Re: [PATCH v8 05/15] bugreport: add uname info

Quoted 9 lines · Junio C Hamano

bugreport.c line 18

Quoted 2 lines · Emily Shaffer
bugreport.c +2 -0
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");

Is that "-a" portable across systems? That is, when given "-a", would "uname" on all platforms show sysname, release, version and machine in that order and nothing else?

Quoted 3 lines

If we are merely saying "in this section, we are showing a selected subset from what we learned about the system with uname(2)", perhaps just stop at saying "uname: "?

Sure, that's good enough for me. I think actually 'uname -a' on a GNU system isn't going to match what's output here anyways, since I don't show the nodename and 'uname -a' would.

Quoted 5 lines

bugreport.c line 20

Quoted 3 lines
bugreport.c +2 -0
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);

i18n/l10n? Don't we want to use strerror() here?

Much much earlier[1] in the series, you suggested not i18n/l10n the contents of the bugreport, as this list would probably be ill-equipped to process such a report. Since it's being added to sys_info, that means this line is bound for the finished report, not for the user.

Speaking of locale, strerror() will print the error in the current locale; if we're still assuming "the list only takes reports in English" should I be explicit and use strerror_l()? Should I print strerror() in current locale and include the errno, e.g. ("uname() failed with '%s' (%d)\n", strerror(errno), errno)?

[1] https://lore.kernel.org/git/[email protected]

#270 lore raw

PATCH v8 6/15 bugreport: add compiler info

In reply to Emily Shaffer

To help pinpoint the source of a regression, it is useful to know some info about the compiler which the user's Git client was built with. By adding a generic get_compiler_info() in 'compat/' we can choose which relevant information to share per compiler; to get started, let's demonstrate the version of glibc if the user built with 'gcc'.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     |  5 +++++
 compat/compiler.h               | 27 +++++++++++++++++++++++++++
 3 files changed, 33 insertions(+)
 create mode 100644 compat/compiler.h
Documentation/git-bugreport.txt +1 -0
index 17b0d14e8d..643d1b2884 100644
@@ -27,6 +27,7 @@ The following information is captured automatically:
 
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
+ - Compiler-specific info string
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
bugreport.c +5 -0
index 06a63cc283..9a470c5588 100644
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include "compat/compiler.h"
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -24,6 +25,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
compat/compiler.h new file +27 -0
new file mode 100644
index 0000000000..ef41177233
@@ -0,0 +1,27 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
+#ifdef __GNUC__
+	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
+#endif
+}
+
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addstr(info, "get_compiler_info() not implemented");
+}
+
+#endif
+
+#endif /* COMPILER_H */
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#271 lore raw

Re: [PATCH v8 06/15] bugreport: add compiler info

Quoted 11 lines · Emily Shaffer
compat/compiler.h new file +5 -0
new file mode 100644
index 0000000000..ef41177233
@@ -0,0 +1,27 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"

compat/compiler.h line 15

Quoted 10 lines
compat/compiler.h new file +10 -0
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
+#ifdef __GNUC__
+	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
+#endif

I think we're better having

- get_compiler_info placed under #ifdef __GNUC__ gate,
- get_libc_info under #ifdef __GLIBC

Then have a function to merge those information together. Correct me if I were wrong, as it is now,
this is only useful for Linux people with glibc.

Anyway, clang also defines __GNUC__ and __GNUC_MINOR__, IIRC, FreeBSD people started to move to use clang instead of gcc as their default compiler.

Gentoo forks are also known to use clang to compile their ports.

Signature · 1 line
Danh
#272 lore raw

Re: [PATCH v8 06/15] bugreport: add compiler info

Quoted 32 lines · Danh Doan
Quoted 11 lines · Emily Shaffer
compat/compiler.h new file +5 -0
new file mode 100644
index 0000000000..ef41177233
@@ -0,0 +1,27 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"

compat/compiler.h line 15

Quoted 10 lines
compat/compiler.h new file +10 -0
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
+#ifdef __GNUC__
+	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
+#endif

I think we're better having

- get_compiler_info placed under #ifdef __GNUC__ gate,
- get_libc_info under #ifdef __GLIBC

Then have a function to merge those information together. Correct me if I were wrong, as it is now,
this is only useful for Linux people with glibc.

Quoted 5 lines

Anyway, clang also defines __GNUC__ and __GNUC_MINOR__, IIRC, FreeBSD people started to move to use clang instead of gcc as their default compiler.

Gentoo forks are also known to use clang to compile their ports.

Yeah, between your comments now and Dscho's comments on this patch in the last rollup, I'll go ahead and refactor it like you suggest.

- Emily

#273 lore raw

Re: [PATCH v8 06/15] bugreport: add compiler info

In reply to Emily Shaffer
Quoted 29 lines · Emily Shaffer
bugreport.c +4 -0
@@ -24,6 +25,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
compat/compiler.h new file +12 -0
new file mode 100644
index 0000000000..ef41177233
@@ -0,0 +1,27 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());

compat/compiler.h line 22

Quoted 10 lines
compat/compiler.h new file +10 -0
+#ifdef __GNUC__
+	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
+#endif
+}
+
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addstr(info, "get_compiler_info() not implemented");

i18n/l10n? Also don't deliberately leave the buffer end with an incomplete line like so---doing so _requires_ the caller above to have strbuf_complete_line() call, but there is no reason to do so.

strbuf_complete_line() does make sense if you insert something that is not controlled by us (e.g. "Please type whatever other comments you may have" to let the user give us free-form text, which may or may not end with a complete line), but otherwise it probably is a sign of being unnecessary sloppy.

compat/compiler.h line 23

Quoted 1 line
compat/compiler.h new file +1 -0
+}

OK, so the idea is to insert "#elif ..." and definition of get_compiler_info() for non GLIBC systems?

I am not sure why you want to make this a header with static inline implementations. Is it expected for this to be included from multiple source files? It would be more understandable if these were in the same file as where get_system_info() is, perhaps immediately before that function.

Also, it would probably be easier to manage if you have two separate helpers, one for what compiler is in use, and the other for what libc is in use.

compat/compiler.h line 27

Quoted 4 lines
compat/compiler.h new file +4 -0
+
+#endif
+
+#endif /* COMPILER_H */

Thanks.

#274 lore raw

Re: [PATCH v8 06/15] bugreport: add compiler info

In reply to Emily Shaffer

bugreport.c static void get_system_info(struct strbuf *sys_info) · line 31

Quoted 3 lines · Emily Shaffer
bugreport.c +3 -0
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);

Continuing the response to 04/15 review, I do not think this is a good use of complete_line(), either. On an exotic platform you haven't seen, get_compiler_info() might stuff nothing in the output buffer, in which case we are left with an incomplete line that just says "compiler info: ", and it might be better not to leave that incomplete line hanging around by calling complete_line(), but an even better solution would be to make sure get_compiler_info() explicitly say it encountered a system totally new to it. And at that point, as long as everybody in get_compiler_info() ends its output with a complete line, there is no need for complete_line() on the caller's side.

Of course, you can make it a convention that all case arms or ifdef blocks in get_compiler_info() uniformly end what they have to say with an incomplete line and make it a responsibility of the caller to terminate the incomplete line. But in that case, you'd not be using complete_line(), but strbuf_addch('\n').

#275 lore raw

PATCH v8 7/15 bugreport: add git-remote-https version

In reply to Emily Shaffer

It's possible for git-remote-curl to be built separately from git; in that case we want to know what version of cURL is used by git-remote-curl, not necessarily which version was present at git-bugreport's build time. So instead, ask git-remote-curl for the version information it knows about.

Today, "git-remote-http" and "git-remote-https" are aliased to "git-remote-curl"; but in case we rely on a different library than cURL in the future, let's not explicitly reference cURL from bugreport.

For longevity purposes, invoke the alias "git-remote-https" instead of "git-remote-http".

Since it could have been built at a different time, also report the version and built-from commit of git-remote-curl alongside the cURL info.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 16 ++++++++++++++++
 remote-curl.c                   |  8 ++++++++
 3 files changed, 25 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 643d1b2884..8c7cb9a5d4 100644
@@ -28,6 +28,7 @@ The following information is captured automatically:
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
+ - 'git remote-https --build-info'
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
bugreport.c +16 -0
index 9a470c5588..fb2adfdf14 100644
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include "compat/compiler.h"
+#include "run-command.h"
+
+static void get_git_remote_https_version_info(struct strbuf *version_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	cp.git_cmd = 1;
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, version_info, 0))
+	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -29,6 +41,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "compiler info: ");
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+	get_git_remote_https_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
remote-curl.c +8 -0
index 8eb96152f5..73e52175c0 100644
@@ -17,6 +17,7 @@
 #include "protocol.h"
 #include "quote.h"
 #include "transport.h"
+#include "version.h"
 
 static struct remote *remote;
 /* always ends with a trailing slash */
@@ -1375,6 +1376,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {
+		printf("git-http-fetch version: %s\n", git_version_string);
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#276 lore raw

Re: [PATCH v8 07/15] bugreport: add git-remote-https version

Quoted 37 lines · Emily Shaffer
bugreport.c +14 -0
+static void get_git_remote_https_version_info(struct strbuf *version_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	cp.git_cmd = 1;
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, version_info, 0))
+	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -29,6 +41,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "compiler info: ");
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+	get_git_remote_https_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
remote-curl.c +1 -0
index 8eb96152f5..73e52175c0 100644
@@ -17,6 +17,7 @@
 #include "protocol.h"
 #include "quote.h"
 #include "transport.h"
+#include "version.h"
 
 static struct remote *remote;
 /* always ends with a trailing slash */

remote-curl.c int cmd_main(int argc, const char **argv) · line 1380

Quoted 5 lines
remote-curl.c +2 -0
@@ -1375,6 +1376,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {
+		printf("git-http-fetch version: %s\n", git_version_string);

We are letting bugreport grab this information from remote-curl, and they are different binaries, so git_version_string we see here is that of the remote-curl. Good.

remote-curl.c int cmd_main(int argc, const char **argv) · line 1388

Quoted 8 lines
remote-curl.c +5 -0
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they

Makes sense, except that it is not clear what our overall stance on i18n/l10n of the bugreport output.

I think there are two schools of thought.

- You can stick to C local, with an expectation that developers can read reports in C locale. This will allow developers to avoid having to read reports written in a language they do not understand, and also makes it easier to mechanically process reports.

- You can make sure what matters in the report is localized to the end-users' locale. This will avoid forcing end-users to send out a report that they may not even able to read (which is what happens if we did not do i18n/l10n).

I am not sure which way you are aiming at. The boilerplate text we saw in a very early part of the series were marked N_() but some of the messages in later parts of the series are not.

#277 lore raw

Re: [PATCH v8 07/15] bugreport: add git-remote-https version

Quoted 69 lines · Junio C Hamano
Quoted 37 lines · Emily Shaffer
bugreport.c +14 -0
+static void get_git_remote_https_version_info(struct strbuf *version_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	cp.git_cmd = 1;
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, version_info, 0))
+	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -29,6 +41,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "compiler info: ");
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+	get_git_remote_https_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
remote-curl.c +1 -0
index 8eb96152f5..73e52175c0 100644
@@ -17,6 +17,7 @@
 #include "protocol.h"
 #include "quote.h"
 #include "transport.h"
+#include "version.h"
 
 static struct remote *remote;
 /* always ends with a trailing slash */

remote-curl.c int cmd_main(int argc, const char **argv) · line 1380

Quoted 5 lines
remote-curl.c +2 -0
@@ -1375,6 +1376,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {
+		printf("git-http-fetch version: %s\n", git_version_string);

We are letting bugreport grab this information from remote-curl, and they are different binaries, so git_version_string we see here is that of the remote-curl. Good.

remote-curl.c int cmd_main(int argc, const char **argv) · line 1388

Quoted 8 lines
remote-curl.c +5 -0
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they

Makes sense, except that it is not clear what our overall stance on i18n/l10n of the bugreport output.

I think there are two schools of thought.

- You can stick to C local, with an expectation that developers can read reports in C locale. This will allow developers to avoid having to read reports written in a language they do not understand, and also makes it easier to mechanically process reports.

Quoted 8 lines

- You can make sure what matters in the report is localized to the end-users' locale. This will avoid forcing end-users to send out a report that they may not even able to read (which is what happens if we did not do i18n/l10n).

I am not sure which way you are aiming at. The boilerplate text we saw in a very early part of the series were marked N_() but some of the messages in later parts of the series are not.

I mentioned it in another reply to you at a different point in the v8 conversation; I took away the localization on the boilerplate at your suggestion. Unfortunately I agree that we wouldn't be equipped to handle reports in other languages. Not being able to understand the user-fillable part of the report extends, I think, to not being able to understand the diagnostic info. If the point of the report is to skip back-and-forth, then writing back to say "actually, run all this stuff manually so we can see the output in English" is not going to achieve that goal.

- Emily

#278 lore raw

Re: [PATCH v8 07/15] bugreport: add git-remote-https version

Quoted 2 lines · Emily Shaffer

... Unfortunately I agree that we wouldn't be equipped to handle reports in other languages.

I actually was anticipating a far better world ;-)

There is no reason to limit the recipient of reports only to *us*. Use of Git and population proficient in Git would become wide enough that we should be able to expect that users who speak language X can be helped by experts who speak the same language X with their issues.

#279 lore raw

Re: [PATCH v8 07/15] bugreport: add git-remote-https version

Quoted 5 lines · Junio C Hamano
Quoted 2 lines · Emily Shaffer

... Unfortunately I agree that we wouldn't be equipped to handle reports in other languages.

Quoted 7 lines

I actually was anticipating a far better world ;-)

There is no reason to limit the recipient of reports only to *us*. Use of Git and population proficient in Git would become wide enough that we should be able to expect that users who speak language X can be helped by experts who speak the same language X with their issues.

Hm. I guess I got the opposite impression from you way back in v1. I do wish it had been communicated a little more clearly; it's frustrating to perceive a reversal after seven months of review. But that's probably on my own reading comprehension :)

Well, I certainly don't mind - but I did have a pretty long conversation with Jonathan Nieder last week about whether it's feasible to make the bugreport extremely stable while also being locale aware. As I understood it, he worried about someone having a misconfigured locale causing the bugreport tool to crash when it tries to set the locale.

But I'm having trouble figuring out how that can happen. It looks like we use libintl.h to do almost all of our string localization, which I have to assume is pretty robust. Or to put it another way, if the user's environment has broken libintl.h, then it seems like they also have bigger problems outside of Git which they would notice already.

(Side note: I went down a rabbit hole of trying to break my locale, and did manage to generate some garbage by setting 'LANG' to a non-UTF-8 character set and setting 'LANGUAGE' to a character set which does require UTF-8, that is, 'LANG=es_MX LANGUAGE=ko git status'.)

How about if I localize the bugreport template, headers, and formatted comments (e.g. "3745 local loose objects"), and include a tip in 'git help bugreport' suggesting that if it doesn't look right, maybe the user wants to run it with 'LANG= LANGUAGE= git bugreport' to ensure it actually gets generated?

I had another thought, actually, that this is maybe semantically similar problem to the malformed config we discussed earlier in the review. Does it make sense to include some kind of --safemode flag to 'git' which asks it to not perform localization and not read configs? I would propose adding it just to git-bugreport, and I could work around some locale weirdness that way, but with a broken config the attempt dies before git-bugreport binary is invoked at all.

- Emily

#280 lore raw

Re: [PATCH v8 07/15] bugreport: add git-remote-https version

Quoted 5 lines · Emily Shaffer

How about if I localize the bugreport template, headers, and formatted comments (e.g. "3745 local loose objects"), and include a tip in 'git help bugreport' suggesting that if it doesn't look right, maybe the user wants to run it with 'LANG= LANGUAGE= git bugreport' to ensure it actually gets generated?

I think that will be what is going to happen anyway in the real world. We'd spend a reasonable effort for localization (and I personally am perfectly OK if the effort for the initial round is "almost zero"), but make sure C-locale is left as an escape hatch.

Quoted 4 lines

I had another thought, actually, that this is maybe semantically similar problem to the malformed config we discussed earlier in the review. Does it make sense to include some kind of --safemode flag to 'git' which asks it to not perform localization and not read configs?

I suspect that "git --safemode" would always end up being buggy than running "git" under LANG=C LC_ALL=C GIT_CONFIG=/dev/null or something like that ;-)

Wouldn't it defeat more than 30% of the value of the tool if we do not read and report the contents of the configuration file(s)?

#281 lore raw

Re: [PATCH v8 07/15] bugreport: add git-remote-https version

Quoted 8 lines · Junio C Hamano
Quoted 5 lines · Emily Shaffer

How about if I localize the bugreport template, headers, and formatted comments (e.g. "3745 local loose objects"), and include a tip in 'git help bugreport' suggesting that if it doesn't look right, maybe the user wants to run it with 'LANG= LANGUAGE= git bugreport' to ensure it actually gets generated?

Quoted 4 lines

I think that will be what is going to happen anyway in the real world. We'd spend a reasonable effort for localization (and I personally am perfectly OK if the effort for the initial round is "almost zero"), but make sure C-locale is left as an escape hatch.

OK.

Quoted 6 lines
Quoted 5 lines

I had another thought, actually, that this is maybe semantically similar problem to the malformed config we discussed earlier in the review. Does it make sense to include some kind of --safemode flag to 'git' which asks it to not perform localization and not read configs?

Quoted 6 lines

I suspect that "git --safemode" would always end up being buggy than running "git" under LANG=C LC_ALL=C GIT_CONFIG=/dev/null or something like that ;-)

Wouldn't it defeat more than 30% of the value of the tool if we do not read and report the contents of the configuration file(s)?

If you mean "the tool" = "Git", I won't argue with you. But if you mean "the tool" = "git-bugreport", I'd say that 70% of the value of the tool is better than 0% because my config is broken and I don't know why or how.

- Emily

#282 lore raw

Re: [PATCH v8 07/15] bugreport: add git-remote-https version

In reply to Emily Shaffer
Quoted 4 lines · Emily Shaffer

Hm. I guess I got the opposite impression from you way back in v1. I do wish it had been communicated a little more clearly; it's frustrating to perceive a reversal after seven months of review. But that's probably on my own reading comprehension :)

Well, seven months is a long time for anybody to learn from repeated reading and gained experience to form an opinion and get affected by others' opinions. In any case, this is one of the reasons why I try to discourage people to have too many topics in flight before moving on to a new and different topic---the risk of ending up with fliping and flopping is just too high when you give people chance to forget. Rather, I'd prefer to see something simple to land first and then later refined.

On this particular issue, I actually do not have a preference. As long as the topic has a coherent position/stance, any one of

(a) we do as much i18n as possible to help end-users, or

(b) we stay away from i18n to ensure the reports are machine readable, or

(c) somewhere in between with a clear criterion where you are drawing the line (e.g. "the introductory text is what we want the end-user to read, so it is i18ned, but the report about their environment are primarily for our use and we avoid localizing so that we can process mechanically"),

is fine. The important point is that we choose what we do with a solid guiding principle behind the decision.

In practice, every string in bugreport.c you have control over the use (or non-use) of _() around it, but codepaths that you call from existing parts of the system are likely to have their messages localized if they are meant for Porcelain use. So from that point of view, (a) would be easier to arrange than (b), I suspect.

Thanks.

#283 lore raw

Re: [PATCH v8 07/15] bugreport: add git-remote-https version

Quoted 7 lines · Junio C Hamano
Quoted 4 lines · Emily Shaffer

Hm. I guess I got the opposite impression from you way back in v1. I do wish it had been communicated a little more clearly; it's frustrating to perceive a reversal after seven months of review. But that's probably on my own reading comprehension :)

Quoted 8 lines

Well, seven months is a long time for anybody to learn from repeated reading and gained experience to form an opinion and get affected by others' opinions. In any case, this is one of the reasons why I try to discourage people to have too many topics in flight before moving on to a new and different topic---the risk of ending up with fliping and flopping is just too high when you give people chance to forget. Rather, I'd prefer to see something simple to land first and then later refined.

I think it's exactly what happened with this topic. It's easy to say "don't have too many things out at once" in theory - but becomes less easy when you combine it with things like quarterly planning at dayjob, etc. In fact I have been trying to revive and push through old topics I had in flight because I had this same realization - my six topics were only each updated every month or so, meaning everyone (including me) forgot what was going on from revision to revision.

I wonder whether it makes sense to try and finalize the bare minimums of this feature, and then move the rest of the work to their own topics. I see a couple benefits:

- Users can start trying out 'git bugreport' when they write us mails much sooner. Even the template alone would increase the quality of a number of the reports I see come into this list.
- It may help insulate this topic from 'feature creep,' which in my opinion has plagued it from the start. While the entire topic of 15 patches is in flux, it's hard for someone else to write their own patch adding, say, midex/commit-graph support, and so I feel obligated to write it myself - which of course increases the amount of time the topic lives on in review.
- Of course smaller topics are easier to review, and a commit-graph expert is much more likely to click on the subject "[PATCH]
bugreport: gather commit-graph diagnostics" than they are to click on the subject "[PATCH] add bugreport utility". That is, I'm guessing there will be more reviews on the semantics of the patch (is it the right diagnostic info? is it useful?) than there have been so far.

The downside is mailing list churn, and the possibility that with a minimum 'git-bugreport' checked in I find something better to do. I'm not too worried about the former; after all, that's what this list is for, right? As for the latter, after 7 months of flip and flop I don't think the risk is greater than it is now anyways ;) I think it makes sense to send the following topics:

/* To me, this sounds like minimum viable bugreport. "What went wrong,
* and how do I build the same binary as you so I can reproduce it?"
add git-bugreport tool:
- bugreport: add tool to generate debugging info
- bugreport: gather git version and build info
- bugreport: add uname info
- bugreport: add compiler info

bugreport: add git-remote-https version

bugreport: include shell info
- help: add shell-path to --build-options
- bugreport: include user interactive shell

bugreport: include config info
- help: move list_config_help to builtin/help
- bugreport: generate config safelist based on docs
- bugreport: add config values from safelist

bugreport: collect list of populated hooks

bugreport: include object store info
- bugreport: count loose objects
- bugreport: add packed object summary
- bugreport: list contents of $OBJDIR/info

I'd like to send that first topic as vN+1 of this one, and I can hold the rest of the branches locally until we get it ironed out; once that topic makes it through to 'next' then I can send the rest again. What do you think?

Quoted 14 lines

On this particular issue, I actually do not have a preference. As long as the topic has a coherent position/stance, any one of

(a) we do as much i18n as possible to help end-users, or

(b) we stay away from i18n to ensure the reports are machine readable, or

(c) somewhere in between with a clear criterion where you are drawing the line (e.g. "the introductory text is what we want the end-user to read, so it is i18ned, but the report about their environment are primarily for our use and we avoid localizing so that we can process mechanically"),

Quoted 8 lines

is fine. The important point is that we choose what we do with a solid guiding principle behind the decision.

In practice, every string in bugreport.c you have control over the use (or non-use) of _() around it, but codepaths that you call from existing parts of the system are likely to have their messages localized if they are meant for Porcelain use. So from that point of view, (a) would be easier to arrange than (b), I suspect.

Hm. Sounds convincing enough to me.

- Emily

#284 lore raw

PATCH v8 8/15 bugreport: include user interactive shell

In reply to Emily Shaffer

It's possible a user may complain about the way that Git interacts with their interactive shell, e.g. autocompletion or shell prompt. In that case, it's useful for us to know which shell they're using interactively.

---
 Documentation/git-bugreport.txt | 1 +
 bugreport.c                     | 5 +++++
 2 files changed, 6 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 8c7cb9a5d4..f7a13c0374 100644
@@ -29,6 +29,7 @@ The following information is captured automatically:
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
  - 'git remote-https --build-info'
+ - $SHELL
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
bugreport.c +5 -0
index fb2adfdf14..f143b9a8f8 100644
@@ -21,6 +21,7 @@ static void get_git_remote_https_version_info(struct strbuf *version_info)
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct utsname uname_info;
+	char *shell = NULL;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
@@ -42,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
 
+	shell = getenv("SHELL");
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    shell ? shell : "<unset>");
+
 	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
 	get_git_remote_https_version_info(sys_info);
 	strbuf_complete_line(sys_info);
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#285 lore raw

PATCH v8 9/15 bugreport: generate config safelist based on docs

In reply to Emily Shaffer

Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "annotate" macro, which is a no-op in AsciiDoc and AsciiDoctor.

Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.

However, it's possible that in some situations, an organization may be less concerned with privacy of things like remote URLs and branch names, and more concerned with ease of diagnosis for their support staff. In those cases, it may make more sense for that organization to modify the code to use a blocklist. To that end, we should try to mark configs which are definitely safe, and configs which are definitely unsafe, and leave blank configs which are somewhere in between. To mark a config as safe, add "annotate:bugreport[include]" to the corresponding line in the config documentation; to mark it as unsafe, add "annotate:bugreport[exclude]" instead.

Generating bugreport-config-safelist.h at build time by grepping the documentation for this new macro helps us prevent staleness. The macro itself is a no-op and should not alter the appearance of the documentation in either AsciiDoc or AsciiDoctor, confirmable by running:

  cd Documentation
  ./doc-diff --asciidoctor HEAD^ HEAD
  ./doc-diff --asciidoc HEAD^ HEAD

Diffing the rendered HTML shows that only inline comments were added, which shouldn't be a problem.

Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.

---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  9 ++++
 Documentation/asciidoctor-extensions.rb |  7 ++++
 Documentation/config/sendemail.txt      | 56 ++++++++++++-------------
 Makefile                                |  7 ++++
 generate-bugreport-config-safelist.sh   | 18 ++++++++
 6 files changed, 70 insertions(+), 28 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh
.gitignore +1 -0
index d89bf9e11e..bd2f49b996 100644
@@ -192,6 +192,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
Documentation/asciidoc.conf +9 -0
index 8fc4b67081..663e06481f 100644
@@ -6,9 +6,14 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The annotate macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources to populate
+# things like the bugreport safelist.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>annotate):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +33,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[annotate-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +101,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[annotate-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
Documentation/asciidoctor-extensions.rb +7 -0
index d906a00803..03c80af0e5 100644
@@ -39,10 +39,17 @@ module Git
         output
       end
     end
+
+    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, target, attrs)
+        ""
+      end
+    end
   end
 end
 
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  inline_macro Git::Documentation::AnnotateProcessor, :annotate
 end
Documentation/config/sendemail.txt +28 -28

83 lines not on this page · raw message

Makefile +7 -0
index 9e6705061d..6bdd3b9337 100644
@@ -818,6 +818,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2167,6 +2168,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
generate-bugreport-config-safelist.sh new file +18 -0
new file mode 100755
index 0000000000..a9e5b6b2a0
@@ -0,0 +1,18 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the annotate-bugreport macro
+sed -n 's/^\([^ ]*\)  *annotate:bugreport\[include\].* ::$/  "\1",/p' \
+	| sort
+
+cat <<EOF
+};
+EOF
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#286 lore raw

Re: [PATCH v8 09/15] bugreport: generate config safelist based on docs

generate-bugreport-config-safelist.sh line 3

Quoted 6 lines · Emily Shaffer
generate-bugreport-config-safelist.sh new file +3 -0
@@ -0,0 +1,18 @@
+#!/bin/sh
+
+cat <<EOF

Quote the EOF if you are not doing parameter expansion in HERE-DOC.

generate-bugreport-config-safelist.sh line 15

Quoted 12 lines
generate-bugreport-config-safelist.sh new file +12 -0
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the annotate-bugreport macro
+sed -n 's/^\([^ ]*\)  *annotate:bugreport\[include\].* ::$/  "\1",/p' \
+	| sort
+

End the first line with a pipe '|' instead of a backslash, and lose the pipe from the beginning of the second line.

Quoted 3 lines
generate-bugreport-config-safelist.sh new file +3 -0
+cat <<EOF
+};
+EOF
#287 lore raw

Re: [PATCH v8 09/15] bugreport: generate config safelist based on docs

In reply to Emily Shaffer

Hi Emily,

Makefile VCSSVN_LIB = vcs-svn/lib.a · line 824

Quoted 12 lines · Emily Shaffer
Makefile +1 -0
index 9e6705061d..6bdd3b9337 100644
@@ -818,6 +818,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \

In order to pretend that we actually care about developers on Windows, let's squash this in?

-- snipsnap --
Subject: [PATCH] fixup??? bugreport: generate config safelist based on docs

The Visual Studio build is a special beast: as we cannot assume the presence of any Unix tools on Windows, we have to commit all of the files generated via shell scripts.

These two generated header files are no exception.

---
 config.mak.uname | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
config.mak.uname +4 -2
index 6d58d22cd5a..f1f36e43e47 100644
@@ -788,8 +788,10 @@ vcxproj:
 	git add -f git/LinkOrCopyBuiltins.targets git-remote-http/LinkOrCopyRemoteHttp.targets
 
 	# Add command-list.h
-	$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 command-list.h
-	git add -f command-list.h
+	$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 command-list.h \
+		config-list.h bugreport-config-safelist.h
+	git add -f command-list.h \
+	 	config-list.h bugreport-config-safelist.h
 
 	# Add scripts
 	rm -f perl/perl.mak

--
2.25.1.windows.1

#288 lore raw

Re: [PATCH v8 09/15] bugreport: generate config safelist based on docs

Quoted 18 lines · Johannes Schindelin

Subject: [PATCH] fixup??? bugreport: generate config safelist based on docs

The Visual Studio build is a special beast: as we cannot assume the presence of any Unix tools on Windows, we have to commit all of the files generated via shell scripts.

These two generated header files are no exception.

---
 config.mak.uname | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
config.mak.uname +0 -0
index 6d58d22cd5a..f1f36e43e47 100644

config.mak.uname vcxproj: · line 794

Quoted 9 lines
config.mak.uname +4 -2
@@ -788,8 +788,10 @@ vcxproj:
 	git add -f git/LinkOrCopyBuiltins.targets git-remote-http/LinkOrCopyRemoteHttp.targets
 
 	# Add command-list.h
-	$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 command-list.h
-	git add -f command-list.h
+	$(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 command-list.h \
+		config-list.h bugreport-config-safelist.h
+	git add -f command-list.h \
+	 	config-list.h bugreport-config-safelist.h

Two important questions are

- Is $(GENERATED_H) visible to you at this point in this makefile snippet that is included from the main Makefile?

- Currently the list of files listed on $(GENERATED_H) match what you are building and adding here. Is there a reason to believe two would ever diverge?

If the answers to the above are yes and no, there is an obvious futureproofing of the suggested patch, i.e.

- $(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 command-list.h
- git add -f command-list.h
+ $(MAKE) MSVC=1 SKIP_VCPKG=1 prefix=/mingw64 $(GENERATED_H)
+ git add -f $(GENERATED_H)

#289 lore raw

PATCH v8 10/15 bugreport: add config values from safelist

In reply to Emily Shaffer

Teach bugreport to gather the values of config options which are present in 'bugreport-config-safelist.h', and show their origin scope.

Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.

Rather than including the path to someone's config, which can reveal filesystem layout and project names, just name the scope (e.g. system, global, local) of the config source.

---
 Documentation/git-bugreport.txt |  1 +
 Makefile                        |  2 ++
 bugreport.c                     | 34 +++++++++++++++++++++++++++++++++
 3 files changed, 37 insertions(+)
Documentation/git-bugreport.txt +1 -0
index f7a13c0374..efe9719ea4 100644
@@ -30,6 +30,7 @@ The following information is captured automatically:
  - Compiler-specific info string
  - 'git remote-https --build-info'
  - $SHELL
+ - Selected config values
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
Makefile +2 -0
index 6bdd3b9337..bf4f3ede2a 100644
@@ -2135,6 +2135,8 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
+bugreport.sp bugreport.s bugreport.o: bugreport-config-safelist.h
+
 builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
bugreport.c +34 -0
index f143b9a8f8..70fb3b8fa6 100644
@@ -6,6 +6,9 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -52,6 +55,29 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	size_t idx;
+	struct string_list_item *it = NULL;
+	struct key_value_info *kv_info = NULL;
+
+	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
+		const struct string_list *list =
+			git_config_get_value_multi(bugreport_config_safelist[idx]);
+
+		if (!list)
+			continue;
+
+		strbuf_addf(config_info, "%s:\n", bugreport_config_safelist[idx]);
+		for_each_string_list_item(it, list) {
+			kv_info = it->util;
+			strbuf_addf(config_info, "  %s (%s)\n", it->string,
+				    kv_info ? config_scope_name(kv_info->scope)
+					    : "source unknown");
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -94,6 +120,7 @@ int cmd_main(int argc, const char **argv)
 	char *option_output = NULL;
 	char *option_suffix = "%F-%H%M";
 	struct stat statbuf;
+	int nongit_ok = 0;
 
 	const struct option bugreport_options[] = {
 		OPT_STRING('o', "output-directory", &option_output, N_("path"),
@@ -102,6 +129,10 @@ int cmd_main(int argc, const char **argv)
 			   N_("specify a strftime format suffix for the filename")),
 		OPT_END()
 	};
+
+	/* Prerequisite for hooks and config checks */
+	setup_git_directory_gently(&nongit_ok);
+
 	argc = parse_options(argc, argv, "", bugreport_options,
 			     bugreport_usage, 0);
 
@@ -132,6 +163,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#290 lore raw

Re: [PATCH v8 10/15] bugreport: add config values from safelist

Quoted 9 lines · Emily Shaffer
bugreport.c +1 -0
@@ -94,6 +120,7 @@ int cmd_main(int argc, const char **argv)
 	char *option_output = NULL;
 	char *option_suffix = "%F-%H%M";
 	struct stat statbuf;
+	int nongit_ok = 0;
 
 	const struct option bugreport_options[] = {
 		OPT_STRING('o', "output-directory", &option_output, N_("path"),

bugreport.c int cmd_main(int argc, const char **argv) · line 134

Quoted 6 lines
bugreport.c +3 -0
@@ -102,6 +129,10 @@ int cmd_main(int argc, const char **argv)
 			   N_("specify a strftime format suffix for the filename")),
 		OPT_END()
 	};
+
+	/* Prerequisite for hooks and config checks */
+	setup_git_directory_gently(&nongit_ok);

Now this starts to break "-o" option when the command is run from a subdirectory of a git working tree, I suspect.

The setup() will discover that you are in a Git controlled working tree, goes up to the top-level of the working tree, so a user may expect

	cd t && git bugreport -o frotz

to create a new directory t/frotz, but it would instead create the directory frotz next to t/, no?

Using OPT_FILENAME and feeding correct prefix to parse_options() will correct this.

Quoted 2 lines
bugreport.c +0 -0
 	argc = parse_options(argc, argv, "", bugreport_options,
 			     bugreport_usage, 0);
#291 lore raw

PATCH v8 11/15 bugreport: collect list of populated hooks

In reply to Emily Shaffer

Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong.

Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot. To reduce the amount of code humans need to read, we turn the list into a string_list and iterate over it (as we are calling the same find_hook operation on each string). However, since bugreport should primarily be called by the user, the performance loss from massaging the string seems acceptable.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 53 +++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+)
Documentation/git-bugreport.txt +1 -0
index efe9719ea4..4d01528540 100644
@@ -31,6 +31,7 @@ The following information is captured automatically:
  - 'git remote-https --build-info'
  - $SHELL
  - Selected config values
+ - A list of enabled hooks
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
bugreport.c +53 -0

71 lines not on this page · raw message

Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#292 lore raw

Re: [PATCH v8 11/15] bugreport: collect list of populated hooks

bugreport.c static void get_safelisted_config(struct strbuf *config_info) · line 87

Quoted 6 lines · Emily Shaffer
bugreport.c +6 -0
+static void get_populated_hooks(struct strbuf *hook_info, int nongit)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hooks`.
+	 */

It may want to become a NEEDSWORK comment.

Quoted 13 lines
bugreport.c +13 -0
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"

bugreport.c static void get_safelisted_config(struct strbuf *config_info) · line 110

Quoted 10 lines
bugreport.c +10 -0
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";

Typo here?

bugreport.c static void get_safelisted_config(struct strbuf *config_info) · line 123

Quoted 13 lines
bugreport.c +13 -0
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+
+
+	if (nongit) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {

I do not get why you want to use string_list for this, especially if you need to use string_list_split.

To me,

	int i;
	const char *hook[] = {
		"applypatch-msg",
	        "pre-applypatch",
		...
		"post-index-change",
	};

	for (i = 0; i < ARRAY_SIZE(hook); i++) {
		if (hook[i] is enabled)
			strbuf_addf(hook_info, "%s\n", hook[i]);
	}

would be far easier to understand. Do you have an external source that can feed you a single long string of comma separated hook names in mind, so that the initialization of *hooks will become simpler that way, or something?

Quoted 20 lines
bugreport.c +10 -0
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -166,6 +216,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Enabled Hooks");
+	get_populated_hooks(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
#293 lore raw

Re: [PATCH v8 11/15] bugreport: collect list of populated hooks

Quoted 10 lines · Junio C Hamano

bugreport.c static void get_safelisted_config(struct strbuf *config_info) · line 87

Quoted 6 lines · Emily Shaffer
bugreport.c +6 -0
+static void get_populated_hooks(struct strbuf *hook_info, int nongit)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hooks`.
+	 */

It may want to become a NEEDSWORK comment.

Oh, good point. I'll rephrase it.

Quoted 18 lines
Quoted 18 lines
bugreport.c +17 -0
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
Quoted 8 lines

bugreport.c static void get_safelisted_config(struct strbuf *config_info) · line 110

Quoted 6 lines
bugreport.c +6 -0
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";

Typo here?

Yep, I imagine from trying to press 'x' to remove the ',' from my list.

Quoted 33 lines
Quoted 4 lines
bugreport.c +3 -0
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+

bugreport.c static void get_safelisted_config(struct strbuf *config_info) · line 123

Quoted 10 lines
bugreport.c +10 -0
+
+	if (nongit) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {

I do not get why you want to use string_list for this, especially if you need to use string_list_split.

To me,

	int i;
	const char *hook[] = {
		"applypatch-msg",
	        "pre-applypatch",
		...
		"post-index-change",
	};

	for (i = 0; i < ARRAY_SIZE(hook); i++) {
		if (hook[i] is enabled)
			strbuf_addf(hook_info, "%s\n", hook[i]);
	}
Quoted 4 lines

would be far easier to understand. Do you have an external source that can feed you a single long string of comma separated hook names in mind, so that the initialization of *hooks will become simpler that way, or something?

Huh. I'm having a hard time remembering why I did it this way. I think I wanted to use the string_list foreach? But you're right that it is pretty hard to understand, plus expensive; I'll just use an array like you suggest.

Quoted 21 lines
Quoted 21 lines
bugreport.c +10 -0
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -166,6 +216,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Enabled Hooks");
+	get_populated_hooks(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
#294 lore raw

PATCH v8 12/15 bugreport: count loose objects

In reply to Emily Shaffer

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 52 +++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 4d01528540..4fe1c60506 100644
@@ -32,6 +32,7 @@ The following information is captured automatically:
  - $SHELL
  - Selected config values
  - A list of enabled hooks
+ - The number of loose objects in the repository
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
bugreport.c +52 -0

70 lines not on this page · raw message

Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#295 lore raw

Re: [PATCH v8 12/15] bugreport: count loose objects

Quoted 39 lines · Emily Shaffer

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 52 +++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 4d01528540..4fe1c60506 100644
@@ -32,6 +32,7 @@ The following information is captured automatically:
  - $SHELL
  - Selected config values
  - A list of enabled hooks
+ - The number of loose objects in the repository
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
bugreport.c +1 -0
index b5a0714a7f..fb7bc72723 100644
@@ -10,6 +10,7 @@
 #include "bugreport-config-safelist.h"
 #include "khash.h"
 #include "run-command.h"
+#include "object-store.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -128,6 +129,54 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 }
 

bugreport.c static void get_populated_hooks(struct strbuf *hook_info, int nongit) · line 141

Quoted 10 lines
bugreport.c +10 -0
+static int loose_object_cb(const struct object_id *oid, const char *path,
+			   void *data) {
+	int *loose_object_count = data;
+
+	if (loose_object_count) {
+		(*loose_object_count)++;
+		return 0;
+	}
+
+	return 1;

What is the point of returning 1 here to abort the iteration early? Wouldn't it be a BUG() if this callback ends up gettting called with NULL in data?

Quoted 50 lines
bugreport.c +40 -0
+}
+
+static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
+
+	int local_loose_object_count = 0, total_loose_object_count = 0;
+	int local_count_questionable = 0, total_count_questionable = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	local_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&local_loose_object_count,
+					FOR_EACH_OBJECT_LOCAL_ONLY);
+
+	total_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&total_loose_object_count,
+					0);
+	strbuf_addf(obj_info, "%d local loose objects%s\n",
+		    local_loose_object_count,
+		    local_count_questionable ? " (problem during count)" : "");
+
+	strbuf_addf(obj_info, "%d alternate loose objects%s\n",
+		    total_loose_object_count - local_loose_object_count,
+		    (local_count_questionable || total_count_questionable)
+			? " (problem during count)"
+			: "");
+
+	strbuf_addf(obj_info, "%d total loose objects%s\n",
+		    total_loose_object_count,
+		    total_count_questionable ? " (problem during count)" : "");
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -219,6 +268,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Enabled Hooks");
 	get_populated_hooks(&buffer, nongit_ok);
 
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
#296 lore raw

Re: [PATCH v8 12/15] bugreport: count loose objects

Quoted 41 lines · Junio C Hamano
Quoted 39 lines · Emily Shaffer

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 52 +++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 4d01528540..4fe1c60506 100644
@@ -32,6 +32,7 @@ The following information is captured automatically:
  - $SHELL
  - Selected config values
  - A list of enabled hooks
+ - The number of loose objects in the repository
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
bugreport.c +1 -0
index b5a0714a7f..fb7bc72723 100644
@@ -10,6 +10,7 @@
 #include "bugreport-config-safelist.h"
 #include "khash.h"
 #include "run-command.h"
+#include "object-store.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -128,6 +129,54 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 }
 
Quoted 14 lines

bugreport.c static void get_populated_hooks(struct strbuf *hook_info, int nongit) · line 141

Quoted 10 lines
bugreport.c +10 -0
+static int loose_object_cb(const struct object_id *oid, const char *path,
+			   void *data) {
+	int *loose_object_count = data;
+
+	if (loose_object_count) {
+		(*loose_object_count)++;
+		return 0;
+	}
+
+	return 1;

What is the point of returning 1 here to abort the iteration early? Wouldn't it be a BUG() if this callback ends up gettting called with NULL in data?

Hrm. Wouldn't BUG() throw away the rest of the generated report?

Maybe it's better, in NULL case, to indicate an error occurred there and write that information into the report, and continue gathering the rest of the info.

#297 lore raw

Re: [PATCH v8 12/15] bugreport: count loose objects

Quoted 57 lines · Emily Shaffer
Quoted 41 lines · Junio C Hamano
Quoted 39 lines · Emily Shaffer

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 52 +++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)
Documentation/git-bugreport.txt +1 -0
index 4d01528540..4fe1c60506 100644
@@ -32,6 +32,7 @@ The following information is captured automatically:
  - $SHELL
  - Selected config values
  - A list of enabled hooks
+ - The number of loose objects in the repository
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
bugreport.c +1 -0
index b5a0714a7f..fb7bc72723 100644
@@ -10,6 +10,7 @@
 #include "bugreport-config-safelist.h"
 #include "khash.h"
 #include "run-command.h"
+#include "object-store.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -128,6 +129,54 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 }
 
Quoted 14 lines

bugreport.c static void get_populated_hooks(struct strbuf *hook_info, int nongit) · line 141

Quoted 10 lines
bugreport.c +10 -0
+static int loose_object_cb(const struct object_id *oid, const char *path,
+			   void *data) {
+	int *loose_object_count = data;
+
+	if (loose_object_count) {
+		(*loose_object_count)++;
+		return 0;
+	}
+
+	return 1;

What is the point of returning 1 here to abort the iteration early? Wouldn't it be a BUG() if this callback ends up gettting called with NULL in data?

Quoted 5 lines

Hrm. Wouldn't BUG() throw away the rest of the generated report?

Maybe it's better, in NULL case, to indicate an error occurred there and write that information into the report, and continue gathering the rest of the info.

Oh, hm. Now that I look twice, you're saying "bugreport gave its own callback a NULL". I guess it could also be that for_each_loose_object() decided to give a NULL context instead of the one we gave...

Even so, I dislike the idea of BUG() called from the tool you are trying to use to report a BUG() message. We do already report if 1 was returned "(problem during count)". I suppose I can try and make a more specific error if we know the callback data was definitely the problem.

- Emily

#298 lore raw

PATCH v8 13/15 bugreport: add packed object summary

In reply to Emily Shaffer

Alongside the loose object counts, it can be useful to show the number of packs and packed objects. This way we can check whether the repo has an appropriate ratio of packed to loose objects to help determine whether it's behaving correctly.

Add a utility to easily traverse all packfiles in a given repository. Use it in packfile.c and remove a redundant call to prepare_packed_git(), which is already called in get_all_packs().

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 30 ++++++++++++++++++++++++++++++
 object-store.h                  |  6 ++++++
 packfile.c                      |  3 +--
 4 files changed, 38 insertions(+), 2 deletions(-)
Documentation/git-bugreport.txt +1 -0
index 4fe1c60506..eb41f0677f 100644
@@ -33,6 +33,7 @@ The following information is captured automatically:
  - Selected config values
  - A list of enabled hooks
  - The number of loose objects in the repository
+ - The number of packs and packed objects in the repository
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
bugreport.c +30 -0
index fb7bc72723..71191f1331 100644
@@ -177,6 +177,33 @@ static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
 		    total_count_questionable ? " (problem during count)" : "");
 }
 
+static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
+{
+	struct packed_git *pack = NULL;
+	int pack_count = 0;
+	int object_count = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	for_each_pack(the_repository, pack) {
+		pack_count++;
+		/*
+		 * To accurately count how many objects are packed, look inside
+		 * the packfile's index.
+		 */
+		open_pack_index(pack);
+		object_count += pack->num_objects;
+	}
+
+	strbuf_addf(obj_info, "%d total packs (%d objects)\n", pack_count,
+		    object_count);
+
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -271,6 +298,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
object-store.h +6 -0
index 5b047637e3..f881a1756e 100644
@@ -7,6 +7,7 @@
 #include "sha1-array.h"
 #include "strbuf.h"
 #include "thread-utils.h"
+#include "packfile.h"
 
 struct object_directory {
 	struct object_directory *next;
@@ -447,4 +448,9 @@ int for_each_object_in_pack(struct packed_git *p,
 int for_each_packed_object(each_packed_object_fn, void *,
 			   enum for_each_object_flags flags);
 
+#define for_each_pack(repo, pack) 		\
+		for (pack = get_all_packs(repo);\
+		     pack;			\
+		     pack = pack->next)
+
 #endif /* OBJECT_STORE_H */
packfile.c +1 -2
index 99dd1a7d09..95afcc1187 100644
@@ -2095,8 +2095,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data,
 	int r = 0;
 	int pack_errors = 0;
 
-	prepare_packed_git(the_repository);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for_each_pack(the_repository, p) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#299 lore raw

Re: [PATCH v8 13/15] bugreport: add packed object summary

Quoted 16 lines · Emily Shaffer
bugreport.c +16 -0
+static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
+{
+	struct packed_git *pack = NULL;
+	int pack_count = 0;
+	int object_count = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	for_each_pack(the_repository, pack) {
+		pack_count++;
+		/*
+		 * To accurately count how many objects are packed, look inside

bugreport.c static void get_loose_object_summary(struct strbuf *obj_info, int nongit) { · line 205

Quoted 10 lines
bugreport.c +10 -0
+		 * the packfile's index.
+		 */
+		open_pack_index(pack);
+		object_count += pack->num_objects;
+	}
+
+	strbuf_addf(obj_info, "%d total packs (%d objects)\n", pack_count,
+		    object_count);
+
+}

Makes sense.

object-store.h int for_each_object_in_pack(struct packed_git *p, · line 454

Quoted 8 lines
object-store.h +4 -0
@@ -447,4 +448,9 @@ int for_each_object_in_pack(struct packed_git *p,
 int for_each_packed_object(each_packed_object_fn, void *,
 			   enum for_each_object_flags flags);
 
+#define for_each_pack(repo, pack) 		\
+		for (pack = get_all_packs(repo);\
+		     pack;			\
+		     pack = pack->next)

I generally avoid #define'ing a control loop pseudo-syntax unless it makes the resulting code hide overly ugly implementation detail.

for_each_string_list() is there to hide the fact that items are stored in an embedded array whose name is .items and size is .nr that is sufficiently ugnly to expose, but iterating over packs does not look so bad.

If you MUST have this as a pseudo-syntax macro, we need

- to match for_each_string_list_item(), have iterating variable 'pack' as the first parameter, and the scope of what's iterated 'repo' as the second.

- to make sure the syntax works correctly even if a parameter is *not* a simple identifier (I think the above is OK, but there may be cases that it does not work well).

Regarding the latter, the way 'item' is incremented at the end of iteration in for_each_string_list_item() is subtle and correct.

#define for_each_string_list_item(item,list)            \
	for (item = (list)->items;                      \
	     item && item < (list)->items + (list)->nr; \
	     ++item)

You would break it if you changed pre-increment to post-increment for a user like this:

	struct string_list *list;
	struct string_list_item *i, **p;
	p = &i;

	for_each_string_list_item(*p, list) {
		...
	}

because ++*p is ++(*p), while *p++ is (*p)++, and we do want the former (i.e. increment the memory cell pointed at by pointer p).

Personally, I would prefer not to introduce this macro if I were working on this topic.

Quoted 15 lines
object-store.h +0 -0
 #endif /* OBJECT_STORE_H */
packfile.c +1 -2
index 99dd1a7d09..95afcc1187 100644
@@ -2095,8 +2095,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data,
 	int r = 0;
 	int pack_errors = 0;
 
-	prepare_packed_git(the_repository);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for_each_pack(the_repository, p) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
#300 lore raw

Re: [PATCH v8 13/15] bugreport: add packed object summary

Quoted 77 lines · Junio C Hamano
Quoted 16 lines · Emily Shaffer
bugreport.c +16 -0
+static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
+{
+	struct packed_git *pack = NULL;
+	int pack_count = 0;
+	int object_count = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	for_each_pack(the_repository, pack) {
+		pack_count++;
+		/*
+		 * To accurately count how many objects are packed, look inside

bugreport.c static void get_loose_object_summary(struct strbuf *obj_info, int nongit) { · line 205

Quoted 10 lines
bugreport.c +10 -0
+		 * the packfile's index.
+		 */
+		open_pack_index(pack);
+		object_count += pack->num_objects;
+	}
+
+	strbuf_addf(obj_info, "%d total packs (%d objects)\n", pack_count,
+		    object_count);
+
+}

Makes sense.

object-store.h int for_each_object_in_pack(struct packed_git *p, · line 454

Quoted 8 lines
object-store.h +4 -0
@@ -447,4 +448,9 @@ int for_each_object_in_pack(struct packed_git *p,
 int for_each_packed_object(each_packed_object_fn, void *,
 			   enum for_each_object_flags flags);
 
+#define for_each_pack(repo, pack) 		\
+		for (pack = get_all_packs(repo);\
+		     pack;			\
+		     pack = pack->next)

I generally avoid #define'ing a control loop pseudo-syntax unless it makes the resulting code hide overly ugly implementation detail.

for_each_string_list() is there to hide the fact that items are stored in an embedded array whose name is .items and size is .nr that is sufficiently ugnly to expose, but iterating over packs does not look so bad.

If you MUST have this as a pseudo-syntax macro, we need

- to match for_each_string_list_item(), have iterating variable 'pack' as the first parameter, and the scope of what's iterated 'repo' as the second.

- to make sure the syntax works correctly even if a parameter is *not* a simple identifier (I think the above is OK, but there may be cases that it does not work well).

Regarding the latter, the way 'item' is incremented at the end of iteration in for_each_string_list_item() is subtle and correct.

#define for_each_string_list_item(item,list)            \
	for (item = (list)->items;                      \
	     item && item < (list)->items + (list)->nr; \
	     ++item)

You would break it if you changed pre-increment to post-increment for a user like this:

	struct string_list *list;
	struct string_list_item *i, **p;
	p = &i;

	for_each_string_list_item(*p, list) {
		...
	}
Quoted 5 lines

because ++*p is ++(*p), while *p++ is (*p)++, and we do want the former (i.e. increment the memory cell pointed at by pointer p).

Personally, I would prefer not to introduce this macro if I were working on this topic.

Ah, I thought this is the kind of thing you meant here[1]. But I agree with what you point out about the shortcomings of this kind of macro. The implementation details are not so ugly; I'll drop it.

- Emily

[1] https://lore.kernel.org/git/[email protected]

Quoted 16 lines
Quoted 16 lines
object-store.h +0 -0
 #endif /* OBJECT_STORE_H */
packfile.c +1 -2
index 99dd1a7d09..95afcc1187 100644
@@ -2095,8 +2095,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data,
 	int r = 0;
 	int pack_errors = 0;
 
-	prepare_packed_git(the_repository);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for_each_pack(the_repository, p) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
#301 lore raw

PATCH v8 14/15 bugreport: list contents of $OBJDIR/info

In reply to Emily Shaffer

Miscellaneous information used about the object store can end up in .git/objects/info; this can help us understand what may be going on with the object store when the user is reporting a bug. Otherwise, it could be difficult to track down what is going wrong with an object which isn't kept locally to .git/objects/ or .git/objects/pack. Having some understanding of where the user's objects may be kept can save us some hops during the bug reporting process.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 54 +++++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)
Documentation/git-bugreport.txt +1 -0
index eb41f0677f..a7ef3360d2 100644
@@ -34,6 +34,7 @@ The following information is captured automatically:
  - A list of enabled hooks
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
+ - A list of the contents of .git/objects/info (or equivalent)
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
bugreport.c +54 -0

66 lines not on this page · raw message

Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#302 lore raw

Re: [PATCH v8 14/15] bugreport: list contents of $OBJDIR/info

bugreport.c static void get_packed_object_summary(struct strbuf *obj_info, int nongit) · line 216

Quoted 10 lines · Emily Shaffer
bugreport.c +10 -0
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+					     struct strbuf *dirpath)
+{
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
+
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;

No error detected?

Quoted 6 lines
bugreport.c +6 -0
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;

bugreport.c static void get_packed_object_summary(struct strbuf *obj_info, int nongit) · line 233

Quoted 10 lines
bugreport.c +10 -0
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);

Shouldn't we be sorting the output?

bugreport.c static void get_packed_object_summary(struct strbuf *obj_info, int nongit) · line 237

Quoted 4 lines
bugreport.c +4 -0
+	}
+
+	closedir(dir);
+}

Hmph, it is somewhat sad that we have to reinvent a degraded copy of "ls -aR" like this.

bugreport.c static void get_packed_object_summary(struct strbuf *obj_info, int nongit) · line 251

Quoted 13 lines
bugreport.c +13 -0
+static void get_object_info_summary(struct strbuf *obj_info, int nongit)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - object info unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");

Would it help to use git_path() for this, perhaps like git_path("objects/info/")?

By the way, do we store anything worth knowing in $GIT_DIR/info too? Perhaps the per-repo attributes and excludes files whose presence may hint at another question to ask when a user reports unexpected behaviour from .gitignore?

Quoted 17 lines
bugreport.c +7 -0
+	list_contents_of_dir_recursively(obj_info, &dirpath);
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -301,6 +352,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Packed Object Summary");
 	get_packed_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
#303 lore raw

PATCH v8 15/15 bugreport: summarize contents of alternates file

In reply to Emily Shaffer

In some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead. Since paths to alternates may be sensitive, we'll instead count how many alternates have been specified and note how many of them exist or are broken.

While object-cache.h describes a function "foreach_alt_odb()", this function does not provide information on broken alternates, which are skipped over in "link_alt_odb_entry()". Since the goal is to identify missing alternates, we can gather the contents of .git/objects/info/alternates manually.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 45 +++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)
Documentation/git-bugreport.txt +1 -0
index a7ef3360d2..1f60fb9456 100644
@@ -35,6 +35,7 @@ The following information is captured automatically:
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
  - A list of the contents of .git/objects/info (or equivalent)
+ - The number of valid and invalid alternates
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
bugreport.c +45 -0

57 lines not on this page · raw message

Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#304 lore raw

Re: [PATCH v8 15/15] bugreport: summarize contents of alternates file

Hi Emily,

Quoted 7 lines · Emily Shaffer

In some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead. Since paths to alternates may be sensitive, we'll instead count how many alternates have been specified and note how many of them exist or are broken.

Quoted 5 lines

While object-cache.h describes a function "foreach_alt_odb()", this function does not provide information on broken alternates, which are skipped over in "link_alt_odb_entry()". Since the goal is to identify missing alternates, we can gather the contents of .git/objects/info/alternates manually.

Makes sense.

bugreport.c static void get_object_info_summary(struct strbuf *obj_info, int nongit) · line 261

Quoted 12 lines
bugreport.c +4 -0
index 1d61e0f642..1640a71086 100644
@@ -255,6 +255,48 @@ static void get_object_info_summary(struct strbuf *obj_info, int nongit)
 	strbuf_release(&dirpath);
 }
 
+static void get_alternates_summary(struct strbuf *alternates_info, int nongit)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;

I am not sure that those variables and the parameter need to repeat "alternate", I find it rather distracting. If it were me, I would rename the parameter to info, the first strbuf to path and the second to line. This function is so specific to read the alternates file that it is quite obvious what their roles are.

Quoted 12 lines
bugreport.c +12 -0
+	FILE *file;
+	size_t exists = 0, broken = 0;
+
+	if (nongit) {
+		strbuf_addstr(alternates_info,
+			"not run from a git repository - alternates unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");

bugreport.c static void get_object_info_summary(struct strbuf *obj_info, int nongit) · line 283

Quoted 10 lines
bugreport.c +10 -0
+
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))

Should we make sure that the alternate is actually valid objects directory here? Like, look whether it is a directory, not a file or a (possibly dangling) symbolic link. This seems to be the only check alt_odb_usable() performs, so that should probably be good enough here, too.

bugreport.c static void get_object_info_summary(struct strbuf *obj_info, int nongit) · line 290

Quoted 7 lines
bugreport.c +7 -0
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",

Sadly, %zd is not portable. Therefore, pu (and es/bugreport) do not even _build_ on Windows. I need this to make it work:

-- snip --

bugreport.c +5 -4
index 3770aa73fae..5033668e22f 100644
@@ -287,10 +287,11 @@ static void get_alternates_summary(struct strbuf *alternates_info, int nongit)
 	}
 
 	strbuf_addf(alternates_info,
-		    "%zd alternates found (%zd working, %zd broken)\n",
-		    exists + broken,
-		    exists,
-		    broken);
+		    "%"PRIuMAX" alternates found "
+		    "(%"PRIuMAX" working, %"PRIuMAX" broken)\n",
+		    (uintmax_t)(exists + broken),
+		    (uintmax_t)exists,
+		    (uintmax_t)broken);
 
 	fclose(file);
 	strbuf_release(&alternate);

-- snap --

Could you incorporate that into the next iteration, please?

Thanks,
Dscho

Quoted 26 lines
bugreport.c +12 -1
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -355,6 +397,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
--

2.25.0.265.gbab2e86ba0-goog

#305 lore raw

Re: [PATCH v8 15/15] bugreport: summarize contents of alternates file

In reply to Emily Shaffer
Quoted 6 lines · Emily Shaffer
bugreport.c +6 -0
+static void get_alternates_summary(struct strbuf *alternates_info, int nongit)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;

bugreport.c static void get_object_info_summary(struct strbuf *obj_info, int nongit) · line 273

Quoted 10 lines
bugreport.c +10 -0
+
+	if (nongit) {
+		strbuf_addstr(alternates_info,
+			"not run from a git repository - alternates unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");

git_path()?

bugreport.c static void get_object_info_summary(struct strbuf *obj_info, int nongit) · line 284

Quoted 10 lines
bugreport.c +10 -0
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;

F_OK reported as "exists" makes quite a lot of sense.

bugreport.c static void get_object_info_summary(struct strbuf *obj_info, int nongit) · line 286

Quoted 2 lines
bugreport.c +2 -0
+		else
+			broken++;

So, shouldn't this rather be "missing"?

bugreport.c static void get_object_info_summary(struct strbuf *obj_info, int nongit) · line 293

Quoted 7 lines
bugreport.c +7 -0
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);

I don't see the point of using size_t for this. Just use int for both, like you did for object count in the step that counds loose and packed objects.

Quoted 18 lines
bugreport.c +8 -0
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -355,6 +397,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
#306 lore raw

v9 cover add git-bugreport tool

In reply to Emily Shaffer

I did run this set through the Azure Pipelines CI [2] and hope that means it's platform-safe; if that's not the case, please let me know your concerns.

For the relevant patches, I made the following changes since v8:

- More portable strftime format
- Use open() instead of fopen_for_writing() in order to specify O_CREAT | O_EXCL | O_WRONLY.
- Fix a use-after-free.
- Add i18n/l10n macro all over the place; per [3] let's translate the whole contents of the report, because it's easy to force to a different language and we may not be the only ones reading reports
- Use a lot less strbuf_complete_line() - when we're making library calls, we have control over the format, so we do know whether it ends in \n or not; let's not waste the time checking it.
- Minor nit on uname header phrasing
- Reword the structure of compat/compiler.h to separate libc and compiler.
- Add MSVC reporting in compat/compiler.h (thanks Dscho).

Thanks all.
- Emily

[1]: https://lore.kernel.org/git/[email protected]
[2]: https://github.com/gitgitgadget/git/pull/566
[3]: https://lore.kernel.org/git/[email protected]

Emily Shaffer (5):
help: move list_config_help to builtin/help
Note that this patch is still required for this series because the help library (for version/build info) is unusable without it.
bugreport: add tool to generate debugging info
bugreport: gather git version and build info
bugreport: add uname info
bugreport: add compiler info

 .gitignore                      |   2 +
 Documentation/git-bugreport.txt |  52 ++++++++++++
 Makefile                        |  18 +++-
 bugreport.c                     | 142 ++++++++++++++++++++++++++++++++
 builtin/help.c                  |  86 +++++++++++++++++++
 command-list.txt                |   1 +
 compat/compiler.h               |  38 +++++++++
 generate-cmdlist.sh             |  19 -----
 generate-configlist.sh          |  21 +++++
 help.c                          | 131 +++++++----------------------
 help.h                          |   2 +-
 strbuf.c                        |   4 +
 strbuf.h                        |   1 +
 t/t0091-bugreport.sh            |  61 ++++++++++++++
 14 files changed, 452 insertions(+), 126 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100644 compat/compiler.h
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#307 lore raw

PATCH v9 1/5 help: move list_config_help to builtin/help

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o, builtin/clean.o, and builtin/config.o. This meant that help.o was unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present).

When command-list.h is included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule.

---
 .gitignore             |  1 +
 Makefile               | 13 +++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 21 +++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 118 insertions(+), 108 deletions(-)
 create mode 100755 generate-configlist.sh
.gitignore +1 -0
index aebe7c0908..ea97de83f3 100644
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
Makefile +10 -3
index 6134104ae6..c552312d3f 100644
@@ -815,6 +815,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2132,7 +2133,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2152,6 +2153,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2785,7 +2792,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2807,7 +2814,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
builtin/help.c +86 -0

98 lines not on this page · raw message

generate-cmdlist.sh +0 -19
index 71158f7d8b..45fecf8bdf 100755
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
generate-configlist.sh new file +21 -0
new file mode 100755
index 0000000000..8692fe5cf4
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	sed 's/^.*$/	"&",/'
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
help.c +0 -85

91 lines not on this page · raw message

help.h +0 -1
index 7a455beeb7..9071894e8c 100644
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#308 lore raw

PATCH v9 2/5 bugreport: add tool to generate debugging info

In reply to Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.

Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.

---
 .gitignore                      |   1 +
 Documentation/git-bugreport.txt |  46 ++++++++++++++
 Makefile                        |   5 ++
 bugreport.c                     | 105 ++++++++++++++++++++++++++++++++
 command-list.txt                |   1 +
 strbuf.c                        |   4 ++
 strbuf.h                        |   1 +
 t/t0091-bugreport.sh            |  61 +++++++++++++++++++
 8 files changed, 224 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh
.gitignore +1 -0
index ea97de83f3..d89bf9e11e 100644
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
Documentation/git-bugreport.txt new file +46 -0

46 lines not on this page · raw message

Makefile +5 -0
index c552312d3f..9e6705061d 100644
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2461,6 +2462,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
bugreport.c new file +105 -0

105 lines not on this page · raw message

command-list.txt +1 -0
index 2087894655..185e5e3f05 100644
@@ -54,6 +54,7 @@ git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
 git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
+git-bugreport                           ancillaryinterrogators
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
 git-check-attr                          purehelpers
strbuf.c +4 -0
index f19da55b07..f1d66c7848 100644
@@ -539,6 +539,10 @@ ssize_t strbuf_write(struct strbuf *sb, FILE *f)
 	return sb->len ? fwrite(sb->buf, 1, sb->len, f) : 0;
 }
 
+ssize_t strbuf_write_fd(struct strbuf *sb, int fd)
+{
+	return sb->len ? write(fd, sb->buf, sb->len) : 0;
+}
 
 #define STRBUF_MAXLINK (2*PATH_MAX)
 
strbuf.h +1 -0
index aae7ac3a82..bbf6204de7 100644
@@ -462,6 +462,7 @@ int strbuf_readlink(struct strbuf *sb, const char *path, size_t hint);
  * NUL bytes.
  */
 ssize_t strbuf_write(struct strbuf *sb, FILE *stream);
+ssize_t strbuf_write_fd(struct strbuf *sb, int fd);
 
 /**
  * Read a line from a FILE *, overwriting the existing contents of
t/t0091-bugreport.sh new file +61 -0

61 lines not on this page · raw message

Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#309 lore raw

Hi Emily,

bugreport.c line 42

Quoted 1 line · Emily Shaffer
bugreport.c new file +1 -0
+	char *option_suffix = "%F-%H%M";

Unfortunately, this still is not an strftime format we can use on Windows. I still needed to fix that in the previously-mentioned manner to fix the test run on Windows.

Thanks,
Dscho

#310 lore raw
In reply to Emily Shaffer

Hi,

Quoted 4 lines · Emily Shaffer
 .gitignore                      |   1 +
 Documentation/git-bugreport.txt |  46 ++++++++++++++
 Makefile                        |   5 ++
 bugreport.c                     | 105 ++++++++++++++++++++++++++++++++
Quoted 8 lines
 command-list.txt                |   1 +
 strbuf.c                        |   4 ++
 strbuf.h                        |   1 +
 t/t0091-bugreport.sh            |  61 +++++++++++++++++++
 8 files changed, 224 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh

Hmm. I am still _quite_ convinced that this would be much better as a built-in. Remember, non-built-ins come with a footprint, and I do not necessarily think that you will want to spend 3MB on a git-bugreport executable when you could have it for a couple dozen kilobytes inside git instead.

Ciao,
Dscho

Quoted 309 lines
.gitignore +1 -0
index ea97de83f3..d89bf9e11e 100644
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
Documentation/git-bugreport.txt new file +46 -0
new file mode 100644
index 0000000000..1f9fde5cde
@@ -0,0 +1,46 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [(-o | --output-directory) <path>] [(-s | --suffix) <format>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+This tool is invoked via the typical Git setup process, which means that in some
+cases, it might not be able to launch - for example, if a relevant config file
+is unreadable. In this kind of scenario, it may be helpful to manually gather
+the kind of information listed above when manually asking for help.
+
+OPTIONS
+-------
+-o <path>::
+--output-directory <path>::
+	Place the resulting bug report file in `<path>` instead of the root of
+	the Git repository.
+
+-s <format>::
+--suffix <format>::
+	Specify an alternate suffix for the bugreport name, to create a file
+	named 'git-bugreport-<formatted suffix>'. This should take the form of a
+	link:strftime[3] format string; the current local time will be used.
+
+GIT
+---
+Part of the linkgit:git[1] suite
Makefile +5 -0
index c552312d3f..9e6705061d 100644
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2461,6 +2462,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
bugreport.c new file +105 -0
new file mode 100644
index 0000000000..f473d606f2
@@ -0,0 +1,105 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to share.\n");
+
+	strbuf_addstr(template, _(template_text));
+	return 0;
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	int report = -1;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+	char *option_suffix = "%F-%H%M";
+	int nongit_ok = 0;
+	const char *prefix = NULL;
+	const char *user_relative_path = NULL;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output-directory", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_STRING('s', "suffix", &option_suffix, N_("format"),
+			   N_("specify a strftime format suffix for the filename")),
+		OPT_END()
+	};
+
+	prefix = setup_git_directory_gently(&nongit_ok);
+
+	argc = parse_options(argc, argv, prefix, bugreport_options,
+			     bugreport_usage, 0);
+
+	/* Prepare the path to put the result */
+	strbuf_addstr(&report_path,
+		      prefix_filename(prefix,
+				      option_output ? option_output : ""));
+	strbuf_complete(&report_path, '/');
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+	switch (safe_create_leading_directories(report_path.buf)) {
+	case SCLD_OK:
+	case SCLD_EXISTS:
+		break;
+	default:
+		die(_("could not create leading directories for '%s'"),
+		    report_path.buf);
+	}
+
+	/* Prepare the report contents */
+	get_bug_template(&buffer);
+
+	/* fopen doesn't offer us an O_EXCL alternative, except with glibc. */
+	report = open(report_path.buf, O_CREAT | O_EXCL | O_WRONLY, 0666);
+
+	if (report < 0) {
+		UNLEAK(report_path);
+		die(_("couldn't create a new file at '%s'"), report_path.buf);
+	}
+
+	strbuf_write_fd(&buffer, report);
+	close(report);
+
+	/*
+	 * We want to print the path relative to the user, but we still need the
+	 * path relative to us to give to the editor.
+	 */
+	if (!(prefix && skip_prefix(report_path.buf, prefix, &user_relative_path)))
+		user_relative_path = report_path.buf;
+	fprintf(stderr, _("Created new report at '%s'.\n"),
+		user_relative_path);
+
+	UNLEAK(buffer);
+	UNLEAK(report_path);
+	return !!launch_editor(report_path.buf, NULL, NULL);
+}
command-list.txt +1 -0
index 2087894655..185e5e3f05 100644
@@ -54,6 +54,7 @@ git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
 git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
+git-bugreport                           ancillaryinterrogators
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
 git-check-attr                          purehelpers
strbuf.c +4 -0
index f19da55b07..f1d66c7848 100644
@@ -539,6 +539,10 @@ ssize_t strbuf_write(struct strbuf *sb, FILE *f)
 	return sb->len ? fwrite(sb->buf, 1, sb->len, f) : 0;
 }
 
+ssize_t strbuf_write_fd(struct strbuf *sb, int fd)
+{
+	return sb->len ? write(fd, sb->buf, sb->len) : 0;
+}
 
 #define STRBUF_MAXLINK (2*PATH_MAX)
 
strbuf.h +1 -0
index aae7ac3a82..bbf6204de7 100644
@@ -462,6 +462,7 @@ int strbuf_readlink(struct strbuf *sb, const char *path, size_t hint);
  * NUL bytes.
  */
 ssize_t strbuf_write(struct strbuf *sb, FILE *stream);
+ssize_t strbuf_write_fd(struct strbuf *sb, int fd);
 
 /**
  * Read a line from a FILE *, overwriting the existing contents of
t/t0091-bugreport.sh new file +61 -1
new file mode 100755
index 0000000000..65f664fdac
@@ -0,0 +1,61 @@
+#!/bin/sh
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+
+check_all_headers_populated () {
+	while read -r line
+	do
+		if test "$(grep "$HEADER_PATTERN" "$line")"
+		then
+			echo "$line"
+			read -r nextline
+			if test -z "$nextline"; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport -s check-headers &&
+	check_all_headers_populated <git-bugreport-check-headers.txt &&
+	test_when_finished rm git-bugreport-check-headers.txt
+'
+
+test_expect_success 'dies if file with same name as report already exists' '
+	>>git-bugreport-duplicate.txt &&
+	test_must_fail git bugreport --suffix duplicate &&
+	test_when_finished rm git-bugreport-duplicate.txt
+'
+
+test_expect_success '--output-directory puts the report in the provided dir' '
+	git bugreport -o foo/ &&
+	test_path_is_file foo/git-bugreport-* &&
+	test_when_finished rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	test_i18ngrep usage output &&
+	test_path_is_missing git-bugreport-*
+'
+
+test_expect_success 'runs outside of a git dir' '
+	nongit git bugreport &&
+	test_when_finished rm non-repo/git-bugreport-*
+'
+
+test_expect_success 'can create leading directories outside of a git dir' '
+	nongit git bugreport -o foo/bar/baz &&
+	test_when_finished rm -fr foo/bar/baz
+'
+
+
+test_done
--

2.25.0.265.gbab2e86ba0-goog

#311 lore raw
Quoted 17 lines · Johannes Schindelin

Hi,

Quoted 9 lines · Emily Shaffer
  .gitignore                      |   1 +
  Documentation/git-bugreport.txt |  46 ++++++++++++++
  Makefile                        |   5 ++
  bugreport.c                     | 105 ++++++++++++++++++++++++++++++++
  command-list.txt                |   1 +
  strbuf.c                        |   4 ++
  strbuf.h                        |   1 +
  t/t0091-bugreport.sh            |  61 +++++++++++++++++++
  8 files changed, 224 insertions(+)
Quoted 3 lines

create mode 100644 Documentation/git-bugreport.txt create mode 100644 bugreport.c
create mode 100755 t/t0091-bugreport.sh

Quoted 8 lines

Hmm. I am still _quite_ convinced that this would be much better as a built-in. Remember, non-built-ins come with a footprint, and I do not necessarily think that you will want to spend 3MB on a git-bugreport executable when you could have it for a couple dozen kilobytes inside git instead.

Ciao,
Dscho

Having this command be a stand-alone exe rather than a builtin allows it to have a different linkage. For example, you could include the libcurl and other libraries that are only linked into the transports. And then report version numbers for them if you wanted.

Jeff

#312 lore raw

Hi Jeff,

Quoted 23 lines · Jeff Hostetler
Quoted 22 lines · Johannes Schindelin
Quoted 9 lines · Emily Shaffer
  .gitignore                      |   1 +
  Documentation/git-bugreport.txt |  46 ++++++++++++++
  Makefile                        |   5 ++
  bugreport.c                     | 105 ++++++++++++++++++++++++++++++++
  command-list.txt                |   1 +
  strbuf.c                        |   4 ++
  strbuf.h                        |   1 +
  t/t0091-bugreport.sh            |  61 +++++++++++++++++++
  8 files changed, 224 insertions(+)
Quoted 3 lines

create mode 100644 Documentation/git-bugreport.txt create mode 100644 bugreport.c
create mode 100755 t/t0091-bugreport.sh

Hmm. I am still _quite_ convinced that this would be much better as a built-in. Remember, non-built-ins come with a footprint, and I do not necessarily think that you will want to spend 3MB on a git-bugreport executable when you could have it for a couple dozen kilobytes inside git instead.

Quoted 7 lines
Quoted 2 lines

Ciao,
Dscho

Having this command be a stand-alone exe rather than a builtin allows it to have a different linkage. For example, you could include the libcurl and other libraries that are only linked into the transports. And then report version numbers for them if you wanted.

While that is true, I would argue that this is quite likely using the _wrong_ information if the user has a different git-remote-https in their PATH. In other words, that information could _mislead_ the recipient of the bug report.

Now, other people might argue that the different linkage lets git-bugreport maybe work where git would fail to load a required dynamic library. But again, I would counter that this is a false
assumption: since git-bugreport relies on libgit.a, it essentially _has_ to have the same linkage as git, or a superset thereof.

So: No, I really think this is going the wrong direction.

If we want bugreport to be a first-class citizen, we should treat it as such. That entails making it a built-in. That entails teaching git-remote-https (and potentially other non-builtins) to sprout that option to enquire other information that should be included in the generated part of the bug report.

Ciao,
Dscho

#313 lore raw
Quoted 2 lines · Johannes Schindelin

If we want bugreport to be a first-class citizen, we should treat it as such. That entails making it a built-in.

The question is, for "bugreport", if it is better to be standalone or to be built-in, and I do not think 'first-class' has anything to do with the question. The transport based on curl-library are separate binaries but they are of course first-class citizens.

Pushing and fetching will happen a lot less often than local operations like "add", "commit", "show", and not paying start-up latencies to load code and link libraries that are not necessary for local operations in built-in, while paying the cost of having to duplicate the memory and disk footprint necessary for libgit.a stuff by making the transport binaries separate from the built-ins, was conscious implementation decision we made earlier to prioritize the reduction of overhead of subcommands for local operation. Yes, the decision de-prioritizes the transport, but that does not mean they are not first-class citizens. It just means that the start-up latency for them (e.g. linking with libcurl) is not huge burden for the overall time cost for the network transfer, and it is necessary cost for them to perform what they need to do anyway, while it is something subcommands for local operations should not have to pay.

Quoted 4 lines

That entails teaching
git-remote-https (and potentially other non-builtins) to sprout that option to enquire other information that should be included in the generated part of the bug report.

I think later rounds of "git bugreports" already did so.

As I mentioned in a separate message to JeffH, this is a good thing to do, whether "git bugreport" is a built-in or a standalone.

#314 lore raw
In reply to Jeff Hostetler
Quoted 4 lines · Jeff Hostetler

Having this command be a stand-alone exe rather than a builtin allows it to have a different linkage. For example, you could include the libcurl and other libraries that are only linked into the transports. And then report version numbers for them if you wanted.

I actually do not think that is a good rationale. Unless your version of "git bugreport" links into the same binary as the "transports", it still is possible that the version of cURL (for example) "git bugreport" can learn internally from may not have anything to do with the version of the library used by the transports.

Of course, making "bugreport" a built-in, i.e. the same binary as the non-transport part of Git, is not a solution for that issue, either. As Dscho suggested and recent rounds of "git bugreport" implements, teaching the transport binaries an option to report relevant pieces of information regarding the libraries they use, and making "git bugreport" ask them, is a very good solution for that.

What makes it possible by making "git bugreport" stand-alone is for it to link with libraries that the remainder of Git, including the transports that link with libcurl, has no business linking with (a library to obtain system details for diagnostic purposes, for example). Early versions of "git bugreport" may not link with anything special, but making sure it starts and stays standalone leaves it easier to do so _without_ having to worry about splitting the code that started as a built-in out to make it independent later.

#315 lore raw
Quoted 21 lines · Junio C Hamano
Quoted 4 lines · Jeff Hostetler

Having this command be a stand-alone exe rather than a builtin allows it to have a different linkage. For example, you could include the libcurl and other libraries that are only linked into the transports. And then report version numbers for them if you wanted.

I actually do not think that is a good rationale. Unless your version of "git bugreport" links into the same binary as the "transports", it still is possible that the version of cURL (for example) "git bugreport" can learn internally from may not have anything to do with the version of the library used by the transports.

Of course, making "bugreport" a built-in, i.e. the same binary as the non-transport part of Git, is not a solution for that issue, either. As Dscho suggested and recent rounds of "git bugreport" implements, teaching the transport binaries an option to report relevant pieces of information regarding the libraries they use, and making "git bugreport" ask them, is a very good solution for that.

Quoted 10 lines

What makes it possible by making "git bugreport" stand-alone is for it to link with libraries that the remainder of Git, including the transports that link with libcurl, has no business linking with (a library to obtain system details for diagnostic purposes, for example). Early versions of "git bugreport" may not link with anything special, but making sure it starts and stays standalone leaves it easier to do so _without_ having to worry about splitting the code that started as a built-in out to make it independent later.

Yeah, that makes sense.
Thanks
Jeff

#316 lore raw
In reply to Junio C Hamano

Hi Junio,

Quoted 5 lines · Junio C Hamano

What makes it possible by making "git bugreport" stand-alone is for it to link with libraries that the remainder of Git, including the transports that link with libcurl, has no business linking with (a library to obtain system details for diagnostic purposes, for example).

That would, however, make git-bugreport more fragile than git, i.e. the former might fail to launch under more circumstances than the latter.

Not a good idea for a tool meant to help reporting bugs, to be more fragile to even _start_ than git.

A tool that is intended to help users come up with bug reports, despite the fact that users are totally unfamiliar with the implementation details of Git and hence unable to judge competently what information to include or to omit, needs to be _robust_.

You do not want to end up, for example, calling a stale git-bugreport from an earlier make install (or from a different installation altogether). That would give the recipient not only bogus information, it would be outright misleading.

And that's what I meant by "a stand-alone git-bugreport would not be a 1st-class citizen".

I would see it as a serious design issue if this command is anything but a built-in.

Ciao,
Dscho

#317 lore raw
Quoted 4 lines · Johannes Schindelin

Hi Junio,

On Fri, 6 Mar 2020, Junio C Hamano wrote:

Quoted 8 lines
Quoted 5 lines

What makes it possible by making "git bugreport" stand-alone is for it to link with libraries that the remainder of Git, including the transports that link with libcurl, has no business linking with (a library to obtain system details for diagnostic purposes, for example).

That would, however, make git-bugreport more fragile than git, i.e. the former might fail to launch under more circumstances than the latter.

That's a bug. You can go fix it when it happens.

#318 lore raw

Hi Junio,

Quoted 10 lines · Junio C Hamano
Quoted 8 lines · Johannes Schindelin
Quoted 5 lines · Junio C Hamano

What makes it possible by making "git bugreport" stand-alone is for it to link with libraries that the remainder of Git, including the transports that link with libcurl, has no business linking with (a library to obtain system details for diagnostic purposes, for example).

Quoted 4 lines
Quoted 2 lines

That would, however, make git-bugreport more fragile than git, i.e. the former might fail to launch under more circumstances than the latter.

That's a bug. You can go fix it when it happens.

Heh... yeah, that would be a bug, and the user would not be able to report it via git bugreport...

Which is the whole point of my complaint.

Isn't it obvious that we should not have an independent git-bugreport by now? With a stand-alone git-bugreport, we might

- fail to load the .so files under more circumstances than git would (since we link to libgit.a, we cannot have a subset of dependencies, only a superset, or the same),

- launch a stale git-bugreport from a completely different Git,

- make users angry for wasting 3MB of their diskspace for git-bugreport when only a dozen kilobyte would suffice.

On the other hand, if we make git-bugreport a built-in, I cannot see any downsides.

For me, therefore, having it as a built-in is a clear win. What am I missing?

Ciao,
Dscho

#319 lore raw
Quoted 5 lines · Johannes Schindelin

On the other hand, if we make git-bugreport a built-in, I cannot see any downsides.

For me, therefore, having it as a built-in is a clear win. What am I missing?

The right sense of relative importance between efficiently running the rest of Git by not bloting the main binary and making sure not to ship Git that does not run unless "git bugreport" runs (which makes sure that "bugreport" runs) is what you are missing, I suspect.

Another thing is that you are giving "git bugreport" too much weight and too little credit to inexperienced users, by assuming that we will never hear from them when "bugreport" is incapable to run for them. They will report, with or without "git bugreport", and the more important thing you seem to be missing is that after the initial contact it would become an interactive process---there is no reason to prioritize to ensure that the initial contact has everything that is needed to diagnose any issue without further interaction. "With my build, even 'bugreport' dumps core." is perfectly good place to start.

Besides, wouldn't the ones on platforms, on which "git bugreport" may have trouble running, i.e. the ones on minority and exotic platforms, tend to be self sufficient and capable (both diagnosing the issue for themselves, and reaching out to us as appropriate) ones in practice (e.g. I have NonStop folks in mind)?

#320 lore raw

Hi Junio,

Quoted 5 lines · Junio C Hamano
Quoted 3 lines · Johannes Schindelin

On the other hand, if we make git-bugreport a built-in, I cannot see any downsides.

Quoted 8 lines
Quoted 2 lines

For me, therefore, having it as a built-in is a clear win. What am I missing?

The right sense of relative importance between efficiently running the rest of Git by not bloting the main binary and making sure not to ship Git that does not run unless "git bugreport" runs (which makes sure that "bugreport" runs) is what you are missing, I suspect.

By that reasoning, git bugreport should not be included in core Git.

Quoted 11 lines

Another thing is that you are giving "git bugreport" too much weight and too little credit to inexperienced users, by assuming that we will never hear from them when "bugreport" is incapable to run for them. They will report, with or without "git bugreport", and the more important thing you seem to be missing is that after the initial contact it would become an interactive process---there is no reason to prioritize to ensure that the initial contact has everything that is needed to diagnose any issue without further interaction. "With my build, even 'bugreport' dumps core." is perfectly good place to start.

Quoted 5 lines

Besides, wouldn't the ones on platforms, on which "git bugreport" may have trouble running, i.e. the ones on minority and exotic platforms, tend to be self sufficient and capable (both diagnosing the issue for themselves, and reaching out to us as appropriate) ones in practice (e.g. I have NonStop folks in mind)?

Yes, I can agree that inexperienced users will not give up and keep up the conversation until they see their problems fixed.

I can also agree that inexperienced users will report bugs even if it is not made super easy for them to do so.

I can agree to that _iff_ I ignore my entire experience maintaining Git for Windows, that is.

Sarcasm aside, I think that you underestimate the importance of a good bug reporting tool like git bugreport, and I suspect that Emily does not.

Ciao,
Dscho

#321 lore raw
Quoted 7 lines · Johannes Schindelin
Quoted 5 lines

The right sense of relative importance between efficiently running the rest of Git by not bloting the main binary and making sure not to ship Git that does not run unless "git bugreport" runs (which makes sure that "bugreport" runs) is what you are missing, I suspect.

By that reasoning, git bugreport should not be included in core Git.

The world does not have to be black-or-white. "git bugreport" that covers mainstream platforms in widely-used configurations may be only an 80% solution, but that is better than nothing at all.

Quoted 11 lines
Quoted 11 lines

Another thing is that you are giving "git bugreport" too much weight and too little credit to inexperienced users, by assuming that we will never hear from them when "bugreport" is incapable to run for them. They will report, with or without "git bugreport", and the more important thing you seem to be missing is that after the initial contact it would become an interactive process---there is no reason to prioritize to ensure that the initial contact has everything that is needed to diagnose any issue without further interaction. "With my build, even 'bugreport' dumps core." is perfectly good place to start.

Quoted 8 lines
Quoted 5 lines

Besides, wouldn't the ones on platforms, on which "git bugreport" may have trouble running, i.e. the ones on minority and exotic platforms, tend to be self sufficient and capable (both diagnosing the issue for themselves, and reaching out to us as appropriate) ones in practice (e.g. I have NonStop folks in mind)?

Yes, I can agree that inexperienced users will not give up and keep up the conversation until they see their problems fixed.

Inexperienced users won't be on minority platforms for which we do not have enough resource to get "git bugreport" to run adequetly in the first place. Even if those on minority platforms whose userbase are all technically incapable ones, if they do not help us help them, what can "git bugreport" can do, and more importantly, how would it make a difference between "bugreport" being a built-in vs a standalone?

At this point, I'd have to say that your quibbling does not deserve a serious response and it would be better use of my time to disengage from this thread.

#322 lore raw

Hi Junio,

Quoted 3 lines · Junio C Hamano

At this point, I'd have to say that your quibbling does not deserve a serious response and it would be better use of my time to disengage from this thread.

I thought we were discussing whether it makes sense to leave git-bugreport as a stand-alone executable, or rather make it a built-in.

If my attempts to convince you that it would be better as a built-in strike you as "quibbling", then I should probably pull out from trying to review and help with this here patch series. I really thought I was helping. My apologies for annoying you instead.

Ciao,
Dscho

#323 lore raw
In reply to Johannes Schindelin
Quoted 17 lines · Johannes Schindelin

Hi,

Quoted 4 lines · Emily Shaffer
 .gitignore                      |   1 +
 Documentation/git-bugreport.txt |  46 ++++++++++++++
 Makefile                        |   5 ++
 bugreport.c                     | 105 ++++++++++++++++++++++++++++++++
Quoted 8 lines
 command-list.txt                |   1 +
 strbuf.c                        |   4 ++
 strbuf.h                        |   1 +
 t/t0091-bugreport.sh            |  61 +++++++++++++++++++
 8 files changed, 224 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh
Quoted 5 lines

Hmm. I am still _quite_ convinced that this would be much better as a built-in. Remember, non-built-ins come with a footprint, and I do not necessarily think that you will want to spend 3MB on a git-bugreport executable when you could have it for a couple dozen kilobytes inside git instead.

This is the kind of stuff I really wanted to get straightened out by sending the smaller changeset, so I'm glad to be having this conversation (again, and hopefully for the last time). I read the replies to this mail, which I'm truncating as I think many of them are distracting rather than discussion-worthy, and have tried to summarize:

Builtin:
+ Don't have to call out-of-process to identify 'git version --build-options'
+ Better assurance that we aren't shipping a broken bugreport alongside a new version
- Binary bloat, possible startup time hit
? Libraries will behave identically to where the user is seeing issues
(This point is a possible pro but also a possible con; see similar point in standalone list)

Standalone:
+ Could investigate libraries who aren't behaving the way they should. (This seems like it'd be better addressed by regression tests, and if we're so sure that git-bugreport is working with the info at hand correctly, why don't we just write the git binary that way too?)
+ Avoid binary bloat in the main executable.
- Have to ship a big standalone binary instead.
- To get accurate version/build info, need to query the Git executable in a new process

Of course if I missed capturing something, please add/correct. Some of these concerns are quantifiable - binary size and overhead, for example
- so I'm planning on doing some experiments in the coming days. I plan put together the handful of patches in the latest version of the topic in both standalone and builtin mode, and then gather the following info:

- Binary size of 'git'
- Binary size of 'git-bugreport' when applicable
- Time for "make" following clean
- Time for 'git status' on an identical real-world repo (I'll use ours, without touching the repo state e.g. changing branch or fetching in between)
- Time to editor for 'git bugreport' with the same setup as previous
- Peak memory footprint during 'git status'

And of course, if there's more to compare that I can quantify, please let me know that too. I expect I'll be ready to run the experiments by Monday (taking some time off tomorrow) so there's time for me to hear about other concerns.

I'll hold off on sharing my own preference until after we've got some benchmarking to look at, so I can understand the whole picture.

- Emily

#324 lore raw
Quoted 8 lines · Emily Shaffer

Builtin:
+ Don't have to call out-of-process to identify 'git version --build-options'
+ Better assurance that we aren't shipping a broken bugreport alongside a new version
- Binary bloat, possible startup time hit
? Libraries will behave identically to where the user is seeing issues
(This point is a possible pro but also a possible con; see similar point in standalone list)

- Makes it hard to pull in libraries that "bugreport" alone will want to use in the future without negatively impacting the rest of Git.

Quoted 1 line

- Time to editor for 'git bugreport' with the same setup as previous

This is something we absolutely should totally ignore. Even if the bulitin version takes 1 sec to spawn an editor while a standalone one took only 0.1 sec, if other criteria you listed that are more important favours the builtin solution, we should pick it.

In any case, I think we've wasted enough time on this. Let's see a minimum working version that won't break/affect the rest of Git and ship it standalone.

Thanks.

#325 lore raw
Quoted 11 lines · Junio C Hamano
Quoted 8 lines · Emily Shaffer

Builtin:
+ Don't have to call out-of-process to identify 'git version --build-options'
+ Better assurance that we aren't shipping a broken bugreport alongside a new version
- Binary bloat, possible startup time hit
? Libraries will behave identically to where the user is seeing issues
(This point is a possible pro but also a possible con; see similar point in standalone list)

Quoted 3 lines

- Makes it hard to pull in libraries that "bugreport" alone will want to use in the future without negatively impacting the rest of Git.

And of course we should never do that, lest git bugreport won't even load because of it while git would.

So even in the purely speculative scenario that we _wanted_ to add more libraries (which we don't, why should we?), we wouldn't actually do it.

Quoted 10 lines
Quoted 1 line

- Time to editor for 'git bugreport' with the same setup as previous

This is something we absolutely should totally ignore. Even if the bulitin version takes 1 sec to spawn an editor while a standalone one took only 0.1 sec, if other criteria you listed that are more important favours the builtin solution, we should pick it.

In any case, I think we've wasted enough time on this. Let's see a minimum working version that won't break/affect the rest of Git and ship it standalone.

I still disagree with that, but it seems that no amount of arguments will convince Junio, and he's dead set on the standalone version.

#326 lore raw
Quoted 8 lines · Johannes Schindelin
Quoted 3 lines · Junio C Hamano

In any case, I think we've wasted enough time on this. Let's see a minimum working version that won't break/affect the rest of Git and ship it standalone.

I still disagree with that, but it seems that no amount of arguments will convince Junio, and he's dead set on the standalone version.

In this case, I'll leave the build settings alone and focus on other comments on this patch. Will try to work on it today.

- Emily

#327 lore raw
In reply to Emily Shaffer

Hi Emily,

Quoted 38 lines · Emily Shaffer
Quoted 15 lines · Johannes Schindelin
Quoted 4 lines · Emily Shaffer
 .gitignore                      |   1 +
 Documentation/git-bugreport.txt |  46 ++++++++++++++
 Makefile                        |   5 ++
 bugreport.c                     | 105 ++++++++++++++++++++++++++++++++
Quoted 8 lines
 command-list.txt                |   1 +
 strbuf.c                        |   4 ++
 strbuf.h                        |   1 +
 t/t0091-bugreport.sh            |  61 +++++++++++++++++++
 8 files changed, 224 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh
Quoted 5 lines

Hmm. I am still _quite_ convinced that this would be much better as a built-in. Remember, non-built-ins come with a footprint, and I do not necessarily think that you will want to spend 3MB on a git-bugreport executable when you could have it for a couple dozen kilobytes inside git instead.

This is the kind of stuff I really wanted to get straightened out by sending the smaller changeset, so I'm glad to be having this conversation (again, and hopefully for the last time). I read the replies to this mail, which I'm truncating as I think many of them are distracting rather than discussion-worthy, and have tried to summarize:

Builtin:
+ Don't have to call out-of-process to identify 'git version --build-options'
+ Better assurance that we aren't shipping a broken bugreport alongside a new version
- Binary bloat, possible startup time hit
? Libraries will behave identically to where the user is seeing issues
(This point is a possible pro but also a possible con; see similar point in standalone list)

Quoted 5 lines

Standalone:
+ Could investigate libraries who aren't behaving the way they should. (This seems like it'd be better addressed by regression tests, and if we're so sure that git-bugreport is working with the info at hand correctly, why don't we just write the git binary that way too?)

No, you're linking with libgit.a, you will have _at least_ the same set of dependencies as git. I made that point before.

Quoted 1 line

+ Avoid binary bloat in the main executable.

What? What bloat? Have you measured this?

Quoted 3 lines

- Have to ship a big standalone binary instead.
- To get accurate version/build info, need to query the Git executable in a new process

Worse. You might pick up inconsistent info from unrelated Git versions, and would be none the wiser. I made that point before.

For example, if you call git bugreport in a Git version that does not even include git-bugreport, you could pick up a git-bugreport from a _different_ Git version. This is a very real scenario on Windows, where we have many applications that ship with their own minimal version of Git for Windows.

Quoted 6 lines

Of course if I missed capturing something, please add/correct. Some of these concerns are quantifiable - binary size and overhead, for example
- so I'm planning on doing some experiments in the coming days. I plan put together the handful of patches in the latest version of the topic in both standalone and builtin mode, and then gather the following info:

Quoted 8 lines

- Binary size of 'git'
- Binary size of 'git-bugreport' when applicable
- Time for "make" following clean
- Time for 'git status' on an identical real-world repo (I'll use ours, without touching the repo state e.g. changing branch or fetching in between)
- Time to editor for 'git bugreport' with the same setup as previous
- Peak memory footprint during 'git status'

Something that you cannot quantify, of course, is the robustness. Which in my mind is more important than all of the above, and of course you would only be able to guarantee that with a built-in version.

Quoted 7 lines

And of course, if there's more to compare that I can quantify, please let me know that too. I expect I'll be ready to run the experiments by Monday (taking some time off tomorrow) so there's time for me to hear about other concerns.

I'll hold off on sharing my own preference until after we've got some benchmarking to look at, so I can understand the whole picture.

Why? Why not strong-arm your preference? Junio and I are not shy doing the same, and those are _your_ patches. Junio is clearly not interested in the command at all, but you are clearly interested in it [*1*]. You should have more than just the final say over this.

Ciao,
Dscho

Footnote *1*: I am *obviously* interested in this because it might potentially make my life as Git for Windows' maintainer a lot easier. If it is robust enough that I don't get bug reports about git-bugreport itself.

#328 lore raw
Quoted 10 lines · Johannes Schindelin

Hi Emily,

Quoted 2 lines · Emily Shaffer

I'll hold off on sharing my own preference until after we've got some benchmarking to look at, so I can understand the whole picture.

Why? Why not strong-arm your preference? Junio and I are not shy doing the same, and those are _your_ patches. Junio is clearly not interested in the command at all, but you are clearly interested in it [*1*]. You should have more than just the final say over this.

We are different people, and we approach problems and disagreements differently. I am almost never the type to strong-arm my preference; I'd rather listen to other arguments and look at data, which is why I proposed the experiments I mentioned here. As for you and Junio not being shy, as you put it, I found most of the discussion offputting, and the tone used absolutely contributed to me finding other things to work on instead of coming back to this minefield sooner.

- Emily

#329 lore raw
In reply to Emily Shaffer
Quoted 3 lines · Emily Shaffer

This is the kind of stuff I really wanted to get straightened out by sending the smaller changeset, so I'm glad to be having this conversation (again, and hopefully for the last time).

I actually have a suspicion that "git bugreport" that is spawned via "git" wrapper is a bad idea (in other words, /usr/bin/git-bug that is totally standalone may be better).

The thing is, anything launched by "git" as its subcommand (be it standalone or builtin) sees an environment already modified by "git", so inspecting say "$PATH" from "git bugreport" (be it standalone or builtin) does not show what the end-user, who may be having trouble, actually has. The mangling of $PATH alone happens to be simple and (we may think) we can easily reverse without losing bits, but there would probably other differences that we think is so subtle and insignificant, right now in this discussion without having actual end-user who is having trouble in front of us, that having "git" layer in between may hide from us.

Having "git-bug" a totally separate tool, that does not even go through git.c::cmd_main(), would also allow (and force) us to treat "git" just like any system component on the end-users' system whose versions and configurations may affect the use of "git" by the end user. The tool, instead of relying on git_version_string[] that is determined at the compile time, would ask "git version" just like the end-users would when asked by us "what version of git do you run?" It also means that "git-bug" can be updated at different cadence and the mismatch of versions between it and "git" does not matter at all.

Having said all that, after suggesting to make the tool even more distant from the remainder of the binary, quite honestly, I do not even want to see us wasting any more time on builtin/standalone at this point, and instead would like to see reports from the end-users produced by a lets-start-small version of the tool and measure how having the tool actually helps.

Thanks.

#330 lore raw

Hi Junio,

Quoted 9 lines · Junio C Hamano
Quoted 3 lines · Emily Shaffer

This is the kind of stuff I really wanted to get straightened out by sending the smaller changeset, so I'm glad to be having this conversation (again, and hopefully for the last time).

I actually have a suspicion that "git bugreport" that is spawned via "git" wrapper is a bad idea (in other words, /usr/bin/git-bug that is totally standalone may be better).

The obvious downside of /usr/bin/git-bug, of course, is that it has no way to provide accurate data regarding, say, the cURL version in use.

That's a pretty severe downside if you are truly interested in helping bug reporters and bug receivers.

Ciao,
Dscho

Quoted 32 lines

The thing is, anything launched by "git" as its subcommand (be it standalone or builtin) sees an environment already modified by "git", so inspecting say "$PATH" from "git bugreport" (be it standalone or builtin) does not show what the end-user, who may be having trouble, actually has. The mangling of $PATH alone happens to be simple and (we may think) we can easily reverse without losing bits, but there would probably other differences that we think is so subtle and insignificant, right now in this discussion without having actual end-user who is having trouble in front of us, that having "git" layer in between may hide from us.

Having "git-bug" a totally separate tool, that does not even go through git.c::cmd_main(), would also allow (and force) us to treat "git" just like any system component on the end-users' system whose versions and configurations may affect the use of "git" by the end user. The tool, instead of relying on git_version_string[] that is determined at the compile time, would ask "git version" just like the end-users would when asked by us "what version of git do you run?" It also means that "git-bug" can be updated at different cadence and the mismatch of versions between it and "git" does not matter at all.

Having said all that, after suggesting to make the tool even more distant from the remainder of the binary, quite honestly, I do not even want to see us wasting any more time on builtin/standalone at this point, and instead would like to see reports from the end-users produced by a lets-start-small version of the tool and measure how having the tool actually helps.

Thanks.

#331 lore raw
Quoted 6 lines · Johannes Schindelin
Quoted 3 lines

I actually have a suspicion that "git bugreport" that is spawned via "git" wrapper is a bad idea (in other words, /usr/bin/git-bug that is totally standalone may be better).

The obvious downside of /usr/bin/git-bug, of course, is that it has no way to provide accurate data regarding, say, the cURL version in use.

Sorry, but I do not see what's new in your argument this time.

I thought we've already established that the best solution for the "accurate data regarding, say, the cURL version in use" is to use your earlier idea, i.e. give an interface to git-remote-curl so that "git bugreport" can ask it such details, because "git bugreport" that is known by git.c::cmd_main(), whether it is builtin or standalone, is *NOT* linked to the transport anyway.

And the same interface can be used by an independent "git-bug", or even by the end user who is sitting at a terminal when asked by git developers "what version of curl library does your build link with?".

#332 lore raw

Hi Junio,

Quoted 18 lines · Junio C Hamano
Quoted 6 lines · Johannes Schindelin
Quoted 3 lines

I actually have a suspicion that "git bugreport" that is spawned via "git" wrapper is a bad idea (in other words, /usr/bin/git-bug that is totally standalone may be better).

The obvious downside of /usr/bin/git-bug, of course, is that it has no way to provide accurate data regarding, say, the cURL version in use.

Sorry, but I do not see what's new in your argument this time.

I thought we've already established that the best solution for the "accurate data regarding, say, the cURL version in use" is to use your earlier idea, i.e. give an interface to git-remote-curl so that "git bugreport" can ask it such details, because "git bugreport" that is known by git.c::cmd_main(), whether it is builtin or standalone, is *NOT* linked to the transport anyway.

Quoted 4 lines

And the same interface can be used by an independent "git-bug", or even by the end user who is sitting at a terminal when asked by git developers "what version of curl library does your build link with?".

You are _asking_ for version mismatch here. If git-bug is a totally standalone program, then it cannot rely at all on git to give it the information it wants. Forward compatibility becomes another concern.

In other words, all the avoidably incurred complexity will come back to make the entire git bugreport not worth all the effort Emily put into it.

Ciao,
Dscho

#333 lore raw

PATCH v9 3/5 bugreport: gather git version and build info

In reply to Emily Shaffer

Knowing which version of Git a user has and how it was built allows us to more precisely pin down the circumstances when a certain issue occurs, so teach bugreport how to tell us the same output as 'git version --build-options'.

It's not ideal to directly call 'git version --build-options' because that output goes to stdout. Instead, wrap the version string in a helper within help.[ch] library, and call that helper from within the bugreport library.

---
 Documentation/git-bugreport.txt |  4 +++
 bugreport.c                     | 19 +++++++++++++-
 help.c                          | 46 ++++++++++++++++++++-------------
 help.h                          |  1 +
 4 files changed, 51 insertions(+), 19 deletions(-)
Documentation/git-bugreport.txt +4 -0
index 1f9fde5cde..f44ae8cbe7 100644
@@ -23,6 +23,10 @@ The following information is requested from the user:
  - Expected behavior
  - Actual behavior
 
+The following information is captured automatically:
+
+ - 'git version --build-options'
+
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
 is unreadable. In this kind of scenario, it may be helpful to manually gather
bugreport.c +18 -1
index f473d606f2..c20437dc6a 100644
@@ -1,8 +1,17 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, _("git version:\n"));
+	get_version_info(sys_info, 1);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
@@ -32,6 +41,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -79,6 +93,9 @@ int cmd_main(int argc, const char **argv)
 	/* Prepare the report contents */
 	get_bug_template(&buffer);
 
+	get_header(&buffer, _("System Info"));
+	get_system_info(&buffer);
+
 	/* fopen doesn't offer us an O_EXCL alternative, except with glibc. */
 	report = open(report_path.buf, O_CREAT | O_EXCL | O_WRONLY, 0666);
 
help.c +28 -18

61 lines not on this page · raw message

help.h +1 -0
index 9071894e8c..500521b908 100644
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void get_version_info(struct strbuf *buf, int show_build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog
#334 lore raw

Re: [PATCH v9 3/5] bugreport: gather git version and build info

Documentation/git-bugreport.txt The following information is requested from the user: · line 28

Quoted 3 lines · Emily Shaffer
Documentation/git-bugreport.txt +3 -0
+The following information is captured automatically:
+
+ - 'git version --build-options'

Good.

Do make sure to give us the same information as if the end user typed that command from the command line. There may be more than one directory on $PATH that has "git" in it; we want to see the one that is giving trouble to the end user, not necessarily the one that came from the same source as running "git bugreport" was compiled from, so it is a good idea not to rely on the contents of static git_version_string[] array that came in libgit.a that the bugreport tool happened to be linked with.

It may not be in the scope of this step, but we'd want to see versions of executables for the python interpreter (2vs3), the tcl/tk interpreter (i.e. wish), the shell (i.e. bash, dash, etc.).

In addition to them, we would want to see versions of some of our own binaries (i.e. "git", and transports that are separate executable files). It would lead to a confusing diagnositic session if the end-user's PATH has /usr/bin and then $HOME/bin, and the end-user has a newer version of "git" with "bugreport" only in $HOME/bin. The user may think s/he is using the latest self-compiled "git", and "bugreport" might even agree if it trusts anything determined when "bugreport" was compiled, but in reality the version of "git" used largely was coming from "/usr/bin" except for any new subcommands, or something silly like that, and asking the environment just like the end user does is the only way to make sure we are getting the true piture.

Thanks.

#335 lore raw

PATCH v9 4/5 bugreport: add uname info

In reply to Emily Shaffer

The contents of uname() can give us some insight into what sort of system the user is running on, and help us replicate their setup if need be. The domainname field is not guaranteed to be available, so don't collect it.

---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 16 +++++++++++++++-
 2 files changed, 16 insertions(+), 1 deletion(-)
Documentation/git-bugreport.txt +1 -0
index f44ae8cbe7..17b0d14e8d 100644
@@ -26,6 +26,7 @@ The following information is requested from the user:
 The following information is captured automatically:
 
  - 'git version --build-options'
+ - uname sysname, release, version, and machine strings
 
 This tool is invoked via the typical Git setup process, which means that in some
 cases, it might not be able to launch - for example, if a relevant config file
bugreport.c +15 -1
index c20437dc6a..857b4f712c 100644
@@ -7,10 +7,24 @@
 
 static void get_system_info(struct strbuf *sys_info)
 {
+	struct utsname uname_info;
+
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, _("git version:\n"));
 	get_version_info(sys_info, 1);
-	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, _("uname() failed with error '%s' (%d)\n"),
+			    strerror(errno),
+			    errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
Signature · 1 line
2.25.0.265.gbab2e86ba0-goog