Skip to contentSkip to conversation

bugreport: add tool to generate debugging info

PATCH Superseded Emily Shaffer 13 participants 373 messages Aug 15, 2019 – Apr 7, 2020 Page 1 of 3

Subsystems (2)

v1 of 13 Superseded Latest is v13 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 bugreport: add tool to generate debugging info

Make it easier for users who encounter a bug to send a report by collecting some state information, intended to be viewed by humans familiar with Git.

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Also, teach Git to write down its own version, the version of some of its dependencies, the operating system information, the effective gitconfig, the configured hooks, and the contents of $GIT_DIR/logs. Finally, make sure Git asks the user to review the contents of the report after it's generated.

If users can send us a well-written bug report complete with system information, a remote we may be able to clone, and a reflog showing us where the problem occurred, we can reduce the number of questions and answers which must travel between the reporter and the Git contributor.

Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.

---
There are some things I'm not certain about from this patch, I'd appreciate feedback.

- Do we want to advertise the Git mailing list for bug reports?
- Which config options should we filter to avoid accidentally receiving credentials?
- At the moment, the test is trying to check "everything we thought we would populate got populated" - that seemed to me like it would hold up best to changes to the set of information being collected. But maybe there's something more robust we can do.

And of course, if there are suggestions for other things to include in the report I'm interested in adding.

An example of the output can be found here:
https://gist.github.com/nasamuffin/2e320892f5c2147e829cbcf5bd0759a2

Thanks.
- Emily

 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 48 ++++++++++++++++++
 Makefile                        |  1 +
 command-list.txt                |  1 +
 git-bugreport.sh                | 86 +++++++++++++++++++++++++++++++++
 t/t0091-bugreport.sh            | 41 ++++++++++++++++
 6 files changed, 178 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100755 git-bugreport.sh
 create mode 100755 t/t0091-bugreport.sh
.gitignore +1 -0
index 521d8f4fb4..b4f5433084 100644
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
Documentation/git-bugreport.txt new file +48 -0

48 lines not on this page · raw message

Makefile +1 -0
index f9255344ae..69801a1c45 100644
@@ -606,6 +606,7 @@ TEST_PROGRAMS_NEED_X =
 unexport CDPATH
 
 SCRIPT_SH += git-bisect.sh
+SCRIPT_SH += git-bugreport.sh
 SCRIPT_SH += git-difftool--helper.sh
 SCRIPT_SH += git-filter-branch.sh
 SCRIPT_SH += git-merge-octopus.sh
command-list.txt +1 -0
index a9ac72bef4..be5a605047 100644
@@ -54,6 +54,7 @@ git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
 git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
+git-bugreport                           ancillaryinterrogators
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
 git-check-attr                          purehelpers
git-bugreport.sh new file +86 -0

86 lines not on this page · raw message

t/t0091-bugreport.sh new file +41 -0

41 lines not on this page · raw message

Signature · 1 line
2.23.0.rc1.153.gdeed80330f-goog
#1 lore raw
Quoted 3 lines · Emily Shaffer

Make it easier for users who encounter a bug to send a report by collecting some state information, intended to be viewed by humans familiar with Git.

This is an excellent idea! VFS for Git has a similar "diagnose" command that collects logs, config, and other details (like packfile sizes and loose object counts). That has been a critical tool for supporting users.

Quoted 16 lines

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Also, teach Git to write down its own version, the version of some of its dependencies, the operating system information, the effective gitconfig, the configured hooks, and the contents of $GIT_DIR/logs. Finally, make sure Git asks the user to review the contents of the report after it's generated.

If users can send us a well-written bug report complete with system information, a remote we may be able to clone, and a reflog showing us where the problem occurred, we can reduce the number of questions and answers which must travel between the reporter and the Git contributor.

Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.

Quoted 6 lines

---
There are some things I'm not certain about from this patch, I'd appreciate feedback.

- Do we want to advertise the Git mailing list for bug reports?

That is possible. Isn't there another mailing list for git users?

I could see a patch added on top of this for git-for-windows/git that changes the instructions to create issues on GitHub.

Quoted 2 lines

- Which config options should we filter to avoid accidentally receiving credentials?

The remote URLs are pretty sensitive. Not only do users sometimes put passwords or PATs into their URLs, the literal name of the repo could be a secret.

git-bugreport.sh

Quoted 120 lines

- At the moment, the test is trying to check "everything we thought we would populate got populated" - that seemed to me like it would hold up best to changes to the set of information being collected. But maybe there's something more robust we can do.

And of course, if there are suggestions for other things to include in the report I'm interested in adding.

An example of the output can be found here:
https://gist.github.com/nasamuffin/2e320892f5c2147e829cbcf5bd0759a2

Thanks.
- Emily

 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 48 ++++++++++++++++++
 Makefile                        |  1 +
 command-list.txt                |  1 +
 git-bugreport.sh                | 86 +++++++++++++++++++++++++++++++++
 t/t0091-bugreport.sh            | 41 ++++++++++++++++
 6 files changed, 178 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100755 git-bugreport.sh
 create mode 100755 t/t0091-bugreport.sh
.gitignore +1 -0
index 521d8f4fb4..b4f5433084 100644
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
Documentation/git-bugreport.txt new file +48 -0
new file mode 100644
index 0000000000..c5f45bbee8
@@ -0,0 +1,48 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [-o | --output <path>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+The following information is captured automatically:
+
+ - Git version (`git version --build-options`)
+ - Machine information (`uname -a`)
+ - Versions of various dependencies
+ - Git config contents (`git config --show-origin --list`)
+ - The contents of all configured git-hooks in `.git/hooks/`
+ - The contents of `.git/logs`
+
+OPTIONS
+-------
+-o [<path>]::
+--output [<path>]::
+	Place the resulting bug report file in <path> instead of the root of the
+	Git repository.
+
+NOTE
+----
+Bug reports can be sent to [email protected].
+
+GIT
+---
+Part of the linkgit:git[1] suite
Makefile +1 -0
index f9255344ae..69801a1c45 100644
@@ -606,6 +606,7 @@ TEST_PROGRAMS_NEED_X =
 unexport CDPATH
 
 SCRIPT_SH += git-bisect.sh
+SCRIPT_SH += git-bugreport.sh
 SCRIPT_SH += git-difftool--helper.sh
 SCRIPT_SH += git-filter-branch.sh
 SCRIPT_SH += git-merge-octopus.sh
command-list.txt +1 -0
index a9ac72bef4..be5a605047 100644
@@ -54,6 +54,7 @@ git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
 git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
+git-bugreport                           ancillaryinterrogators
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
 git-check-attr                          purehelpers
git-bugreport.sh new file +0 -0
new file mode 100755
index 0000000000..2200703a51

At first I was alarmed by "What? another shell script?" but this command should prioritize flexibility and extensibility over speed. Running multiple processes shouldn't be too taxing for what we are trying to do here.

Quoted 31 lines
git-bugreport.sh new file +30 -0
@@ -0,0 +1,86 @@
+#!/bin/sh
+
+print_filenames_and_content() {
+while read -r line; do
+	echo "$line";
+	echo "========";
+	cat "$line";
+	echo;
+done
+}
+
+generate_report_text() {
+
+	# Generate a form for the user to fill out with their issue.
+	gettextln "Thank you for filling out a Git bug report!"
+	gettextln "Please answer the following questions to help us understand your issue."
+	echo
+	gettextln "What did you do before the bug happened? (Steps to reproduce your issue)"
+	echo
+	gettextln "What did you expect to happen? (Expected behavior)"
+	echo
+	gettextln "What happened instead? (Actual behavior)"
+	echo
+	gettextln "Anything else you want to add:"
+	echo
+	gettextln "Please review the rest of the bug report below."
+	gettextln "You can delete any lines you don't wish to send."
+	echo
+
+	echo "[System Information]"

git-bugreport.sh line 40

Quoted 10 lines
git-bugreport.sh new file +10 -0
+	git version --build-options
+	uname -a
+	curl-config --version
+	ldd --version
+	echo
+
+	echo "[Git Config]"
+	# TODO: Pass this through grep -v to avoid users sending us their credentials.
+	git config --show-origin --list
+	echo

Config options to consider stripping out:

	*url*
	*pass* (anything "password" but also "sendmail.smtppass")

git-bugreport.sh line 44

Quoted 3 lines
git-bugreport.sh new file +3 -0
+	echo "[Configured Hooks]"
+	find "$GIT_DIR/hooks/" -type f | grep -v "\.sample$" | print_filenames_and_content
+	echo

Remove the sample hooks, but focus on the others. Will this look like garbage if a hook is a binary file?

git-bugreport.sh line 48

Quoted 4 lines
git-bugreport.sh new file +4 -0
+
+	echo "[Git Logs]"
+	find "$GIT_DIR/logs" -type f | print_filenames_and_content
+	echo

As mentioned before, I've sometimes found it helpful to know the data shape for the object store. Having a few extra steps such as the following could be nice:

echo "[Loose Objects]"
for objdir in $(find "$GIT_DIR/objects/??" -type d) do
echo "$objdir: $(ls $objdir | wc -l)"
done
echo

echo "[Pack Data]"
ls -l "$GIT_DIR/objects/pack"
echo

echo "[Object Info Data]"
ls -lR "$GIT_DIR/objects/info"
echo

echo "[Alternates File]"
echo "========"
cat "$GIT_DIR/objects/info/alternates"
echo

That last one will collect information on the commit-graph file, even if it is an incremental file.

Quoted 76 lines
git-bugreport.sh new file +38 -0
+
+}
+
+USAGE="[-o | --output <path>]"
+
+SUBDIRECTORY_OK=t
+OPTIONS_SPEC=
+. git-sh-setup
+. git-sh-i18n
+
+basedir="$PWD"
+while :
+do
+	case "$1" in
+	-o|--output)
+		shift
+		basedir="$1"
+		shift
+		continue
+		;;
+	"")
+		break
+		;;
+	*)
+		usage
+		;;
+	esac
+done
+
+
+# Create bugreport file
+BUGREPORT_FILE="$basedir/git-bugreport-$(whoami)-$(hostname)-$(date -Iminutes)"
+
+generate_report_text >$BUGREPORT_FILE
+
+git_editor $BUGREPORT_FILE
+
+eval_gettextln "Your new bug report is in \$BUGREPORT_FILE."
t/t0091-bugreport.sh new file +31 -0
new file mode 100755
index 0000000000..6eb2ee4f66
@@ -0,0 +1,41 @@
+#!/bin/bash
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if [$(grep $HEADER_PATTERN $line)]; then
+			read -r nextline
+			if [-z $nextline]; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	check_all_headers_populated <git-bugreport-* &&
+	rm git-bugreport-*
+'
+
+test_expect_success '--output puts the report in the provided dir' '
+	mkdir foo/ &&
+	git bugreport -o foo/ &&
+	test -f foo/git-bugreport-* &&

t/t0091-bugreport.sh line 41

Quoted 10 lines
t/t0091-bugreport.sh new file +10 -0
+	rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	grep usage output &&
+	test ! -f git-bugreport-*
+'
+
+test_done

I think this is a great start, and I'll take some time later to try it out.

Thanks,
-Stolee

#2 lore raw
Quoted 4 lines · Derrick Stolee

Config options to consider stripping out:

	*url*
	*pass* (anything "password" but also "sendmail.smtppass")

Blacklisting? I wonder if users feel safer if these are limited to known-benign ones.

Quoted 6 lines

git-bugreport.sh line 44

Quoted 3 lines
git-bugreport.sh new file +3 -0
+	echo "[Configured Hooks]"
+	find "$GIT_DIR/hooks/" -type f | grep -v "\.sample$" | print_filenames_and_content
+	echo

Remove the sample hooks, but focus on the others. Will this look like garbage if a hook is a binary file?

This makes me feel very nervous. $GIT_DIR/hooks/ are private and people can hardcode credentials in them; $GIT_DIR/hooks/pre-foo may be written toread from $GIT_DIR/hooks/mypassword with the knowledge that there won't be any "mypassword" hook.

#3 lore raw
Quoted 9 lines · Junio C Hamano
Quoted 4 lines · Derrick Stolee

Config options to consider stripping out:

	*url*
	*pass* (anything "password" but also "sendmail.smtppass")

Blacklisting? I wonder if users feel safer if these are limited to known-benign ones.

I think a whitelist of config options to print would grow stale immediately, and the options we're missing would be very likely to be configs to turn on new experimental features - which is probably what we most want the bugreport for.

Quoted 8 lines
Quoted 7 lines

git-bugreport.sh line 44

Quoted 4 lines
git-bugreport.sh new file +3 -0
+	echo "[Configured Hooks]"
+	find "$GIT_DIR/hooks/" -type f | grep -v "\.sample$" | print_filenames_and_content
+	echo

Remove the sample hooks, but focus on the others. Will this look like garbage if a hook is a binary file?

Quoted 4 lines

This makes me feel very nervous. $GIT_DIR/hooks/ are private and people can hardcode credentials in them; $GIT_DIR/hooks/pre-foo may be written toread from $GIT_DIR/hooks/mypassword with the knowledge that there won't be any "mypassword" hook.

Hmm. I think the list of valid hooks isn't one that changes often, but it's also not enumerated in some machine-parseable way - it exists in Documentation/githooks.txt but that's all. I'd still be a little worried about bitrot... I think it's probably better to list the filenames in $GIT_DIR/hooks but not print their contents. I'll modify it.

- Emily

#4 lore raw
Quoted 4 lines · Emily Shaffer

I think a whitelist of config options to print would grow stale immediately, and the options we're missing would be very likely to be configs to turn on new experimental features - which is probably what we most want the bugreport for.

The implementation of your "git bugreport" command comes from the same version of Git as such an experimental feature you want feedback to is shipped with, so I am not sure where your concern about staleness comes from.

If you really care about getting quality reports, you need to earn users' trust (one way of doing so would be to maintain a good "list of benign configuration knobs whose values help diagnosing issues"), and you need to make sure we obtain relevant pieces of information. Both of these things are something you need to actively work on.

We have trained ourselves to the point that we consider it a bug if you add a new command git-bugreport without adding a corresponding pattern in the .gitignore file. And thanks to that discipline, we have been fairly good at keeping .gitignore up to date. I do not see a reason why we cannot do something similar to the registry of configuration variables that we care about, which may be shipped as part of "git bugreport" command.

#5 lore raw
Quoted 18 lines · Junio C Hamano
Quoted 4 lines · Emily Shaffer

I think a whitelist of config options to print would grow stale immediately, and the options we're missing would be very likely to be configs to turn on new experimental features - which is probably what we most want the bugreport for.

The implementation of your "git bugreport" command comes from the same version of Git as such an experimental feature you want feedback to is shipped with, so I am not sure where your concern about staleness comes from.

If you really care about getting quality reports, you need to earn users' trust (one way of doing so would be to maintain a good "list of benign configuration knobs whose values help diagnosing issues"), and you need to make sure we obtain relevant pieces of information. Both of these things are something you need to actively work on.

Quoted 7 lines

We have trained ourselves to the point that we consider it a bug if you add a new command git-bugreport without adding a corresponding pattern in the .gitignore file. And thanks to that discipline, we have been fairly good at keeping .gitignore up to date. I do not see a reason why we cannot do something similar to the registry of configuration variables that we care about, which may be shipped as part of "git bugreport" command.

I think comparing this habit to the .gitignore isn't quite fair - .gitignore tells me I forgot to add my new command binary to it, when I run git status to see what I need to add to my commit with new command.

But, I agree that it's not going to be possible to create a completely leakproof blacklisting heuristic here. So I'll use a whitelist for the next patchset.

- Emily

#6 lore raw
Quoted 4 lines · Emily Shaffer

I think comparing this habit to the .gitignore isn't quite fair - .gitignore tells me I forgot to add my new command binary to it, when I run git status to see what I need to add to my commit with new command.

That is why I said that we need to actively work on, if we care about getting quality reports.

I do not think it is unreasonable to expect the build procedure for "git bugreport" to involve scanning in Documentation/config/ to pick up variable names, annotated in such a way that is invisible to AsciiDoc to allow us tell which ones are sensitive and which ones are not. A test in t/ could even check if a documented configuration variable has such an annotation. A commit that adds configuration variables without documentiong them does exist, but variables without documentation are (1) bugs, and (2) are not worth serious engineering effort on until they get documented.

I am not saying that you must implement the whitelist exactly like the above. I am not even saying that this must be whitelist and not blacklist---you'd have the same issue maintaining the list anyway.

The above is merely to illustrate the reason why I think that the kind of "active work" to make sure that the list will not go stale would be feasible.

Thanks.

#7 lore raw
Quoted 10 lines · Junio C Hamano
Quoted 4 lines · Emily Shaffer

I think comparing this habit to the .gitignore isn't quite fair - .gitignore tells me I forgot to add my new command binary to it, when I run git status to see what I need to add to my commit with new command.

That is why I said that we need to actively work on, if we care about getting quality reports.

Quoted 9 lines

I do not think it is unreasonable to expect the build procedure for "git bugreport" to involve scanning in Documentation/config/ to pick up variable names, annotated in such a way that is invisible to AsciiDoc to allow us tell which ones are sensitive and which ones are not. A test in t/ could even check if a documented configuration variable has such an annotation. A commit that adds configuration variables without documentiong them does exist, but variables without documentation are (1) bugs, and (2) are not worth serious engineering effort on until they get documented.

Interesting. I think I have an idea for a way to do this, but it ends up fairly large; I'll send proof of concept as a follow-on patch to this one. Thanks for the suggestions.

- Emily

#8 lore raw
In reply to Derrick Stolee

Hi,

Quoted 11 lines · Derrick Stolee

git-bugreport.sh

Quoted 5 lines · Emily Shaffer
git-bugreport.sh new file +0 -0
new file mode 100755
index 0000000000..2200703a51

At first I was alarmed by "What? another shell script?" but this command should prioritize flexibility and extensibility over speed. Running multiple processes shouldn't be too taxing for what we are trying to do here.

Git for Windows sometimes receives bug reports about Bash not being able to start (usually it is a DLL base address problem, related to the way Cygwin and MSYS2 emulate fork()).

In such a case, git bugreport would only be able to offer a reason for yet another bug report instead of adding useful metadata.

Something to keep in mind when deciding how to implement this command.

Ciao,
Dscho

#9 lore raw
Quoted 16 lines · Johannes Schindelin

Hi,

Quoted 11 lines · Derrick Stolee

git-bugreport.sh

Quoted 5 lines · Emily Shaffer
git-bugreport.sh new file +0 -0
new file mode 100755
index 0000000000..2200703a51

At first I was alarmed by "What? another shell script?" but this command should prioritize flexibility and extensibility over speed. Running multiple processes shouldn't be too taxing for what we are trying to do here.

Quoted 3 lines

Git for Windows sometimes receives bug reports about Bash not being able to start (usually it is a DLL base address problem, related to the way Cygwin and MSYS2 emulate fork()).

Hmm. In a case like this, then, how is someone using GfW at all? Embarrassingly, I don't actually have a way to try it out for myself. It seems there's no GUI, and users are using it through the command line in mingw, so when you say "bash doesn't start" do you mean "users can't use the mingw prompt at all"?

Quoted 8 lines

In such a case, git bugreport would only be able to offer a reason for yet another bug report instead of adding useful metadata.

Something to keep in mind when deciding how to implement this command.

Ciao,
Dscho

Yeah, that's an interesting point, thanks for bringing it up. So, in the case when bash won't start at all, what does that failure look like? How much of Git can users still use? For example, would they be able to get far enough to run a binary git-bugreport?

- Emily

#10 lore raw

Hi Emily,

Quoted 16 lines · Emily Shaffer
Quoted 15 lines · Johannes Schindelin
Quoted 11 lines · Derrick Stolee

git-bugreport.sh

Quoted 5 lines · Emily Shaffer
git-bugreport.sh new file +0 -0
new file mode 100755
index 0000000000..2200703a51

At first I was alarmed by "What? another shell script?" but this command should prioritize flexibility and extensibility over speed. Running multiple processes shouldn't be too taxing for what we are trying to do here.

Quoted 5 lines
Quoted 3 lines

Git for Windows sometimes receives bug reports about Bash not being able to start (usually it is a DLL base address problem, related to the way Cygwin and MSYS2 emulate fork()).

Hmm. In a case like this, then, how is someone using GfW at all?

On Windows, there are native command-line interpreters available that are _not_ Bash: PowerShell and CMD.

These days, you can get away with using Git _without_ a working Unix shell most of the time. There are only preciously few commands left that are still written as scripts, and most of these seem to be used less often than one might think:

- submodule
- bisect
- filter-branch
- instaweb
- mergetool
- some uncommon merge strategies
- rebase -p (already deprecated)
- several CVS/Subversion/Arch/Quilt integrations
- send-email (only relevant for mailing list centric projects, again, not very common any longer)
- request-pull (I would not be surprised if this is specific for the Linux project, as if that project was even close to the main user of Git)

So as long as you don't use submodules, and as long as you are unaware of the bisect command, which would comprise the majority of Git users in my experience, you can totally use Git for Windows without using Bash or Perl, i.e. without using the Cygwin/MSYS2 runtime that seems to be one of the common causes for trouble in Git for Windows.

Quoted 4 lines

Embarrassingly, I don't actually have a way to try it out for myself. It seems there's no GUI, and users are using it through the command line in mingw, so when you say "bash doesn't start" do you mean "users can't use the mingw prompt at all"?

There is no MinGW prompt.

MinGW stands for "Minimal GNU on Windows", but it really refers to a way to compile native Win32 programs via GCC. Meaning that some of the POSIX functionality Unix/Linux developers have come to expect is unavailable. For example, fork(). Which means that there is no native port of Bash to Windows, at least not that _I_ am aware of.

Git for Windows comes with a "Git Bash", which is essentially a Bash built against the MSYS2 runtime (i.e. it is much more like a Cygwin executable than like a Win32 executable).

The common way to run Git for Windows is actually via git.exe, independent of what particular command-line interpreter you prefer.

Quoted 9 lines
Quoted 4 lines

In such a case, git bugreport would only be able to offer a reason for yet another bug report instead of adding useful metadata.

Something to keep in mind when deciding how to implement this command.

Yeah, that's an interesting point, thanks for bringing it up. So, in the case when bash won't start at all, what does that failure look like? How much of Git can users still use? For example, would they be able to get far enough to run a binary git-bugreport?

See above.

If I were you, I would not write git bugreport as anything but a built-in, written in C. Unless there are _really_ good reasons not to [*1*]. And quite honestly, I don't see any such reasons.

Ciao,
Dscho

Footnote *1*: One good reason to use the MSYS2 Bash would be to be able to use MSYS2's POSIX emulation layer, e.g. to talk to the SSH agent (that is also an MSYS2 program, meaning that it communicates via emulated Unix sockets, which is a facility not available to native Win32 programs).

Another valid reason to use the MSYS2 Bash is to be able to talk to the (emulated) pseudo TTYs provided by the MSYS2 runtime, e.g. when running inside a MinTTY window, which is the default for Git Bash in Git for Windows.

#11 lore raw
In reply to Derrick Stolee
Quoted 3 lines · Derrick Stolee
Quoted 1 line

- Do we want to advertise the Git mailing list for bug reports?

That is possible. Isn't there another mailing list for git users?

I know there's an IRC channel for Git users, I dunno about mailing list. I'm worried that places I name as points of contact will grow stale, although I suppose #git on freenode isn't really going anywhere, for example. (But as a counterexample, I hope that nobody sends something like this to #git-devel, which seems to have a lower population than this mailing list.)

Quoted 3 lines

I could see a patch added on top of this for git-for-windows/git that changes the instructions to create issues on GitHub.

Indeed; I imagine we'd probably like to patch it to ask for bugs in our bug tracker.

Quoted 6 lines
Quoted 3 lines

- Which config options should we filter to avoid accidentally receiving credentials?

The remote URLs are pretty sensitive. Not only do users sometimes put passwords or PATs into their URLs, the literal name of the repo could be a secret.

Now here's where I start to wonder. We often debug internally by asking for the remote URL and replicating the issue there, which is why I mentioned it explicitly in the commit message. But I hadn't considered folks including the password in the URL.

Well, I suppose anybody can keep a local patch to change the config filter pattern. I'll try to make it easy to spot and modify.

[snip]

Quoted 3 lines

At first I was alarmed by "What? another shell script?" but this command should prioritize flexibility and extensibility over speed. Running multiple processes shouldn't be too taxing for what we are trying to do here.

If shell scripts are entirely deprecated I can convert it, but doing it in C seemed like overkill when I really just wanted "what are all the commands we would ask the user to run and tell us the output?". I figured also that it would be a little more immune to bitrot to output the contents of porcelain commands here.

Quoted 9 lines

git-bugreport.sh line 40

Quoted 4 lines
git-bugreport.sh new file +4 -0
+	echo "[Git Config]"
+	# TODO: Pass this through grep -v to avoid users sending us their credentials.
+	git config --show-origin --list
+	echo

Config options to consider stripping out:

	*url*
	*pass* (anything "password" but also "sendmail.smtppass")

Done, thanks.

Quoted 7 lines

git-bugreport.sh line 44

Quoted 4 lines
git-bugreport.sh new file +3 -0
+	echo "[Configured Hooks]"
+	find "$GIT_DIR/hooks/" -type f | grep -v "\.sample$" | print_filenames_and_content
+	echo

Remove the sample hooks, but focus on the others. Will this look like garbage if a hook is a binary file?

Yeah, I'm sure it will. I'll add a check to print_filenames_and_content() so it can tell us if there is a hook installed there, even if we can't see the content.

Quoted 9 lines

git-bugreport.sh line 48

Quoted 5 lines
git-bugreport.sh new file +4 -0
+
+	echo "[Git Logs]"
+	find "$GIT_DIR/logs" -type f | print_filenames_and_content
+	echo

As mentioned before, I've sometimes found it helpful to know the data shape for the object store. Having a few extra steps such as the following could be nice:

Quoted 4 lines
	echo "[Loose Objects]"
	for objdir in $(find "$GIT_DIR/objects/??" -type d)
	do
		echo "$objdir: $(ls $objdir | wc -l)"

echo "$objdir: $(ls $objdir | wc -l) objects
I'll add context so we don't need to have the bugreport script memorized in order to read a bugreport :)

Quoted 11 lines

done
echo

echo "[Pack Data]"
ls -l "$GIT_DIR/objects/pack"
echo

echo "[Object Info Data]"
ls -lR "$GIT_DIR/objects/info"
echo

Quoted 7 lines

echo "[Alternates File]"
echo "========"
cat "$GIT_DIR/objects/info/alternates"
echo

That last one will collect information on the commit-graph file, even if it is an incremental file.

Thanks Stolee, these are awesome and exactly the kind of feedback I was hoping for.

Quoted 5 lines

I think this is a great start, and I'll take some time later to try it out.

Thanks,
-Stolee

Awesome. I'm excited to hear how it goes.

- Emily

#12 lore raw
In reply to Emily Shaffer
Quoted 20 lines · Emily Shaffer
Documentation/git-bugreport.txt new file +14 -0
new file mode 100644
index 0000000000..c5f45bbee8
@@ -0,0 +1,48 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [-o | --output <path>]
+
+DESCRIPTION
+-----------

Documentation/git-bugreport.txt line 24

Quoted 10 lines
Documentation/git-bugreport.txt new file +10 -0
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior

+ - How the above two are different

It is often helpful to have users explain how the expected and actual are different in their own words.

Documentation/git-bugreport.txt line 44

Quoted 3 lines
Documentation/git-bugreport.txt new file +3 -0
+NOTE
+----
+Bug reports can be sent to [email protected].

I am not sure if this belongs here.

Quoted 6 lines
git-bugreport.sh new file +0 -0
new file mode 100755
index 0000000000..2200703a51

git-bugreport.sh line 10

Quoted 10 lines
git-bugreport.sh new file +10 -0
@@ -0,0 +1,86 @@
+#!/bin/sh
+
+print_filenames_and_content() {
+while read -r line; do
+	echo "$line";
+	echo "========";
+	cat "$line";
+	echo;
+done
+}

Style.

- have SP on both sides of ()
- one more HT indent for the function body
- "do" on its own line
- no unnecessary semicolons when LF would do

You probably are better off asking xargs to do this instead of relying on "read -r".

	find ... | xargs -n 1 sh -c 'echo "$1" && cat "$1"' -

or something like that, perhaps.

Quoted 8 lines
git-bugreport.sh new file +8 -0
+
+generate_report_text() {
+
+	# Generate a form for the user to fill out with their issue.
+	gettextln "Thank you for filling out a Git bug report!"
+	gettextln "Please answer the following questions to help us understand your issue."
+	echo
+	gettextln "What did you do before the bug happened? (Steps to reproduce your issue)"

git-bugreport.sh line 28

Quoted 10 lines
git-bugreport.sh new file +10 -0
+	echo
+	gettextln "What did you expect to happen? (Expected behavior)"
+	echo
+	gettextln "What happened instead? (Actual behavior)"
+	echo
+	gettextln "Anything else you want to add:"
+	echo
+	gettextln "Please review the rest of the bug report below."
+	gettextln "You can delete any lines you don't wish to send."
+	echo

Would we on the receiving end be able to tell these section headers in translated to 47 different languages? I am sure that i18n is used here to encourage non-C-locale users to file bugs in their own languages, but are we prepared to react to them?

git-bugreport.sh line 34

Quoted 5 lines
git-bugreport.sh new file +5 -0
+	echo "[System Information]"
+	git version --build-options
+	uname -a
+	curl-config --version
+	ldd --version

"curl-config: command not found" may be clear enough, but would there be a case where errors from two or more consecutive commands in the above sequence would make the output confusing to the person sitting on the receiving end? Would it help, as a convention, to always ahve "echo [what am I reporting]" before each of these commands?

git-bugreport.sh line 80

Quoted 2 lines
git-bugreport.sh new file +2 -0
+# Create bugreport file
+BUGREPORT_FILE="$basedir/git-bugreport-$(whoami)-$(hostname)-$(date -Iminutes)"

How portable is -Iminutes?

t/t0091-bugreport.sh line 1

Quoted 7 lines
t/t0091-bugreport.sh new file +1 -0
new file mode 100755
index 0000000000..6eb2ee4f66
@@ -0,0 +1,41 @@
+#!/bin/bash

Make sure /bin/sh suffices to run the test script.

t/t0091-bugreport.sh line 13

Quoted 11 lines
t/t0091-bugreport.sh new file +11 -0
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if [$(grep $HEADER_PATTERN $line)]; then

Documentation/CodingGuidelines

I am not sure if the traits this test script checks about the contents of the output is all that interesting. Whenever we add new sections to the main command because we want other kinds of information collected, we'd update this test script because otherwise the test would fail, but would it result in quality bugreport tool, or is it just additional busywork?

If we decide later that a header and its body needs to be separated with a blank line for better readablity, the check done here would also need to be updated, but again, that does not feel like anything more than just busywork to me.

The tests for "-o" and unknown options do make sense, though.

Thanks.

Quoted 35 lines
t/t0091-bugreport.sh new file +35 -0
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if [$(grep $HEADER_PATTERN $line)]; then
+			read -r nextline
+			if [-z $nextline]; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	check_all_headers_populated <git-bugreport-* &&
+	rm git-bugreport-*
+'
+
+test_expect_success '--output puts the report in the provided dir' '
+	mkdir foo/ &&
+	git bugreport -o foo/ &&
+	test -f foo/git-bugreport-* &&
+	rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	grep usage output &&
+	test ! -f git-bugreport-*
+'
+
+test_done
#13 lore raw
Quoted 35 lines · Junio C Hamano
Quoted 20 lines · Emily Shaffer
Documentation/git-bugreport.txt new file +14 -0
new file mode 100644
index 0000000000..c5f45bbee8
@@ -0,0 +1,48 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [-o | --output <path>]
+
+DESCRIPTION
+-----------

Documentation/git-bugreport.txt line 24

Quoted 10 lines
Documentation/git-bugreport.txt new file +10 -0
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior

+ - How the above two are different

Quoted 2 lines

It is often helpful to have users explain how the expected and actual are different in their own words.

Good point; added.

Quoted 6 lines

Documentation/git-bugreport.txt line 44

Quoted 4 lines
Documentation/git-bugreport.txt new file +3 -0
+NOTE
+----
+Bug reports can be sent to [email protected].

I am not sure if this belongs here.

Sure, I wasn't certain either. Would you rather I remove the "what to do with this bugreport" NOTE section entirely? I worry that folks will think this generated text file is being magically reported behind the scenes, or that they won't know what to do with the file now that it's created.

I suppose I did mention in the DESCRIPTION section that the user can share the generated text file to report a bug. Maybe I should remove "for example to the Git ML" from there as well.

Quoted 25 lines
Quoted 7 lines
git-bugreport.sh new file +0 -0
new file mode 100755
index 0000000000..2200703a51

git-bugreport.sh line 10

Quoted 10 lines
git-bugreport.sh new file +10 -0
@@ -0,0 +1,86 @@
+#!/bin/sh
+
+print_filenames_and_content() {
+while read -r line; do
+	echo "$line";
+	echo "========";
+	cat "$line";
+	echo;
+done
+}

Style.

- have SP on both sides of ()
- one more HT indent for the function body
- "do" on its own line
- no unnecessary semicolons when LF would do

Quoted 6 lines

You probably are better off asking xargs to do this instead of relying on "read -r".

	find ... | xargs -n 1 sh -c 'echo "$1" && cat "$1"' -

or something like that, perhaps.

Hm. In responding to Stolee's comments, I ended up replacing "cat $line" with another function, and it seems xargs isn't happy directing to functions which haven't been exported, because it spins a new process for each argument it's passed. I had originally intended to do this with xargs but then didn't want to pack many lines into one xargs call.

The way the code is in PS1 I think it's a reasonable replacement, but the code in PS2 is less so - there's a check for whether a file given is binary or whether it exists at all (the former for hooks, which I think you were worried about, and the latter for the alternates file, which I found I do not have.)

Why is "read -r" unreliable? Is it better to export a function and use xargs, compared to using "read -r" in a loop like this? I was worried that exporting a function is similar to namespace pollution in C++, but maybe I'm mistaken.

Quoted 16 lines
Quoted 16 lines
git-bugreport.sh new file +15 -0
+
+generate_report_text() {
+
+	# Generate a form for the user to fill out with their issue.
+	gettextln "Thank you for filling out a Git bug report!"
+	gettextln "Please answer the following questions to help us understand your issue."
+	echo
+	gettextln "What did you do before the bug happened? (Steps to reproduce your issue)"
+	echo
+	gettextln "What did you expect to happen? (Expected behavior)"
+	echo
+	gettextln "What happened instead? (Actual behavior)"
+	echo
+	gettextln "Anything else you want to add:"
+	echo
Quoted 8 lines

git-bugreport.sh line 28

Quoted 3 lines
git-bugreport.sh new file +3 -0
+	gettextln "Please review the rest of the bug report below."
+	gettextln "You can delete any lines you don't wish to send."
+	echo

Would we on the receiving end be able to tell these section headers in translated to 47 different languages? I am sure that i18n is used here to encourage non-C-locale users to file bugs in their own languages, but are we prepared to react to them?

I hope that we could accept reports in many languages with online translation as a guide, but practically speaking, machine-learning-powered online translation may not be as good for technical topics as it is for conversational.

At the same time, it seems unnecessarily restrictive to ask for bugs to come in English. My heart thinks it's better to encourage other languages too.

Quoted 13 lines

git-bugreport.sh line 34

Quoted 6 lines
git-bugreport.sh new file +5 -0
+	echo "[System Information]"
+	git version --build-options
+	uname -a
+	curl-config --version
+	ldd --version

"curl-config: command not found" may be clear enough, but would there be a case where errors from two or more consecutive commands in the above sequence would make the output confusing to the person sitting on the receiving end? Would it help, as a convention, to always ahve "echo [what am I reporting]" before each of these commands?

Yeah, I think that's a good point.

In responding to Stolee's review, I mentioned adding additional context to some command's output so that "those reviewing the report don't need to be intimately familiar with what the bugreport script runs". But it might actually be most useful to echo the command that's being run everywhere.

I imagine it's fine to include the [] header as well, to make it easier for the user to understand what was generated.

Quoted 5 lines

git-bugreport.sh line 80

Quoted 3 lines
git-bugreport.sh new file +2 -0
+# Create bugreport file
+BUGREPORT_FILE="$basedir/git-bugreport-$(whoami)-$(hostname)-$(date -Iminutes)"

How portable is -Iminutes?

Ah, it seems while it works in MingW and bash, it doesn't work in zsh. I'll figure something else out. Thanks.

Quoted 10 lines

t/t0091-bugreport.sh line 1

Quoted 8 lines
t/t0091-bugreport.sh new file +1 -0
new file mode 100755
index 0000000000..6eb2ee4f66
@@ -0,0 +1,41 @@
+#!/bin/bash

Make sure /bin/sh suffices to run the test script.

Will do. Thanks.

Quoted 15 lines

t/t0091-bugreport.sh line 13

Quoted 12 lines
t/t0091-bugreport.sh new file +11 -0
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if [$(grep $HEADER_PATTERN $line)]; then

Documentation/CodingGuidelines

Quoted 6 lines

I am not sure if the traits this test script checks about the contents of the output is all that interesting. Whenever we add new sections to the main command because we want other kinds of information collected, we'd update this test script because otherwise the test would fail, but would it result in quality bugreport tool, or is it just additional busywork?

I'm a little confused about this - I checked for a general [...] header standing alone on a line specifically so that we wouldn't need to update the test script when adding a new section. What do you see that will grow stale?

Quoted 5 lines

If we decide later that a header and its body needs to be separated with a blank line for better readablity, the check done here would also need to be updated, but again, that does not feel like anything more than just busywork to me.

Sure, I agree with that.

So, what's your suggestion? Not to check the output at all? (This may actually be fine; it occurred to me while reading your review that if a user is filing a bug report about something, one of the diagnostic commands in bugreport might be what's broken for them. So perhaps it should be tolerant to missing information...)

Quoted 4 lines

The tests for "-o" and unknown options do make sense, though.

Thanks.

Thanks for reviewing.

- Emily

#14 lore raw
Quoted 8 lines · Emily Shaffer
Quoted 5 lines

Documentation/git-bugreport.txt line 44

Quoted 3 lines
Documentation/git-bugreport.txt new file +3 -0
+NOTE
+----
+Bug reports can be sent to [email protected].

I am not sure if this belongs here.

Sure, I wasn't certain either. Would you rather I remove the "what to do with this bugreport" NOTE section entirely?

Not really. You are invoking an editor to let the user edit the pre-populated report, and I would imagine that a comment in that file would be the best place to give instructions, not a manpage for "git bugreport" command.

Quoted 5 lines

So, what's your suggestion? Not to check the output at all? (This may actually be fine; it occurred to me while reading your review that if a user is filing a bug report about something, one of the diagnostic commands in bugreport might be what's broken for them. So perhaps it should be tolerant to missing information...)

Right.

#15 lore raw
Quoted 8 lines · Junio C Hamano
Quoted 6 lines · Emily Shaffer
Quoted 5 lines

Documentation/git-bugreport.txt line 44

Quoted 3 lines
Documentation/git-bugreport.txt new file +3 -0
+NOTE
+----
+Bug reports can be sent to [email protected].

I am not sure if this belongs here.

Quoted 7 lines
Quoted 2 lines

Sure, I wasn't certain either. Would you rather I remove the "what to do with this bugreport" NOTE section entirely?

Not really. You are invoking an editor to let the user edit the pre-populated report, and I would imagine that a comment in that file would be the best place to give instructions, not a manpage for "git bugreport" command.

Oh, I see! In that case, do you still want the Git mailing list address shown in the report text?

Quoted 8 lines
Quoted 6 lines

So, what's your suggestion? Not to check the output at all? (This may actually be fine; it occurred to me while reading your review that if a user is filing a bug report about something, one of the diagnostic commands in bugreport might be what's broken for them. So perhaps it should be tolerant to missing information...)

Right.

Ok, will do.

#16 lore raw

v2 cover add git-bugreport tool

In reply to Emily Shaffer

Having read Dscho's response after already writing this reroll, I think it is a compelling case to rewrite as a C builtin, although I'm worried a little about the best way to continue to call porcelain commands to protect against bitrot.

Regardless, I think the proof of concept in patch 2/2 still deserves review; likely a reimplementation in C would see git-bugreport-config-whitelist generated in a similar way, but as a header instead of a textfile. So I'm still very interested in comments on the direction and the Makefile changes there.

The patch 1/2 changes are primarily based on Stolee and Junio's suggestions; thanks both. There's some rewording of manpage and commit message, as well as fixes when I realized this command did not work as expected from a worktree created with git worktree add. I also moved the reflog contents to the very bottom of the output as they're likely harmless, but very verbose; I didn't want the user to stop reading and never examine the shorter and scarier generated output, like the config.

Thanks for thoughts.
- Emily

Emily Shaffer (2):
bugreport: add tool to generate debugging info
bugreport: generate config whitelist based on docs

 .gitignore                             |   2 +
 Documentation/config/sendemail.txt     |  68 ++++++------
 Documentation/git-bugreport.txt        |  51 +++++++++
 Makefile                               |  10 +-
 bugreport-generate-config-whitelist.sh |   4 +
 command-list.txt                       |   1 +
 git-bugreport.sh                       | 139 +++++++++++++++++++++++++
 git-sh-setup.sh                        |   2 +
 t/t0091-bugreport.sh                   |  25 +++++
 9 files changed, 267 insertions(+), 35 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100755 bugreport-generate-config-whitelist.sh
 create mode 100755 git-bugreport.sh
 create mode 100755 t/t0091-bugreport.sh
Signature · 1 line
2.23.0.rc1.153.gdeed80330f-goog
#17 lore raw

PATCH v2 1/2 bugreport: add tool to generate debugging info

Make it easier for users who encounter a bug to send a report by collecting some state information, intended to be viewed by humans familiar with Git.

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Also, teach Git to write down its own version, the version of some of its dependencies, the operating system information, select entries from the gitconfig, the configured hooks, and the contents of $GIT_DIR/logs. Finally, make sure Git asks the user to review the contents of the report after it's generated.

If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.

Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.

---
 .gitignore                      |   1 +
 Documentation/git-bugreport.txt |  51 ++++++++++++
 Makefile                        |   1 +
 command-list.txt                |   1 +
 git-bugreport-config-whitelist  |  22 +++++
 git-bugreport.sh                | 139 ++++++++++++++++++++++++++++++++
 git-sh-setup.sh                 |   2 +
 t/t0091-bugreport.sh            |  25 ++++++
 8 files changed, 242 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 git-bugreport-config-whitelist
 create mode 100755 git-bugreport.sh
 create mode 100755 t/t0091-bugreport.sh
.gitignore +1 -0
index 521d8f4fb4..b4f5433084 100644
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
Documentation/git-bugreport.txt new file +51 -0

51 lines not on this page · raw message

Makefile +1 -0
index f9255344ae..69801a1c45 100644
@@ -606,6 +606,7 @@ TEST_PROGRAMS_NEED_X =
 unexport CDPATH
 
 SCRIPT_SH += git-bisect.sh
+SCRIPT_SH += git-bugreport.sh
 SCRIPT_SH += git-difftool--helper.sh
 SCRIPT_SH += git-filter-branch.sh
 SCRIPT_SH += git-merge-octopus.sh
command-list.txt +1 -0
index a9ac72bef4..be5a605047 100644
@@ -54,6 +54,7 @@ git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
 git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
+git-bugreport                           ancillaryinterrogators
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
 git-check-attr                          purehelpers
git-bugreport-config-whitelist new file +22 -0
new file mode 100644
index 0000000000..e4f07f7175
@@ -0,0 +1,22 @@
+http.version
+protocol.version
+protocol.persistent-https.allow
+protocol.rpc.allow
+protocol.sso.allow
+submodule.repolike
+trace2.eventtarget
+trace2.configparams
+color.ui
+core.pager
+sendemail.smtpencryption
+sendemail.smtpserver
+sendemail.smtpserverport
+sendemail.smtpsslcertpath
+credential.helper
+merge.tool
+grep.linenumber
+rerere.enabled
+core.repositoryformatversion
+core.filemode
+core.bare
+core.logallrefupdates
git-bugreport.sh new file +139 -0

139 lines not on this page · raw message

git-sh-setup.sh +2 -0
index 10d9764185..1db8af0d76 100644
@@ -379,6 +379,8 @@ git_dir_init () {
 		exit 1
 	}
 	: "${GIT_OBJECT_DIRECTORY="$(git rev-parse --git-path objects)"}"
+        : "${GIT_LOGS_DIRECTORY="$(git rev-parse --git-path logs)"}"
+        : "${GIT_HOOKS_DIRECTORY="$(git rev-parse --git-path hooks)"}"
 }
 
 if test -z "$NONGIT_OK"
t/t0091-bugreport.sh new file +25 -0
new file mode 100755
index 0000000000..e4e2d1a927
@@ -0,0 +1,25 @@
+#!/bin/sh
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+test_expect_success 'creates a report without crashing' '
+	git bugreport &&
+	rm git-bugreport-*
+'
+
+test_expect_success '--output puts the report in the provided dir' '
+	mkdir foo/ &&
+	git bugreport -o foo/ &&
+	test -f foo/git-bugreport-* &&
+	rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	grep usage output &&
+	test ! -f git-bugreport-*
+'
+
+test_done
Signature · 1 line
2.23.0.rc1.153.gdeed80330f-goog
#18 lore raw

PATCH v2 2/2 bugreport: generate config whitelist based on docs

In reply to Emily Shaffer

Add a new step to the build to generate a whitelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the whitelist by annotating their documentation in Documentation/config with the line "// bugreport-include".

Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blacklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a whitelist is more reliable.

In order to prevent staleness of the whitelist, add a mechanism to generate the whitelist from annotations in the config documentation, where contributors are already used to documenting their new config keys.

Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.

---
 .gitignore                             |  1 +
 Documentation/config/sendemail.txt     | 68 +++++++++++++-------------
 Makefile                               |  9 +++-
 bugreport-generate-config-whitelist.sh |  4 ++
 git-bugreport-config-whitelist         | 22 ---------
 5 files changed, 47 insertions(+), 57 deletions(-)
 create mode 100755 bugreport-generate-config-whitelist.sh
 delete mode 100644 git-bugreport-config-whitelist
.gitignore +1 -0
index b4f5433084..65cc74c748 100644
@@ -26,6 +26,7 @@
 /git-blame
 /git-branch
 /git-bugreport
+/git-bugreport-config-whitelist
 /git-bundle
 /git-cat-file
 /git-check-attr
Documentation/config/sendemail.txt +34 -34

97 lines not on this page · raw message

Makefile +8 -1
index 69801a1c45..a62bbefb8f 100644
@@ -639,6 +639,10 @@ SCRIPT_PYTHON += git-p4.py
 SCRIPT_SH_GEN = $(patsubst %.sh,%,$(SCRIPT_SH))
 SCRIPT_PERL_GEN = $(patsubst %.perl,%,$(SCRIPT_PERL))
 SCRIPT_PYTHON_GEN = $(patsubst %.py,%,$(SCRIPT_PYTHON))
+SCRIPT_DEPENDENCIES = git-bugreport-config-whitelist
+
+$(SCRIPT_DEPENDENCIES): Documentation/config/*.txt
+	sh bugreport-generate-config-whitelist.sh
 
 # Individual rules to allow e.g.
 # "make -C ../.. SCRIPT_PERL=contrib/foo/bar.perl build-perl-script"
@@ -656,17 +660,20 @@ install-perl-script: $(SCRIPT_PERL_GEN)
 install-python-script: $(SCRIPT_PYTHON_GEN)
 	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
 
-.PHONY: clean-perl-script clean-sh-script clean-python-script
+.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
 clean-sh-script:
 	$(RM) $(SCRIPT_SH_GEN)
 clean-perl-script:
 	$(RM) $(SCRIPT_PERL_GEN)
 clean-python-script:
 	$(RM) $(SCRIPT_PYTHON_GEN)
+clean-script-dependencies:
+	$(RM) $(SCRIPT_DEPENDENCIES)
 
 SCRIPTS = $(SCRIPT_SH_GEN) \
 	  $(SCRIPT_PERL_GEN) \
 	  $(SCRIPT_PYTHON_GEN) \
+	  $(SCRIPT_DEPENDENCIES) \
 	  git-instaweb
 
 ETAGS_TARGET = TAGS
bugreport-generate-config-whitelist.sh new file +4 -0
new file mode 100755
index 0000000000..ca6b232024
@@ -0,0 +1,4 @@
+#!/bin/sh
+
+grep -RhPo ".*(?=:: \/\/ bugreport-include)" Documentation/config \
+  >git-bugreport-config-whitelist
git-bugreport-config-whitelist deleted +0 -22
deleted file mode 100644
index e4f07f7175..0000000000
@@ -1,22 +0,0 @@
-http.version
-protocol.version
-protocol.persistent-https.allow
-protocol.rpc.allow
-protocol.sso.allow
-submodule.repolike
-trace2.eventtarget
-trace2.configparams
-color.ui
-core.pager
-sendemail.smtpencryption
-sendemail.smtpserver
-sendemail.smtpserverport
-sendemail.smtpsslcertpath
-credential.helper
-merge.tool
-grep.linenumber
-rerere.enabled
-core.repositoryformatversion
-core.filemode
-core.bare
-core.logallrefupdates
Signature · 1 line
2.23.0.rc1.153.gdeed80330f-goog
#19 lore raw

Re: [PATCH v2 2/2] bugreport: generate config whitelist based on docs

Quoted 6 lines · Emily Shaffer

Add a new step to the build to generate a whitelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the whitelist by annotating their documentation in Documentation/config with the line "// bugreport-include".

These "// bugreport-include" show up in the rendered manpages, both with AsciiDoc and Asciidoctor. :-(

Documentation/config/sendemail.txt line 2

Quoted 6 lines
Documentation/config/sendemail.txt +1 -1
@@ -1,63 +1,63 @@
-sendemail.identity::
+sendemail.identity:: // bugreport-exclude
        A configuration identity. When given, causes values in the

If I put each comment on a line of its own (after the config option, but I suppose before would work the same way), Asciidoctor truly ignores them and everything's fine. And AsciiDoc renders this one and others like it ok.

Documentation/config/sendemail.txt line 5

Quoted 6 lines
Documentation/config/sendemail.txt +3 -3
-sendemail.aliasesFile::
-sendemail.aliasFileType::
-sendemail.annotate::
+sendemail.aliasesFile:: // bugreport-exclude
+sendemail.aliasFileType:: // bugreport-exclude
+sendemail.annotate:: // bugreport-include

However, AsciiDoc (version 8.6.10) seems to effectively replace those comments with an empty line during processing, and it makes quite the difference here. Instead of these appearing in a compact comma-separated list, they are treated as individual items in the description list with no supporting content.

FWIW, I like the idea of annotating things here to make it harder to forget this whitelisting when adding a new config item.

Below is what I came up with as an alternative approach. Feel free to steal, squash and/or ignore as you see fit.

BTW, I'm not sure the grep invocation is portable (-R ? -h ? -P ? -o ?). We should probably also do the usual "create a candidate output file, then move it into place" dance for robustness.

I do think we should test the generated whitelist in some minimal way, e.g., to check that it does contain something which objectively belongs in the whitelist and -- more importantly IMHO -- does *not* contain something that shouldn't be there, such as sendemail.smtpPass.

Martin

-- >8 --
Subject: [PATCH] Use a bugreport:include/exclude macro instead

Implement a no-op "bugreport" macro for use as "bugreport:include[x]" to annotate the config keys that should be included in the automatically generated whitelist. Use "exclude" for the others.

With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc seems to want something between the brackets. A bit unfortunate, but not a huge problem -- we'll just provide an "x".

"doc-diff" reports that this macro doesn't render at all. That is, these are both empty after this commit:

  cd Documentation
  ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
  ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD

Diffing the rendered HTML shows that there is some small amount of whitespace and comments added. That shouldn't be a problem.

We could perhaps let the implementation verify that the "action" is one of "include" and "exclude". For the Asciidoctor implementation that should be straightforward, but for AsciiDoc I don't immediately know how to do it. Anyway, if someone stumbles on the keyboard and writes "bugreport:icndule", they'll "only" miss out on the config key being included in the whitelist. If this were a blacklist, the consequences of a misspelled target could be a lot more severe.

Disclaimer: This is the outcome of a combination of copy-paste and trial and error -- better solutions might be possible...

---
I suppose this could also be "annotate:bugreport[ignore]" to avoid the "[x]" and to place all of this under a more general "annotate" macro.

 Documentation/asciidoc.conf             |  8 +++
 Documentation/asciidoctor-extensions.rb |  8 +++
 Documentation/config/sendemail.txt      | 68 ++++++++++++-------------
 bugreport-generate-config-whitelist.sh  |  2 +-
 4 files changed, 51 insertions(+), 35 deletions(-)
Documentation/asciidoc.conf +8 -0
index 2c16c536ba..5a3c5ef028 100644
@@ -6,9 +6,13 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The bugreport macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +32,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[bugreport-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[bugreport-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
Documentation/asciidoctor-extensions.rb +8 -0
index 0089e0cfb8..54cbd5ddaf 100644
@@ -20,9 +20,17 @@ module Git
         end
       end
     end
+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, action, attrs)
+        ""
+      end
+    end
   end
 end
 
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
+  # The bugreport macro does nothing as far as rendering is
+  # concerned -- we just grep for it in the sources.
+  inline_macro Git::Documentation::BugReportProcessor, :bugreport
 end
Documentation/config/sendemail.txt +34 -34

97 lines not on this page · raw message

bugreport-generate-config-whitelist.sh +1 -1
index ca6b232024..896b838cfa 100755
@@ -1,4 +1,4 @@
 #!/bin/sh
 
-grep -RhPo ".*(?=:: \/\/ bugreport-include)" Documentation/config \
+grep -RhPo ".*(?= +bugreport:include.* ::)" Documentation/config \
   >git-bugreport-config-whitelist
Signature · 1 line
2.23.0
#20 lore raw

Re: [PATCH v2 2/2] bugreport: generate config whitelist based on docs

Quoted 10 lines · Martin Ågren
Quoted 6 lines · Emily Shaffer

Add a new step to the build to generate a whitelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the whitelist by annotating their documentation in Documentation/config with the line "// bugreport-include".

These "// bugreport-include" show up in the rendered manpages, both with AsciiDoc and Asciidoctor. :-(

Hmm, I don't see it in the troff or the HTML with asciidoc using make
- but I do see with asciidoctor, or asciidoc -d html5. Nice catch, thanks.

Quoted 26 lines

Documentation/config/sendemail.txt line 2

Quoted 7 lines
Documentation/config/sendemail.txt +1 -1
@@ -1,63 +1,63 @@
-sendemail.identity::
+sendemail.identity:: // bugreport-exclude
        A configuration identity. When given, causes values in the

If I put each comment on a line of its own (after the config option, but I suppose before would work the same way), Asciidoctor truly ignores them and everything's fine. And AsciiDoc renders this one and others like it ok.

Documentation/config/sendemail.txt line 5

Quoted 6 lines
Documentation/config/sendemail.txt +3 -3
-sendemail.aliasesFile::
-sendemail.aliasFileType::
-sendemail.annotate::
+sendemail.aliasesFile:: // bugreport-exclude
+sendemail.aliasFileType:: // bugreport-exclude
+sendemail.annotate:: // bugreport-include

However, AsciiDoc (version 8.6.10) seems to effectively replace those comments with an empty line during processing, and it makes quite the difference here. Instead of these appearing in a compact comma-separated list, they are treated as individual items in the description list with no supporting content.

Quoted 5 lines

FWIW, I like the idea of annotating things here to make it harder to forget this whitelisting when adding a new config item.

Below is what I came up with as an alternative approach. Feel free to steal, squash and/or ignore as you see fit.

This is cool - I'll add this to the commit chain and build on top of it. Thanks!

Quoted 2 lines

BTW, I'm not sure the grep invocation is portable (-R ? -h ? -P ? -o ?).

Yeah, I'll see what I can do about that (recursive, no filename, Perl-compatible regex, match only) - I can probably replace this with something like find | xargs pgrep - I'll keep digging. Thanks.

Quoted 7 lines

We should probably also do the usual "create a candidate output file, then move it into place" dance for robustness.

I do think we should test the generated whitelist in some minimal way, e.g., to check that it does contain something which objectively belongs in the whitelist and -- more importantly IMHO -- does *not* contain something that shouldn't be there, such as sendemail.smtpPass.

Good point, I'll add a test for this.

Thanks very much for this!
- Emily

#21 lore raw

v3 cover add git-bugreport tool

In reply to Emily Shaffer

Thanks for the patience with the long wait, all. Here's an attempt at the rewrite in C; I think it does verbatim what the sh version did except that this doesn't print the reflog - Jonathan Nieder was good enough to point out to me that folks probably don't want to share their commit subjects all willy-nilly if they're working on something secretive.

For the most part I hoped to do this stuff stupidly - that is to say, independent of the libraries within Git - in an effort to both avoid bitrot, and see what is (as opposed to what the libraries think should be).

The one common patch with v2 is "generate config whitelist based on docs", which I have not changed at all.

I look forward to everyone's suggestions.

- Emily

Emily Shaffer (9):
bugreport: add tool to generate debugging info
bugreport: generate config whitelist based on docs
bugreport: add version and system information
bugreport: add config values from whitelist
bugreport: collect list of populated hooks
bugreport: count loose objects
bugreport: add packed object summary
bugreport: list contents of $OBJDIR/info
bugreport: print contents of alternates file

 .gitignore                             |   1 +
 Documentation/config/sendemail.txt     |  68 +++---
 Makefile                               |  11 +-
 bugreport-generate-config-whitelist.sh |   4 +
 bugreport.c                            | 314 +++++++++++++++++++++++++
 bugreport.h                            |  44 ++++
 builtin.h                              |   1 +
 builtin/bugreport.c                    |  90 +++++++
 git.c                                  |   1 +
 9 files changed, 499 insertions(+), 35 deletions(-)
 create mode 100755 bugreport-generate-config-whitelist.sh
 create mode 100644 bugreport.c
 create mode 100644 bugreport.h
 create mode 100644 builtin/bugreport.c
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
#22 lore raw

PATCH v3 1/9 bugreport: add tool to generate debugging info

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.

Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.

---
 Makefile            |  1 +
 builtin.h           |  1 +
 builtin/bugreport.c | 50 +++++++++++++++++++++++++++++++++++++++++++++
 git.c               |  1 +
 4 files changed, 53 insertions(+)
 create mode 100644 builtin/bugreport.c
Makefile +1 -0
index 58b92af54b..132e2a52da 100644
@@ -1039,6 +1039,7 @@ BUILTIN_OBJS += builtin/archive.o
 BUILTIN_OBJS += builtin/bisect--helper.o
 BUILTIN_OBJS += builtin/blame.o
 BUILTIN_OBJS += builtin/branch.o
+BUILTIN_OBJS += builtin/bugreport.o
 BUILTIN_OBJS += builtin/bundle.o
 BUILTIN_OBJS += builtin/cat-file.o
 BUILTIN_OBJS += builtin/check-attr.o
builtin.h +1 -0
index 5cf5df69f7..c6373d3289 100644
@@ -135,6 +135,7 @@ int cmd_archive(int argc, const char **argv, const char *prefix);
 int cmd_bisect__helper(int argc, const char **argv, const char *prefix);
 int cmd_blame(int argc, const char **argv, const char *prefix);
 int cmd_branch(int argc, const char **argv, const char *prefix);
+int cmd_bugreport(int argc, const char **argv, const char *prefix);
 int cmd_bundle(int argc, const char **argv, const char *prefix);
 int cmd_cat_file(int argc, const char **argv, const char *prefix);
 int cmd_checkout(int argc, const char **argv, const char *prefix);
builtin/bugreport.c new file +50 -0

50 lines not on this page · raw message

git.c +1 -0
index ce6ab0ece2..2d6a64f019 100644
@@ -473,6 +473,7 @@ static struct cmd_struct commands[] = {
 	{ "bisect--helper", cmd_bisect__helper, RUN_SETUP },
 	{ "blame", cmd_blame, RUN_SETUP },
 	{ "branch", cmd_branch, RUN_SETUP | DELAY_PAGER_CONFIG },
+	{ "bugreport", cmd_bugreport, RUN_SETUP },
 	{ "bundle", cmd_bundle, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "cat-file", cmd_cat_file, RUN_SETUP },
 	{ "check-attr", cmd_check_attr, RUN_SETUP },
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
#23 lore raw
Quoted 52 lines · Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.

Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.

---
 Makefile            |  1 +
 builtin.h           |  1 +
 builtin/bugreport.c | 50 +++++++++++++++++++++++++++++++++++++++++++++
 git.c               |  1 +
 4 files changed, 53 insertions(+)
 create mode 100644 builtin/bugreport.c
Makefile +1 -0
index 58b92af54b..132e2a52da 100644
@@ -1039,6 +1039,7 @@ BUILTIN_OBJS += builtin/archive.o
 BUILTIN_OBJS += builtin/bisect--helper.o
 BUILTIN_OBJS += builtin/blame.o
 BUILTIN_OBJS += builtin/branch.o
+BUILTIN_OBJS += builtin/bugreport.o
 BUILTIN_OBJS += builtin/bundle.o
 BUILTIN_OBJS += builtin/cat-file.o
 BUILTIN_OBJS += builtin/check-attr.o
builtin.h +1 -0
index 5cf5df69f7..c6373d3289 100644
@@ -135,6 +135,7 @@ int cmd_archive(int argc, const char **argv, const char *prefix);
 int cmd_bisect__helper(int argc, const char **argv, const char *prefix);
 int cmd_blame(int argc, const char **argv, const char *prefix);
 int cmd_branch(int argc, const char **argv, const char *prefix);
+int cmd_bugreport(int argc, const char **argv, const char *prefix);
 int cmd_bundle(int argc, const char **argv, const char *prefix);
 int cmd_cat_file(int argc, const char **argv, const char *prefix);
 int cmd_checkout(int argc, const char **argv, const char *prefix);
builtin/bugreport.c new file +0 -0
new file mode 100644
index 0000000000..2ef16440a0

builtin/bugreport.c line 6

Quoted 6 lines
builtin/bugreport.c new file +6 -0
@@ -0,0 +1,50 @@
+#include "builtin.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+int get_bug_template(struct strbuf *template)

Compilation fails here for me with:
builtin/bugreport.c:6:5: error: no previous prototype
for ‘get_bug_template’ [-Werror=missing-prototypes]

Can you make this function static?

Quoted 58 lines
builtin/bugreport.c new file +44 -0
+{
+	const char template_text[] =
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to send.\n";
+
+	strbuf_reset(template);
+	strbuf_add(template, template_text, strlen(template_text));
+	return 0;
+}
+
+int cmd_bugreport(int argc, const char **argv, const char *prefix)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+	report = fopen_for_writing(report_path.buf);
+
+	get_bug_template(&buffer);
+	strbuf_write(&buffer, report);
+
+	fclose(report);
+
+	launch_editor(report_path.buf, NULL, NULL);
+	return 0;
+}
git.c +1 -0
index ce6ab0ece2..2d6a64f019 100644
@@ -473,6 +473,7 @@ static struct cmd_struct commands[] = {
 	{ "bisect--helper", cmd_bisect__helper, RUN_SETUP },
 	{ "blame", cmd_blame, RUN_SETUP },
 	{ "branch", cmd_branch, RUN_SETUP | DELAY_PAGER_CONFIG },
+	{ "bugreport", cmd_bugreport, RUN_SETUP },
 	{ "bundle", cmd_bundle, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "cat-file", cmd_cat_file, RUN_SETUP },
 	{ "check-attr", cmd_check_attr, RUN_SETUP },
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog

Can you also add /git-bugreport to .gitignore?

#24 lore raw
In reply to Emily Shaffer
Quoted 3 lines · Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

It makes sense, but I do not think of any good reason why this should be implemented as a builtin. I'd expect it would probably need to collect more info on the running environment than otherwise necessary for the regular Git operation, and perhaps you'd want to even link with libraries that are not needed for the regular Git operation to achieve that.

Can you make it a standalone binary instead to avoid bloat of the main "git" binary?

Thanks.

#25 lore raw
Quoted 13 lines · Junio C Hamano
Quoted 3 lines · Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

It makes sense, but I do not think of any good reason why this should be implemented as a builtin. I'd expect it would probably need to collect more info on the running environment than otherwise necessary for the regular Git operation, and perhaps you'd want to even link with libraries that are not needed for the regular Git operation to achieve that.

Quoted 2 lines

Can you make it a standalone binary instead to avoid bloat of the main "git" binary?

Sure. This would fix some other issues (needing to link against curl to get the curl version, which is exactly what you implied). I wasn't certain which circumstances a standalone binary was preferred, but I agree with your reasoning here for sure.

- Emily

#26 lore raw

Hi,

Quoted 14 lines · Emily Shaffer
Quoted 13 lines · Junio C Hamano
Quoted 3 lines · Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

It makes sense, but I do not think of any good reason why this should be implemented as a builtin. I'd expect it would probably need to collect more info on the running environment than otherwise necessary for the regular Git operation, and perhaps you'd want to even link with libraries that are not needed for the regular Git operation to achieve that.

Quoted 7 lines
Quoted 2 lines

Can you make it a standalone binary instead to avoid bloat of the main "git" binary?

Sure. This would fix some other issues (needing to link against curl to get the curl version, which is exactly what you implied). I wasn't certain which circumstances a standalone binary was preferred, but I agree with your reasoning here for sure.

FWIW I disagree with the idea that a tiny built-in command like bugreport would "bloat" the main git binary.

In contrast, I think that stand-alone commands _do_ bloat. Look here:

$ ls -lh git-daemon.exe git-credential-store.exe
-rwxr-xr-x 1 me 4096 1.8M Nov  6 13:43 git-credential-store.exe*
-rwxr-xr-x 1 me 4096 1.8M Nov  6 13:43 git-daemon.exe*

In other words, even a super simple stand-alone like credential-store (the credential-store.c file has only 198 lines!) weighs in with almost two megabytes.

So I fear that the claim that a stand-alone command would add less bloat than a built-in one, especially for a relatively small thing like bugreport has it exactly backwards.

Ciao,
Dscho

#27 lore raw
Quoted 4 lines · Johannes Schindelin

FWIW I disagree with the idea that a tiny built-in command like bugreport would "bloat" the main git binary.

In contrast, I think that stand-alone commands _do_ bloat. Look here:

I probably should have clarified that "bloat" in the context of this discussion is not about on-disk space. It is about resources needed to run "git status/diff/etc" that are everyday local commands that are expected to be lightweight, i.e. the same criteria applied when making the networking part (which the user expects to be coffee time) separate from them.

#28 lore raw

Hi Junio,

Quoted 5 lines · Junio C Hamano
Quoted 3 lines · Johannes Schindelin

FWIW I disagree with the idea that a tiny built-in command like bugreport would "bloat" the main git binary.

Quoted 8 lines
Quoted 1 line

In contrast, I think that stand-alone commands _do_ bloat. Look here:

I probably should have clarified that "bloat" in the context of this discussion is not about on-disk space. It is about resources needed to run "git status/diff/etc" that are everyday local commands that are expected to be lightweight, i.e. the same criteria applied when making the networking part (which the user expects to be coffee time) separate from them.

I guess I still do not understand.

Are you suggesting that bugreport would be unwelcome as a built-in? If so, I would really like to know why. Because I think it would make for a very fine built-in. I interact with users all the time, and a good tool to provide good information to accompany a bug report is definitely something that would improve the current situation.

Ciao,
Dscho

#29 lore raw
In reply to Emily Shaffer

Hi Emily,

Quoted 2 lines · Emily Shaffer

[...] some other issues (needing to link against curl to get the curl version, which is exactly what you implied) [...]

I did suggest on IRC to teach git-remote-https an option where it prints the cURL version (and build options) and exits.

This would have the further advantage of making sure that the correct information is included. If you have two separate binaries that both link to cURL, they could still be linked statically, to different versions of cURL (this could happen e.g. if you have a git-remote-https from a different build in your path).

In short: I still think that it would make for a much better idea to query the git-remote-https executable for the version information than to link bugreport to libcurl.

Ciao,
Dscho

#30 lore raw
Quoted 2 lines · Johannes Schindelin

I did suggest on IRC to teach git-remote-https an option where it prints the cURL version (and build options) and exits.

I like that. You ask the exact binary what (it thinks) it uses, so that there won't be skew between the view by "git remote-http" and "git bugreport" on the world.

#31 lore raw
In reply to Johannes Schindelin
Quoted 7 lines · Johannes Schindelin

Hi Emily,

Quoted 2 lines · Emily Shaffer

[...] some other issues (needing to link against curl to get the curl version, which is exactly what you implied) [...]

Quoted 8 lines

I did suggest on IRC to teach git-remote-https an option where it prints the cURL version (and build options) and exits.

This would have the further advantage of making sure that the correct information is included. If you have two separate binaries that both link to cURL, they could still be linked statically, to different versions of cURL (this could happen e.g. if you have a git-remote-https from a different build in your path).

Yeah, it's a good point and an angle I hadn't thought of. Thanks.

Quoted 3 lines

In short: I still think that it would make for a much better idea to query the git-remote-https executable for the version information than to link bugreport to libcurl.

Will do - the git-bugreport standalone will invoke the git-remote-https standalone to ask for version info.

- Emily

#32 lore raw
In reply to Emily Shaffer
Quoted 5 lines · Emily Shaffer
Quoted 2 lines

Can you make it a standalone binary instead to avoid bloat of the main "git" binary?

Sure. This would fix some other issues (needing to link against curl to get the curl version, which is exactly what you implied).

Hmph, I actually was not thinking about the cURL library.

I imagined that your tool may need to learn more about the system in order to make the report and for that there may be libraries that makes it easy than say reading directly from the /proc filesystem etc. that you may end up using. Unlike cURL, such a library would have no use in the rest of Git anywhere.

#33 lore raw

PATCH v3 2/9 bugreport: generate config whitelist based on docs

In reply to Emily Shaffer

Add a new step to the build to generate a whitelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the whitelist by annotating their documentation in Documentation/config with the line "// bugreport-include".

Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blacklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a whitelist is more reliable.

In order to prevent staleness of the whitelist, add a mechanism to generate the whitelist from annotations in the config documentation, where contributors are already used to documenting their new config keys.

Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.

---
 .gitignore                             |  1 +
 Documentation/config/sendemail.txt     | 68 +++++++++++++-------------
 Makefile                               |  9 +++-
 bugreport-generate-config-whitelist.sh |  4 ++
 4 files changed, 47 insertions(+), 35 deletions(-)
 create mode 100755 bugreport-generate-config-whitelist.sh
.gitignore +1 -0
index 89b3b79c1a..055a84c4a8 100644
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
Documentation/config/sendemail.txt +34 -34

97 lines not on this page · raw message

Makefile +8 -1
index 132e2a52da..78767ecdde 100644
@@ -634,6 +634,10 @@ SCRIPT_PYTHON += git-p4.py
 SCRIPT_SH_GEN = $(patsubst %.sh,%,$(SCRIPT_SH))
 SCRIPT_PERL_GEN = $(patsubst %.perl,%,$(SCRIPT_PERL))
 SCRIPT_PYTHON_GEN = $(patsubst %.py,%,$(SCRIPT_PYTHON))
+SCRIPT_DEPENDENCIES = git-bugreport-config-whitelist
+
+$(SCRIPT_DEPENDENCIES): Documentation/config/*.txt
+	sh bugreport-generate-config-whitelist.sh
 
 # Individual rules to allow e.g.
 # "make -C ../.. SCRIPT_PERL=contrib/foo/bar.perl build-perl-script"
@@ -651,17 +655,20 @@ install-perl-script: $(SCRIPT_PERL_GEN)
 install-python-script: $(SCRIPT_PYTHON_GEN)
 	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
 
-.PHONY: clean-perl-script clean-sh-script clean-python-script
+.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
 clean-sh-script:
 	$(RM) $(SCRIPT_SH_GEN)
 clean-perl-script:
 	$(RM) $(SCRIPT_PERL_GEN)
 clean-python-script:
 	$(RM) $(SCRIPT_PYTHON_GEN)
+clean-script-dependencies:
+	$(RM) $(SCRIPT_DEPENDENCIES)
 
 SCRIPTS = $(SCRIPT_SH_GEN) \
 	  $(SCRIPT_PERL_GEN) \
 	  $(SCRIPT_PYTHON_GEN) \
+	  $(SCRIPT_DEPENDENCIES) \
 	  git-instaweb
 
 ETAGS_TARGET = TAGS
bugreport-generate-config-whitelist.sh new file +4 -0
new file mode 100755
index 0000000000..ca6b232024
@@ -0,0 +1,4 @@
+#!/bin/sh
+
+grep -RhPo ".*(?=:: \/\/ bugreport-include)" Documentation/config \
+  >git-bugreport-config-whitelist
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
#34 lore raw

Re: [PATCH v3 2/9] bugreport: generate config whitelist based on docs

Hi Emily,

bugreport-generate-config-whitelist.sh line 3

Quoted 9 lines · Emily Shaffer
bugreport-generate-config-whitelist.sh new file +3 -0
new file mode 100755
index 0000000000..ca6b232024
@@ -0,0 +1,4 @@
+#!/bin/sh
+
+grep -RhPo ".*(?=:: \/\/ bugreport-include)" Documentation/config \

I am rather certain that -P is not supported by BSD grep (see
https://man.openbsd.org/grep).

Why not something portable, e.g.

	find Documentation/config -type f -exec cat {} \; |
	sed -n 's/^\(.*\):: \/\/ bugreport-include$/\1/p' \

?

Ciao,
Dscho

Quoted 5 lines
bugreport-generate-config-whitelist.sh new file +1 -1
+  >git-bugreport-config-whitelist
--

2.24.0.rc0.303.g954a862665-goog

#35 lore raw

PATCH v3 3/9 bugreport: add version and system information

In reply to Emily Shaffer

Teach bugreport how to collect the Git, curl, and ldd versions, as well as the uname string.

Learning the uname and versions in the user's environment will enable us to reproduce behavior reported by the user; the versions will also give us a good starting place while bisecting a newly introduced bug.

Put this functionality in a library, so that other commands can easily include this debug info if necessary.

---
 Makefile            |  1 +
 bugreport.c         | 55 +++++++++++++++++++++++++++++++++++++++++++++
 bugreport.h         |  7 ++++++
 builtin/bugreport.c | 13 +++++++++++
 4 files changed, 76 insertions(+)
 create mode 100644 bugreport.c
 create mode 100644 bugreport.h
Makefile +1 -0
index 78767ecdde..045b22cb67 100644
@@ -844,6 +844,7 @@ LIB_OBJS += bisect.o
 LIB_OBJS += blame.o
 LIB_OBJS += blob.o
 LIB_OBJS += branch.o
+LIB_OBJS += bugreport.o
 LIB_OBJS += bulk-checkin.o
 LIB_OBJS += bundle.o
 LIB_OBJS += cache-tree.o
bugreport.c new file +55 -0

55 lines not on this page · raw message

bugreport.h new file +7 -0
new file mode 100644
index 0000000000..ba216acf3f
@@ -0,0 +1,7 @@
+#include "strbuf.h"
+
+/**
+ * Adds the Git version, `uname -a`, and `curl-config --version` to sys_info.
+ * The previous contents of sys_info will be discarded.
+ */
+void get_system_info(struct strbuf *sys_info);
builtin/bugreport.c +13 -0
index 2ef16440a0..7232d31be7 100644
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "bugreport.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
@@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+void add_header(FILE *report, const char *title)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	strbuf_addf(&buffer, "\n\n[%s]\n", title);
+	strbuf_write(&buffer, report);
+}
+
 int cmd_bugreport(int argc, const char **argv, const char *prefix)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -43,6 +51,11 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_bug_template(&buffer);
 	strbuf_write(&buffer, report);
 
+	// add other contents
+	add_header(report, "System Info");
+	get_system_info(&buffer);
+	strbuf_write(&buffer, report);
+
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
#36 lore raw

Re: [PATCH v3 3/9] bugreport: add version and system information

Hi Emily,

Quoted 11 lines · Emily Shaffer
bugreport.c new file +5 -0
new file mode 100644
index 0000000000..ada54fe583
@@ -0,0 +1,55 @@
+#include "cache.h"
+
+#include "bugreport.h"
+#include "help.h"
+#include "run-command.h"

bugreport.c line 15

Quoted 10 lines
bugreport.c new file +10 -0
+#include "version.h"
+
+void get_system_info(struct strbuf *sys_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+	struct strbuf std_out = STRBUF_INIT;
+
+	strbuf_reset(sys_info);
+
+	// get git version from native cmd

Please use C-style comments instead of C++ ones.

bugreport.c line 23

Quoted 8 lines
bugreport.c new file +8 -0
+	strbuf_addstr(sys_info, "git version: ");
+	strbuf_addstr(sys_info, git_version_string);
+	strbuf_complete_line(sys_info);
+
+	// system call for other version info
+	argv_array_push(&cp.args, "uname");
+	argv_array_push(&cp.args, "-a");
+	capture_command(&cp, &std_out, 0);

Mmmkay. I am not too much of a fan of relying on the uname executable, as it can very well be a 32-bit uname.exe on Windows, which obviously would _not_ report the architecture of the machine, but something misleading.

Why not use the uname() function (that we even define in compat/mingw.c)?

Also, why not include the same information as git version --build-options, most importantly GIT_HOST_CPU?

In any case, if you are capturing the output of uname -a, you definitely will want to pass the RUN_COMMAND_STDOUT_TO_STDERR flag (in case, say, the MSYS2 uname.exe fails with those dreaded Cygwin error messages like "*** fatal error - add_item ("\??\D:\git\installation\Git", "/", ...) failed, errno 1").

bugreport.c line 35

Quoted 12 lines
bugreport.c new file +12 -0
+
+	strbuf_addstr(sys_info, "uname -a: ");
+	strbuf_addbuf(sys_info, &std_out);
+	strbuf_complete_line(sys_info);
+
+	argv_array_clear(&cp.args);
+	strbuf_reset(&std_out);
+
+
+	argv_array_push(&cp.args, "curl-config");
+	argv_array_push(&cp.args, "--version");
+	capture_command(&cp, &std_out, 0);

This will be almost certainly be incorrect, as most _regular_ users won't have curl-config. I know, it is easy to forget that most Git users are no hard-core C developers ;-)

A better alternative would be to use curl_version(), guarded, of course, by #ifndef NO_CURL...

bugreport.c line 47

Quoted 12 lines
bugreport.c new file +12 -0
+
+	strbuf_addstr(sys_info, "curl-config --version: ");
+	strbuf_addbuf(sys_info, &std_out);
+	strbuf_complete_line(sys_info);
+
+	argv_array_clear(&cp.args);
+	strbuf_reset(&std_out);
+
+
+	argv_array_push(&cp.args, "ldd");
+	argv_array_push(&cp.args, "--version");
+	capture_command(&cp, &std_out, 0);

Again, this command will only be present in few setups. I am not actually sure that the output of this is interesting to begin with.

What I _do_ think is that a much more interesting piece of information would be the exact GLIBC version, the OS name and the OS version.

Quoted 32 lines
bugreport.c new file +8 -0
+
+	strbuf_addstr(sys_info, "ldd --version: ");
+	strbuf_addbuf(sys_info, &std_out);
+	strbuf_complete_line(sys_info);
+
+	argv_array_clear(&cp.args);
+	strbuf_reset(&std_out);
+}
bugreport.h new file +7 -0
new file mode 100644
index 0000000000..ba216acf3f
@@ -0,0 +1,7 @@
+#include "strbuf.h"
+
+/**
+ * Adds the Git version, `uname -a`, and `curl-config --version` to sys_info.
+ * The previous contents of sys_info will be discarded.
+ */
+void get_system_info(struct strbuf *sys_info);
builtin/bugreport.c +1 -0
index 2ef16440a0..7232d31be7 100644
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "bugreport.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"

builtin/bugreport.c int get_bug_template(struct strbuf *template) · line 35

Quoted 8 lines
builtin/bugreport.c +5 -0
@@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+void add_header(FILE *report, const char *title)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	strbuf_addf(&buffer, "\n\n[%s]\n", title);
+	strbuf_write(&buffer, report);

This leaks buffer. Why not write into report via fprintf() directly?

Ciao,
Dscho

Quoted 21 lines
builtin/bugreport.c +7 -1
+}
+
 int cmd_bugreport(int argc, const char **argv, const char *prefix)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -43,6 +51,11 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_bug_template(&buffer);
 	strbuf_write(&buffer, report);
 
+	// add other contents
+	add_header(report, "System Info");
+	get_system_info(&buffer);
+	strbuf_write(&buffer, report);
+
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
--

2.24.0.rc0.303.g954a862665-goog

#37 lore raw

Re: [PATCH v3 3/9] bugreport: add version and system information

Quoted 42 lines · Johannes Schindelin

Hi Emily,

Quoted 11 lines · Emily Shaffer
bugreport.c new file +5 -0
new file mode 100644
index 0000000000..ada54fe583
@@ -0,0 +1,55 @@
+#include "cache.h"
+
+#include "bugreport.h"
+#include "help.h"
+#include "run-command.h"

bugreport.c line 15

Quoted 10 lines
bugreport.c new file +10 -0
+#include "version.h"
+
+void get_system_info(struct strbuf *sys_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+	struct strbuf std_out = STRBUF_INIT;
+
+	strbuf_reset(sys_info);
+
+	// get git version from native cmd

Please use C-style comments instead of C++ ones.

bugreport.c line 23

Quoted 8 lines
bugreport.c new file +8 -0
+	strbuf_addstr(sys_info, "git version: ");
+	strbuf_addstr(sys_info, git_version_string);
+	strbuf_complete_line(sys_info);
+
+	// system call for other version info
+	argv_array_push(&cp.args, "uname");
+	argv_array_push(&cp.args, "-a");
+	capture_command(&cp, &std_out, 0);

Mmmkay. I am not too much of a fan of relying on the uname executable, as it can very well be a 32-bit uname.exe on Windows, which obviously would _not_ report the architecture of the machine, but something misleading.

Quoted 2 lines

Why not use the uname() function (that we even define in compat/mingw.c)?

Very glad to have your review and point this kind of thing out. :) It simplifies the code, too. I wasn't sure about these system calls, so I appreciate you suggesting alternatives.

Quoted 10 lines

Also, why not include the same information as git version --build-options, most importantly GIT_HOST_CPU?

In any case, if you are capturing the output of uname -a, you definitely will want to pass the RUN_COMMAND_STDOUT_TO_STDERR flag (in case, say, the MSYS2 uname.exe fails with those dreaded Cygwin error messages like "*** fatal error - add_item ("\??\D:\git\installation\Git", "/", ...) failed, errno 1").

Quoted 16 lines

bugreport.c line 35

Quoted 12 lines
bugreport.c new file +12 -0
+
+	strbuf_addstr(sys_info, "uname -a: ");
+	strbuf_addbuf(sys_info, &std_out);
+	strbuf_complete_line(sys_info);
+
+	argv_array_clear(&cp.args);
+	strbuf_reset(&std_out);
+
+
+	argv_array_push(&cp.args, "curl-config");
+	argv_array_push(&cp.args, "--version");
+	capture_command(&cp, &std_out, 0);

This will be almost certainly be incorrect, as most _regular_ users won't have curl-config. I know, it is easy to forget that most Git users are no hard-core C developers ;-)

Heresy! :)

Quoted 4 lines

A better alternative would be to use curl_version(), guarded, of course, by #ifndef NO_CURL...

Quoted 15 lines

bugreport.c line 47

Quoted 12 lines
bugreport.c new file +12 -0
+
+	strbuf_addstr(sys_info, "curl-config --version: ");
+	strbuf_addbuf(sys_info, &std_out);
+	strbuf_complete_line(sys_info);
+
+	argv_array_clear(&cp.args);
+	strbuf_reset(&std_out);
+
+
+	argv_array_push(&cp.args, "ldd");
+	argv_array_push(&cp.args, "--version");
+	capture_command(&cp, &std_out, 0);

Again, this command will only be present in few setups. I am not actually sure that the output of this is interesting to begin with.

It was a suggestion, I believe, from Jonathan Nieder.

Quoted 3 lines

What I _do_ think is that a much more interesting piece of information would be the exact GLIBC version, the OS name and the OS version.

The glibc version is easy; I've done that. It certainly looks nicer than the ldd call.

I guess I may be missing something, because as I start to look into how to the OS info, I fall down a hole of many, many preprocessor defines to check. If that's the approach you want me to take, just say the word, but it will be ugly :) I suppose I had hoped the uname info would give us a close enough idea that full OS info isn't necessary.

Quoted 36 lines
Quoted 36 lines
bugreport.c new file +8 -0
+
+	strbuf_addstr(sys_info, "ldd --version: ");
+	strbuf_addbuf(sys_info, &std_out);
+	strbuf_complete_line(sys_info);
+
+	argv_array_clear(&cp.args);
+	strbuf_reset(&std_out);
+}
bugreport.h new file +7 -0
new file mode 100644
index 0000000000..ba216acf3f
@@ -0,0 +1,7 @@
+#include "strbuf.h"
+
+/**
+ * Adds the Git version, `uname -a`, and `curl-config --version` to sys_info.
+ * The previous contents of sys_info will be discarded.
+ */
+void get_system_info(struct strbuf *sys_info);
builtin/bugreport.c +1 -0
index 2ef16440a0..7232d31be7 100644
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "bugreport.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
@@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
Quoted 8 lines

builtin/bugreport.c int get_bug_template(struct strbuf *template) · line 35

Quoted 5 lines
builtin/bugreport.c +5 -0
+void add_header(FILE *report, const char *title)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	strbuf_addf(&buffer, "\n\n[%s]\n", title);
+	strbuf_write(&buffer, report);

This leaks buffer. Why not write into report via fprintf() directly?

Rather, to match the style of the rest of the builtin, modified get_header to add the header to a passed-in strbuf instead of modifying the file directly.

Quoted 20 lines

Ciao,
Dscho

Quoted 16 lines
builtin/bugreport.c +7 -0
+}
+
 int cmd_bugreport(int argc, const char **argv, const char *prefix)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -43,6 +51,11 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_bug_template(&buffer);
 	strbuf_write(&buffer, report);
 
+	// add other contents
+	add_header(report, "System Info");
+	get_system_info(&buffer);
+	strbuf_write(&buffer, report);
+
 	fclose(report);
 
Quoted 5 lines
Quoted 5 lines
builtin/bugreport.c +0 -1
 	launch_editor(report_path.buf, NULL, NULL);
--

2.24.0.rc0.303.g954a862665-goog

Based on Dscho's comments, each piece of system info is gathered differently enough that I will do each in an independent commit now.

#38 lore raw

Re: [PATCH v3 3/9] bugreport: add version and system information

Hi Emily,

Quoted 25 lines · Emily Shaffer
Quoted 14 lines · Johannes Schindelin
Quoted 12 lines · Emily Shaffer
bugreport.c new file +6 -0
new file mode 100644
index 0000000000..ada54fe583
[...]
+	strbuf_addstr(sys_info, "curl-config --version: ");
+	strbuf_addbuf(sys_info, &std_out);
+	strbuf_complete_line(sys_info);
+
+	argv_array_clear(&cp.args);
+	strbuf_reset(&std_out);
Quoted 8 lines

bugreport.c line 47

Quoted 5 lines
bugreport.c new file +5 -0
+
+
+	argv_array_push(&cp.args, "ldd");
+	argv_array_push(&cp.args, "--version");
+	capture_command(&cp, &std_out, 0);

Again, this command will only be present in few setups. I am not actually sure that the output of this is interesting to begin with.

Quoted 1 line

It was a suggestion, I believe, from Jonathan Nieder.

Yes, I guess Jonathan builds their Git locally, too.

It _is_ easy to forget that most users find this too involved to even try.

Nothing like reading through a bug tracker quite frequently to learn about the actual troubles actual users have :-)

Quoted 11 lines
Quoted 2 lines

What I _do_ think is that a much more interesting piece of information would be the exact GLIBC version, the OS name and the OS version.

The glibc version is easy; I've done that. It certainly looks nicer than the ldd call.

I guess I may be missing something, because as I start to look into how to the OS info, I fall down a hole of many, many preprocessor defines to check. If that's the approach you want me to take, just say the word, but it will be ugly :) I suppose I had hoped the uname info would give us a close enough idea that full OS info isn't necessary.

We could go down the pre-processor route, but that would give us the OS name and version at build time, not at run time. I think we will be mostly interested in the latter, though.

We might need to enhance our uname() emulation in compat/mingw.c, but I think we already have enough information there.

When it comes to glibc, I think gnu_get_libc_version() would get us what we want. A trickier thing might be to determine whether we're actually linking against glibc; I do not want to break musl builds again, I already did that inadvertently when requiring REG_STARTEND back in the days.

Quoted 23 lines
Quoted 14 lines
Quoted 14 lines
builtin/bugreport.c +1 -0
index 2ef16440a0..7232d31be7 100644
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "bugreport.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
@@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
Quoted 8 lines

builtin/bugreport.c int get_bug_template(struct strbuf *template) · line 35

Quoted 5 lines
builtin/bugreport.c +5 -0
+void add_header(FILE *report, const char *title)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	strbuf_addf(&buffer, "\n\n[%s]\n", title);
+	strbuf_write(&buffer, report);

This leaks buffer. Why not write into report via fprintf() directly?

Quoted 3 lines

Rather, to match the style of the rest of the builtin, modified get_header to add the header to a passed-in strbuf instead of modifying the file directly.

Hmm. It makes the code less elegant in my opinion. I would rather either render the entire bug report into a single strbuf and then write it via write_in_full(), or use fprintf() directly.

Ciao,
Dscho

#39 lore raw

Re: [PATCH v3 3/9] bugreport: add version and system information

Quoted 23 lines · Johannes Schindelin
Quoted 23 lines
Quoted 22 lines
Quoted 11 lines
builtin/bugreport.c +1 -0
index 2ef16440a0..7232d31be7 100644
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "bugreport.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"

builtin/bugreport.c int get_bug_template(struct strbuf *template) · line 35

Quoted 8 lines
builtin/bugreport.c +5 -0
@@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+void add_header(FILE *report, const char *title)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	strbuf_addf(&buffer, "\n\n[%s]\n", title);
+	strbuf_write(&buffer, report);

This leaks buffer. Why not write into report via fprintf() directly?

Quoted 7 lines
Quoted 3 lines

Rather, to match the style of the rest of the builtin, modified get_header to add the header to a passed-in strbuf instead of modifying the file directly.

Hmm. It makes the code less elegant in my opinion. I would rather either render the entire bug report into a single strbuf and then write it via write_in_full(), or use fprintf() directly.

Yeah, that sounds good. I will do that. :)

#40 lore raw

Re: [PATCH v3 3/9] bugreport: add version and system information

In reply to Emily Shaffer

On 2019.10.24 19:51, Emily Shaffer wrote:
[snip]

Quoted 11 lines
builtin/bugreport.c +1 -0
index 2ef16440a0..7232d31be7 100644
@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "bugreport.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"

builtin/bugreport.c int get_bug_template(struct strbuf *template) · line 31

Quoted 4 lines
builtin/bugreport.c +1 -0
@@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+void add_header(FILE *report, const char *title)

I get the same compilation error here, can you make add_header() static as well?

#41 lore raw

PATCH v3 4/9 bugreport: add config values from whitelist

In reply to Emily Shaffer

Teach bugreport to gather the values of config options which are present in 'git-bugreport-config-whitelist'.

Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.

Reading the whitelist into memory and sorting it saves us time - since git_config_bugreport() is called for every option the user has configured, limiting the file IO to one open/read/close and performing option lookup in sublinear time is a useful optimization.

---
 bugreport.c         | 50 +++++++++++++++++++++++++++++++++++++++++++++
 bugreport.h         |  7 +++++++
 builtin/bugreport.c |  4 ++++
 3 files changed, 61 insertions(+)
bugreport.c +50 -0
index ada54fe583..afa4836ab1 100644
@@ -1,10 +1,24 @@
 #include "cache.h"
 
 #include "bugreport.h"
+#include "config.h"
+#include "exec-cmd.h"
 #include "help.h"
 #include "run-command.h"
 #include "version.h"
 
+/**
+ * A sorted list of config options which we will add to the bugreport. Managed
+ * by 'gather_whitelist(...)'.
+ */
+struct string_list whitelist = STRING_LIST_INIT_DUP;
+struct strbuf configs_and_values = STRBUF_INIT;
+
+// git version --build-options
+// uname -a
+// curl-config --version
+// ldd --version
+// echo $SHELL
 void get_system_info(struct strbuf *sys_info)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;
@@ -53,3 +67,39 @@ void get_system_info(struct strbuf *sys_info)
 	argv_array_clear(&cp.args);
 	strbuf_reset(&std_out);
 }
+
+void gather_whitelist(struct strbuf *path)
+{
+	struct strbuf tmp = STRBUF_INIT;
+	strbuf_read_file(&tmp, path->buf, 0);
+	string_list_init(&whitelist, 1);
+	string_list_split(&whitelist, tmp.buf, '\n', -1);
+	string_list_sort(&whitelist);
+}
+
+int git_config_bugreport(const char *var, const char *value, void *cb)
+{
+	if (string_list_has_string(&whitelist, var)) {
+		strbuf_addf(&configs_and_values,
+			    "%s : %s\n",
+			    var, value);
+	}
+
+	return 0;
+}
+
+void get_whitelisted_config(struct strbuf *config_info)
+{
+	struct strbuf path = STRBUF_INIT;
+
+	strbuf_addstr(&path, git_exec_path());
+	strbuf_addstr(&path, "/git-bugreport-config-whitelist");
+
+	gather_whitelist(&path);
+	strbuf_init(&configs_and_values, whitelist.nr);
+
+	git_config(git_config_bugreport, NULL);
+
+	strbuf_reset(config_info);
+	strbuf_addbuf(config_info, &configs_and_values);
+}
bugreport.h +7 -0
index ba216acf3f..7413e7e1be 100644
@@ -5,3 +5,10 @@
  * The previous contents of sys_info will be discarded.
  */
 void get_system_info(struct strbuf *sys_info);
+
+/**
+ * Adds the values of the config items listed in
+ * 'git-bugreport-config-whitelist' to config_info. The previous contents of
+ * config_info will be discarded.
+ */
+void get_whitelisted_config(struct strbuf *sys_info);
builtin/bugreport.c +4 -0
index 7232d31be7..70fe0d2b85 100644
@@ -56,6 +56,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_system_info(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Whitelisted Config");
+	get_whitelisted_config(&buffer);
+	strbuf_write(&buffer, report);
+
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
#42 lore raw

Re: [PATCH v3 4/9] bugreport: add config values from whitelist

Hi Emily,

Quoted 7 lines · Emily Shaffer

Teach bugreport to gather the values of config options which are present in 'git-bugreport-config-whitelist'.

Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.

Should we still have the // bugreport-exclude comments, then?

Quoted 5 lines

Reading the whitelist into memory and sorting it saves us time - since git_config_bugreport() is called for every option the user has configured, limiting the file IO to one open/read/close and performing option lookup in sublinear time is a useful optimization.

Maybe we even want a hashmap? That would reduce the time complexity even further.

Quoted 16 lines
bugreport.c +4 -0
index ada54fe583..afa4836ab1 100644
@@ -1,10 +1,24 @@
 #include "cache.h"
 
 #include "bugreport.h"
+#include "config.h"
+#include "exec-cmd.h"
 #include "help.h"
 #include "run-command.h"
 #include "version.h"
 
+/**
+ * A sorted list of config options which we will add to the bugreport. Managed

bugreport.c line 21

Quoted 10 lines
bugreport.c +10 -0
+ * by 'gather_whitelist(...)'.
+ */
+struct string_list whitelist = STRING_LIST_INIT_DUP;
+struct strbuf configs_and_values = STRBUF_INIT;
+
+// git version --build-options
+// uname -a
+// curl-config --version
+// ldd --version
+// echo $SHELL

These comments probably want to move to a single, C style comment, and they probably want to be introduced together with get_system_info().

I also have to admit that I might have missed where $SHELL was added to the output...

Quoted 13 lines
bugreport.c +6 -0
 void get_system_info(struct strbuf *sys_info)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;
@@ -53,3 +67,39 @@ void get_system_info(struct strbuf *sys_info)
 	argv_array_clear(&cp.args);
 	strbuf_reset(&std_out);
 }
+
+void gather_whitelist(struct strbuf *path)
+{
+	struct strbuf tmp = STRBUF_INIT;
+	strbuf_read_file(&tmp, path->buf, 0);
+	string_list_init(&whitelist, 1);

bugreport.c void get_system_info(struct strbuf *sys_info) · line 85

Quoted 10 lines
bugreport.c +10 -0
+	string_list_split(&whitelist, tmp.buf, '\n', -1);
+	string_list_sort(&whitelist);
+}
+
+int git_config_bugreport(const char *var, const char *value, void *cb)
+{
+	if (string_list_has_string(&whitelist, var)) {
+		strbuf_addf(&configs_and_values,
+			    "%s : %s\n",
+			    var, value);

A quite useful piece of information would be the config source. Not sure whether we can do that outside of config.c yet...

bugreport.c void get_system_info(struct strbuf *sys_info) · line 96

Quoted 11 lines
bugreport.c +11 -0
+	}
+
+	return 0;
+}
+
+void get_whitelisted_config(struct strbuf *config_info)
+{
+	struct strbuf path = STRBUF_INIT;
+
+	strbuf_addstr(&path, git_exec_path());
+	strbuf_addstr(&path, "/git-bugreport-config-whitelist");

Hmm. I would have expected this patch to come directly after the patch 2/9 that generates that white-list, and I would also have expected that to be pre-sorted, and compiled in.

Do you want users to _edit_ the file in the exec path? In general, that path will be write-protected, though. A better alternative would probably be to compile in a hard-coded list, and to allow including more values e.g. by offering command-line options to specify config setting patterns. But if we allow patterns, we might actually want to have those exclusions to prevent sensitive data from being included.

Quoted 14 lines
bugreport.c +9 -0
+
+	gather_whitelist(&path);
+	strbuf_init(&configs_and_values, whitelist.nr);
+
+	git_config(git_config_bugreport, NULL);
+
+	strbuf_reset(config_info);
+	strbuf_addbuf(config_info, &configs_and_values);
+}
bugreport.h +0 -0
index ba216acf3f..7413e7e1be 100644

bugreport.h line 9

Quoted 5 lines
bugreport.h +2 -0
@@ -5,3 +5,10 @@
  * The previous contents of sys_info will be discarded.
  */
 void get_system_info(struct strbuf *sys_info);
+
+/**

I also frequently use JavaDoc-style /**, but I am not sure that this is actually desired in Git's source code ;-)

Quoted 10 lines
bugreport.h +5 -0
+ * Adds the values of the config items listed in
+ * 'git-bugreport-config-whitelist' to config_info. The previous contents of
+ * config_info will be discarded.
+ */
+void get_whitelisted_config(struct strbuf *sys_info);
builtin/bugreport.c +0 -0
index 7232d31be7..70fe0d2b85 100644

builtin/bugreport.c int cmd_bugreport(int argc, const char **argv, const char *prefix) · line 59

Quoted 4 lines
builtin/bugreport.c +1 -0
@@ -56,6 +56,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_system_info(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Whitelisted Config");

Quite honestly, I would like to avoid the term "whitelist" for good. How about "Selected config settings" instead?

Thanks,
Dscho

Quoted 10 lines
builtin/bugreport.c +3 -1
+	get_whitelisted_config(&buffer);
+	strbuf_write(&buffer, report);
+
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
--

2.24.0.rc0.303.g954a862665-goog

#43 lore raw

Re: [PATCH v3 4/9] bugreport: add config values from whitelist

Quoted 1 line · Johannes Schindelin

Hi Emily,

Sorry for the delay in replying. This work has been backburnered and this mail slipped through the cracks.

Quoted 6 lines · Emily Shaffer
Quoted 4 lines

Teach bugreport to gather the values of config options which are present in 'git-bugreport-config-whitelist'.

Quoted 6 lines
Quoted 4 lines

Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.

Should we still have the // bugreport-exclude comments, then?

They were optional (useless) before too. I can remove them if you want; I suppose I like the idea of having precedent if someone wants to build their own internal version with opt-out configs rather than opt-in. I can remove them if we want; it doesn't matter very much to me either way.

Quoted 9 lines
Quoted 6 lines

Reading the whitelist into memory and sorting it saves us time - since git_config_bugreport() is called for every option the user has configured, limiting the file IO to one open/read/close and performing option lookup in sublinear time is a useful optimization.

Maybe we even want a hashmap? That would reduce the time complexity even further.

Sure, we can do it. I'll make that change.

Quoted 15 lines
Quoted 15 lines
bugreport.c +2 -0
index ada54fe583..afa4836ab1 100644
@@ -1,10 +1,24 @@
 #include "cache.h"
 
 #include "bugreport.h"
+#include "config.h"
+#include "exec-cmd.h"
 #include "help.h"
 #include "run-command.h"
 #include "version.h"
 
Quoted 15 lines

bugreport.c line 21

Quoted 12 lines
bugreport.c +12 -0
+/**
+ * A sorted list of config options which we will add to the bugreport. Managed
+ * by 'gather_whitelist(...)'.
+ */
+struct string_list whitelist = STRING_LIST_INIT_DUP;
+struct strbuf configs_and_values = STRBUF_INIT;
+
+// git version --build-options
+// uname -a
+// curl-config --version
+// ldd --version
+// echo $SHELL

These comments probably want to move to a single, C style comment, and they probably want to be introduced together with get_system_info().

Yeah, it's stale and has been removed now. It was less commentary and more todo list for author ;)

Quoted 3 lines

I also have to admit that I might have missed where $SHELL was added to the output...

I skipped it entirely since bugreport doesn't run in shell anymore. If you have advice for gathering the user's shell I can try to add it; is there such a difference between, say, a Debian user using bash and a Debian user using zsh? I suppose it could be useful if someone has an issue with GIT_PS1, or with autocompletion. I'll look into gathering it.

Quoted 19 lines
Quoted 19 lines
bugreport.c +11 -0
 void get_system_info(struct strbuf *sys_info)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;
@@ -53,3 +67,39 @@ void get_system_info(struct strbuf *sys_info)
 	argv_array_clear(&cp.args);
 	strbuf_reset(&std_out);
 }
+
+void gather_whitelist(struct strbuf *path)
+{
+	struct strbuf tmp = STRBUF_INIT;
+	strbuf_read_file(&tmp, path->buf, 0);
+	string_list_init(&whitelist, 1);
+	string_list_split(&whitelist, tmp.buf, '\n', -1);
+	string_list_sort(&whitelist);
+}
+
+int git_config_bugreport(const char *var, const char *value, void *cb)
Quoted 8 lines

bugreport.c void get_system_info(struct strbuf *sys_info) · line 85

Quoted 5 lines
bugreport.c +5 -0
+{
+	if (string_list_has_string(&whitelist, var)) {
+		strbuf_addf(&configs_and_values,
+			    "%s : %s\n",
+			    var, value);

A quite useful piece of information would be the config source. Not sure whether we can do that outside of config.c yet...

It's possible. I can add it.

Quoted 17 lines

bugreport.c void get_system_info(struct strbuf *sys_info) · line 96

Quoted 12 lines
bugreport.c +11 -0
+	}
+
+	return 0;
+}
+
+void get_whitelisted_config(struct strbuf *config_info)
+{
+	struct strbuf path = STRBUF_INIT;
+
+	strbuf_addstr(&path, git_exec_path());
+	strbuf_addstr(&path, "/git-bugreport-config-whitelist");

Hmm. I would have expected this patch to come directly after the patch 2/9 that generates that white-list, and I would also have expected that to be pre-sorted, and compiled in.

Quoted 6 lines

Do you want users to _edit_ the file in the exec path? In general, that path will be write-protected, though. A better alternative would probably be to compile in a hard-coded list, and to allow including more values e.g. by offering command-line options to specify config setting patterns. But if we allow patterns, we might actually want to have those exclusions to prevent sensitive data from being included.

Hm, interesting. Do we have precedent for compiling in a header generated during the build process? I think I saw one when I was adding this script - I'll take a look.

Quoted 37 lines
Quoted 15 lines
bugreport.c +9 -0
+
+	gather_whitelist(&path);
+	strbuf_init(&configs_and_values, whitelist.nr);
+
+	git_config(git_config_bugreport, NULL);
+
+	strbuf_reset(config_info);
+	strbuf_addbuf(config_info, &configs_and_values);
+}
bugreport.h +0 -0
index ba216acf3f..7413e7e1be 100644

bugreport.h line 9

Quoted 5 lines
bugreport.h +2 -0
@@ -5,3 +5,10 @@
  * The previous contents of sys_info will be discarded.
  */
 void get_system_info(struct strbuf *sys_info);
+
+/**

I also frequently use JavaDoc-style /**, but I am not sure that this is actually desired in Git's source code ;-)

Quoted 13 lines
bugreport.h +5 -0
+ * Adds the values of the config items listed in
+ * 'git-bugreport-config-whitelist' to config_info. The previous contents of
+ * config_info will be discarded.
+ */
+void get_whitelisted_config(struct strbuf *sys_info);
builtin/bugreport.c +0 -0
index 7232d31be7..70fe0d2b85 100644
@@ -56,6 +56,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_system_info(&buffer);
 	strbuf_write(&buffer, report);
 
Quoted 4 lines

builtin/bugreport.c int cmd_bugreport(int argc, const char **argv, const char *prefix) · line 59

Quoted 1 line
builtin/bugreport.c +1 -0
+	add_header(report, "Whitelisted Config");

Quite honestly, I would like to avoid the term "whitelist" for good. How about "Selected config settings" instead?

Will do - thanks for the callout.

Quoted 14 lines

Thanks,
Dscho

Quoted 10 lines
builtin/bugreport.c +3 -1
+	get_whitelisted_config(&buffer);
+	strbuf_write(&buffer, report);
+
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
--

2.24.0.rc0.303.g954a862665-goog

#44 lore raw

Re: [PATCH v3 4/9] bugreport: add config values from whitelist

Hi Emily,

Quoted 13 lines · Emily Shaffer
Quoted 11 lines · Johannes Schindelin
Quoted 7 lines · Emily Shaffer

Teach bugreport to gather the values of config options which are present in 'git-bugreport-config-whitelist'.

Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.

Quoted 7 lines
Quoted 1 line

Should we still have the // bugreport-exclude comments, then?

They were optional (useless) before too. I can remove them if you want; I suppose I like the idea of having precedent if someone wants to build their own internal version with opt-out configs rather than opt-in. I can remove them if we want; it doesn't matter very much to me either way.

How are you guaranteeing that this information does not become stale, like, immediately?

If you _still_ insist on keeping those comments even after answering this question, at least please turn them into C-style comments: we have no C++-style comments in Git and want to keep it this way.

Quoted 25 lines
Quoted 18 lines

bugreport.c void get_system_info(struct strbuf *sys_info) · line 96

Quoted 13 lines
bugreport.c +11 -0
[...]
+	}
+
+	return 0;
+}
+
+void get_whitelisted_config(struct strbuf *config_info)
+{
+	struct strbuf path = STRBUF_INIT;
+
+	strbuf_addstr(&path, git_exec_path());
+	strbuf_addstr(&path, "/git-bugreport-config-whitelist");

Hmm. I would have expected this patch to come directly after the patch 2/9 that generates that white-list, and I would also have expected that to be pre-sorted, and compiled in.

Quoted 6 lines

Do you want users to _edit_ the file in the exec path? In general, that path will be write-protected, though. A better alternative would probably be to compile in a hard-coded list, and to allow including more values e.g. by offering command-line options to specify config setting patterns. But if we allow patterns, we might actually want to have those exclusions to prevent sensitive data from being included.

Quoted 3 lines

Hm, interesting. Do we have precedent for compiling in a header generated during the build process? I think I saw one when I was adding this script - I'll take a look.

Yes, we do. The entire command-list.h is generated during the build. Look for GENERATED_H in the Makefile.

Ciao,
Dscho

#45 lore raw

Re: [PATCH v3 4/9] bugreport: add config values from whitelist

In reply to Emily Shaffer

On 2019.10.24 19:51, Emily Shaffer wrote:
[snip]

Quoted 30 lines
bugreport.c +14 -0
index ada54fe583..afa4836ab1 100644
@@ -1,10 +1,24 @@
 #include "cache.h"
 
 #include "bugreport.h"
+#include "config.h"
+#include "exec-cmd.h"
 #include "help.h"
 #include "run-command.h"
 #include "version.h"
 
+/**
+ * A sorted list of config options which we will add to the bugreport. Managed
+ * by 'gather_whitelist(...)'.
+ */
+struct string_list whitelist = STRING_LIST_INIT_DUP;
+struct strbuf configs_and_values = STRBUF_INIT;
+
+// git version --build-options
+// uname -a
+// curl-config --version
+// ldd --version
+// echo $SHELL
 void get_system_info(struct strbuf *sys_info)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;

bugreport.c void get_system_info(struct strbuf *sys_info) · line 71

Quoted 5 lines
bugreport.c +2 -0
@@ -53,3 +67,39 @@ void get_system_info(struct strbuf *sys_info)
 	argv_array_clear(&cp.args);
 	strbuf_reset(&std_out);
 }
+
+void gather_whitelist(struct strbuf *path)

This and git_config_bugreport() below should both be static as well. Rather than repeating advice on the later patches, I'll just note that any new functions that don't show up in the corresponding .h file should be marked static.

#46 lore raw

Re: [PATCH v3 4/9] bugreport: add config values from whitelist

Quoted 4 lines · Josh Steadmon

This and git_config_bugreport() below should both be static as well. Rather than repeating advice on the later patches, I'll just note that any new functions that don't show up in the corresponding .h file should be marked static.

Good advice.

More importantly, given that "git bugreport" itself has no service of itself to offer other existing parts of Git (it is just a "gather various pieces of information from different places, and then produce a text file output" application), I do not see much point in it having its own header file that others would #include (i.e. the include file is to define services that are offered by it). If there are common enough service routines invented to support the need of bugreport.c (e.g. perhaps it wants to give more info than what is currently available via the existing API on the contents of in-core index), by definition of being them common enough, they should be added to the header that can be used by both bugreport.c and other existing users of the same subsystem (e.g. if it is about in-core index, perhaps cache.h).

It makes perfect sense for bugreport.c to #include header files for the Git internals to collect pieces of information from inside Git, though.

#47 lore raw

Re: [PATCH v3 4/9] bugreport: add config values from whitelist

Quoted 9 lines · Junio C Hamano
Quoted 4 lines · Josh Steadmon

This and git_config_bugreport() below should both be static as well. Rather than repeating advice on the later patches, I'll just note that any new functions that don't show up in the corresponding .h file should be marked static.

Good advice.

Quoted 13 lines

More importantly, given that "git bugreport" itself has no service of itself to offer other existing parts of Git (it is just a "gather various pieces of information from different places, and then produce a text file output" application), I do not see much point in it having its own header file that others would #include (i.e. the include file is to define services that are offered by it). If there are common enough service routines invented to support the need of bugreport.c (e.g. perhaps it wants to give more info than what is currently available via the existing API on the contents of in-core index), by definition of being them common enough, they should be added to the header that can be used by both bugreport.c and other existing users of the same subsystem (e.g. if it is about in-core index, perhaps cache.h).

Are you asking me to have only builtin/bugreport.c and implement each function there, and eliminate both <basedir>/bugreport.[ch]?

It may be true that deciding to put this functionality into a library "in case someone needs it later" was premature, so I can do that - I just want to make sure I understand you.

- Emily

#48 lore raw

PATCH v3 5/9 bugreport: collect list of populated hooks

In reply to Emily Shaffer

Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong.

Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot. To reduce the amount of code humans need to read, we turn the list into a string_list and iterate over it (as we are calling the same find_hook operation on each string). However, since bugreport should primarily be called by the user, the performance loss from massaging the string seems acceptable.

---
 bugreport.c         | 44 ++++++++++++++++++++++++++++++++++++++++++++
 bugreport.h         |  6 ++++++
 builtin/bugreport.c |  4 ++++
 3 files changed, 54 insertions(+)
bugreport.c +44 -0
index afa4836ab1..9d7f44ff28 100644
@@ -103,3 +103,47 @@ void get_whitelisted_config(struct strbuf *config_info)
 	strbuf_reset(config_info);
 	strbuf_addbuf(config_info, &configs_and_values);
 }
+
+void get_populated_hooks(struct strbuf *hook_info)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hook`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+
+	strbuf_reset(hook_info);
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
bugreport.h +6 -0
index 7413e7e1be..942a5436e3 100644
@@ -12,3 +12,9 @@ void get_system_info(struct strbuf *sys_info);
  * config_info will be discarded.
  */
 void get_whitelisted_config(struct strbuf *sys_info);
+
+/**
+ * Adds the paths to all configured hooks (but not their contents). The previous
+ * contents of hook_info will be discarded.
+ */
+void get_populated_hooks(struct strbuf *hook_info);
builtin/bugreport.c +4 -0
index 70fe0d2b85..a0eefba498 100644
@@ -60,6 +60,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_whitelisted_config(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Populated Hooks");
+	get_populated_hooks(&buffer);
+	strbuf_write(&buffer, report);
+
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
#49 lore raw

Re: [PATCH v3 5/9] bugreport: collect list of populated hooks

Hi Emily,

Quoted 7 lines · Emily Shaffer

Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong.

Quoted 7 lines

Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot. To reduce the amount of code humans need to read, we turn the list into a string_list and iterate over it (as we are calling the same find_hook operation on each string). However, since bugreport should primarily be called by the user, the performance loss from massaging the string seems acceptable.

Good idea!

Quoted 36 lines
---
 bugreport.c         | 44 ++++++++++++++++++++++++++++++++++++++++++++
 bugreport.h         |  6 ++++++
 builtin/bugreport.c |  4 ++++
 3 files changed, 54 insertions(+)
bugreport.c +20 -0
index afa4836ab1..9d7f44ff28 100644
@@ -103,3 +103,47 @@ void get_whitelisted_config(struct strbuf *config_info)
 	strbuf_reset(config_info);
 	strbuf_addbuf(config_info, &configs_and_values);
 }
+
+void get_populated_hooks(struct strbuf *hook_info)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hook`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"

bugreport.c void get_whitelisted_config(struct strbuf *config_info) · line 135

Quoted 10 lines
bugreport.c +10 -0
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";

Well, this is disappointing: I tried to come up with a scripted way to extract the commit names from our source code, and I failed! This is where I gave up:

	git grep run_hook |
	sed -n 's/.*run_hook[^(]*([^,]*, *\([^,]*\).*/\1/p' |
	sed -e '/^name$/d' -e '/^const char \*name$/d' \
		-e 's/push_to_checkout_hook/"push-to-checkout"/' |
	sort |
	uniq

This prints only the following hook names:

	"applypatch-msg"
	"post-applypatch"
	"post-checkout"
	"post-index-change"
	"post-merge"
	"pre-applypatch"
	"pre-auto-gc"
	"pre-rebase"
	"prepare-commit-msg"
	"push-to-checkout"

But at least it was easy to script the extracting from the Documentation:

	sed -n '/^[a-z]/{N;/\n~~~/{s/\n.*//;p}}' \
		Documentation/githooks.txt

bugreport.c void get_whitelisted_config(struct strbuf *config_info) · line 143

Quoted 8 lines
bugreport.c +8 -0
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+
+	strbuf_reset(hook_info);
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {

This should definitely be done at compile time, I think. We should be able to generate an appropriate header via something like this:

	cat >hook-names.h <<-EOF
	static const char *hook_names[] = {
	$(sed -n '/^[a-z]/{N;/\n~~~/{s/\n.*/",/;s/^/	"/;p}}' \
		Documentation/githooks.txt)
	};
	EOF

Then you would use a simple for() loop using ARRAY_SIZE() to iterate over the hook names.

Quoted 25 lines
bugreport.c +6 -0
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
bugreport.h +6 -0
index 7413e7e1be..942a5436e3 100644
@@ -12,3 +12,9 @@ void get_system_info(struct strbuf *sys_info);
  * config_info will be discarded.
  */
 void get_whitelisted_config(struct strbuf *sys_info);
+
+/**
+ * Adds the paths to all configured hooks (but not their contents). The previous
+ * contents of hook_info will be discarded.
+ */
+void get_populated_hooks(struct strbuf *hook_info);
builtin/bugreport.c +0 -0
index 70fe0d2b85..a0eefba498 100644

builtin/bugreport.c int cmd_bugreport(int argc, const char **argv, const char *prefix) · line 63

Quoted 4 lines
builtin/bugreport.c +1 -0
@@ -60,6 +60,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_whitelisted_config(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Populated Hooks");

Wait! I should have stumbled over this in an earlier patch. The add_header() function should not take a FILE * parameter at all, but instead an struct strbuf * one!

Ciao,
Dscho

Quoted 10 lines
builtin/bugreport.c +3 -1
+	get_populated_hooks(&buffer);
+	strbuf_write(&buffer, report);
+
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
--

2.24.0.rc0.303.g954a862665-goog

#50 lore raw

Re: [PATCH v3 5/9] bugreport: collect list of populated hooks

Quoted 92 lines · Johannes Schindelin

Hi Emily,

Quoted 7 lines · Emily Shaffer

Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong.

Quoted 7 lines

Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot. To reduce the amount of code humans need to read, we turn the list into a string_list and iterate over it (as we are calling the same find_hook operation on each string). However, since bugreport should primarily be called by the user, the performance loss from massaging the string seems acceptable.

Good idea!

Quoted 36 lines
---
 bugreport.c         | 44 ++++++++++++++++++++++++++++++++++++++++++++
 bugreport.h         |  6 ++++++
 builtin/bugreport.c |  4 ++++
 3 files changed, 54 insertions(+)
bugreport.c +20 -0
index afa4836ab1..9d7f44ff28 100644
@@ -103,3 +103,47 @@ void get_whitelisted_config(struct strbuf *config_info)
 	strbuf_reset(config_info);
 	strbuf_addbuf(config_info, &configs_and_values);
 }
+
+void get_populated_hooks(struct strbuf *hook_info)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hook`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"

bugreport.c void get_whitelisted_config(struct strbuf *config_info) · line 135

Quoted 10 lines
bugreport.c +10 -0
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";

Well, this is disappointing: I tried to come up with a scripted way to extract the commit names from our source code, and I failed! This is where I gave up:

	git grep run_hook |
	sed -n 's/.*run_hook[^(]*([^,]*, *\([^,]*\).*/\1/p' |
	sed -e '/^name$/d' -e '/^const char \*name$/d' \
		-e 's/push_to_checkout_hook/"push-to-checkout"/' |
	sort |
	uniq

This prints only the following hook names:

	"applypatch-msg"
	"post-applypatch"
	"post-checkout"
	"post-index-change"
	"post-merge"
	"pre-applypatch"
	"pre-auto-gc"
	"pre-rebase"
	"prepare-commit-msg"
	"push-to-checkout"
Quoted 6 lines

But at least it was easy to script the extracting from the Documentation:

	sed -n '/^[a-z]/{N;/\n~~~/{s/\n.*//;p}}' \
		Documentation/githooks.txt

To be totally frank, I'm not keen on spending a lot of time with scripting this. My parallel work with config-based hooks will also involve an in-code source of truth for available hooknames; I'd like to punt on some scripting, putting it in the makefile, etc blah since I know I'm planning to fix this particular annoyance at the source - and since it looks like bugreport will stay in the review phase for at least a little longer.

Quoted 69 lines

bugreport.c void get_whitelisted_config(struct strbuf *config_info) · line 143

Quoted 8 lines
bugreport.c +8 -0
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+
+	strbuf_reset(hook_info);
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {

This should definitely be done at compile time, I think. We should be able to generate an appropriate header via something like this:

	cat >hook-names.h <<-EOF
	static const char *hook_names[] = {
	$(sed -n '/^[a-z]/{N;/\n~~~/{s/\n.*/",/;s/^/	"/;p}}' \
		Documentation/githooks.txt)
	};
	EOF

Then you would use a simple for() loop using ARRAY_SIZE() to iterate over the hook names.

Quoted 25 lines
bugreport.c +6 -0
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
bugreport.h +6 -0
index 7413e7e1be..942a5436e3 100644
@@ -12,3 +12,9 @@ void get_system_info(struct strbuf *sys_info);
  * config_info will be discarded.
  */
 void get_whitelisted_config(struct strbuf *sys_info);
+
+/**
+ * Adds the paths to all configured hooks (but not their contents). The previous
+ * contents of hook_info will be discarded.
+ */
+void get_populated_hooks(struct strbuf *hook_info);
builtin/bugreport.c +0 -0
index 70fe0d2b85..a0eefba498 100644

builtin/bugreport.c int cmd_bugreport(int argc, const char **argv, const char *prefix) · line 63

Quoted 4 lines
builtin/bugreport.c +1 -0
@@ -60,6 +60,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_whitelisted_config(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Populated Hooks");

Wait! I should have stumbled over this in an earlier patch. The add_header() function should not take a FILE * parameter at all, but instead an struct strbuf * one!

Ciao,
Dscho

Quoted 10 lines
builtin/bugreport.c +3 -1
+	get_populated_hooks(&buffer);
+	strbuf_write(&buffer, report);
+
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
--

2.24.0.rc0.303.g954a862665-goog

#51 lore raw

Re: [PATCH v3 5/9] bugreport: collect list of populated hooks

Hi Emily,

Quoted 74 lines · Emily Shaffer
Quoted 65 lines · Johannes Schindelin
Quoted 28 lines · Emily Shaffer
bugreport.c +20 -0
index afa4836ab1..9d7f44ff28 100644
@@ -103,3 +103,47 @@ void get_whitelisted_config(struct strbuf *config_info)
 	strbuf_reset(config_info);
 	strbuf_addbuf(config_info, &configs_and_values);
 }
+
+void get_populated_hooks(struct strbuf *hook_info)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hook`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"

bugreport.c void get_whitelisted_config(struct strbuf *config_info) · line 135

Quoted 10 lines
bugreport.c +10 -0
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";

Well, this is disappointing: I tried to come up with a scripted way to extract the commit names from our source code, and I failed! This is where I gave up:

	git grep run_hook |
	sed -n 's/.*run_hook[^(]*([^,]*, *\([^,]*\).*/\1/p' |
	sed -e '/^name$/d' -e '/^const char \*name$/d' \
		-e 's/push_to_checkout_hook/"push-to-checkout"/' |
	sort |
	uniq

This prints only the following hook names:

	"applypatch-msg"
	"post-applypatch"
	"post-checkout"
	"post-index-change"
	"post-merge"
	"pre-applypatch"
	"pre-auto-gc"
	"pre-rebase"
	"prepare-commit-msg"
	"push-to-checkout"
Quoted 6 lines

But at least it was easy to script the extracting from the Documentation:

	sed -n '/^[a-z]/{N;/\n~~~/{s/\n.*//;p}}' \
		Documentation/githooks.txt
Quoted 7 lines

To be totally frank, I'm not keen on spending a lot of time with scripting this. My parallel work with config-based hooks will also involve an in-code source of truth for available hooknames; I'd like to punt on some scripting, putting it in the makefile, etc blah since I know I'm planning to fix this particular annoyance at the source - and since it looks like bugreport will stay in the review phase for at least a little longer.

Fair enough, especially if it addresses my complaint about the scriptability.

Ciao,
Dscho

#52 lore raw

PATCH v3 6/9 bugreport: count loose objects

In reply to Emily Shaffer

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

Rather than directly invoking 'git-count-objects', which may sometimes fail unexpectedly on Git for Windows, manually count the contents of .git/objects. Additionally, since we may wish to inspect other directories' contents for bugreport in the future, put the directory listing into a helper function.

---
 bugreport.c         | 72 +++++++++++++++++++++++++++++++++++++++++++++
 bugreport.h         |  6 ++++
 builtin/bugreport.c |  4 +++
 3 files changed, 82 insertions(+)
bugreport.c +72 -0
index 9d7f44ff28..54e1d47103 100644
@@ -5,8 +5,11 @@
 #include "exec-cmd.h"
 #include "help.h"
 #include "run-command.h"
+#include "strbuf.h"
 #include "version.h"
 
+#include "dirent.h"
+
 /**
  * A sorted list of config options which we will add to the bugreport. Managed
  * by 'gather_whitelist(...)'.
@@ -147,3 +150,72 @@ void get_populated_hooks(struct strbuf *hook_info)
 		}
 	}
 }
+
+/**
+ * Fill 'contents' with the contents of the dir at 'dirpath'.
+ * If 'filter' is nonzero, the contents are filtered on d_type as 'type' - see
+ * 'man readdir'. opendir() doesn't take string length as an arg, so don't
+ * bother passing it in.
+ */
+void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath,
+			  int filter, unsigned char type)
+{
+	struct dirent *dir = NULL;
+	DIR *dh = NULL;
+
+	dh = opendir(dirpath->buf);
+	while (dh && (dir = readdir(dh))) {
+		if (!filter || type == dir->d_type) {
+			string_list_append(contents, dir->d_name);
+		}
+	}
+}
+
+
+void get_object_counts(struct strbuf *obj_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+	struct strbuf std_out = STRBUF_INIT;
+
+	argv_array_push(&cp.args, "count-objects");
+	argv_array_push(&cp.args, "-vH");
+	cp.git_cmd = 1;
+	capture_command(&cp, &std_out, 0);
+
+	strbuf_reset(obj_info);
+	strbuf_addstr(obj_info, "git-count-objects -vH:\n");
+	strbuf_addbuf(obj_info, &std_out);
+}
+
+void get_loose_object_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+	struct string_list subdirs = STRING_LIST_INIT_DUP;
+	struct string_list_item *subdir;
+
+	strbuf_reset(obj_info);
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+
+	list_contents_of_dir(&subdirs, &dirpath, 1, DT_DIR);
+
+	for_each_string_list_item(subdir, &subdirs)
+	{
+		struct strbuf subdir_buf = STRBUF_INIT;
+		struct string_list objects = STRING_LIST_INIT_DUP;
+
+		/*
+		 * Only interested in loose objects - so dirs named with the
+		 * first byte of the object ID
+		 */
+		if (strlen(subdir->string) != 2 || !strcmp(subdir->string, ".."))
+			continue;
+
+		strbuf_addbuf(&subdir_buf, &dirpath);
+		strbuf_addstr(&subdir_buf, subdir->string);
+		list_contents_of_dir(&objects, &subdir_buf, 0, 0);
+		strbuf_addf(obj_info, "%s: %d objects\n", subdir->string,
+			    objects.nr);
+	}
+}
bugreport.h +6 -0
index 942a5436e3..09ad0c2599 100644
@@ -18,3 +18,9 @@ void get_whitelisted_config(struct strbuf *sys_info);
  * contents of hook_info will be discarded.
  */
 void get_populated_hooks(struct strbuf *hook_info);
+
+/**
+ * Adds the output of `git count-object -vH`. The previous contents of hook_info
+ * will be discarded.
+ */
+void get_loose_object_summary(struct strbuf *obj_info);
builtin/bugreport.c +4 -0
index a0eefba498..b2ab194207 100644
@@ -64,6 +64,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_populated_hooks(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Object Counts");
+	get_loose_object_summary(&buffer);
+	strbuf_write(&buffer, report);
+
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
#53 lore raw

Re: [PATCH v3 6/9] bugreport: count loose objects

Hi Emily,

Quoted 9 lines · Emily Shaffer

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

Rather than directly invoking 'git-count-objects', which may sometimes fail unexpectedly on Git for Windows, manually count the contents of .git/objects. Additionally, since we may wish to inspect other directories' contents for bugreport in the future, put the directory listing into a helper function.

Thank you, much appreciated!

I guess the next step is to count the number of packs, and the number of submodules ;-)

Quoted 13 lines
---
 bugreport.c         | 72 +++++++++++++++++++++++++++++++++++++++++++++
 bugreport.h         |  6 ++++
 builtin/bugreport.c |  4 +++
 3 files changed, 82 insertions(+)
bugreport.c +0 -0
index 9d7f44ff28..54e1d47103 100644

bugreport.c line 8

Quoted 4 lines
bugreport.c +1 -0
@@ -5,8 +5,11 @@
 #include "exec-cmd.h"
 #include "help.h"
 #include "run-command.h"
+#include "strbuf.h"

Why not append this to the end of the #include list, as is common in Git's commit history?

bugreport.c line 11

Quoted 3 lines
bugreport.c +1 -0
 #include "version.h"
 
+#include "dirent.h"

This header (although with pointy brackets instead of double quotes) is already included in git-compat-util.h

bugreport.c void get_populated_hooks(struct strbuf *hook_info) · line 155

Quoted 11 lines
bugreport.c +4 -0
+
 /**
  * A sorted list of config options which we will add to the bugreport. Managed
  * by 'gather_whitelist(...)'.
@@ -147,3 +150,72 @@ void get_populated_hooks(struct strbuf *hook_info)
 		}
 	}
 }
+
+/**
+ * Fill 'contents' with the contents of the dir at 'dirpath'.

Since you start this comment in JavaDoc style, there should be an almost empty line after this one ("almost" because it still contains the asterisk, of course).

bugreport.c void get_populated_hooks(struct strbuf *hook_info) · line 160

Quoted 5 lines
bugreport.c +5 -0
+ * If 'filter' is nonzero, the contents are filtered on d_type as 'type' - see
+ * 'man readdir'. opendir() doesn't take string length as an arg, so don't
+ * bother passing it in.
+ */
+void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath,

Shouldn't this be static?

Quoted 5 lines
bugreport.c +5 -0
+			  int filter, unsigned char type)
+{
+	struct dirent *dir = NULL;
+	DIR *dh = NULL;
+

bugreport.c void get_populated_hooks(struct strbuf *hook_info) · line 175

Quoted 10 lines
bugreport.c +10 -0
+	dh = opendir(dirpath->buf);
+	while (dh && (dir = readdir(dh))) {
+		if (!filter || type == dir->d_type) {
+			string_list_append(contents, dir->d_name);
+		}
+	}
+}
+
+
+void get_object_counts(struct strbuf *obj_info)

Oops. This function is no longer used.

Quoted 34 lines
bugreport.c +34 -0
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+	struct strbuf std_out = STRBUF_INIT;
+
+	argv_array_push(&cp.args, "count-objects");
+	argv_array_push(&cp.args, "-vH");
+	cp.git_cmd = 1;
+	capture_command(&cp, &std_out, 0);
+
+	strbuf_reset(obj_info);
+	strbuf_addstr(obj_info, "git-count-objects -vH:\n");
+	strbuf_addbuf(obj_info, &std_out);
+}
+
+void get_loose_object_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+	struct string_list subdirs = STRING_LIST_INIT_DUP;
+	struct string_list_item *subdir;
+
+	strbuf_reset(obj_info);
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+
+	list_contents_of_dir(&subdirs, &dirpath, 1, DT_DIR);
+
+	for_each_string_list_item(subdir, &subdirs)
+	{
+		struct strbuf subdir_buf = STRBUF_INIT;
+		struct string_list objects = STRING_LIST_INIT_DUP;
+
+		/*
+		 * Only interested in loose objects - so dirs named with the

bugreport.c void get_populated_hooks(struct strbuf *hook_info) · line 219

Quoted 10 lines
bugreport.c +10 -0
+		 * first byte of the object ID
+		 */
+		if (strlen(subdir->string) != 2 || !strcmp(subdir->string, ".."))
+			continue;
+
+		strbuf_addbuf(&subdir_buf, &dirpath);
+		strbuf_addstr(&subdir_buf, subdir->string);
+		list_contents_of_dir(&objects, &subdir_buf, 0, 0);
+		strbuf_addf(obj_info, "%s: %d objects\n", subdir->string,
+			    objects.nr);

Hmm. Not only does this leak objects, it also throws away the contents that we so painfully constructed.

Wouldn't it make more sense to do something like this instead?

static int is_hex(const char *string, size_t count)

{
	for (; count; string++, count--)
		if (hexval(*string) < 0)
			return 0;
	return 1;
}

static ssize_t count_loose_objects(struct strbuf *objects_path)

{
	ssize_t ret = 0;
	size_t len;
	struct dirent *d;
	DIR *dir, *subdir;

	dir = opendir(objects_path->buf);
	if (!dir)
		return -1;

	strbuf_complete(objects_path, '/');
	len = objects_path->len;
	while ((d = readdir(dir))) {
		if (d->d_type != DT_DIR)
			continue;
		strbuf_setlen(objects_path, len);
		strbuf_addstr(objects_path, d->d_name);
		subdir = opendir(objects_path->buf);
		if (!subdir)
			continue;
		while ((d = readdir(subdir)))
			if (d->dt_type == DT_REG &&
			    is_hex(dir->d_name, the_repository->hash_algo->hexsz))
				ret++;
		closedir(subdir);
	}
	closedir(dir);
	strbuf_reset(objects_path, len);
	return ret;
}

Ciao,
Dscho

Quoted 35 lines
bugreport.c +2 -0
+	}
+}
bugreport.h +6 -0
index 942a5436e3..09ad0c2599 100644
@@ -18,3 +18,9 @@ void get_whitelisted_config(struct strbuf *sys_info);
  * contents of hook_info will be discarded.
  */
 void get_populated_hooks(struct strbuf *hook_info);
+
+/**
+ * Adds the output of `git count-object -vH`. The previous contents of hook_info
+ * will be discarded.
+ */
+void get_loose_object_summary(struct strbuf *obj_info);
builtin/bugreport.c +4 -1
index a0eefba498..b2ab194207 100644
@@ -64,6 +64,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_populated_hooks(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Object Counts");
+	get_loose_object_summary(&buffer);
+	strbuf_write(&buffer, report);
+
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
--

2.24.0.rc0.303.g954a862665-goog

#54 lore raw

Re: [PATCH v3 6/9] bugreport: count loose objects

Quoted 139 lines · Johannes Schindelin

Hi Emily,

Quoted 9 lines · Emily Shaffer

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

Rather than directly invoking 'git-count-objects', which may sometimes fail unexpectedly on Git for Windows, manually count the contents of .git/objects. Additionally, since we may wish to inspect other directories' contents for bugreport in the future, put the directory listing into a helper function.

Thank you, much appreciated!

I guess the next step is to count the number of packs, and the number of submodules ;-)

Quoted 13 lines
---
 bugreport.c         | 72 +++++++++++++++++++++++++++++++++++++++++++++
 bugreport.h         |  6 ++++
 builtin/bugreport.c |  4 +++
 3 files changed, 82 insertions(+)
bugreport.c +0 -0
index 9d7f44ff28..54e1d47103 100644

bugreport.c line 8

Quoted 4 lines
bugreport.c +1 -0
@@ -5,8 +5,11 @@
 #include "exec-cmd.h"
 #include "help.h"
 #include "run-command.h"
+#include "strbuf.h"

Why not append this to the end of the #include list, as is common in Git's commit history?

bugreport.c line 11

Quoted 3 lines
bugreport.c +1 -0
 #include "version.h"
 
+#include "dirent.h"

This header (although with pointy brackets instead of double quotes) is already included in git-compat-util.h

bugreport.c void get_populated_hooks(struct strbuf *hook_info) · line 155

Quoted 11 lines
bugreport.c +4 -0
+
 /**
  * A sorted list of config options which we will add to the bugreport. Managed
  * by 'gather_whitelist(...)'.
@@ -147,3 +150,72 @@ void get_populated_hooks(struct strbuf *hook_info)
 		}
 	}
 }
+
+/**
+ * Fill 'contents' with the contents of the dir at 'dirpath'.

Since you start this comment in JavaDoc style, there should be an almost empty line after this one ("almost" because it still contains the asterisk, of course).

bugreport.c void get_populated_hooks(struct strbuf *hook_info) · line 160

Quoted 5 lines
bugreport.c +5 -0
+ * If 'filter' is nonzero, the contents are filtered on d_type as 'type' - see
+ * 'man readdir'. opendir() doesn't take string length as an arg, so don't
+ * bother passing it in.
+ */
+void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath,

Shouldn't this be static?

Quoted 5 lines
bugreport.c +5 -0
+			  int filter, unsigned char type)
+{
+	struct dirent *dir = NULL;
+	DIR *dh = NULL;
+

bugreport.c void get_populated_hooks(struct strbuf *hook_info) · line 175

Quoted 10 lines
bugreport.c +10 -0
+	dh = opendir(dirpath->buf);
+	while (dh && (dir = readdir(dh))) {
+		if (!filter || type == dir->d_type) {
+			string_list_append(contents, dir->d_name);
+		}
+	}
+}
+
+
+void get_object_counts(struct strbuf *obj_info)

Oops. This function is no longer used.

Quoted 34 lines
bugreport.c +34 -0
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+	struct strbuf std_out = STRBUF_INIT;
+
+	argv_array_push(&cp.args, "count-objects");
+	argv_array_push(&cp.args, "-vH");
+	cp.git_cmd = 1;
+	capture_command(&cp, &std_out, 0);
+
+	strbuf_reset(obj_info);
+	strbuf_addstr(obj_info, "git-count-objects -vH:\n");
+	strbuf_addbuf(obj_info, &std_out);
+}
+
+void get_loose_object_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+	struct string_list subdirs = STRING_LIST_INIT_DUP;
+	struct string_list_item *subdir;
+
+	strbuf_reset(obj_info);
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+
+	list_contents_of_dir(&subdirs, &dirpath, 1, DT_DIR);
+
+	for_each_string_list_item(subdir, &subdirs)
+	{
+		struct strbuf subdir_buf = STRBUF_INIT;
+		struct string_list objects = STRING_LIST_INIT_DUP;
+
+		/*
+		 * Only interested in loose objects - so dirs named with the

bugreport.c void get_populated_hooks(struct strbuf *hook_info) · line 219

Quoted 10 lines
bugreport.c +10 -0
+		 * first byte of the object ID
+		 */
+		if (strlen(subdir->string) != 2 || !strcmp(subdir->string, ".."))
+			continue;
+
+		strbuf_addbuf(&subdir_buf, &dirpath);
+		strbuf_addstr(&subdir_buf, subdir->string);
+		list_contents_of_dir(&objects, &subdir_buf, 0, 0);
+		strbuf_addf(obj_info, "%s: %d objects\n", subdir->string,
+			    objects.nr);

Hmm. Not only does this leak objects, it also throws away the contents that we so painfully constructed.

Wouldn't it make more sense to do something like this instead?

Quoted 7 lines

static int is_hex(const char *string, size_t count)

{
	for (; count; string++, count--)
		if (hexval(*string) < 0)
			return 0;
	return 1;
}

True if the string matches [0-9a-fA-F]*, false otherwise.

Quoted 13 lines

static ssize_t count_loose_objects(struct strbuf *objects_path)

{
	ssize_t ret = 0;
	size_t len;
	struct dirent *d;
	DIR *dir, *subdir;

	dir = opendir(objects_path->buf);
	if (!dir)
		return -1;

	strbuf_complete(objects_path, '/');

starting with .git/objects/...

Quoted 2 lines
	len = objects_path->len;
	while ((d = readdir(dir))) {

For all contents of dir...

Quoted 2 lines
		if (d->d_type != DT_DIR)
			continue;

..which are directories...

Quoted 3 lines
		strbuf_setlen(objects_path, len);
		strbuf_addstr(objects_path, d->d_name);
		subdir = opendir(objects_path->buf);

show all the contents of that subdirectory.

Quoted 3 lines
		if (!subdir)
			continue;
		while ((d = readdir(subdir)))

for each regular file there,

Quoted 1 line
			if (d->dt_type == DT_REG &&

if it's a regular file,

Quoted 1 line
			    is_hex(dir->d_name, the_repository->hash_algo->hexsz))

and the directory is named like an object prefix,

Quoted 1 line
				ret++;

increase the total count of numbers of loose objects.

Quoted 6 lines
		closedir(subdir);
	}
	closedir(dir);
	strbuf_reset(objects_path, len);
	return ret;
}

The foremost difference here is that the loose object count previously was not given in total - instead, it was divided up by object prefix. I can't speak to whether that's actually very important to know about, but the original request from stolee was to have the summary by dirname. That's certainly still possible with a light modification to this code.

(Suggestion from Stolee, some months ago earlier in this thread:)

As mentioned before, I've sometimes found it helpful to know the data shape for the object store. Having a few extra steps such as the following could be nice:

          echo "[Loose Objects]"
          for objdir in $(find "$GIT_DIR/objects/??" -type d)
          do
                  echo "$objdir: $(ls $objdir | wc -l)"
          done
          echo

...

Checking that the directory we're about to inspect is hexval rather than that it's only two characters long is also an interesting point. I'd probably rather do both, though, since I think we both missed futureproofing by a little bit.

Dropping the many string_list is fine by me - call it object-oriented habits dying hard.

I worry somewhat on delving into every directory and only afterwards checking whether the directory is one we care about, but that's an easy modification too to your basic suggestion ("don't use a string list for that, silly").

Thanks. I'll make the changes with the modifications I mentioned and add your Helped-by line on this commit.

- Emily

Quoted 39 lines

Ciao,
Dscho

Quoted 35 lines
bugreport.c +2 -0
+	}
+}
bugreport.h +6 -0
index 942a5436e3..09ad0c2599 100644
@@ -18,3 +18,9 @@ void get_whitelisted_config(struct strbuf *sys_info);
  * contents of hook_info will be discarded.
  */
 void get_populated_hooks(struct strbuf *hook_info);
+
+/**
+ * Adds the output of `git count-object -vH`. The previous contents of hook_info
+ * will be discarded.
+ */
+void get_loose_object_summary(struct strbuf *obj_info);
builtin/bugreport.c +4 -1
index a0eefba498..b2ab194207 100644
@@ -64,6 +64,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_populated_hooks(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Object Counts");
+	get_loose_object_summary(&buffer);
+	strbuf_write(&buffer, report);
+
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
--

2.24.0.rc0.303.g954a862665-goog

#55 lore raw

Re: [PATCH v3 6/9] bugreport: count loose objects

In reply to Emily Shaffer

On 2019.10.24 19:51, Emily Shaffer wrote:
[snip]

Quoted 5 lines
bugreport.h +0 -0
index 942a5436e3..09ad0c2599 100644

bugreport.h void get_whitelisted_config(struct strbuf *sys_info); · line 26

Quoted 9 lines
bugreport.h +6 -0
@@ -18,3 +18,9 @@ void get_whitelisted_config(struct strbuf *sys_info);
  * contents of hook_info will be discarded.
  */
 void get_populated_hooks(struct strbuf *hook_info);
+
+/**
+ * Adds the output of `git count-object -vH`. The previous contents of hook_info
+ * will be discarded.
+ */
+void get_loose_object_summary(struct strbuf *obj_info);

Looks like a copy/paste typo here, shouldn't "hook_info" be "obj_info" in the comment?

#56 lore raw

PATCH v3 7/9 bugreport: add packed object summary

In reply to Emily Shaffer

Alongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.

---
 bugreport.c         | 21 +++++++++++++++++++++
 bugreport.h         |  6 ++++++
 builtin/bugreport.c |  4 ++++
 3 files changed, 31 insertions(+)
bugreport.c +21 -0
index 54e1d47103..79ddb8baaa 100644
@@ -219,3 +219,24 @@ void get_loose_object_summary(struct strbuf *obj_info)
 			    objects.nr);
 	}
 }
+
+void get_packed_object_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+	struct string_list contents = STRING_LIST_INIT_DUP;
+	struct string_list_item *entry;
+
+	strbuf_reset(obj_info);
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "pack/");
+	list_contents_of_dir(&contents, &dirpath, 0, 0);
+
+	// list full contents of $GIT_OBJECT_DIRECTORY/pack/
+	for_each_string_list_item(entry, &contents) {
+		strbuf_addbuf(obj_info, &dirpath);
+		strbuf_addstr(obj_info, entry->string);
+		strbuf_complete_line(obj_info);
+	}
+}
bugreport.h +6 -0
index 09ad0c2599..11ff7df41b 100644
@@ -24,3 +24,9 @@ void get_populated_hooks(struct strbuf *hook_info);
  * will be discarded.
  */
 void get_loose_object_summary(struct strbuf *obj_info);
+
+/**
+ * Adds a list of the contents of '.git/objects/pack'. The previous contents of
+ * hook_info will be discarded.
+ */
+void get_packed_object_summary(struct strbuf *obj_info);
builtin/bugreport.c +4 -0
index b2ab194207..da91a3944e 100644
@@ -68,6 +68,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_loose_object_summary(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+	strbuf_write(&buffer, report);
+
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
#57 lore raw

Re: [PATCH v3 7/9] bugreport: add packed object summary

Hi Emily,

Quoted 3 lines · Emily Shaffer

Alongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.

Yes, I was write! Packs are next ;-)

Quoted 27 lines
---
 bugreport.c         | 21 +++++++++++++++++++++
 bugreport.h         |  6 ++++++
 builtin/bugreport.c |  4 ++++
 3 files changed, 31 insertions(+)
bugreport.c +11 -0
index 54e1d47103..79ddb8baaa 100644
@@ -219,3 +219,24 @@ void get_loose_object_summary(struct strbuf *obj_info)
 			    objects.nr);
 	}
 }
+
+void get_packed_object_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+	struct string_list contents = STRING_LIST_INIT_DUP;
+	struct string_list_item *entry;
+
+	strbuf_reset(obj_info);
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');

bugreport.c void get_loose_object_summary(struct strbuf *obj_info) · line 242

Quoted 10 lines
bugreport.c +10 -0
+	strbuf_addstr(&dirpath, "pack/");
+	list_contents_of_dir(&contents, &dirpath, 0, 0);
+
+	// list full contents of $GIT_OBJECT_DIRECTORY/pack/
+	for_each_string_list_item(entry, &contents) {
+		strbuf_addbuf(obj_info, &dirpath);
+		strbuf_addstr(obj_info, entry->string);
+		strbuf_complete_line(obj_info);
+	}
+}

Okay, but I think that you will want to discern between regular .pack files, .idx files and tmp_* files.

Quoted 19 lines
bugreport.h +6 -0
index 09ad0c2599..11ff7df41b 100644
@@ -24,3 +24,9 @@ void get_populated_hooks(struct strbuf *hook_info);
  * will be discarded.
  */
 void get_loose_object_summary(struct strbuf *obj_info);
+
+/**
+ * Adds a list of the contents of '.git/objects/pack'. The previous contents of
+ * hook_info will be discarded.
+ */
+void get_packed_object_summary(struct strbuf *obj_info);
builtin/bugreport.c +0 -0
index b2ab194207..da91a3944e 100644

builtin/bugreport.c int cmd_bugreport(int argc, const char **argv, const char *prefix) · line 74

Quoted 7 lines
builtin/bugreport.c +4 -0
@@ -68,6 +68,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_loose_object_summary(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+	strbuf_write(&buffer, report);
+

Hmm. At this point, I am unclear whether you want to write into an strbuf, or directly into a FILE *? I would rather have only one, not a mix.

Ciao,
Dscho

Quoted 7 lines
builtin/bugreport.c +0 -1
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
--

2.24.0.rc0.303.g954a862665-goog

#58 lore raw

Re: [PATCH v3 7/9] bugreport: add packed object summary

Quoted 42 lines · Johannes Schindelin

Hi Emily,

Quoted 3 lines · Emily Shaffer

Alongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.

Yes, I was write! Packs are next ;-)

Quoted 32 lines
---
 bugreport.c         | 21 +++++++++++++++++++++
 bugreport.h         |  6 ++++++
 builtin/bugreport.c |  4 ++++
 3 files changed, 31 insertions(+)
bugreport.c +16 -0
index 54e1d47103..79ddb8baaa 100644
@@ -219,3 +219,24 @@ void get_loose_object_summary(struct strbuf *obj_info)
 			    objects.nr);
 	}
 }
+
+void get_packed_object_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+	struct string_list contents = STRING_LIST_INIT_DUP;
+	struct string_list_item *entry;
+
+	strbuf_reset(obj_info);
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "pack/");
+	list_contents_of_dir(&contents, &dirpath, 0, 0);
+
+	// list full contents of $GIT_OBJECT_DIRECTORY/pack/
+	for_each_string_list_item(entry, &contents) {
Quoted 8 lines

bugreport.c void get_loose_object_summary(struct strbuf *obj_info) · line 242

Quoted 5 lines
bugreport.c +5 -0
+		strbuf_addbuf(obj_info, &dirpath);
+		strbuf_addstr(obj_info, entry->string);
+		strbuf_complete_line(obj_info);
+	}
+}

Okay, but I think that you will want to discern between regular .pack files, .idx files and tmp_* files.

Discern in what way? How would you like to see them treated separately? They're all being listed here, not just counted, so it seems to me like I can read the generated bugreport and see which files are index, pack, or temporary here.

Quoted 42 lines
Quoted 20 lines
bugreport.h +6 -0
index 09ad0c2599..11ff7df41b 100644
@@ -24,3 +24,9 @@ void get_populated_hooks(struct strbuf *hook_info);
  * will be discarded.
  */
 void get_loose_object_summary(struct strbuf *obj_info);
+
+/**
+ * Adds a list of the contents of '.git/objects/pack'. The previous contents of
+ * hook_info will be discarded.
+ */
+void get_packed_object_summary(struct strbuf *obj_info);
builtin/bugreport.c +0 -0
index b2ab194207..da91a3944e 100644

builtin/bugreport.c int cmd_bugreport(int argc, const char **argv, const char *prefix) · line 74

Quoted 7 lines
builtin/bugreport.c +4 -0
@@ -68,6 +68,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_loose_object_summary(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+	strbuf_write(&buffer, report);
+

Hmm. At this point, I am unclear whether you want to write into an strbuf, or directly into a FILE *? I would rather have only one, not a mix.

Ciao,
Dscho

Quoted 7 lines
builtin/bugreport.c +0 -1
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
--

2.24.0.rc0.303.g954a862665-goog

#59 lore raw

Re: [PATCH v3 7/9] bugreport: add packed object summary

Hi Emily,

Quoted 52 lines · Emily Shaffer
Quoted 42 lines · Johannes Schindelin

Hi Emily,

Quoted 3 lines · Emily Shaffer

Alongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.

Yes, I was write! Packs are next ;-)

Quoted 32 lines
---
 bugreport.c         | 21 +++++++++++++++++++++
 bugreport.h         |  6 ++++++
 builtin/bugreport.c |  4 ++++
 3 files changed, 31 insertions(+)
bugreport.c +16 -0
index 54e1d47103..79ddb8baaa 100644
@@ -219,3 +219,24 @@ void get_loose_object_summary(struct strbuf *obj_info)
 			    objects.nr);
 	}
 }
+
+void get_packed_object_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+	struct string_list contents = STRING_LIST_INIT_DUP;
+	struct string_list_item *entry;
+
+	strbuf_reset(obj_info);
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "pack/");
+	list_contents_of_dir(&contents, &dirpath, 0, 0);
+
+	// list full contents of $GIT_OBJECT_DIRECTORY/pack/
+	for_each_string_list_item(entry, &contents) {
Quoted 8 lines

bugreport.c void get_loose_object_summary(struct strbuf *obj_info) · line 242

Quoted 5 lines
bugreport.c +5 -0
+		strbuf_addbuf(obj_info, &dirpath);
+		strbuf_addstr(obj_info, entry->string);
+		strbuf_complete_line(obj_info);
+	}
+}

Okay, but I think that you will want to discern between regular .pack files, .idx files and tmp_* files.

Quoted 4 lines

Discern in what way? How would you like to see them treated separately? They're all being listed here, not just counted, so it seems to me like I can read the generated bugreport and see which files are index, pack, or temporary here.

I take your word for it (sorry, it's been half an eternity since I wrapped my head around the diff, I forgotten pretty much all about it ;-))

Ciao,
Dscho

Quoted 44 lines
Quoted 43 lines
Quoted 21 lines
bugreport.h +6 -0
index 09ad0c2599..11ff7df41b 100644
@@ -24,3 +24,9 @@ void get_populated_hooks(struct strbuf *hook_info);
  * will be discarded.
  */
 void get_loose_object_summary(struct strbuf *obj_info);
+
+/**
+ * Adds a list of the contents of '.git/objects/pack'. The previous contents of
+ * hook_info will be discarded.
+ */
+void get_packed_object_summary(struct strbuf *obj_info);
builtin/bugreport.c +0 -0
index b2ab194207..da91a3944e 100644

builtin/bugreport.c int cmd_bugreport(int argc, const char **argv, const char *prefix) · line 74

Quoted 7 lines
builtin/bugreport.c +4 -0
@@ -68,6 +68,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_loose_object_summary(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+	strbuf_write(&buffer, report);
+

Hmm. At this point, I am unclear whether you want to write into an strbuf, or directly into a FILE *? I would rather have only one, not a mix.

Ciao,
Dscho

Quoted 7 lines
builtin/bugreport.c +0 -1
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
--

2.24.0.rc0.303.g954a862665-goog

#60 lore raw

Re: [PATCH v3 7/9] bugreport: add packed object summary

Quoted 56 lines · Johannes Schindelin

Hi Emily,

Quoted 52 lines · Emily Shaffer
Quoted 50 lines · Johannes Schindelin

Hi Emily,

Quoted 3 lines · Emily Shaffer

Alongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.

Yes, I was write! Packs are next ;-)

Quoted 27 lines
---
 bugreport.c         | 21 +++++++++++++++++++++
 bugreport.h         |  6 ++++++
 builtin/bugreport.c |  4 ++++
 3 files changed, 31 insertions(+)
bugreport.c +11 -0
index 54e1d47103..79ddb8baaa 100644
@@ -219,3 +219,24 @@ void get_loose_object_summary(struct strbuf *obj_info)
 			    objects.nr);
 	}
 }
+
+void get_packed_object_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+	struct string_list contents = STRING_LIST_INIT_DUP;
+	struct string_list_item *entry;
+
+	strbuf_reset(obj_info);
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');

bugreport.c void get_loose_object_summary(struct strbuf *obj_info) · line 242

Quoted 10 lines
bugreport.c +10 -0
+	strbuf_addstr(&dirpath, "pack/");
+	list_contents_of_dir(&contents, &dirpath, 0, 0);
+
+	// list full contents of $GIT_OBJECT_DIRECTORY/pack/
+	for_each_string_list_item(entry, &contents) {
+		strbuf_addbuf(obj_info, &dirpath);
+		strbuf_addstr(obj_info, entry->string);
+		strbuf_complete_line(obj_info);
+	}
+}

Okay, but I think that you will want to discern between regular .pack files, .idx files and tmp_* files.

Quoted 7 lines
Quoted 4 lines

Discern in what way? How would you like to see them treated separately? They're all being listed here, not just counted, so it seems to me like I can read the generated bugreport and see which files are index, pack, or temporary here.

I take your word for it (sorry, it's been half an eternity since I wrapped my head around the diff, I forgotten pretty much all about it ;-))

That makes two of us. :)

Quoted 48 lines

Ciao,
Dscho

Quoted 44 lines
Quoted 43 lines
Quoted 21 lines
bugreport.h +6 -0
index 09ad0c2599..11ff7df41b 100644
@@ -24,3 +24,9 @@ void get_populated_hooks(struct strbuf *hook_info);
  * will be discarded.
  */
 void get_loose_object_summary(struct strbuf *obj_info);
+
+/**
+ * Adds a list of the contents of '.git/objects/pack'. The previous contents of
+ * hook_info will be discarded.
+ */
+void get_packed_object_summary(struct strbuf *obj_info);
builtin/bugreport.c +0 -0
index b2ab194207..da91a3944e 100644

builtin/bugreport.c int cmd_bugreport(int argc, const char **argv, const char *prefix) · line 74

Quoted 7 lines
builtin/bugreport.c +4 -0
@@ -68,6 +68,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_loose_object_summary(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+	strbuf_write(&buffer, report);
+

Hmm. At this point, I am unclear whether you want to write into an strbuf, or directly into a FILE *? I would rather have only one, not a mix.

Ciao,
Dscho

Quoted 7 lines
builtin/bugreport.c +0 -1
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
--

2.24.0.rc0.303.g954a862665-goog

#61 lore raw

PATCH v3 8/9 bugreport: list contents of $OBJDIR/info

In reply to Emily Shaffer

Miscellaneous information used about the object store can end up in .git/objects/info; this can help us understand what may be going on with the object store when the user is reporting a bug. Otherwise, it could be difficult to track down what is going wrong with an object which isn't kept locally to .git/objects/ or .git/objects/pack. Having some understanding of where the user's objects may be kept can save us some hops during the bug reporting process.

---
 bugreport.c         | 58 +++++++++++++++++++++++++++++++++++++++++++++
 bugreport.h         |  6 +++++
 builtin/bugreport.c |  7 ++++++
 3 files changed, 71 insertions(+)
bugreport.c +58 -0
index 79ddb8baaa..ce15904fec 100644
@@ -6,6 +6,7 @@
 #include "help.h"
 #include "run-command.h"
 #include "strbuf.h"
+#include "string-list.h"
 #include "version.h"
 
 #include "dirent.h"
@@ -171,6 +172,41 @@ void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath,
 	}
 }
 
+/**
+ * Fills 'contents' with a list of all directories within the provided
+ * directory, recursing into each directory.
+ */
+void list_contents_of_dir_recursively(struct string_list *contents,
+				      struct strbuf *dirpath)
+{
+	struct string_list current_contents = STRING_LIST_INIT_DUP;
+	struct string_list current_subdirs = STRING_LIST_INIT_DUP;
+	struct string_list_item *it;
+	struct strbuf buf = STRBUF_INIT;
+
+	list_contents_of_dir(&current_contents, dirpath, 0, 0);
+	for_each_string_list_item(it, &current_contents) {
+		strbuf_reset(&buf);
+		strbuf_addbuf(&buf, dirpath);
+		strbuf_complete(&buf, '/');
+		strbuf_addstr(&buf, it->string);
+
+		string_list_append(contents, buf.buf);
+	}
+
+	list_contents_of_dir(&current_subdirs, dirpath, 1, DT_DIR);
+	for_each_string_list_item(it, &current_subdirs) {
+		if (strcmp(it->string, ".") == 0
+		    || strcmp(it->string, "..") == 0)
+			continue;
+		strbuf_reset(&buf);
+		strbuf_addbuf(&buf, dirpath);
+		strbuf_complete(&buf, '/');
+		strbuf_addstr(&buf, it->string);
+
+		list_contents_of_dir_recursively(contents, &buf);
+	}
+}
 
 void get_object_counts(struct strbuf *obj_info)
 {
@@ -240,3 +276,25 @@ void get_packed_object_summary(struct strbuf *obj_info)
 		strbuf_complete_line(obj_info);
 	}
 }
+
+void get_object_info_summary(struct strbuf *obj_info)
+{
+	// strbuf += GITDIR/info/:
+	// recursively list contents of $GIT_OBJECT_DIRECTORY/info
+	struct strbuf dirpath = STRBUF_INIT;
+	struct string_list contents = STRING_LIST_INIT_DUP;
+	struct string_list_item *entry;
+
+	strbuf_reset(obj_info);
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(&contents, &dirpath);
+
+	for_each_string_list_item(entry, &contents) {
+		strbuf_addstr(obj_info, entry->string);
+		strbuf_complete_line(obj_info);
+	}
+}
bugreport.h +6 -0
index 11ff7df41b..4f5e2d1b9a 100644
@@ -30,3 +30,9 @@ void get_loose_object_summary(struct strbuf *obj_info);
  * hook_info will be discarded.
  */
 void get_packed_object_summary(struct strbuf *obj_info);
+
+/**
+ * Adds a list of all contents (recursively) of '.git/objects/info'. The
+ * previous contents of hook_info will be discarded.
+ */
+void get_object_info_summary(struct strbuf *obj_info);
builtin/bugreport.c +7 -0
index da91a3944e..8aad33a9b0 100644
@@ -72,6 +72,13 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_packed_object_summary(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Object Info Data");
+	get_object_info_summary(&buffer);
+	strbuf_write(&buffer, report);
+
+	// Close file
+	// open file in editor
+	launch_editor(report_path, NULL, NULL);
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
#62 lore raw

Re: [PATCH v3 8/9] bugreport: list contents of $OBJDIR/info

Hi Emily,

Quoted 7 lines · Emily Shaffer

Miscellaneous information used about the object store can end up in .git/objects/info; this can help us understand what may be going on with the object store when the user is reporting a bug. Otherwise, it could be difficult to track down what is going wrong with an object which isn't kept locally to .git/objects/ or .git/objects/pack. Having some understanding of where the user's objects may be kept can save us some hops during the bug reporting process.

Makes sense.

Quoted 23 lines
---
 bugreport.c         | 58 +++++++++++++++++++++++++++++++++++++++++++++
 bugreport.h         |  6 +++++
 builtin/bugreport.c |  7 ++++++
 3 files changed, 71 insertions(+)
bugreport.c +1 -0
index 79ddb8baaa..ce15904fec 100644
@@ -6,6 +6,7 @@
 #include "help.h"
 #include "run-command.h"
 #include "strbuf.h"
+#include "string-list.h"
 #include "version.h"
 
 #include "dirent.h"
@@ -171,6 +172,41 @@ void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath,
 	}
 }

bugreport.c void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath, · line 183

Quoted 10 lines
bugreport.c +9 -0
 
+/**
+ * Fills 'contents' with a list of all directories within the provided
+ * directory, recursing into each directory.
+ */
+void list_contents_of_dir_recursively(struct string_list *contents,
+				      struct strbuf *dirpath)
+{
+	struct string_list current_contents = STRING_LIST_INIT_DUP;
+	struct string_list current_subdirs = STRING_LIST_INIT_DUP;

Rather than populating those string lists, why not write into the contents directly, _while_ traversing the directories?

But if you really, really want to use string lists (hey, I'm a fan, I introduced them to Git's source code, after all), then please make sure to release them in the end, too.

bugreport.c void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath, · line 192

Quoted 9 lines
bugreport.c +9 -0
+	struct string_list_item *it;
+	struct strbuf buf = STRBUF_INIT;
+
+	list_contents_of_dir(&current_contents, dirpath, 0, 0);
+	for_each_string_list_item(it, &current_contents) {
+		strbuf_reset(&buf);
+		strbuf_addbuf(&buf, dirpath);
+		strbuf_complete(&buf, '/');
+		strbuf_addstr(&buf, it->string);

You don't need a separate strbuf for that: just complete dirpath with a trailing slash, remember the length, then in each iteration reset the length and append the current subdirectory name.

Quoted 35 lines
bugreport.c +28 -0
+
+		string_list_append(contents, buf.buf);
+	}
+
+	list_contents_of_dir(&current_subdirs, dirpath, 1, DT_DIR);
+	for_each_string_list_item(it, &current_subdirs) {
+		if (strcmp(it->string, ".") == 0
+		    || strcmp(it->string, "..") == 0)
+			continue;
+		strbuf_reset(&buf);
+		strbuf_addbuf(&buf, dirpath);
+		strbuf_complete(&buf, '/');
+		strbuf_addstr(&buf, it->string);
+
+		list_contents_of_dir_recursively(contents, &buf);
+	}
+}
 
 void get_object_counts(struct strbuf *obj_info)
 {
@@ -240,3 +276,25 @@ void get_packed_object_summary(struct strbuf *obj_info)
 		strbuf_complete_line(obj_info);
 	}
 }
+
+void get_object_info_summary(struct strbuf *obj_info)
+{
+	// strbuf += GITDIR/info/:
+	// recursively list contents of $GIT_OBJECT_DIRECTORY/info
+	struct strbuf dirpath = STRBUF_INIT;
+	struct string_list contents = STRING_LIST_INIT_DUP;
+	struct string_list_item *entry;
+
+	strbuf_reset(obj_info);
+

bugreport.c void get_packed_object_summary(struct strbuf *obj_info) · line 299

Quoted 10 lines
bugreport.c +10 -0
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(&contents, &dirpath);
+
+	for_each_string_list_item(entry, &contents) {
+		strbuf_addstr(obj_info, entry->string);
+		strbuf_complete_line(obj_info);
+	}

As mentioned earlier, I would rather write things into obj_info directly, without detouring to a string list.

Or, if you decide that you want to write to a FILE * anyway, I'd rather avoid the detour to the strbuf altogether.

Ciao,
Dscho

Quoted 37 lines
bugreport.c +1 -0
+}
bugreport.h +6 -0
index 11ff7df41b..4f5e2d1b9a 100644
@@ -30,3 +30,9 @@ void get_loose_object_summary(struct strbuf *obj_info);
  * hook_info will be discarded.
  */
 void get_packed_object_summary(struct strbuf *obj_info);
+
+/**
+ * Adds a list of all contents (recursively) of '.git/objects/info'. The
+ * previous contents of hook_info will be discarded.
+ */
+void get_object_info_summary(struct strbuf *obj_info);
builtin/bugreport.c +7 -1
index da91a3944e..8aad33a9b0 100644
@@ -72,6 +72,13 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_packed_object_summary(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Object Info Data");
+	get_object_info_summary(&buffer);
+	strbuf_write(&buffer, report);
+
+	// Close file
+	// open file in editor
+	launch_editor(report_path, NULL, NULL);
 	fclose(report);
 
 	launch_editor(report_path.buf, NULL, NULL);
--

2.24.0.rc0.303.g954a862665-goog

#63 lore raw

PATCH v3 9/9 bugreport: print contents of alternates file

In reply to Emily Shaffer

In some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead.

---
 bugreport.c         | 14 ++++++++++++++
 bugreport.h         |  6 ++++++
 builtin/bugreport.c |  4 ++++
 3 files changed, 24 insertions(+)
bugreport.c +14 -0
index ce15904fec..a7bdc72b7f 100644
@@ -298,3 +298,17 @@ void get_object_info_summary(struct strbuf *obj_info)
 		strbuf_complete_line(obj_info);
 	}
 }
+
+void get_alternates_file(struct strbuf *alternates_info)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	strbuf_reset(alternates_info);
+	strbuf_addbuf(alternates_info, &alternates_path);
+	strbuf_complete_line(alternates_info);
+	strbuf_read_file(alternates_info, alternates_path.buf, 0);
+}
bugreport.h +6 -0
index 4f5e2d1b9a..74d1f79960 100644
@@ -36,3 +36,9 @@ void get_packed_object_summary(struct strbuf *obj_info);
  * previous contents of hook_info will be discarded.
  */
 void get_object_info_summary(struct strbuf *obj_info);
+
+/**
+ * Adds the contents of '.git/info/alternates'. The previous contents of
+ * alternates_info will be discarded.
+ */
+void get_alternates_file(struct strbuf *alt_info);
builtin/bugreport.c +4 -0
index 8aad33a9b0..0784bdc42a 100644
@@ -76,6 +76,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_object_info_summary(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Alternates File");
+	get_alternates_file(&buffer);
+	strbuf_write(&buffer, report);
+
 	// Close file
 	// open file in editor
 	launch_editor(report_path, NULL, NULL);
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
#64 lore raw

Re: [PATCH v3 9/9] bugreport: print contents of alternates file

Hi Emily,

Quoted 4 lines · Emily Shaffer

In some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead.

Doesn't this open the possibility of leaking project's (possibly NDA'ed) names?

I could imagine that we might rather want to count the alternates, and maybe separate into those alternates that actually exist and alternates that do not exist (which would produce a warning that the user might have trained themselves to ignore).

Ciao,
Dscho

Quoted 63 lines
---
 bugreport.c         | 14 ++++++++++++++
 bugreport.h         |  6 ++++++
 builtin/bugreport.c |  4 ++++
 3 files changed, 24 insertions(+)
bugreport.c +14 -0
index ce15904fec..a7bdc72b7f 100644
@@ -298,3 +298,17 @@ void get_object_info_summary(struct strbuf *obj_info)
 		strbuf_complete_line(obj_info);
 	}
 }
+
+void get_alternates_file(struct strbuf *alternates_info)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	strbuf_reset(alternates_info);
+	strbuf_addbuf(alternates_info, &alternates_path);
+	strbuf_complete_line(alternates_info);
+	strbuf_read_file(alternates_info, alternates_path.buf, 0);
+}
bugreport.h +6 -0
index 4f5e2d1b9a..74d1f79960 100644
@@ -36,3 +36,9 @@ void get_packed_object_summary(struct strbuf *obj_info);
  * previous contents of hook_info will be discarded.
  */
 void get_object_info_summary(struct strbuf *obj_info);
+
+/**
+ * Adds the contents of '.git/info/alternates'. The previous contents of
+ * alternates_info will be discarded.
+ */
+void get_alternates_file(struct strbuf *alt_info);
builtin/bugreport.c +4 -1
index 8aad33a9b0..0784bdc42a 100644
@@ -76,6 +76,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)
 	get_object_info_summary(&buffer);
 	strbuf_write(&buffer, report);
 
+	add_header(report, "Alternates File");
+	get_alternates_file(&buffer);
+	strbuf_write(&buffer, report);
+
 	// Close file
 	// open file in editor
 	launch_editor(report_path, NULL, NULL);
--

2.24.0.rc0.303.g954a862665-goog

#65 lore raw

Re: [PATCH v3 9/9] bugreport: print contents of alternates file

Quoted 9 lines · Johannes Schindelin

Hi Emily,

Quoted 4 lines · Emily Shaffer

In some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead.

Quoted 6 lines

Doesn't this open the possibility of leaking project's (possibly NDA'ed) names?

I could imagine that we might rather want to count the alternates, and maybe separate into those alternates that actually exist and alternates that do not exist (which would produce a warning that the user might have trained themselves to ignore).

Sounds reasonable. Will do.

- Emily

#66 lore raw

Re: [PATCH v3 0/9] add git-bugreport tool

In reply to Emily Shaffer
Quoted 6 lines · Emily Shaffer

Thanks for the patience with the long wait, all. Here's an attempt at the rewrite in C; I think it does verbatim what the sh version did except that this doesn't print the reflog - Jonathan Nieder was good enough to point out to me that folks probably don't want to share their commit subjects all willy-nilly if they're working on something secretive.

Is the goal to give a tool the end users can type "git bugreport<RET>" and automatically send the result to this mailing list (or some bug tracker)? Or is this only about producing a pre-filled bug report template to be slurped into their MUA and then further manually edited before sending it out?

It probably is controversial if we exposed contents of hooks scripts (I can imagine some people may trigger external process by pinging a URL that includes credential material out of laziness), so the presence test you have is probably a good stopping point. I do not know how much it helps to know which hooks exist in order to diagnose an anomaly without knowing what their contents are, but it is a start.

By the way, I doubt that it is the best use of developer time to write these in C---taking various pieces of information from different places to prepare a text file sounds more suited to scripting languages, especially while we are still learning what kind of information we want to collect and how we want to present the final result (iow, for the first handful of years after deploying this command).

Thanks.

#67 lore raw

Re: [PATCH v3 0/9] add git-bugreport tool

Hi Junio,

Quoted 23 lines · Junio C Hamano
Quoted 6 lines · Emily Shaffer

Thanks for the patience with the long wait, all. Here's an attempt at the rewrite in C; I think it does verbatim what the sh version did except that this doesn't print the reflog - Jonathan Nieder was good enough to point out to me that folks probably don't want to share their commit subjects all willy-nilly if they're working on something secretive.

Is the goal to give a tool the end users can type "git bugreport<RET>" and automatically send the result to this mailing list (or some bug tracker)? Or is this only about producing a pre-filled bug report template to be slurped into their MUA and then further manually edited before sending it out?

It probably is controversial if we exposed contents of hooks scripts (I can imagine some people may trigger external process by pinging a URL that includes credential material out of laziness), so the presence test you have is probably a good stopping point. I do not know how much it helps to know which hooks exist in order to diagnose an anomaly without knowing what their contents are, but it is a start.

Quoted 7 lines

By the way, I doubt that it is the best use of developer time to write these in C---taking various pieces of information from different places to prepare a text file sounds more suited to scripting languages, especially while we are still learning what kind of information we want to collect and how we want to present the final result (iow, for the first handful of years after deploying this command).

If you want to limit Git to Linux, then relying on a specific scripting language like Unix shell scripting sounds okay.

If you target more platforms, then a range of scripting languages sounds a lot more sensible, my top choice would be node.js.

I understand that that is not an option here, and there is a strong preference for Unix shell scripting in Git. This is a bit unfortunate:

A good chunk of the bug reports I see in Git for Windows are _about_ problems with Unix shell scripting. There are fork problems, problems where add_item failed (which is one of the least helpful error messages I saw in my entire life), there are problems with anti-malware causing segmentation faults in the Unix shell script interpreter (the MSYS2 Bash), and there are even (believe it or not) problems with _display drivers_. Yes, that is right, display drivers can interfere with Unix shell script execution on Windows. (This is really rare, but hey, bug reports are rare in Git for Windows to begin with...)

All of these problems _prevent_ Unix shell scripts from being interpreted as expected on Windows. Which would ridicule the idea of reporting any of these bugs using a tool that is implemented in Unix shell.

And what is worse than a bug in an application? A bug in the application's very own bug reporting tool.

I pointed this out to Emily, and that is the reason why this amount of work was put in to _improve_ the tool, precisely by _not_ implementing it as a Unix shell script, but instead in plain C (which is a _lot_ more robust).

Ciao,
Dscho

#68 lore raw

v4 cover add git-bugreport tool

In reply to Emily Shaffer

This topic branch depends on the patch mailed in lore.kernel.org/git/[email protected] in order to display scopes for configs gathered during "bugreport: add config values from safelist".

Quite a few changes since v3. I'll try to summarize the big picture stuff, plus highlight new changes and difference in patches inline in the commit list. I'll also stick an interdiff since v3 at the bottom of this cover letter.

Some overall changes:
- The entire command now exists in one file, bugreport.c, which is compiled as a standalone.
- static-ness of local functions has been cleaned up.
- strbuf lifetime management has been cleaned up.
- Mention of "whitelist" and "blacklist" has been changed to "safelist" and "blocklist" while dealing with Git-approved bugreportable configs. Thanks Dscho for pointing this out; "safelist" makes sense to me as these are configs we have deemed safe for public inspection.
- Inquiry into both Git's $SHELL_PATH and the user's $SHELL is new to v4.
- The 'man git bugreport' page which was lost from the script version of git-bugreport has been recovered, as have the "tests" (which only test whether it runs and doesn't crash).

Emily Shaffer (15):
bugreport: add tool to generate debugging info
Moved from library + builtin to a standalone git-bugreport binary.

help: move list_config_help to builtin/help
Resolves a circular dependency where using help.o required all builtins to exist. This also moves generation of config-list.h into its own script (it was previously roommates with command-list.h).

The following 4 commits were previously a single commit which used run-command.h to call different processes; this has been removed except for the cURL version.

bugreport: gather git version and build info
Additionally capture the "--build-info" output from 'git version' where it was not previously captured.

help: add shell-path to --build-options
Adding $SHELL_PATH to 'git --version --build-options' gets it in the bugreport for free. (This is the shell Git uses, determined at build time.)

bugreport: add uname info
Capture uname from the library instead of invoking a different process.

bugreport: add glibc version
Capture glibc version info from the library instead of invoking the 'ldd' process.

bugreport: add curl version
Capture the cURL version from git-http-fetch, and add a -V flag to that binary for this purpose.

bugreport: include user interactive shell
A new feature; check what $SHELL the user is using (at runtime)

bugreport: generate config safelist based on docs
This commit squashes in suggestions both from Johannes Schindelin and Martin Ågren, hopefully resulting in Asciidoc(tor) not showing the safelist macros, and a more portable safelist generation script. It also puts the safelist, sorted, directly into a header instead of into a text file, saving us a file read at runtime.

bugreport: add config values from safelist
At Dscho's suggestion, now we put these into a map (based on khash and oidset.h's example) for constant-time lookup during the config parse.

bugreport: collect list of populated hooks
Only a minor change - now this works if 'git bugreport' is invoked outside of a Git repo.

bugreport: count loose objects
bugreport: add packed object summary
bugreport: list contents of $OBJDIR/info
For the above 3, the entire 'string_list' has been refactored out; now the obj_info buffer is written to directly during the directory traversal. Also paid more attention to when to free strbuf used for paths.

bugreport: summarize contents of alternates file
Rather than printing the contents of the alternates file, now the number of specified alternate stores are listed, and they are tested to check whether they exist. This way we stop looking at user filepaths (sensitive) and start learning whether they have a broken alternate (new and useful info).

 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   8 +
 Documentation/asciidoctor-extensions.rb |   7 +
 Documentation/config/sendemail.txt      |  68 ++--
 Documentation/git-bugreport.txt         |  43 +++
 Documentation/git-http-fetch.txt        |   5 +
 Makefile                                |  31 +-
 bugreport.c                             | 434 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 generate-bugreport-config-safelist.sh   |  22 ++
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  24 ++
 help.c                                  | 131 ++-----
 help.h                                  |   2 +-
 http-fetch.c                            |  13 +-
 t/t0091-bugreport.sh                    |  41 +++
 16 files changed, 774 insertions(+), 163 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh
.gitignore +2 -0
index 055a84c4a8..30935621d9 100644
@@ -189,7 +189,9 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
Documentation/asciidoc.conf +8 -0
index 8fc4b67081..5d5359fcf9 100644
@@ -6,9 +6,13 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The bugreport macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +32,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[bugreport-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[bugreport-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
Documentation/asciidoctor-extensions.rb +7 -0
index d906a00803..750bdff9af 100644
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, action, attrs)
+        ""
       end
     end
   end
@@ -45,4 +49,7 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  # The bugreport macro does nothing as far as rendering is
+  # concerned -- we just grep for it in the sources.
+  inline_macro Git::Documentation::BugReportProcessor, :bugreport
 end
Documentation/config/sendemail.txt +34 -34

97 lines not on this page · raw message

Documentation/git-bugreport.txt new file +43 -0

43 lines not on this page · raw message

Documentation/git-http-fetch.txt +5 -0
index 666b042679..2894c5e82b 100644
@@ -10,6 +10,7 @@ SYNOPSIS
 --------
 [verse]
 'git http-fetch' [-c] [-t] [-a] [-d] [-v] [-w filename] [--recover] [--stdin] <commit> <url>
+'git http-fetch' [-V]
 
 DESCRIPTION
 -----------
@@ -30,6 +31,10 @@ commit-id::
 -v::
 	Report what is downloaded.
 
+-V::
+	Report information about the version of git-http-fetch, including the
+	versions of its dependencies.
+
 -w <filename>::
         Writes the commit-id into the filename under $GIT_DIR/refs/<filename> on
         the local end after the transfer is complete.
Makefile +25 -13
index 6e361eed64..76dc51e2b1 100644
@@ -634,10 +634,6 @@ SCRIPT_PYTHON += git-p4.py
 SCRIPT_SH_GEN = $(patsubst %.sh,%,$(SCRIPT_SH))
 SCRIPT_PERL_GEN = $(patsubst %.perl,%,$(SCRIPT_PERL))
 SCRIPT_PYTHON_GEN = $(patsubst %.py,%,$(SCRIPT_PYTHON))
-SCRIPT_DEPENDENCIES = git-bugreport-config-whitelist
-
-$(SCRIPT_DEPENDENCIES): Documentation/config/*.txt
-	sh bugreport-generate-config-whitelist.sh
 
 # Individual rules to allow e.g.
 # "make -C ../.. SCRIPT_PERL=contrib/foo/bar.perl build-perl-script"
@@ -662,13 +658,10 @@ clean-perl-script:
 	$(RM) $(SCRIPT_PERL_GEN)
 clean-python-script:
 	$(RM) $(SCRIPT_PYTHON_GEN)
-clean-script-dependencies:
-	$(RM) $(SCRIPT_DEPENDENCIES)
 
 SCRIPTS = $(SCRIPT_SH_GEN) \
 	  $(SCRIPT_PERL_GEN) \
 	  $(SCRIPT_PYTHON_GEN) \
-	  $(SCRIPT_DEPENDENCIES) \
 	  git-instaweb
 
 ETAGS_TARGET = TAGS
@@ -688,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -821,7 +815,9 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -846,7 +842,6 @@ LIB_OBJS += bisect.o
 LIB_OBJS += blame.o
 LIB_OBJS += blob.o
 LIB_OBJS += branch.o
-LIB_OBJS += bugreport.o
 LIB_OBJS += bulk-checkin.o
 LIB_OBJS += bundle.o
 LIB_OBJS += cache-tree.o
@@ -1049,7 +1044,6 @@ BUILTIN_OBJS += builtin/archive.o
 BUILTIN_OBJS += builtin/bisect--helper.o
 BUILTIN_OBJS += builtin/blame.o
 BUILTIN_OBJS += builtin/branch.o
-BUILTIN_OBJS += builtin/bugreport.o
 BUILTIN_OBJS += builtin/bundle.o
 BUILTIN_OBJS += builtin/cat-file.o
 BUILTIN_OBJS += builtin/check-attr.o
@@ -2135,7 +2129,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2155,6 +2149,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2162,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
@@ -2457,6 +2463,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
@@ -2788,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-GEN_HDRS := command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -2811,7 +2821,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -3114,7 +3124,9 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo
+	$(RM) config-list.h command-list.h bugreport-config-safelist.h
+	$(RM) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
bugreport-generate-config-whitelist.sh deleted +0 -4
deleted file mode 100755
index ca6b232024..0000000000
@@ -1,4 +0,0 @@
-#!/bin/sh
-
-grep -RhPo ".*(?=:: \/\/ bugreport-include)" Documentation/config \
-  >git-bugreport-config-whitelist
bugreport.c +312 -192

597 lines not on this page · raw message

bugreport.h deleted +0 -44

44 lines not on this page · raw message

builtin.h +0 -1
index c6373d3289..5cf5df69f7 100644
@@ -135,7 +135,6 @@ int cmd_archive(int argc, const char **argv, const char *prefix);
 int cmd_bisect__helper(int argc, const char **argv, const char *prefix);
 int cmd_blame(int argc, const char **argv, const char *prefix);
 int cmd_branch(int argc, const char **argv, const char *prefix);
-int cmd_bugreport(int argc, const char **argv, const char *prefix);
 int cmd_bundle(int argc, const char **argv, const char *prefix);
 int cmd_cat_file(int argc, const char **argv, const char *prefix);
 int cmd_checkout(int argc, const char **argv, const char *prefix);
builtin/bugreport.c deleted +0 -90

90 lines not on this page · raw message

builtin/help.c +86 -0

98 lines not on this page · raw message

config.c +17 -0
index e7052b3977..baab4a916e 100644
@@ -3312,6 +3312,23 @@ enum config_scope current_config_scope(void)
 		return current_parsing_scope;
 }
 
+const char *config_scope_to_string(enum config_scope scope)
+{
+	switch (scope) {
+	case CONFIG_SCOPE_SYSTEM:
+		return "system";
+	case CONFIG_SCOPE_GLOBAL:
+		return "global";
+	case CONFIG_SCOPE_REPO:
+		return "repo";
+	case CONFIG_SCOPE_CMDLINE:
+		return "cmdline";
+	case CONFIG_SCOPE_UNKNOWN:
+	default:
+		return "unknown";
+	}
+}
+
 int lookup_config(const char **mapping, int nr_mapping, const char *var)
 {
 	int i;
config.h +1 -0
index 91fd4c5e96..c8bf296dcc 100644
@@ -303,6 +303,7 @@ enum config_scope {
 };
 
 enum config_scope current_config_scope(void);
+const char *config_scope_to_string(enum config_scope);
 const char *current_config_origin_type(void);
 const char *current_config_name(void);
 
generate-bugreport-config-safelist.sh new file +22 -0
new file mode 100755
index 0000000000..06b8e0c3c4
@@ -0,0 +1,22 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the bugreport-include macro
+sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |
+sort |
+while read line
+do
+	echo "	\"$line\","
+done
+
+cat <<EOF
+};
+EOF
generate-cmdlist.sh +0 -19
index 71158f7d8b..45fecf8bdf 100755
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
generate-configlist.sh new file +24 -0
new file mode 100755
index 0000000000..eca6a00c30
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
git.c +0 -1
index 2d6a64f019..ce6ab0ece2 100644
@@ -473,7 +473,6 @@ static struct cmd_struct commands[] = {
 	{ "bisect--helper", cmd_bisect__helper, RUN_SETUP },
 	{ "blame", cmd_blame, RUN_SETUP },
 	{ "branch", cmd_branch, RUN_SETUP | DELAY_PAGER_CONFIG },
-	{ "bugreport", cmd_bugreport, RUN_SETUP },
 	{ "bundle", cmd_bundle, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "cat-file", cmd_cat_file, RUN_SETUP },
 	{ "check-attr", cmd_check_attr, RUN_SETUP },
help.c +28 -103

152 lines not on this page · raw message

help.h +1 -1
index 7a455beeb7..54f6b5f793 100644
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
@@ -38,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void list_version_info(struct strbuf *buf, int build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
http-fetch.c +12 -1
index a32ac118d9..31844812a1 100644
@@ -3,9 +3,18 @@
 #include "exec-cmd.h"
 #include "http.h"
 #include "walker.h"
+#include "version.h"
 
 static const char http_fetch_usage[] = "git http-fetch "
-"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
+"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
+
+void NORETURN version_info()
+{
+	printf("git-http-fetch version: %s\n", git_version_string);
+	printf("built from commit: %s\n", git_built_from_commit_string);
+	printf("curl version: %s\n", curl_version());
+	exit(0);
+}
 
 int cmd_main(int argc, const char **argv)
 {
@@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
 		} else if (argv[arg][1] == 'a') {
 		} else if (argv[arg][1] == 'v') {
 			get_verbosely = 1;
+		} else if (argv[arg][1] == 'V') {
+			version_info();
 		} else if (argv[arg][1] == 'w') {
 			write_ref = &argv[arg + 1];
 			arg++;
t/t0091-bugreport.sh new file +41 -0

41 lines not on this page · raw message

Signature · 1 line
2.24.1.735.g03f4e72817-goog
#69 lore raw

PATCH v4 1/15 bugreport: add tool to generate debugging info

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.

Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.

---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 43 ++++++++++++++++++++
 Makefile                        |  5 +++
 bugreport.c                     | 69 +++++++++++++++++++++++++++++++++
 t/t0091-bugreport.sh            | 41 ++++++++++++++++++++
 5 files changed, 159 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh
.gitignore +1 -0
index 89b3b79c1a..055a84c4a8 100644
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
Documentation/git-bugreport.txt new file +43 -0

43 lines not on this page · raw message

Makefile +5 -0
index b7d7374dac..9dff91436e 100644
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2448,6 +2449,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
bugreport.c new file +69 -0

69 lines not on this page · raw message

t/t0091-bugreport.sh new file +41 -0

41 lines not on this page · raw message

Signature · 1 line
2.24.1.735.g03f4e72817-goog
#70 lore raw

PATCH v4 2/15 help: move list_config_help to builtin/help

In reply to Emily Shaffer

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o, builtin/clean.o, and builtin/config.o. This meant that help.o was unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present).

When command-list.h is included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule.

---
 .gitignore             |  1 +
 Makefile               | 16 ++++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 24 ++++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 123 insertions(+), 109 deletions(-)
 create mode 100755 generate-configlist.sh
.gitignore +1 -0
index 055a84c4a8..5dde2cc4c8 100644
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
Makefile +12 -4
index 9dff91436e..c49f55a521 100644
@@ -815,6 +815,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2127,7 +2128,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2147,6 +2148,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2784,7 +2791,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-GEN_HDRS := command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -2807,7 +2814,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -3110,7 +3117,8 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
+	$(RM) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
builtin/help.c +86 -0

98 lines not on this page · raw message

generate-cmdlist.sh +0 -19
index 71158f7d8b..45fecf8bdf 100755
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
generate-configlist.sh new file +24 -0
new file mode 100755
index 0000000000..eca6a00c30
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
help.c +0 -85

91 lines not on this page · raw message

help.h +0 -1
index 7a455beeb7..9071894e8c 100644
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
Signature · 1 line
2.24.1.735.g03f4e72817-goog
#71 lore raw

Re: [PATCH v4 02/15] help: move list_config_help to builtin/help

Quoted 8 lines · Emily Shaffer

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o, builtin/clean.o, and builtin/config.o. This meant that help.o was unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present).

Quoted 8 lines

When command-list.h is included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule.

---

OK. Looks like a clean-up that is very worthy even without the remainder of the topic.

Thanks.

Quoted 360 lines
 .gitignore             |  1 +
 Makefile               | 16 ++++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 24 ++++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 123 insertions(+), 109 deletions(-)
 create mode 100755 generate-configlist.sh
.gitignore +1 -0
index 055a84c4a8..5dde2cc4c8 100644
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
Makefile +12 -4
index 9dff91436e..c49f55a521 100644
@@ -815,6 +815,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2127,7 +2128,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2147,6 +2148,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2784,7 +2791,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-GEN_HDRS := command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -2807,7 +2814,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -3110,7 +3117,8 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
+	$(RM) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
builtin/help.c +86 -0
index e5590d7787..1c5f2b9255 100644
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
generate-cmdlist.sh +0 -19
index 71158f7d8b..45fecf8bdf 100755
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
generate-configlist.sh new file +24 -0
new file mode 100755
index 0000000000..eca6a00c30
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
help.c +0 -85
index cf67624a94..a21487db77 100644
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
help.h +0 -1
index 7a455beeb7..9071894e8c 100644
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
#72 lore raw

Re: [PATCH v4 02/15] help: move list_config_help to builtin/help

Quoted 16 lines · Junio C Hamano
Quoted 14 lines · Emily Shaffer

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o, builtin/clean.o, and builtin/config.o. This meant that help.o was unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present).

When command-list.h is included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule.

Quoted 7 lines
Quoted 2 lines

---

OK. Looks like a clean-up that is very worthy even without the remainder of the topic.

Thanks.

Ok. Are you going to take it on its own? If so, I'll be relying on two branches in pu/ - what's the best way to let you know I need them both if/when I send another version of this chain?

Quoted 361 lines
Quoted 361 lines
 .gitignore             |  1 +
 Makefile               | 16 ++++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 24 ++++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 123 insertions(+), 109 deletions(-)
 create mode 100755 generate-configlist.sh
.gitignore +1 -0
index 055a84c4a8..5dde2cc4c8 100644
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
Makefile +12 -4
index 9dff91436e..c49f55a521 100644
@@ -815,6 +815,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2127,7 +2128,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2147,6 +2148,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2784,7 +2791,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-GEN_HDRS := command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -2807,7 +2814,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -3110,7 +3117,8 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
+	$(RM) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
builtin/help.c +86 -0
index e5590d7787..1c5f2b9255 100644
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
generate-cmdlist.sh +0 -19
index 71158f7d8b..45fecf8bdf 100755
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
generate-configlist.sh new file +24 -0
new file mode 100755
index 0000000000..eca6a00c30
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
help.c +0 -85
index cf67624a94..a21487db77 100644
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
help.h +0 -1
index 7a455beeb7..9071894e8c 100644
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
#73 lore raw

Re: [PATCH v4 02/15] help: move list_config_help to builtin/help

Quoted 6 lines · Emily Shaffer
Quoted 4 lines

OK. Looks like a clean-up that is very worthy even without the remainder of the topic.

Thanks.

Ok. Are you going to take it on its own?

Not really. Unless you abandon the rest of the topic, in which case it may be worth resurrecting this one on its own, that is. But that is not what you plan to do anyway, so I expect to see it as part of an updated series ;-)

#74 lore raw

Re: [PATCH v4 02/15] help: move list_config_help to builtin/help

Quoted 5 lines · Junio C Hamano
Quoted 3 lines · Emily Shaffer
Quoted 3 lines

OK. Looks like a clean-up that is very worthy even without the remainder of the topic.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Thanks.

Ok. Are you going to take it on its own?

Not really. Unless you abandon the rest of the topic, in which case it may be worth resurrecting this one on its own, that is. But that is not what you plan to do anyway, so I expect to see it as part of an updated series ;-)

Ok! That's fine and makes my life easier. Just making sure. :)

- Emily

#75 lore raw

PATCH v4 3/15 bugreport: gather git version and build info

In reply to Emily Shaffer

Knowing which version of Git a user has and how it was built allows us to more precisely pin down the circumstances when a certain issue occurs, so teach bugreport how to tell us the same output as 'git version --build-options'.

It's not ideal to directly call 'git version --build-options' because that output goes to stdout. Instead, wrap the version string in a helper within help.[ch] library, and call that helper from within the bugreport library.

---
 bugreport.c | 23 ++++++++++++++++++++++-
 help.c      | 45 +++++++++++++++++++++++++++------------------
 help.h      |  1 +
 3 files changed, 50 insertions(+), 19 deletions(-)
bugreport.c +22 -1
index 5495b31674..59d8b5a3af 100644
@@ -1,8 +1,20 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	struct strbuf version_info = STRBUF_INIT;
+
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	list_version_info(&version_info, 1);
+	strbuf_addbuf(sys_info, &version_info);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
@@ -32,6 +44,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -60,6 +77,10 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	// add other contents
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
help.c +27 -18

60 lines not on this page · raw message

help.h +1 -0
index 9071894e8c..54f6b5f793 100644
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void list_version_info(struct strbuf *buf, int build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
Signature · 1 line
2.24.1.735.g03f4e72817-goog
#76 lore raw

Re: [PATCH v4 03/15] bugreport: gather git version and build info

Quoted 9 lines · Emily Shaffer

Knowing which version of Git a user has and how it was built allows us to more precisely pin down the circumstances when a certain issue occurs, so teach bugreport how to tell us the same output as 'git version --build-options'.

It's not ideal to directly call 'git version --build-options' because that output goes to stdout. Instead, wrap the version string in a helper within help.[ch] library, and call that helper from within the bugreport library.

Move to strbuf() from stdio makes sense.

bugreport.c int cmd_main(int argc, const char **argv) · line 80

Quoted 1 line
bugreport.c +1 -0
+	// add other contents

Style.

help.h void add_cmdname(struct cmdnames *cmds, const char *name, int len); · line 38

Quoted 7 lines
help.h +0 -0
index 9071894e8c..54f6b5f793 100644
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);

Many new helpers are called get_frotz(), but only one among these is called list_version_info(). I do not think the naming of the get_*() ones, that are private to the bugreport tool, matters that much, but unlike "list_commands()" whose purpose is to list the commands ;-), the new function does not list versions---it just gives information about a single version which is the one that is being run, so perhaps it is a misnomer.

help.h void add_cmdname(struct cmdnames *cmds, const char *name, int len); · line 40

Quoted 2 lines
help.h +1 -0
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void list_version_info(struct strbuf *buf, int build_options);

It is not clear to the readers build_options is a boolean that tells the function to include (or not to include) build options. Perhaps rename it to "int show_build_options" or something?

#77 lore raw

Re: [PATCH v4 03/15] bugreport: gather git version and build info

Quoted 12 lines · Junio C Hamano
Quoted 9 lines · Emily Shaffer

Knowing which version of Git a user has and how it was built allows us to more precisely pin down the circumstances when a certain issue occurs, so teach bugreport how to tell us the same output as 'git version --build-options'.

It's not ideal to directly call 'git version --build-options' because that output goes to stdout. Instead, wrap the version string in a helper within help.[ch] library, and call that helper from within the bugreport library.

Quoted 5 lines

Move to strbuf() from stdio makes sense.

bugreport.c int cmd_main(int argc, const char **argv) · line 80

Quoted 1 line
bugreport.c +1 -0
+	// add other contents

Style.

Sure, dropped this entirely. I think with the helpers the code is self-documenting there.

Quoted 17 lines

help.h void add_cmdname(struct cmdnames *cmds, const char *name, int len); · line 38

Quoted 8 lines
help.h +0 -0
index 9071894e8c..54f6b5f793 100644
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);

Many new helpers are called get_frotz(), but only one among these is called list_version_info(). I do not think the naming of the get_*() ones, that are private to the bugreport tool, matters that much, but unlike "list_commands()" whose purpose is to list the commands ;-), the new function does not list versions---it just gives information about a single version which is the one that is being run, so perhaps it is a misnomer.

Hm, sure. I renamed it to get_version_info(); I had named it list_* because all the other helpers in help.h are named list_*, and it does print more than one piece of info. But I do see your point (all the info is about the same version) so I've renamed it.

Quoted 7 lines

help.h void add_cmdname(struct cmdnames *cmds, const char *name, int len); · line 40

Quoted 3 lines
help.h +1 -0
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void list_version_info(struct strbuf *buf, int build_options);

It is not clear to the readers build_options is a boolean that tells the function to include (or not to include) build options. Perhaps rename it to "int show_build_options" or something?

Agree, done.

#78 lore raw

Re: [PATCH v4 03/15] bugreport: gather git version and build info

In reply to Emily Shaffer

Hi Emily,

Quoted 16 lines · Emily Shaffer
help.c +8 -0
index a21487db77..a43693fca5 100644
@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void list_version_info(struct strbuf *buf, int build_options)
+{
+	strbuf_reset(buf);
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.

help.c const char *help_unknown_cmd(const char *cmd) · line 642

Quoted 10 lines
help.c +10 -0
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addf(buf, "no commit associated with this build\n");

The "StaticAnalysis" job of the Azure Pipeline is not happy with this, claiming that this should be an strbuf_addstr() call instead. For details, see:

https://dev.azure.com/gitgitgadget/8fc4a374-71dc-4558-a5ea-dd1c081ea621/_apis/build/builds/23830/logs/68

Ciao,
Dscho

Quoted 57 lines
help.c +9 -18
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -637,25 +662,9 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	list_version_info(&buf, build_options);
+	printf("%s", buf.buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
help.h +1 -1
index 9071894e8c..54f6b5f793 100644
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void list_version_info(struct strbuf *buf, int build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
--

2.24.1.735.g03f4e72817-goog

#79 lore raw

Re: [PATCH v4 03/15] bugreport: gather git version and build info

Quoted 10 lines · Johannes Schindelin

help.c const char *help_unknown_cmd(const char *cmd) · line 642

Quoted 7 lines
help.c +7 -0
+	if (build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addf(buf, "no commit associated with this build\n");

The "StaticAnalysis" job of the Azure Pipeline is not happy with this, claiming that this should be an strbuf_addstr() call instead.

You mean the "else" clause, right? That feels similar to say

	printf("Hello world\n");

should better be written as

	fputs("Hello world\n", stdout);

which I do not agree with at all. IOW, I view the distinction more like "once it is written one way or the other way, it is not worth spending bits and braincycles to see if it is worth changing it" kind of minor stylistic preference.

#80 lore raw

Re: [PATCH v4 03/15] bugreport: gather git version and build info

Quoted 17 lines · Junio C Hamano
Quoted 10 lines · Johannes Schindelin

help.c const char *help_unknown_cmd(const char *cmd) · line 642

Quoted 7 lines
help.c +7 -0
+	if (build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addf(buf, "no commit associated with this build\n");

The "StaticAnalysis" job of the Azure Pipeline is not happy with this, claiming that this should be an strbuf_addstr() call instead.

You mean the "else" clause, right? That feels similar to say

	printf("Hello world\n");
Quoted 8 lines

should better be written as

	fputs("Hello world\n", stdout);

which I do not agree with at all. IOW, I view the distinction more like "once it is written one way or the other way, it is not worth spending bits and braincycles to see if it is worth changing it" kind of minor stylistic preference.

I think I side with Junio here, although it's true that when strbuf_addstr() exists it doesn't make that much sense to use strbuf_addf(). Since there's some other comments, though, I'll change this too to make your CI shut up. :)

- Emily

#81 lore raw

PATCH v4 4/15 help: add shell-path to --build-options

In reply to Emily Shaffer

It may be useful to know which shell Git was built to try to point to, in the event that shell-based Git commands are failing. $SHELL_PATH is set during the build and used to launch the manpage viewer, as well as by git-compat-util.h, and it's used during tests. 'git version --build-options' is encouraged for use in bug reports, so it makes sense to include this information there.

---
 help.c | 1 +
 1 file changed, 1 insertion(+)
help.c +1 -0
index a43693fca5..6d1bd17971 100644
@@ -642,6 +642,7 @@ void list_version_info(struct strbuf *buf, int build_options)
 			strbuf_addf(buf, "no commit associated with this build\n");
 		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
 		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 }
Signature · 1 line
2.24.1.735.g03f4e72817-goog
#82 lore raw

PATCH v4 5/15 bugreport: add uname info

In reply to Emily Shaffer

The contents of uname() can give us some insight into what sort of system the user is running on, and help us replicate their setup if need be. The domainname field is not guaranteed to be available, so don't collect it.

---
 bugreport.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
bugreport.c +13 -0
index 59d8b5a3af..9c69e3fa34 100644
@@ -8,12 +8,25 @@
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
+	struct utsname uname_info;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	list_version_info(&version_info, 1);
 	strbuf_addbuf(sys_info, &version_info);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.nodename,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
Signature · 1 line
2.24.1.735.g03f4e72817-goog
#83 lore raw

Re: [PATCH v4 05/15] bugreport: add uname info

Quoted 4 lines · Emily Shaffer

The contents of uname() can give us some insight into what sort of system the user is running on, and help us replicate their setup if need be. The domainname field is not guaranteed to be available, so don't collect it.

It was surprising to me that we do use "struct utsname" somewhere in the system ;-) ... so at least we know this is just as portable as the remainder of Git, which is good.

Quoted 35 lines
---
 bugreport.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
bugreport.c +13 -0
index 59d8b5a3af..9c69e3fa34 100644
@@ -8,12 +8,25 @@
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
+	struct utsname uname_info;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	list_version_info(&version_info, 1);
 	strbuf_addbuf(sys_info, &version_info);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.nodename,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
#84 lore raw

Re: [PATCH v4 05/15] bugreport: add uname info

In reply to Emily Shaffer
Quoted 4 lines · Emily Shaffer

The contents of uname() can give us some insight into what sort of system the user is running on, and help us replicate their setup if need be. The domainname field is not guaranteed to be available, so don't collect it.

The manpage on Linux says that it's a GNU extension; on Mac OS it isn't mentioned. So it would be more accurate to say that it's known not to be available on many systems rather than just not being guaranteed.

Besides that I think in some cases it may be considered sensitive information, so another reason to not include it. Perhaps even worthy of being mentioned as the primary reason.

bugreport.c line 26

Quoted 1 line
bugreport.c +1 -0
+			    uname_info.nodename,

I think in some cases this could also be considered as sensitive information, and is unlikely to help in diagnosing problems. So I'd move to exclude this as well.

#85 lore raw

PATCH v4 6/15 bugreport: add glibc version

In reply to Emily Shaffer

To help pinpoint the source of a regression, it is useful to know the version of libc which the user's Git client was built with. Let's include it alongside the other versioning information, which is used to identify how the client was built.

---
 bugreport.c | 5 +++++
 1 file changed, 5 insertions(+)
bugreport.c +5 -0
index 9c69e3fa34..af715dc157 100644
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include <gnu/libc-version.h>
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -27,6 +28,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "glibc version: ");
+	strbuf_addstr(sys_info, gnu_get_libc_version());
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
Signature · 1 line
2.24.1.735.g03f4e72817-goog
#86 lore raw

Re: [PATCH v4 06/15] bugreport: add glibc version

Quoted 9 lines · Emily Shaffer

To help pinpoint the source of a regression, it is useful to know the version of libc which the user's Git client was built with. Let's include it alongside the other versioning information, which is used to identify how the client was built.

---
 bugreport.c | 5 +++++
 1 file changed, 5 insertions(+)

This is the first iffy one in the series. It is unreasonable to assume that we can dictate that Git can be built only on glibc systems, no?

Making this conditional, perhaps make "bugreport.c" depend on "extern void get_compiler_info(struct strbuf *sys_info)" and require the function to be defined in compat/; the glibc variant that uses gnu_get_libc_version() would become just one of them.

Quoted 23 lines
bugreport.c +5 -0
index 9c69e3fa34..af715dc157 100644
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include <gnu/libc-version.h>
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -27,6 +28,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "glibc version: ");
+	strbuf_addstr(sys_info, gnu_get_libc_version());
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
#87 lore raw

Re: [PATCH v4 06/15] bugreport: add glibc version

Quoted 12 lines · Junio C Hamano
Quoted 9 lines · Emily Shaffer

To help pinpoint the source of a regression, it is useful to know the version of libc which the user's Git client was built with. Let's include it alongside the other versioning information, which is used to identify how the client was built.

---
 bugreport.c | 5 +++++
 1 file changed, 5 insertions(+)
Quoted 8 lines

This is the first iffy one in the series. It is unreasonable to assume that we can dictate that Git can be built only on glibc systems, no?

Making this conditional, perhaps make "bugreport.c" depend on "extern void get_compiler_info(struct strbuf *sys_info)" and require the function to be defined in compat/; the glibc variant that uses gnu_get_libc_version() would become just one of them.

Interesting. This sounds like a good approach - thanks, I'll look into it.

- Emily

#88 lore raw

PATCH v4 7/15 bugreport: add curl version

In reply to Emily Shaffer

It's possible for git-http* to be built separately from git; in that case we want to know what version of cURL is used by git-http*, not necessarily which version was present at git-bugreport's build time. So instead, ask git-http-fetch for the version information it knows about.

git-http-fetch was chosen as git-http-backend was described as a server-side implementation, and as an accidental fetch in case of problems was considered less harmful than an accidental push.

Since it could have been built at a different time, also report the version and built-from commit of git-http-fetch alongside the cURL info.

---
 Documentation/git-http-fetch.txt |  5 +++++
 bugreport.c                      | 16 ++++++++++++++++
 http-fetch.c                     | 13 ++++++++++++-
 3 files changed, 33 insertions(+), 1 deletion(-)
Documentation/git-http-fetch.txt +5 -0
index 666b042679..2894c5e82b 100644
@@ -10,6 +10,7 @@ SYNOPSIS
 --------
 [verse]
 'git http-fetch' [-c] [-t] [-a] [-d] [-v] [-w filename] [--recover] [--stdin] <commit> <url>
+'git http-fetch' [-V]
 
 DESCRIPTION
 -----------
@@ -30,6 +31,10 @@ commit-id::
 -v::
 	Report what is downloaded.
 
+-V::
+	Report information about the version of git-http-fetch, including the
+	versions of its dependencies.
+
 -w <filename>::
         Writes the commit-id into the filename under $GIT_DIR/refs/<filename> on
         the local end after the transfer is complete.
bugreport.c +16 -0
index af715dc157..f5598513d9 100644
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include <gnu/libc-version.h>
+#include "run-command.h"
+
+static void get_http_version_info(struct strbuf *http_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "http-fetch");
+	argv_array_push(&cp.args, "-V");
+	if (capture_command(&cp, http_info, 0))
+	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -32,6 +44,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "glibc version: ");
 	strbuf_addstr(sys_info, gnu_get_libc_version());
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
+	get_http_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
http-fetch.c +12 -1
index a32ac118d9..31844812a1 100644
@@ -3,9 +3,18 @@
 #include "exec-cmd.h"
 #include "http.h"
 #include "walker.h"
+#include "version.h"
 
 static const char http_fetch_usage[] = "git http-fetch "
-"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
+"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
+
+void NORETURN version_info()
+{
+	printf("git-http-fetch version: %s\n", git_version_string);
+	printf("built from commit: %s\n", git_built_from_commit_string);
+	printf("curl version: %s\n", curl_version());
+	exit(0);
+}
 
 int cmd_main(int argc, const char **argv)
 {
@@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
 		} else if (argv[arg][1] == 'a') {
 		} else if (argv[arg][1] == 'v') {
 			get_verbosely = 1;
+		} else if (argv[arg][1] == 'V') {
+			version_info();
 		} else if (argv[arg][1] == 'w') {
 			write_ref = &argv[arg + 1];
 			arg++;
Signature · 1 line
2.24.1.735.g03f4e72817-goog
#89 lore raw

Re: [PATCH v4 07/15] bugreport: add curl version

Quoted 6 lines · Emily Shaffer

It's possible for git-http* to be built separately from git; in that case we want to know what version of cURL is used by git-http*, not necessarily which version was present at git-bugreport's build time. So instead, ask git-http-fetch for the version information it knows about.

Quoted 6 lines

git-http-fetch was chosen as git-http-backend was described as a server-side implementation, and as an accidental fetch in case of problems was considered less harmful than an accidental push.

Since it could have been built at a different time, also report the version and built-from commit of git-http-fetch alongside the cURL info.

One possible issue I have is that I was hoping that eventually we can discard "git http-fetch" altogether sometime in the future. Does anybody still use the dumb HTTP transport seriously?

And the first move in that direction would be to allow the system be built without http-fetch, even if git-remote-curl (and its aliases) would still be built to access smart-http transports.

So, I am not sure. This is just the matter of adding an out-of-line hidden option used only for environment inspection, so if it can be done to git-remote-curl, that would probably be much more future proof.

Quoted 9 lines
bugreport.c +1 -0
index af715dc157..f5598513d9 100644
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include <gnu/libc-version.h>
+#include "run-command.h"

bugreport.c line 18

Quoted 10 lines
bugreport.c +10 -0
+
+static void get_http_version_info(struct strbuf *http_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "http-fetch");
+	argv_array_push(&cp.args, "-V");
+	if (capture_command(&cp, http_info, 0))
+	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");

OK. We probably can also take the compile-time "NO_CURL" into account, so that we can tell a misconfigured installation that wanted to have CURL but failed to install a usable http-fetch and an installation that deliberately omitted anything cURL?

Quoted 18 lines
bugreport.c +4 -0
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -32,6 +44,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "glibc version: ");
 	strbuf_addstr(sys_info, gnu_get_libc_version());
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
+	get_http_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
http-fetch.c +0 -0
index a32ac118d9..31844812a1 100644

http-fetch.c line 11

Quoted 10 lines
http-fetch.c +4 -1
@@ -3,9 +3,18 @@
 #include "exec-cmd.h"
 #include "http.h"
 #include "walker.h"
+#include "version.h"
 
 static const char http_fetch_usage[] = "git http-fetch "
-"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
+"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
+
+void NORETURN version_info()

void NORETURN version_info(void)

Quoted 18 lines
http-fetch.c +8 -0
+{
+	printf("git-http-fetch version: %s\n", git_version_string);
+	printf("built from commit: %s\n", git_built_from_commit_string);
+	printf("curl version: %s\n", curl_version());
+	exit(0);
+}
 
 int cmd_main(int argc, const char **argv)
 {
@@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
 		} else if (argv[arg][1] == 'a') {
 		} else if (argv[arg][1] == 'v') {
 			get_verbosely = 1;
+		} else if (argv[arg][1] == 'V') {
+			version_info();
 		} else if (argv[arg][1] == 'w') {
 			write_ref = &argv[arg + 1];
 			arg++;
#90 lore raw

Re: [PATCH v4 07/15] bugreport: add curl version

Quoted 12 lines · Junio C Hamano
Quoted 10 lines · Emily Shaffer

It's possible for git-http* to be built separately from git; in that case we want to know what version of cURL is used by git-http*, not necessarily which version was present at git-bugreport's build time. So instead, ask git-http-fetch for the version information it knows about.

git-http-fetch was chosen as git-http-backend was described as a server-side implementation, and as an accidental fetch in case of problems was considered less harmful than an accidental push.

Quoted 6 lines
Quoted 2 lines

Since it could have been built at a different time, also report the version and built-from commit of git-http-fetch alongside the cURL info.

One possible issue I have is that I was hoping that eventually we can discard "git http-fetch" altogether sometime in the future. Does anybody still use the dumb HTTP transport seriously?

Oh, interesting. I was about to say, "I still use it to fetch, when I don't really care" - but that isn't even true, as I fetch via https (and have so much muscle memory to type https that I don't even notice).`

Quoted 9 lines

And the first move in that direction would be to allow the system be built without http-fetch, even if git-remote-curl (and its aliases) would still be built to access smart-http transports.

So, I am not sure. This is just the matter of adding an out-of-line hidden option used only for environment inspection, so if it can be done to git-remote-curl, that would probably be much more future proof.

Ok. I'll move it.

Quoted 17 lines
Quoted 17 lines
bugreport.c +8 -0
index af715dc157..f5598513d9 100644
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include <gnu/libc-version.h>
+#include "run-command.h"
+
+static void get_http_version_info(struct strbuf *http_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "http-fetch");
Quoted 8 lines

bugreport.c line 18

Quoted 3 lines
bugreport.c +3 -0
+	argv_array_push(&cp.args, "-V");
+	if (capture_command(&cp, http_info, 0))
+	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");

OK. We probably can also take the compile-time "NO_CURL" into account, so that we can tell a misconfigured installation that wanted to have CURL but failed to install a usable http-fetch and an installation that deliberately omitted anything cURL?

Oh, interesting idea! I'll add that.

Quoted 51 lines
Quoted 19 lines
bugreport.c +4 -0
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -32,6 +44,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "glibc version: ");
 	strbuf_addstr(sys_info, gnu_get_libc_version());
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
+	get_http_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
http-fetch.c +0 -0
index a32ac118d9..31844812a1 100644

http-fetch.c line 11

Quoted 10 lines
http-fetch.c +4 -1
@@ -3,9 +3,18 @@
 #include "exec-cmd.h"
 #include "http.h"
 #include "walker.h"
+#include "version.h"
 
 static const char http_fetch_usage[] = "git http-fetch "
-"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
+"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
+
+void NORETURN version_info()

void NORETURN version_info(void)

Quoted 18 lines
http-fetch.c +8 -0
+{
+	printf("git-http-fetch version: %s\n", git_version_string);
+	printf("built from commit: %s\n", git_built_from_commit_string);
+	printf("curl version: %s\n", curl_version());
+	exit(0);
+}
 
 int cmd_main(int argc, const char **argv)
 {
@@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
 		} else if (argv[arg][1] == 'a') {
 		} else if (argv[arg][1] == 'v') {
 			get_verbosely = 1;
+		} else if (argv[arg][1] == 'V') {
+			version_info();
 		} else if (argv[arg][1] == 'w') {
 			write_ref = &argv[arg + 1];
 			arg++;
#91 lore raw

Re: [PATCH v4 07/15] bugreport: add curl version

In reply to Emily Shaffer

Hi Emily,

Quoted 5 lines · Emily Shaffer
http-fetch.c +0 -0
index a32ac118d9..31844812a1 100644

http-fetch.c line 11

Quoted 10 lines
http-fetch.c +4 -1
@@ -3,9 +3,18 @@
 #include "exec-cmd.h"
 #include "http.h"
 #include "walker.h"
+#include "version.h"
 
 static const char http_fetch_usage[] = "git http-fetch "
-"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
+"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
+
+void NORETURN version_info()

Pretty much all the builds in
https://dev.azure.com/gitgitgadget/git/_build/results?buildId=23830&view=logs&jobId=253e5128-1058-5bd4-fdf1-9b556d3207f8&j=fd490c07-0b22-5182-fac9-6d67fe1e939b&t=ce91d5d6-0c55-50f5-8ab9-6695c03ab102
fail because this function definition needs (void) instead of ().

Ciao,
Dscho

Quoted 23 lines
http-fetch.c +8 -1
+{
+	printf("git-http-fetch version: %s\n", git_version_string);
+	printf("built from commit: %s\n", git_built_from_commit_string);
+	printf("curl version: %s\n", curl_version());
+	exit(0);
+}
 
 int cmd_main(int argc, const char **argv)
 {
@@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
 		} else if (argv[arg][1] == 'a') {
 		} else if (argv[arg][1] == 'v') {
 			get_verbosely = 1;
+		} else if (argv[arg][1] == 'V') {
+			version_info();
 		} else if (argv[arg][1] == 'w') {
 			write_ref = &argv[arg + 1];
 			arg++;
--

2.24.1.735.g03f4e72817-goog

#92 lore raw

PATCH v4 8/15 bugreport: include user interactive shell

In reply to Emily Shaffer

It's possible a user may complain about the way that Git interacts with their interactive shell, e.g. autocompletion or shell prompt. In that case, it's useful for us to know which shell they're using interactively.

$SHELL isn't set by every shell, but getenv() returns NULL in the event the variable isn't found, so we'll see a line like "$SHELL:
(null)" to tell us that variable wasn't set.

---
 bugreport.c | 3 +++
 1 file changed, 3 insertions(+)
bugreport.c +3 -0
index f5598513d9..759cc0b0f8 100644
@@ -45,6 +45,9 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, gnu_get_libc_version());
 	strbuf_complete_line(sys_info);
 
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    getenv("SHELL"));
+
 	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
 	get_http_version_info(sys_info);
 	strbuf_complete_line(sys_info);
Signature · 1 line
2.24.1.735.g03f4e72817-goog
#93 lore raw

Re: [PATCH v4 08/15] bugreport: include user interactive shell

Quoted 8 lines · Emily Shaffer

It's possible a user may complain about the way that Git interacts with their interactive shell, e.g. autocompletion or shell prompt. In that case, it's useful for us to know which shell they're using interactively.

$SHELL isn't set by every shell, but getenv() returns NULL in the event the variable isn't found, so we'll see a line like "$SHELL:
(null)" to tell us that variable wasn't set.

Do not depend on vsnprintf() from glibc that accepts NULL and shows "(null)". You can segfault on systems without glibc that way.

Quoted 19 lines
---
 bugreport.c | 3 +++
 1 file changed, 3 insertions(+)
bugreport.c +3 -0
index f5598513d9..759cc0b0f8 100644
@@ -45,6 +45,9 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, gnu_get_libc_version());
 	strbuf_complete_line(sys_info);
 
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    getenv("SHELL"));
+
 	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
 	get_http_version_info(sys_info);
 	strbuf_complete_line(sys_info);
#94 lore raw

PATCH v4 9/15 bugreport: generate config safelist based on docs

In reply to Emily Shaffer

Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "bugreport" macro, which is recognized by AsciiDoc and AsciiDoctor.

Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.

In order to prevent staleness of the safelist, add a mechanism to generate the safelist from annotations in the config documentation, where contributors are already used to documenting their new config keys.

Implement a new no-op "bugreport" macro for use as "bugreport:include[x]" to annotate the config keys that should be included in the automatically generated safelist. Use "exclude" for the others.

With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc seems to want something between the brackets. A bit unfortunate, but not a huge problem -- we'll just provide an "x".

"doc-diff" reports that this macro doesn't render at all. That is, these are both empty after this commit:

  cd Documentation
  ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
  ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD

Diffing the rendered HTML shows that there is some small amount of whitespace and comments added. That shouldn't be a problem.

We could perhaps let the implementation verify that the "action" is one of "include" and "exclude". For the Asciidoctor implementation that should be straightforward, but for AsciiDoc I don't immediately know how to do it. Anyway, if someone stumbles on the keyboard and writes "bugreport:icndule", they'll "only" miss out on the config key being included in the safelist. If this were a blocklist, the consequences of a misspelled target could be a lot more severe.

Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.

---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  8 +++
 Documentation/asciidoctor-extensions.rb |  7 +++
 Documentation/config/sendemail.txt      | 68 ++++++++++++-------------
 Makefile                                | 14 +++--
 generate-bugreport-config-safelist.sh   | 22 ++++++++
 6 files changed, 83 insertions(+), 37 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh
.gitignore +1 -0
index 5dde2cc4c8..30935621d9 100644
@@ -191,6 +191,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
Documentation/asciidoc.conf +8 -0
index 8fc4b67081..5d5359fcf9 100644
@@ -6,9 +6,13 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The bugreport macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +32,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[bugreport-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[bugreport-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
Documentation/asciidoctor-extensions.rb +7 -0
index d906a00803..750bdff9af 100644
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, action, attrs)
+        ""
       end
     end
   end
@@ -45,4 +49,7 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  # The bugreport macro does nothing as far as rendering is
+  # concerned -- we just grep for it in the sources.
+  inline_macro Git::Documentation::BugReportProcessor, :bugreport
 end
Documentation/config/sendemail.txt +34 -34

97 lines not on this page · raw message

Makefile +11 -3
index c49f55a521..76dc51e2b1 100644
@@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
 install-python-script: $(SCRIPT_PYTHON_GEN)
 	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
 
-.PHONY: clean-perl-script clean-sh-script clean-python-script
+.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
 clean-sh-script:
 	$(RM) $(SCRIPT_SH_GEN)
 clean-perl-script:
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
@@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-GEN_HDRS := config-list.h command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo
+	$(RM) config-list.h command-list.h bugreport-config-safelist.h
 	$(RM) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
generate-bugreport-config-safelist.sh new file +22 -0
new file mode 100755
index 0000000000..06b8e0c3c4
@@ -0,0 +1,22 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the bugreport-include macro
+sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |
+sort |
+while read line
+do
+	echo "	\"$line\","
+done
+
+cat <<EOF
+};
+EOF
Signature · 1 line
2.24.1.735.g03f4e72817-goog
#95 lore raw

Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs

Quoted 14 lines · Emily Shaffer
Makefile +1 -1
index c49f55a521..76dc51e2b1 100644
@@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
 install-python-script: $(SCRIPT_PYTHON_GEN)
 	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
 
-.PHONY: clean-perl-script clean-sh-script clean-python-script
+.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
 clean-sh-script:
 	$(RM) $(SCRIPT_SH_GEN)
 clean-perl-script:

Makefile VCSSVN_LIB = vcs-svn/lib.a · line 820

Quoted 4 lines
Makefile +1 -0
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h

OK.

Makefile command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc · line 2169

Quoted 11 lines
Makefile +5 -0
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@

OK.

Makefile command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc · line 2173

Quoted 3 lines
Makefile +0 -0
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\

But bugreport.o needs this *.h file generated before a compiler attempts to produce it out of bugreport.c; that dependency is missing from this update to the Makefile.

Quoted 10 lines
Makefile +1 -1
@@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-GEN_HDRS := config-list.h command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h

Makefile clean: profile-clean coverage-clean cocciclean · line 3128

Quoted 6 lines
Makefile +2 -1
@@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo
+	$(RM) config-list.h command-list.h bugreport-config-safelist.h

It is kind of sad that GEN_HDRS defines the list of build artifact that we should be able to clean, and then we manually list them to be removed here independently. Can we fix it up too?

We probably clean up the build/update procedure around unicode-width.h before we can do so, probably. It may be generatable using contrib/ script, but as far as our normal build is concerned, it is a tracked source and not part of the build byproducts, so we probably would want to remove it from GEN_HDRS. When that happens, we can $(RM) all of the $(GEN_HDRS) in the "clean" target.

#96 lore raw

Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs

Quoted 37 lines · Junio C Hamano
Quoted 14 lines · Emily Shaffer
Makefile +1 -1
index c49f55a521..76dc51e2b1 100644
@@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
 install-python-script: $(SCRIPT_PYTHON_GEN)
 	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
 
-.PHONY: clean-perl-script clean-sh-script clean-python-script
+.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
 clean-sh-script:
 	$(RM) $(SCRIPT_SH_GEN)
 clean-perl-script:

Makefile VCSSVN_LIB = vcs-svn/lib.a · line 820

Quoted 4 lines
Makefile +1 -0
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h

OK.

Makefile command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc · line 2169

Quoted 11 lines
Makefile +5 -0
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@

OK.

Quoted 7 lines

Makefile command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc · line 2173

Quoted 3 lines
Makefile +0 -0
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\

But bugreport.o needs this *.h file generated before a compiler attempts to produce it out of bugreport.c; that dependency is missing from this update to the Makefile.

Hm. Somewhere I misformatted my commits, then. My goal was to add the new header here, but not add it as a dependency to anybody; then, in the next commit, add it as a dependency to git-bugreport and start to use it (commit 1: make the thing; commit 2: use the thing). But now when I look at the next commit, I don't see a Makefile change. So I missed something while I was shuffling around fixups. Thanks for noticing.

Quoted 5 lines
Quoted 5 lines
Makefile +0 -0
@@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
Quoted 16 lines

Makefile clean: profile-clean coverage-clean cocciclean · line 3128

Quoted 12 lines
Makefile +3 -2
-GEN_HDRS := config-list.h command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo
+	$(RM) config-list.h command-list.h bugreport-config-safelist.h

It is kind of sad that GEN_HDRS defines the list of build artifact that we should be able to clean, and then we manually list them to be removed here independently. Can we fix it up too?

Yeah, I can do that. I thought so too when I was updating this.

I *think* what happened is that when I split config-list away (in the earleir commit in this chain) I didn't notice that command-list.h was already in GEN_HDRS, and instead just grepped Makefile for "command-list.h" and added config-list.h next to it. So I'll try to fix it much earlier, in that commit, and then simply add to the appropriate variables in this commit.

Quoted 7 lines

We probably clean up the build/update procedure around unicode-width.h before we can do so, probably. It may be generatable using contrib/ script, but as far as our normal build is concerned, it is a tracked source and not part of the build byproducts, so we probably would want to remove it from GEN_HDRS. When that happens, we can $(RM) all of the $(GEN_HDRS) in the "clean" target.

Noted. Thanks.

- Emily

#97 lore raw

Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs

Quoted 85 lines · Emily Shaffer
Quoted 37 lines · Junio C Hamano
Quoted 14 lines · Emily Shaffer
Makefile +1 -1
index c49f55a521..76dc51e2b1 100644
@@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
 install-python-script: $(SCRIPT_PYTHON_GEN)
 	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
 
-.PHONY: clean-perl-script clean-sh-script clean-python-script
+.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
 clean-sh-script:
 	$(RM) $(SCRIPT_SH_GEN)
 clean-perl-script:

Makefile VCSSVN_LIB = vcs-svn/lib.a · line 820

Quoted 4 lines
Makefile +1 -0
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h

OK.

Makefile command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc · line 2169

Quoted 11 lines
Makefile +5 -0
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@

OK.

Quoted 7 lines

Makefile command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc · line 2173

Quoted 3 lines
Makefile +0 -0
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\

But bugreport.o needs this *.h file generated before a compiler attempts to produce it out of bugreport.c; that dependency is missing from this update to the Makefile.

Hm. Somewhere I misformatted my commits, then. My goal was to add the new header here, but not add it as a dependency to anybody; then, in the next commit, add it as a dependency to git-bugreport and start to use it (commit 1: make the thing; commit 2: use the thing). But now when I look at the next commit, I don't see a Makefile change. So I missed something while I was shuffling around fixups. Thanks for noticing.

Quoted 5 lines
Quoted 5 lines
Makefile +0 -0
@@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
Quoted 16 lines

Makefile clean: profile-clean coverage-clean cocciclean · line 3128

Quoted 12 lines
Makefile +3 -2
-GEN_HDRS := config-list.h command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo
+	$(RM) config-list.h command-list.h bugreport-config-safelist.h

It is kind of sad that GEN_HDRS defines the list of build artifact that we should be able to clean, and then we manually list them to be removed here independently. Can we fix it up too?

Yeah, I can do that. I thought so too when I was updating this.

I *think* what happened is that when I split config-list away (in the earleir commit in this chain) I didn't notice that command-list.h was already in GEN_HDRS, and instead just grepped Makefile for "command-list.h" and added config-list.h next to it. So I'll try to fix it much earlier, in that commit, and then simply add to the appropriate variables in this commit.

Quoted 1 line
Quoted 8 lines
Quoted 6 lines

We probably clean up the build/update procedure around unicode-width.h before we can do so, probably. It may be generatable using contrib/ script, but as far as our normal build is concerned, it is a tracked source and not part of the build byproducts, so we probably would want to remove it from GEN_HDRS. When that happens, we can $(RM) all of the $(GEN_HDRS) in the "clean" target.

Noted. Thanks.

Ah, after I sent this mail I saw your patch. For those playing along at home, https://lore.kernel.org/[email protected]

Will review.

- Emily

#98 lore raw

Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs

In reply to Emily Shaffer

Hi Emily,

Quoted 9 lines · Emily Shaffer
generate-bugreport-config-safelist.sh new file +3 -0
new file mode 100755
index 0000000000..06b8e0c3c4
@@ -0,0 +1,22 @@
+#!/bin/sh
+
+cat <<EOF

generate-bugreport-config-safelist.sh line 13

Quoted 10 lines
generate-bugreport-config-safelist.sh new file +10 -0
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the bugreport-include macro
+sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |

If you use / "\1",/ as replacement, you can totally avoid that ugly while loop (that is unfortunately quite slow in MSYS2/Cygwin). You can still pipe the result into sort just the same.

Ciao,
Dscho

Quoted 13 lines
generate-bugreport-config-safelist.sh new file +9 -1
+sort |
+while read line
+do
+	echo "	\"$line\","
+done
+
+cat <<EOF
+};
+EOF
--

2.24.1.735.g03f4e72817-goog

#99 lore raw

Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs

Quoted 19 lines · Johannes Schindelin

Hi Emily,

Quoted 15 lines · Emily Shaffer
generate-bugreport-config-safelist.sh new file +9 -0
new file mode 100755
index 0000000000..06b8e0c3c4
@@ -0,0 +1,22 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
Quoted 8 lines

generate-bugreport-config-safelist.sh line 13

Quoted 4 lines
generate-bugreport-config-safelist.sh new file +4 -0
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the bugreport-include macro
+sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |

If you use / "\1",/ as replacement, you can totally avoid that ugly while loop (that is unfortunately quite slow in MSYS2/Cygwin). You can still pipe the result into sort just the same.

Oooof. How embarrassing - I will do that.

#100 lore raw

Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs

In reply to Emily Shaffer

Hi Emily,

Documentation/asciidoctor-extensions.rb module Git · line 43

Quoted 12 lines · Emily Shaffer
Documentation/asciidoctor-extensions.rb +4 -0
index d906a00803..750bdff9af 100644
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, action, attrs)
+        ""

The Azure Pipeline fails on this hunk, saying:

/usr/bin/asciidoctor: /home/vsts/work/1/s/Documentation/asciidoctor-extensions.rb:41: class definition in method body (SyntaxError) /home/vsts/work/1/s/Documentation/asciidoctor-extensions.rb:55: syntax error, unexpected end-of-input, expecting keyword_end

See
https://dev.azure.com/gitgitgadget/git/_build/results?buildId=23830&view=logs&jobId=253e5128-1058-5bd4-fdf1-9b556d3207f8&j=253e5128-1058-5bd4-fdf1-9b556d3207f8&t=95c65aec-3627-54dc-fc17-47625f123bb3
for details.

Ciao,
Dscho

Quoted 199 lines
Documentation/asciidoctor-extensions.rb +3 -0
       end
     end
   end
@@ -45,4 +49,7 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  # The bugreport macro does nothing as far as rendering is
+  # concerned -- we just grep for it in the sources.
+  inline_macro Git::Documentation::BugReportProcessor, :bugreport
 end
Documentation/config/sendemail.txt +34 -34
index 0006faf800..92f5082013 100644
@@ -1,63 +1,63 @@
-sendemail.identity::
+sendemail.identity bugreport:exclude[x] ::
 	A configuration identity. When given, causes values in the
 	'sendemail.<identity>' subsection to take precedence over
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption::
+sendemail.smtpEncryption bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
-sendemail.smtpssl (deprecated)::
+sendemail.smtpssl (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for 'sendemail.smtpEncryption = ssl'.
 
-sendemail.smtpsslcertpath::
+sendemail.smtpsslcertpath bugreport:exclude[x] ::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*::
+sendemail.<identity>.* bugreport:exclude[x] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
 	`sendemail.identity`.
 
-sendemail.aliasesFile::
-sendemail.aliasFileType::
-sendemail.annotate::
-sendemail.bcc::
-sendemail.cc::
-sendemail.ccCmd::
-sendemail.chainReplyTo::
-sendemail.confirm::
-sendemail.envelopeSender::
-sendemail.from::
-sendemail.multiEdit::
-sendemail.signedoffbycc::
-sendemail.smtpPass::
-sendemail.suppresscc::
-sendemail.suppressFrom::
-sendemail.to::
-sendemail.tocmd::
-sendemail.smtpDomain::
-sendemail.smtpServer::
-sendemail.smtpServerPort::
-sendemail.smtpServerOption::
-sendemail.smtpUser::
-sendemail.thread::
-sendemail.transferEncoding::
-sendemail.validate::
-sendemail.xmailer::
+sendemail.aliasesFile bugreport:exclude[x] ::
+sendemail.aliasFileType bugreport:exclude[x] ::
+sendemail.annotate bugreport:include[x] ::
+sendemail.bcc bugreport:include[x] ::
+sendemail.cc bugreport:include[x] ::
+sendemail.ccCmd bugreport:include[x] ::
+sendemail.chainReplyTo bugreport:include[x] ::
+sendemail.confirm bugreport:include[x] ::
+sendemail.envelopeSender bugreport:include[x] ::
+sendemail.from bugreport:include[x] ::
+sendemail.multiEdit bugreport:include[x] ::
+sendemail.signedoffbycc bugreport:include[x] ::
+sendemail.smtpPass bugreport:exclude[x] ::
+sendemail.suppresscc bugreport:include[x] ::
+sendemail.suppressFrom bugreport:include[x] ::
+sendemail.to bugreport:include[x] ::
+sendemail.tocmd bugreport:include[x] ::
+sendemail.smtpDomain bugreport:include[x] ::
+sendemail.smtpServer bugreport:include[x] ::
+sendemail.smtpServerPort bugreport:include[x] ::
+sendemail.smtpServerOption bugreport:include[x] ::
+sendemail.smtpUser bugreport:exclude[x] ::
+sendemail.thread bugreport:include[x] ::
+sendemail.transferEncoding bugreport:include[x] ::
+sendemail.validate bugreport:include[x] ::
+sendemail.xmailer bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.
 
-sendemail.signedoffcc (deprecated)::
+sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize bugreport:include[x] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay::
+sendemail.smtpReloginDelay bugreport:include[x] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
Makefile +11 -3
index c49f55a521..76dc51e2b1 100644
@@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
 install-python-script: $(SCRIPT_PYTHON_GEN)
 	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
 
-.PHONY: clean-perl-script clean-sh-script clean-python-script
+.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
 clean-sh-script:
 	$(RM) $(SCRIPT_SH_GEN)
 clean-perl-script:
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
@@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-GEN_HDRS := config-list.h command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo
+	$(RM) config-list.h command-list.h bugreport-config-safelist.h
 	$(RM) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
generate-bugreport-config-safelist.sh new file +22 -1
new file mode 100755
index 0000000000..06b8e0c3c4
@@ -0,0 +1,22 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the bugreport-include macro
+sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |
+sort |
+while read line
+do
+	echo "	\"$line\","
+done
+
+cat <<EOF
+};
+EOF
--

2.24.1.735.g03f4e72817-goog

#101 lore raw

PATCH v4 10/15 bugreport: add config values from safelist

In reply to Emily Shaffer

Teach bugreport to gather the values of config options which are present in 'bugreport-config-safelist.h'.

Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.

Taking the build-time generated array and putting it into a set saves us time - since git_config_bugreport() is called for every option the user has configured, performing option lookup in constant time is a useful optimization.

---
 bugreport.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 71 insertions(+), 1 deletion(-)
bugreport.c +71 -1
index 759cc0b0f8..1fca28f0b9 100644
@@ -6,6 +6,9 @@
 #include "help.h"
 #include <gnu/libc-version.h>
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_http_version_info(struct strbuf *http_info)
 {
@@ -18,6 +21,41 @@ static void get_http_version_info(struct strbuf *http_info)
 	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
 }
 
+KHASH_INIT(cfg_set, const char*, int, 0, kh_str_hash_func, kh_str_hash_equal);
+
+struct cfgset {
+	kh_cfg_set_t set;
+};
+
+struct cfgset safelist;
+
+static void cfgset_init(struct cfgset *set, size_t initial_size)
+{
+	memset(&set->set, 0, sizeof(set->set));
+	if (initial_size)
+		kh_resize_cfg_set(&set->set, initial_size);
+}
+
+static int cfgset_insert(struct cfgset *set, const char *cfg_key)
+{
+	int added;
+	kh_put_cfg_set(&set->set, cfg_key, &added);
+	printf("ESS: added %s\n", cfg_key);
+	return !added;
+}
+
+static int cfgset_contains(struct cfgset *set, const char *cfg_key)
+{
+	khiter_t pos = kh_get_cfg_set(&set->set, cfg_key);
+	return pos != kh_end(&set->set);
+}
+
+static void cfgset_clear(struct cfgset *set)
+{
+	kh_release_cfg_set(&set->set);
+	cfgset_init(set, 0);
+}
+
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
@@ -53,6 +91,36 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void gather_safelist()
+{
+	int index;
+	int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
+	cfgset_init(&safelist, safelist_len);
+	for (index = 0; index < safelist_len; index++)
+		cfgset_insert(&safelist, bugreport_config_safelist[index]);
+
+}
+
+static int git_config_bugreport(const char *var, const char *value, void *cb)
+{
+	struct strbuf *config_info = (struct strbuf *)cb;
+
+	if (cfgset_contains(&safelist, var))
+		strbuf_addf(config_info,
+			    "%s (%s) : %s\n",
+			    var, config_scope_to_string(current_config_scope()),
+			    value);
+
+	return 0;
+}
+
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	gather_safelist();
+	git_config(git_config_bugreport, config_info);
+	cfgset_clear(&safelist);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -114,10 +182,12 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
-	// add other contents
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
Signature · 1 line
2.24.1.735.g03f4e72817-goog
#102 lore raw

Re: [PATCH v4 10/15] bugreport: add config values from safelist

Quoted 8 lines · Emily Shaffer

Teach bugreport to gather the values of config options which are present in 'bugreport-config-safelist.h'.

Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.

Quoted 4 lines

Taking the build-time generated array and putting it into a set saves us time - since git_config_bugreport() is called for every option the user has configured, performing option lookup in constant time is a useful optimization.

Interesting. I actually was expecting the look-up to go the other way around. We know the safe keys, so iterate over them and grab their values, if defined. No need for hashes or anything, but just a simple linear list of safe stuff.

But that is too simple-minded. If we wanted to safelist foo.*.bar, where '*' can be anything, walking on the list of safe variables would not work. We must have a hash table that knows "foo.*.bar" is allowed, and while walking all the configuration keys, when we see foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if it is whitelisted, or something like that.

But then I am not sure if this implementation does something like this for three-level names. If not, I do not see much point in use of the hash there either.

Puzzled.

Quoted 113 lines
---
 bugreport.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 71 insertions(+), 1 deletion(-)
bugreport.c +71 -1
index 759cc0b0f8..1fca28f0b9 100644
@@ -6,6 +6,9 @@
 #include "help.h"
 #include <gnu/libc-version.h>
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_http_version_info(struct strbuf *http_info)
 {
@@ -18,6 +21,41 @@ static void get_http_version_info(struct strbuf *http_info)
 	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
 }
 
+KHASH_INIT(cfg_set, const char*, int, 0, kh_str_hash_func, kh_str_hash_equal);
+
+struct cfgset {
+	kh_cfg_set_t set;
+};
+
+struct cfgset safelist;
+
+static void cfgset_init(struct cfgset *set, size_t initial_size)
+{
+	memset(&set->set, 0, sizeof(set->set));
+	if (initial_size)
+		kh_resize_cfg_set(&set->set, initial_size);
+}
+
+static int cfgset_insert(struct cfgset *set, const char *cfg_key)
+{
+	int added;
+	kh_put_cfg_set(&set->set, cfg_key, &added);
+	printf("ESS: added %s\n", cfg_key);
+	return !added;
+}
+
+static int cfgset_contains(struct cfgset *set, const char *cfg_key)
+{
+	khiter_t pos = kh_get_cfg_set(&set->set, cfg_key);
+	return pos != kh_end(&set->set);
+}
+
+static void cfgset_clear(struct cfgset *set)
+{
+	kh_release_cfg_set(&set->set);
+	cfgset_init(set, 0);
+}
+
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
@@ -53,6 +91,36 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void gather_safelist()
+{
+	int index;
+	int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
+	cfgset_init(&safelist, safelist_len);
+	for (index = 0; index < safelist_len; index++)
+		cfgset_insert(&safelist, bugreport_config_safelist[index]);
+
+}
+
+static int git_config_bugreport(const char *var, const char *value, void *cb)
+{
+	struct strbuf *config_info = (struct strbuf *)cb;
+
+	if (cfgset_contains(&safelist, var))
+		strbuf_addf(config_info,
+			    "%s (%s) : %s\n",
+			    var, config_scope_to_string(current_config_scope()),
+			    value);
+
+	return 0;
+}
+
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	gather_safelist();
+	git_config(git_config_bugreport, config_info);
+	cfgset_clear(&safelist);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -114,10 +182,12 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
-	// add other contents
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
#103 lore raw

Re: [PATCH v4 10/15] bugreport: add config values from safelist

Quoted 15 lines · Junio C Hamano
Quoted 8 lines · Emily Shaffer

Teach bugreport to gather the values of config options which are present in 'bugreport-config-safelist.h'.

Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.

Quoted 4 lines

Taking the build-time generated array and putting it into a set saves us time - since git_config_bugreport() is called for every option the user has configured, performing option lookup in constant time is a useful optimization.

Quoted 4 lines

Interesting. I actually was expecting the look-up to go the other way around. We know the safe keys, so iterate over them and grab their values, if defined. No need for hashes or anything, but just a simple linear list of safe stuff.

Hm. Without looking at the code, I said to myself, "repo_config_get_value() will open all the available config files to find the resolved value, so I don't want to do n*4 file open/closes, I only want to do 4 total."

Now I look at the code and see that the configs are already being read into a hashset before now. So you're right that it doesn't make sense for me to do it this way....

Quoted 7 lines

But that is too simple-minded. If we wanted to safelist foo.*.bar, where '*' can be anything, walking on the list of safe variables would not work. We must have a hash table that knows "foo.*.bar" is allowed, and while walking all the configuration keys, when we see foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if it is whitelisted, or something like that.

...unless we want to use wildcards like you suggest.

But I'm not sure it's a good idea. I envision someone writing another Git add-on, which offers someone to specify "user.password" and automagically do some Bitbucket interaction, or something. (An extreme and misguided example, but I hope the point remains.) Then if I say, "all the user.* configs I can see in 'git help config' are safe, so I will just safelist user.*," I'm not accounting for this other tool's known configs which are a superset of what Git knows and expects.

The point of this safelist-generation exercise is to avoid accidentally printing some sensitive user config, and the point of using a safelist instead of a blocklist is to avoid printing a third-party config we didn't know about (or messing up our pattern matching). So I suggest we avoid pattern matching entirely.

Quoted 6 lines

But then I am not sure if this implementation does something like this for three-level names. If not, I do not see much point in use of the hash there either.

Puzzled.

I'd prefer to solve it by iterating over the compile-time-generated safelist using get_config_value_multi(), which I clock at O(n) for n = safelist length: performing n calls to an O(1) hashset lookup. That saves us the O(n*log(n)) hashset population, and the implementation of Yet Another Set in the source.

Thanks.

- Emily

#104 lore raw

Re: [PATCH v4 10/15] bugreport: add config values from safelist

Quoted 11 lines · Emily Shaffer
Quoted 6 lines

But that is too simple-minded. If we wanted to safelist foo.*.bar, where '*' can be anything, walking on the list of safe variables would not work. We must have a hash table that knows "foo.*.bar" is allowed, and while walking all the configuration keys, when we see foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if it is whitelisted, or something like that.

...unless we want to use wildcards like you suggest.

But I'm not sure it's a good idea. I envision someone writing another Git add-on, which offers someone to specify "user.password" ...

Wildcarding the leaf level of two (or for that matter three) level names like "user.*" in your example is of course a nonsense way to use the safelist. But think about three-level names where the second level is designed to be used for user-supplied token to give things of similar nature a name the user can use to distinguish things. If remote.origin.url is worth reporting, wouldn't remote.upstream.url also be? Shouldn't remote.*.url be the way to say "the URL field for each and every remote is a safe thing to report" in the safelist?

#105 lore raw

Re: [PATCH v4 10/15] bugreport: add config values from safelist

Quoted 14 lines · Junio C Hamano
Quoted 11 lines · Emily Shaffer
Quoted 6 lines

But that is too simple-minded. If we wanted to safelist foo.*.bar, where '*' can be anything, walking on the list of safe variables would not work. We must have a hash table that knows "foo.*.bar" is allowed, and while walking all the configuration keys, when we see foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if it is whitelisted, or something like that.

...unless we want to use wildcards like you suggest.

But I'm not sure it's a good idea. I envision someone writing another Git add-on, which offers someone to specify "user.password" ...

Quoted 9 lines

Wildcarding the leaf level of two (or for that matter three) level names like "user.*" in your example is of course a nonsense way to use the safelist. But think about three-level names where the second level is designed to be used for user-supplied token to give things of similar nature a name the user can use to distinguish things. If remote.origin.url is worth reporting, wouldn't remote.upstream.url also be? Shouldn't remote.*.url be the way to say "the URL field for each and every remote is a safe thing to report" in the safelist?

Bah, somehow this mail slipped through the cracks for me.

When you put it that way, I see how it can be useful. Although I admit to fatigue on this patchset and now I'm hesitant about feature creep. :)

I'm going to send the reroll for v5 as I have it, not having seen this mail, and think more about how I'd approach wildcarding in the middle like this. I want to take a look at the other configs which use a middle level as a user token and try to reason about whether we ought to be collecting them. I'll hope to report back sooner (or later).

- Emily

#106 lore raw

Re: [PATCH v4 10/15] bugreport: add config values from safelist

In reply to Emily Shaffer

Hi Emily,

bugreport.c line 11

Quoted 11 lines · Emily Shaffer
bugreport.c +3 -0
index 759cc0b0f8..1fca28f0b9 100644
@@ -6,6 +6,9 @@
 #include "help.h"
 #include <gnu/libc-version.h>
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"

Seems that this patch makes things fail in the CI build
(https://dev.azure.com/gitgitgadget/git/_build/results?buildId=23830&view=results):

-- snipsnap --
bugreport.c:10:10: fatal error: 'bugreport-config-safelist.h' file not found

#include "bugreport-config-safelist.h"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1 error generated.
Makefile:2395: recipe for target 'bugreport.o' failed
make: *** [bugreport.o] Error 1
make: *** Waiting for unfinished jobs....

#107 lore raw

PATCH v4 11/15 bugreport: collect list of populated hooks

In reply to Emily Shaffer

Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong.

Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot. To reduce the amount of code humans need to read, we turn the list into a string_list and iterate over it (as we are calling the same find_hook operation on each string). However, since bugreport should primarily be called by the user, the performance loss from massaging the string seems acceptable.

---
 bugreport.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 55 insertions(+)
bugreport.c +55 -0
index 1fca28f0b9..f89cb8d754 100644
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "bugreport-config-safelist.h"
 #include "khash.h"
+#include "run-command.h"
 
 static void get_http_version_info(struct strbuf *http_info)
 {
@@ -121,6 +122,57 @@ static void get_safelisted_config(struct strbuf *config_info)
 	cfgset_clear(&safelist);
 }
 
+static void get_populated_hooks(struct strbuf *hook_info)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hook`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+	int nongit_ok;
+
+	setup_git_directory_gently(&nongit_ok);
+
+	if (nongit_ok) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -188,6 +240,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Configured Hooks");
+	get_populated_hooks(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
Signature · 1 line
2.24.1.735.g03f4e72817-goog
#108 lore raw

Re: [PATCH v4 11/15] bugreport: collect list of populated hooks

bugreport.c static void get_safelisted_config(struct strbuf *config_info) · line 130

Quoted 4 lines · Emily Shaffer
bugreport.c +4 -0
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hook`.
+	 */

We probably would want to employ technique similar to what you used for the list of safe configuration we saw in an earlier patch in the series. What's not even documented is not worth reporting ;-)

Quoted 58 lines
bugreport.c +48 -0
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+	int nongit_ok;
+
+	setup_git_directory_gently(&nongit_ok);
+
+	if (nongit_ok) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -188,6 +240,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Configured Hooks");
+	get_populated_hooks(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
#109 lore raw

Re: [PATCH v4 11/15] bugreport: collect list of populated hooks

Quoted 10 lines · Junio C Hamano

bugreport.c static void get_safelisted_config(struct strbuf *config_info) · line 130

Quoted 4 lines · Emily Shaffer
bugreport.c +4 -0
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hook`.
+	 */

We probably would want to employ technique similar to what you used for the list of safe configuration we saw in an earlier patch in the series. What's not even documented is not worth reporting ;-)

I somewhat slightly would rather not, since I hope to include a list of the available hooks in my work with git-hook:
https://lore.kernel.org/[email protected]

It's true that the way this patch is written now is prone to bitrot. It's also true that I could win the lottery tomorrow and never finish my hopes of git-hook implementation. And it's finally true that I don't want to write a fourth build-time-generated header which scrapes docs, especially if I also plan to delete it two months from now.

I wonder whether a better approach might be to drop hook support entirely here, and add it later on when this particular bit of tech debt (no canonical list of hook names) is solved, one way or another.

Another approach may be to just list all contents of $(core.hookdir)/ which doesn't end in ".sample", which is what the bash first attempt of bugreport did. This does mean that those who have something like:

.git/hooks/
pre-commit
pre-commit.d/

pre-commit:
sh pre-commit.d/a
sh pre-commit.d/b

will have their multihook config exposed, although the contents of those hooks won't be.

I'm interested to know everyone else's opinion, because mine is biased by my hope that I can solve all the world's problems with git-hook ;)

- Emily

#110 lore raw

PATCH v4 12/15 bugreport: count loose objects

In reply to Emily Shaffer

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

Rather than directly invoking 'git-count-objects', which may sometimes fail unexpectedly on Git for Windows, manually count the contents of .git/objects. Additionally, since we may wish to inspect other directories' contents for bugreport in the future, put the directory listing into a helper function.

---
 bugreport.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)
bugreport.c +64 -0
index f89cb8d754..3abb83d77f 100644
@@ -173,6 +173,67 @@ static void get_populated_hooks(struct strbuf *hook_info)
 	}
 }
 
+static int is_hex(const char *string, size_t count)
+{
+	for (; count; string++, count--) {
+		if (!isxdigit(*string))
+			return 0;
+	}
+	return 1;
+}
+
+static void get_loose_object_summary(struct strbuf *obj_info) {
+	struct dirent *d = NULL;
+	DIR *dir, *subdir = NULL;
+	size_t dir_len;
+	struct strbuf dirpath = STRBUF_INIT;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+
+	dir = opendir(dirpath.buf);
+	if (!dir) {
+		strbuf_addf(obj_info, "could not open object directory '%s'\n",
+			    dirpath.buf);
+		strbuf_release(&dirpath);
+		return;
+	}
+
+	dir_len = dirpath.len;
+
+	while ((d = readdir(dir))) {
+		int object_count = 0;
+		char subdir_name[3];
+
+		if (d->d_type != DT_DIR)
+			continue;
+
+		if ((strlen(d->d_name) != 2) || (!is_hex(d->d_name, 2)))
+			continue;
+
+		/* copy directory name + \0 */
+		memcpy(subdir_name, d->d_name, 3);
+
+		strbuf_setlen(&dirpath, dir_len);
+		strbuf_addstr(&dirpath, d->d_name);
+
+		subdir = opendir(dirpath.buf);
+		if (!subdir)
+			continue;
+		while ((d = readdir(subdir)))
+			if (d->d_type == DT_REG)
+				object_count++;
+
+		closedir(subdir);
+
+		strbuf_addf(obj_info, "%s: %d\n", subdir_name, object_count);
+	}
+
+
+	closedir(dir);
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -243,6 +304,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Configured Hooks");
 	get_populated_hooks(&buffer);
 
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
Signature · 1 line
2.24.1.735.g03f4e72817-goog
#111 lore raw

Re: [PATCH v4 12/15] bugreport: count loose objects

Quoted 7 lines · Emily Shaffer

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

Rather than directly invoking 'git-count-objects', which may sometimes fail unexpectedly on Git for Windows, manually count the contents of .git/objects.

Is for_each_loose_object() or for_each_loose_file_in_objdir() not sufficient? We really do *not* want a redundant implementation of something we already use elsewhere in the system, especially for the purpose of this program, because you would end up reporting what you computed in a way that may be quite different from what the rest of the system that is actually used by the end users computes.

#112 lore raw

Re: [PATCH v4 12/15] bugreport: count loose objects

Quoted 10 lines · Junio C Hamano
Quoted 7 lines · Emily Shaffer

The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.

Rather than directly invoking 'git-count-objects', which may sometimes fail unexpectedly on Git for Windows, manually count the contents of .git/objects.

Quoted 7 lines

Is for_each_loose_object() or for_each_loose_file_in_objdir() not sufficient? We really do *not* want a redundant implementation of something we already use elsewhere in the system, especially for the purpose of this program, because you would end up reporting what you computed in a way that may be quite different from what the rest of the system that is actually used by the end users computes.

Oh, thanks for the pointer. That looks sufficient - I will investigate the differences between the two and stop filesystem browsing by hand in this case.

- Emily

#113 lore raw

PATCH v4 13/15 bugreport: add packed object summary

In reply to Emily Shaffer

Alongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.

---
 bugreport.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
bugreport.c +31 -0
index 3abb83d77f..992d8f9de7 100644
@@ -234,6 +234,34 @@ static void get_loose_object_summary(struct strbuf *obj_info) {
 	strbuf_release(&dirpath);
 }
 
+static void get_packed_object_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+	struct dirent *d;
+	DIR *dir = NULL;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "pack/");
+
+	dir = opendir(dirpath.buf);
+	if (!dir) {
+		strbuf_addf(obj_info, "could not open packed object directory '%s'\n",
+			    dirpath.buf);
+		strbuf_release(&dirpath);
+		return;
+	}
+
+	while ((d = readdir(dir))) {
+		strbuf_addbuf(obj_info, &dirpath);
+		strbuf_addstr(obj_info, d->d_name);
+		strbuf_complete_line(obj_info);
+	}
+
+	closedir(dir);
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -307,6 +335,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
Signature · 1 line
2.24.1.735.g03f4e72817-goog
#114 lore raw

Re: [PATCH v4 13/15] bugreport: add packed object summary

Quoted 3 lines · Emily Shaffer

Alongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.

Not exactly the same comment, but is in the same spirit, as the previous step applies to this one.

Would it be too much work to libify the bulk of cmd_count_objects() that computes numbers, making the cmd_count_objects() into a thin wrapper that calls the libified "counter/collector" function and prints the resulting numbers received from it?

That way, the get_packed_object_summary() can be another consumer of the same "counter/collector" function, no?

Quoted 54 lines
---
 bugreport.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
bugreport.c +31 -0
index 3abb83d77f..992d8f9de7 100644
@@ -234,6 +234,34 @@ static void get_loose_object_summary(struct strbuf *obj_info) {
 	strbuf_release(&dirpath);
 }
 
+static void get_packed_object_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+	struct dirent *d;
+	DIR *dir = NULL;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "pack/");
+
+	dir = opendir(dirpath.buf);
+	if (!dir) {
+		strbuf_addf(obj_info, "could not open packed object directory '%s'\n",
+			    dirpath.buf);
+		strbuf_release(&dirpath);
+		return;
+	}
+
+	while ((d = readdir(dir))) {
+		strbuf_addbuf(obj_info, &dirpath);
+		strbuf_addstr(obj_info, d->d_name);
+		strbuf_complete_line(obj_info);
+	}
+
+	closedir(dir);
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -307,6 +335,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
#115 lore raw

Re: [PATCH v4 13/15] bugreport: add packed object summary

Quoted 9 lines · Junio C Hamano
Quoted 3 lines · Emily Shaffer

Alongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.

Not exactly the same comment, but is in the same spirit, as the previous step applies to this one.

Quoted 7 lines

Would it be too much work to libify the bulk of cmd_count_objects() that computes numbers, making the cmd_count_objects() into a thin wrapper that calls the libified "counter/collector" function and prints the resulting numbers received from it?

That way, the get_packed_object_summary() can be another consumer of the same "counter/collector" function, no?

That sounds like a good approach. Will do.

Quoted 55 lines
Quoted 55 lines
---
 bugreport.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
bugreport.c +31 -0
index 3abb83d77f..992d8f9de7 100644
@@ -234,6 +234,34 @@ static void get_loose_object_summary(struct strbuf *obj_info) {
 	strbuf_release(&dirpath);
 }
 
+static void get_packed_object_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+	struct dirent *d;
+	DIR *dir = NULL;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "pack/");
+
+	dir = opendir(dirpath.buf);
+	if (!dir) {
+		strbuf_addf(obj_info, "could not open packed object directory '%s'\n",
+			    dirpath.buf);
+		strbuf_release(&dirpath);
+		return;
+	}
+
+	while ((d = readdir(dir))) {
+		strbuf_addbuf(obj_info, &dirpath);
+		strbuf_addstr(obj_info, d->d_name);
+		strbuf_complete_line(obj_info);
+	}
+
+	closedir(dir);
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -307,6 +335,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
#116 lore raw

PATCH v4 14/15 bugreport: list contents of $OBJDIR/info

In reply to Emily Shaffer

Miscellaneous information used about the object store can end up in .git/objects/info; this can help us understand what may be going on with the object store when the user is reporting a bug. Otherwise, it could be difficult to track down what is going wrong with an object which isn't kept locally to .git/objects/ or .git/objects/pack. Having some understanding of where the user's objects may be kept can save us some hops during the bug reporting process.

---
 bugreport.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)
bugreport.c +48 -0
index 992d8f9de7..5d300f1c5f 100644
@@ -262,6 +262,51 @@ static void get_packed_object_summary(struct strbuf *obj_info)
 	strbuf_release(&dirpath);
 }
 
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
+{
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
+
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(obj_info, &dirpath);
+
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -338,6 +383,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Packed Object Summary");
 	get_packed_object_summary(&buffer);
 
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
Signature · 1 line
2.24.1.735.g03f4e72817-goog
#117 lore raw

PATCH v4 15/15 bugreport: summarize contents of alternates file

In reply to Emily Shaffer

In some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead. Since paths to alternates may be sensitive, we'll instead count how many alternates have been specified and note how many of them exist or are broken.

---
 bugreport.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
bugreport.c +39 -0
index 5d300f1c5f..d2a3fb1f67 100644
@@ -307,6 +307,42 @@ static void get_object_info_summary(struct strbuf *obj_info)
 	strbuf_release(&dirpath);
 }
 
+static void get_alternates_summary(struct strbuf *alternates_info)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -386,6 +422,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
Signature · 1 line
2.24.1.735.g03f4e72817-goog
#118 lore raw

v5 cover add git-bugreport tool

In reply to Emily Shaffer

From: Emily Shaffer <[email protected]>

This topic branch depends on the patch mailed in lore.kernel.org/git/[email protected] in order to display scopes for configs gathered during "bugreport: add config values from safelist".

I'll summarize v4-v5. Since v4 has languished for some time, I don't think an interdiff is too helpful, so I won't include one. Bonus, the code is much simplified from some suggestions from Junio on how to inspect objects, so I hope it's easy to review anyways.

Throughout, did some style changes away from C++ bad habits. Otherwise, as listed:

Emily Shaffer (15):
bugreport: add tool to generate debugging info
(no change)
help: move list_config_help to builtin/help

    Changed to agree with f3719846134

bugreport: gather git version and build info
Style only

  help: add shell-path to --build-options
    (no change)

bugreport: add uname info
Removed nodename for privacy reasons

bugreport: add compiler info
Moved glibc tattling into compat/.

I appreciate a close look at this one - I think I understood the right way to go about a compat/ util but it's the first one I've done.

bugreport: add curl version
Moved curl tattling from git-http-fetch into git-remote-curl

bugreport: include user interactive shell
Stop depending on a compiler quirk to save us from segfault

bugreport: generate config safelist based on docs
Changed to agree with f3719846134 Cleaned up script a little

bugreport: add config values from safelist
Made git-bugreport dependent on generated safelist header

bugreport: collect list of populated hooks
No change.
Per https://lore.kernel.org/git/[email protected],
should we even keep this patch?

bugreport: count loose objects
Use helpers from object-store.h instead of manually walking the filesystem.

bugreport: add packed object summary
Use helpers from object-store.h instead of manually walking the filesystem.

bugreport: list contents of $OBJDIR/info
(no change)

bugreport: summarize contents of alternates file
Rephrase commit message to explain why I can't use the helpers in object-store.h.

Thanks.
- Emily

 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   8 +
 Documentation/asciidoctor-extensions.rb |   7 +
 Documentation/config/sendemail.txt      |  68 ++--
 Documentation/git-bugreport.txt         |  43 +++
 Makefile                                |  25 +-
 bugreport.c                             | 427 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 compat/compiler.h                       |  24 ++
 generate-bugreport-config-safelist.sh   |  22 ++
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  24 ++
 help.c                                  | 133 ++------
 help.h                                  |   2 +-
 remote-curl.c                           |   8 +
 t/t0091-bugreport.sh                    |  41 +++
 16 files changed, 780 insertions(+), 160 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100644 compat/compiler.h
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#119 lore raw

PATCH v5 1/15 bugreport: add tool to generate debugging info

From: Emily Shaffer <[email protected]>

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.

Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.

---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 43 ++++++++++++++++++++
 Makefile                        |  5 +++
 bugreport.c                     | 69 +++++++++++++++++++++++++++++++++
 t/t0091-bugreport.sh            | 41 ++++++++++++++++++++
 5 files changed, 159 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh
.gitignore +1 -0
index aebe7c0908..ca301bc890 100644
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
Documentation/git-bugreport.txt new file +43 -0

43 lines not on this page · raw message

Makefile +5 -0
index 09f98b777c..f271619371 100644
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2450,6 +2451,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
bugreport.c new file +69 -0

69 lines not on this page · raw message

t/t0091-bugreport.sh new file +41 -0

41 lines not on this page · raw message

Signature · 1 line
2.25.0.341.g760bfbb309-goog
#120 lore raw
Quoted 6 lines · On Fri, 24 Jan 2020 at 05:56, <[email protected]> wrote:

From: Emily Shaffer <[email protected]>

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

("Later" meaning "later in this series", or "any day now"? ;-) )

Quoted 4 lines

+SYNOPSIS
+--------
+[verse]
+'git bugreport' [-o | --output <path>]

Hmm. Should that be "[(-o | --output) <path>]"?

Documentation/git-bugreport.txt line 18

Quoted 6 lines
Documentation/git-bugreport.txt new file +6 -0
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.

Nice description. Got it.

Documentation/git-bugreport.txt line 32

Quoted 13 lines
Documentation/git-bugreport.txt new file +13 -0
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+The following information is captured automatically:
+
+ - Git version (`git version --build-options`)
+ - Machine information (`uname -a`)
+ - Versions of various dependencies
+ - Git config contents (`git config --show-origin --list`)
+ - The names of all configured git-hooks in `.git/hooks/`

I would have expected these points to appear later, both to make it clear what this does commit does (and not), and to highlight what user-visible (documentation-worthy) changes later commits bring along.

Quoted 4 lines

+OPTIONS

+-------
+-o [<path>]::
+--output [<path>]::

Drop the "[" and "]"? If you give -o, you'd better give a path as well?

Quoted 1 line

+ Place the resulting bug report file in <path> instead of the root of the

<path>

Quoted 2 lines

+"Please review the rest of the bug report below.\n" +"You can delete any lines you don't wish to send.\n");

"send" sounds like we're *just* about to send this report somewhere, but it's "only" going to be written to the disk. Maybe "share", instead?

bugreport.c line 54

Quoted 4 lines
bugreport.c new file +4 -0
+       if (option_output) {
+               strbuf_addstr(&report_path, option_output);
+               strbuf_complete(&report_path, '/');
+       }

I thought I'd use -o to indicate the filename, but it turns out it's the *directory* where the file (of some semi-random, generated name) will end up. Re-reading the docs further up, I can see how this is consistent. I sort of wonder if this should be --output*-directory* for symmetry with git format-patch.

bugreport.c line 60

Quoted 5 lines
bugreport.c new file +5 -0
+       strbuf_addstr(&report_path, "git-bugreport-");
+       strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
+       strbuf_addstr(&report_path, ".txt");
+
+

(Double blank line?)

bugreport.c line 63

Quoted 3 lines
bugreport.c new file +3 -0
+       get_bug_template(&buffer);
+
+       report = fopen_for_writing(report_path.buf);

Report might be NULL here.

If there's already such a file, we overwrite. Should we generate the filename using not just today's date (two bug reports in a day wouldn't be unheard of?) but also something like hh:mm:ss?

bugreport.c line 65

Quoted 2 lines
bugreport.c new file +2 -0
+       strbuf_write(&buffer, report);
+       fclose(report);

Maybe clear the strbuf around here...

bugreport.c line 68

Quoted 2 lines
bugreport.c new file +2 -0
+       launch_editor(report_path.buf, NULL, NULL);
+       return 0;

... and/or UNLEAK it here, together with report_path.

Maybe "return -launch_editor(...)"?

Quoted 1 line

+#!/bin/bash

Use /bin/sh instead?

t/t0091-bugreport.sh line 13

Quoted 7 lines
t/t0091-bugreport.sh new file +7 -0
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+       while read -r line; do
+               if [$(grep $HEADER_PATTERN $line)]; then

I think this is a bash-ism.

t/t0091-bugreport.sh line 15

Quoted 2 lines
t/t0091-bugreport.sh new file +2 -0
+                       read -r nextline
+                       if [-z $nextline]; then

Likewise.

Quoted 4 lines
t/t0091-bugreport.sh new file +4 -0
+                               return 1;
+                       fi
+               fi
+       done

t/t0091-bugreport.sh line 29

Quoted 10 lines
t/t0091-bugreport.sh new file +10 -0
+}
+
+test_expect_success 'creates a report with content in the right places' '
+       git bugreport &&
+       check_all_headers_populated <git-bugreport-* &&
+       rm git-bugreport-*
+'
+
+test_expect_success '--output puts the report in the provided dir' '
+       mkdir foo/ &&

If foo isn't there, do we not create it? Apparently not -- in my testing, we segfault. (We don't check for NULL after opening the file.)

t/t0091-bugreport.sh line 31

Quoted 2 lines
t/t0091-bugreport.sh new file +2 -0
+       git bugreport -o foo/ &&
+       test -f foo/git-bugreport-* &&

test_path_is_file

t/t0091-bugreport.sh line 38

Quoted 7 lines
t/t0091-bugreport.sh new file +7 -0
+       rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+       test_must_fail git bugreport --false 2>output &&
+       grep usage output &&
+       test ! -f git-bugreport-*

test_path_is_missing

Martin

#121 lore raw
Quoted 8 lines · Martin Ågren
Quoted 6 lines · On Fri, 24 Jan 2020 at 05:56, <[email protected]> wrote:

From: Emily Shaffer <[email protected]>

Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.

Quoted 8 lines

("Later" meaning "later in this series", or "any day now"? ;-) )

Quoted 4 lines

+SYNOPSIS
+--------
+[verse]
+'git bugreport' [-o | --output <path>]

Hmm. Should that be "[(-o | --output) <path>]"?

Done.

Quoted 18 lines

Documentation/git-bugreport.txt line 18

Quoted 6 lines
Documentation/git-bugreport.txt new file +6 -0
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.

Nice description. Got it.

Quoted 9 lines
Documentation/git-bugreport.txt new file +9 -0
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+The following information is captured automatically:
+
+ - Git version (`git version --build-options`)
Quoted 8 lines
Quoted 4 lines

+ - Machine information (uname -a)
+ - Versions of various dependencies
+ - Git config contents (git config --show-origin --list)
+ - The names of all configured git-hooks in .git/hooks/

I would have expected these points to appear later, both to make it clear what this does commit does (and not), and to highlight what user-visible (documentation-worthy) changes later commits bring along.

Sure, agreed.

Quoted 7 lines
Quoted 5 lines

+OPTIONS

+-------
+-o [<path>]::
+--output [<path>]::

Drop the "[" and "]"? If you give -o, you'd better give a path as well?

Done.

Quoted 3 lines
Quoted 1 line

+ Place the resulting bug report file in <path> instead of the root of the

<path>

Done.

Quoted 5 lines
Quoted 2 lines

+"Please review the rest of the bug report below.\n" +"You can delete any lines you don't wish to send.\n");

"send" sounds like we're *just* about to send this report somewhere, but it's "only" going to be written to the disk. Maybe "share", instead?

Nice turn of phrase, done.

Quoted 10 lines

bugreport.c line 54

Quoted 4 lines
bugreport.c new file +4 -0
+       if (option_output) {
+               strbuf_addstr(&report_path, option_output);
+               strbuf_complete(&report_path, '/');
+       }

I thought I'd use -o to indicate the filename, but it turns out it's the *directory* where the file (of some semi-random, generated name) will end up. Re-reading the docs further up, I can see how this is consistent. I sort of wonder if this should be --output*-directory* for symmetry with git format-patch.

Sure.

Quoted 7 lines

bugreport.c line 60

Quoted 5 lines
bugreport.c new file +5 -0
+       strbuf_addstr(&report_path, "git-bugreport-");
+       strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
+       strbuf_addstr(&report_path, ".txt");
+
+

(Double blank line?)

Done.

Quoted 5 lines

bugreport.c line 63

Quoted 3 lines
bugreport.c new file +3 -0
+       get_bug_template(&buffer);
+
+       report = fopen_for_writing(report_path.buf);

Report might be NULL here.

Nice.

Quoted 3 lines

If there's already such a file, we overwrite. Should we generate the filename using not just today's date (two bug reports in a day wouldn't be unheard of?) but also something like hh:mm:ss?

Sure. For the sake of brevity I'll probably neglect seconds; I hope someone is spending more than 1 minute filling in the provided form.

I'm a little worried about including : in a filename, so I went for 'git-bugreport-YYYY-MM-DD-HHMM' (24-hour).

As I started to write a test to ensure duplicate filenames were handled well, Jonathan Tan pointed out that it would be easy to add an arg like --suffix to allow specifying a custom strftime string. That would allow users to easily create a file named git-bugreport-fetch-failing.txt or git-bugreport-March-19.txt or whatever they want; it also makes testing easy. So I'll add this for the next rollup.

Quoted 4 lines

bugreport.c line 65

Quoted 3 lines
bugreport.c new file +2 -0
+       strbuf_write(&buffer, report);
+       fclose(report);
Quoted 8 lines

Maybe clear the strbuf around here...

bugreport.c line 68

Quoted 2 lines
bugreport.c new file +2 -0
+       launch_editor(report_path.buf, NULL, NULL);
+       return 0;

... and/or UNLEAK it here, together with report_path.

Maybe "return -launch_editor(...)"?

Hm, sure. I see that builtin/tag.c does mark strbufs this way, so I don't see a problem using UNLEAK and tail-calling launch_editor().

As a final bonus, I also added a line to report to stderr the name of the file that was created. I noticed it's sort of unclear what the command actually did otherwise.

Quoted 3 lines
Quoted 1 line

+#!/bin/bash

Use /bin/sh instead?

Yeah, doing so immediately pointed out the bashisms you mentioned, plus some more. :facepalm:

Quoted 25 lines

t/t0091-bugreport.sh line 13

Quoted 8 lines
t/t0091-bugreport.sh new file +7 -0
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+       while read -r line; do
+               if [$(grep $HEADER_PATTERN $line)]; then

I think this is a bash-ism.

t/t0091-bugreport.sh line 15

Quoted 2 lines
t/t0091-bugreport.sh new file +2 -0
+                       read -r nextline
+                       if [-z $nextline]; then

Likewise.

Quoted 9 lines
t/t0091-bugreport.sh new file +9 -0
+                               return 1;
+                       fi
+               fi
+       done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+       git bugreport &&
+       check_all_headers_populated <git-bugreport-* &&
Quoted 8 lines

t/t0091-bugreport.sh line 29

Quoted 5 lines
t/t0091-bugreport.sh new file +5 -0
+       rm git-bugreport-*
+'
+
+test_expect_success '--output puts the report in the provided dir' '
+       mkdir foo/ &&

If foo isn't there, do we not create it? Apparently not -- in my testing, we segfault. (We don't check for NULL after opening the file.)

Yeah, at the moment I just added a die() if we can't open the provided path. I think other utilties can create the path (e.g. git-format-patch) but where it makes sense to do so, I'd prefer to keep bugreport very simple.

I'll die instead of overwriting, too; you're right that spending quite a while on a bug report and then accidentally writing over it with a blank one would be a very bad user experience.

Quoted 5 lines

t/t0091-bugreport.sh line 31

Quoted 3 lines
t/t0091-bugreport.sh new file +2 -0
+       git bugreport -o foo/ &&
+       test -f foo/git-bugreport-* &&

test_path_is_file

Sure.

Quoted 9 lines

t/t0091-bugreport.sh line 38

Quoted 7 lines
t/t0091-bugreport.sh new file +7 -0
+       rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+       test_must_fail git bugreport --false 2>output &&
+       grep usage output &&
+       test ! -f git-bugreport-*

test_path_is_missing

OK.

Thanks very much, Martin, for the thorough review. This is incredibly helpful.

- Emily

#122 lore raw

PATCH v5 2/15 help: move list_config_help to builtin/help

In reply to [email protected]

From: Emily Shaffer <[email protected]>

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o, builtin/clean.o, and builtin/config.o. This meant that help.o was unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present).

When command-list.h is included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule.

---
 .gitignore             |  1 +
 Makefile               | 13 +++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 24 ++++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 121 insertions(+), 108 deletions(-)
 create mode 100755 generate-configlist.sh
.gitignore +1 -0
index ca301bc890..d89bf9e11e 100644
@@ -190,6 +190,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
Makefile +10 -3
index f271619371..a01a050aa3 100644
@@ -815,6 +815,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2129,7 +2130,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2149,6 +2150,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2786,7 +2793,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2808,7 +2815,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
builtin/help.c +86 -0

98 lines not on this page · raw message

generate-cmdlist.sh +0 -19
index 71158f7d8b..45fecf8bdf 100755
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
generate-configlist.sh new file +24 -0
new file mode 100755
index 0000000000..eca6a00c30
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
help.c +0 -85

91 lines not on this page · raw message

help.h +0 -1
index 7a455beeb7..9071894e8c 100644
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#123 lore raw

Re: [PATCH v5 02/15] help: move list_config_help to builtin/help

Quoted 3 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:

To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present).

Nit: I think this could be patch 01/15. It shuffles things around -- *then*, starting with patch 02/15, we can add exciting stuff.

Martin

#124 lore raw

Re: [PATCH v5 02/15] help: move list_config_help to builtin/help

Quoted 7 lines · Martin Ågren
Quoted 3 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:

To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present).

Nit: I think this could be patch 01/15. It shuffles things around -- *then*, starting with patch 02/15, we can add exciting stuff.

Sounds reasonable, here and 04/15. Will do.

#125 lore raw

PATCH v5 3/15 bugreport: gather git version and build info

In reply to [email protected]

From: Emily Shaffer <[email protected]>

Knowing which version of Git a user has and how it was built allows us to more precisely pin down the circumstances when a certain issue occurs, so teach bugreport how to tell us the same output as 'git version --build-options'.

It's not ideal to directly call 'git version --build-options' because that output goes to stdout. Instead, wrap the version string in a helper within help.[ch] library, and call that helper from within the bugreport library.

---
 bugreport.c | 22 +++++++++++++++++++++-
 help.c      | 47 +++++++++++++++++++++++++++++------------------
 help.h      |  1 +
 3 files changed, 51 insertions(+), 19 deletions(-)
bugreport.c +21 -1
index 5495b31674..b1a5a279ac 100644
@@ -1,8 +1,20 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	struct strbuf version_info = STRBUF_INIT;
+
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(&version_info, 1);
+	strbuf_addbuf(sys_info, &version_info);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
@@ -32,6 +44,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -60,6 +77,9 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
help.c +29 -18

62 lines not on this page · raw message

help.h +1 -0
index 9071894e8c..500521b908 100644
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void get_version_info(struct strbuf *buf, int show_build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#126 lore raw

Re: [PATCH v5 03/15] bugreport: gather git version and build info

bugreport.c line 16

Quoted 9 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:
bugreport.c +9 -0
+static void get_system_info(struct strbuf *sys_info)
+{
+       struct strbuf version_info = STRBUF_INIT;
+
+       /* get git version from native cmd */
+       strbuf_addstr(sys_info, "git version:\n");
+       get_version_info(&version_info, 1);
+       strbuf_addbuf(sys_info, &version_info);
+       strbuf_complete_line(sys_info);

This leaks version_info.

Quoted 1 line

+}

Martin

#127 lore raw

Re: [PATCH v5 03/15] bugreport: gather git version and build info

Quoted 12 lines · Martin Ågren

bugreport.c line 16

Quoted 9 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:
bugreport.c +9 -0
+static void get_system_info(struct strbuf *sys_info)
+{
+       struct strbuf version_info = STRBUF_INIT;
+
+       /* get git version from native cmd */
+       strbuf_addstr(sys_info, "git version:\n");
+       get_version_info(&version_info, 1);
+       strbuf_addbuf(sys_info, &version_info);
+       strbuf_complete_line(sys_info);

This leaks version_info.

So it does.

I think the easiest thing is to drop the 'strbuf_reset()' at the top of help.c:get_version_info(), and just pass in sys_info from bugreport.c.

- Emily

#128 lore raw

PATCH v5 4/15 help: add shell-path to --build-options

In reply to [email protected]

From: Emily Shaffer <[email protected]>

It may be useful to know which shell Git was built to try to point to, in the event that shell-based Git commands are failing. $SHELL_PATH is set during the build and used to launch the manpage viewer, as well as by git-compat-util.h, and it's used during tests. 'git version --build-options' is encouraged for use in bug reports, so it makes sense to include this information there.

---
 help.c | 1 +
 1 file changed, 1 insertion(+)
help.c +1 -0
index 156a6cdea3..d3f941824f 100644
@@ -642,6 +642,7 @@ void get_version_info(struct strbuf *buf, int show_build_options)
 			strbuf_addstr(buf, "no commit associated with this build\n");
 		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
 		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 }
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#129 lore raw

Re: [PATCH v5 04/15] help: add shell-path to --build-options

Quoted 6 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:

It may be useful to know which shell Git was built to try to point to, in the event that shell-based Git commands are failing. $SHELL_PATH is set during the build and used to launch the manpage viewer, as well as by git-compat-util.h, and it's used during tests. 'git version --build-options' is encouraged for use in bug reports, so it makes sense to include this information there.

Hmm, similar nit as for patch 02/15: This could go in as, e.g., 02/15 and your 01/15 would be 03/15: This patch would be useful even without git bugreport.

Martin

#130 lore raw

PATCH v5 5/15 bugreport: add uname info

In reply to [email protected]

From: Emily Shaffer <[email protected]>

The contents of uname() can give us some insight into what sort of system the user is running on, and help us replicate their setup if need be. The domainname field is not guaranteed to be available, so don't collect it.

---
 bugreport.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
bugreport.c +12 -0
index b1a5a279ac..720c91e1bd 100644
@@ -8,12 +8,24 @@
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
+	struct utsname uname_info;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	get_version_info(&version_info, 1);
 	strbuf_addbuf(sys_info, &version_info);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#131 lore raw

PATCH v5 6/15 bugreport: add compiler info

In reply to [email protected]

From: Emily Shaffer <[email protected]>

To help pinpoint the source of a regression, it is useful to know some info about the compiler which the user's Git client was built with. By adding a generic get_compiler_info() in 'compat/' we can choose which relevant information to share per compiler; to get started, let's demonstrate the version of glibc if the user built with 'gcc'.

---
 bugreport.c       |  5 +++++
 compat/compiler.h | 24 ++++++++++++++++++++++++
 2 files changed, 29 insertions(+)
 create mode 100644 compat/compiler.h
bugreport.c +5 -0
index 720c91e1bd..818ccb385c 100644
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include "compat/compiler.h"
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -26,6 +27,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
compat/compiler.h new file +24 -0
new file mode 100644
index 0000000000..bda5098e1b
@@ -0,0 +1,24 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
+}
+
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addstr(info, "get_compiler_info() not implemented");
+}
+
+#endif
+
+#endif /* COMPILER_H */
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#132 lore raw

Re: [PATCH v5 06/15] bugreport: add compiler info

compat/compiler.h line 19

Quoted 5 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:
compat/compiler.h new file +5 -0
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+       strbuf_addstr(info, "get_compiler_info() not implemented");

Maybe "no compiler info available" (or s/ available//, or s/available/reported/), or something else more human-readable?

compat/compiler.h line 24

Quoted 5 lines
compat/compiler.h new file +5 -0
+}
+
+#endif
+
+#endif /* COMPILER_H */

Martin

#133 lore raw

Re: [PATCH v5 06/15] bugreport: add compiler info

Quoted 9 lines · Martin Ågren

compat/compiler.h line 19

Quoted 5 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:
compat/compiler.h new file +5 -0
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+       strbuf_addstr(info, "get_compiler_info() not implemented");

Maybe "no compiler info available" (or s/ available//, or s/available/reported/), or something else more human-readable?

Hm. I envisioned the target audience for this as "Git developers" (and that's why I didn't mark the unimplemented string for translation); Git developers know exactly what "get_compiler_info() not implemented" means. But I suppose it's just as easy to grep for one string as the other.

I am hesitant to say "no info available" or "no info reported" when I'm certain there is some environment variable set at build time which contains the info we'd want to show here; it really is just a question of some Git developer having put in the time to implement this function.

So, I think I will leave it. Thanks for your suggestions.

- Emily

#134 lore raw

Re: [PATCH v5 06/15] bugreport: add compiler info

Quoted 18 lines · Emily Shaffer
Quoted 9 lines · Martin Ågren

compat/compiler.h line 19

Quoted 5 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:
compat/compiler.h new file +5 -0
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+       strbuf_addstr(info, "get_compiler_info() not implemented");

Maybe "no compiler info available" (or s/ available//, or s/available/reported/), or something else more human-readable?

Hm. I envisioned the target audience for this as "Git developers" (and that's why I didn't mark the unimplemented string for translation); Git developers know exactly what "get_compiler_info() not implemented" means. But I suppose it's just as easy to grep for one string as the other.

Quoted 6 lines

I am hesitant to say "no info available" or "no info reported" when I'm certain there is some environment variable set at build time which contains the info we'd want to show here; it really is just a question of some Git developer having put in the time to implement this function.

So, I think I will leave it. Thanks for your suggestions.

Ok, that makes sense.

Martin

#135 lore raw

PATCH v5 7/15 bugreport: add curl version

In reply to [email protected]

From: Emily Shaffer <[email protected]>

It's possible for git-remote-curl to be built separately from git; in that case we want to know what version of cURL is used by git-remote-curl, not necessarily which version was present at git-bugreport's build time. So instead, ask git-remote-curl for the version information it knows about.

For longevity purposes, invoke the alias "git-remote-https" instead of "git-remote-http".

Since it could have been built at a different time, also report the version and built-from commit of git-remote-curl alongside the cURL info.

---
 bugreport.c   | 16 ++++++++++++++++
 remote-curl.c |  8 ++++++++
 2 files changed, 24 insertions(+)
bugreport.c +16 -0
index 818ccb385c..73f6d39517 100644
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include "compat/compiler.h"
+#include "run-command.h"
+
+static void get_curl_version_info(struct strbuf *curl_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, curl_info, 0))
+	    strbuf_addstr(curl_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -31,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "compiler info: ");
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+	get_curl_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
remote-curl.c +8 -0
index 350d92a074..c590fbfae3 100644
@@ -17,6 +17,7 @@
 #include "protocol.h"
 #include "quote.h"
 #include "transport.h"
+#include "version.h"
 
 static struct remote *remote;
 /* always ends with a trailing slash */
@@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {
+		printf("git-http-fetch version: %s\n", git_version_string);
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#136 lore raw

Re: [PATCH v5 07/15] bugreport: add curl version

Quoted 14 lines · On Fri, 24 Jan 2020 at 04:41, <[email protected]> wrote:
bugreport.c +10 -0
+static void get_curl_version_info(struct strbuf *curl_info)
+{
+       struct child_process cp = CHILD_PROCESS_INIT;
+
+       argv_array_push(&cp.args, "git");
+       argv_array_push(&cp.args, "remote-https");
+       argv_array_push(&cp.args, "--build-info");
+       if (capture_command(&cp, curl_info, 0))
+           strbuf_addstr(curl_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {

bugreport.c static void get_system_info(struct strbuf *sys_info) · line 48

Quoted 6 lines
bugreport.c +3 -0
@@ -31,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
        strbuf_addstr(sys_info, "compiler info: ");
        get_compiler_info(sys_info);
        strbuf_complete_line(sys_info);
+
+       strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+       get_curl_version_info(sys_info);

The header here looks a lot like an implementation detail of get_curl_version_info(). Or put differently, these risk getting out of sync. Maybe frame the header a bit more human readable: "curl version". But is this "curl version", or more like "git-remote-https version"? There's some discrepancy here.

bugreport.c static void get_system_info(struct strbuf *sys_info) · line 50

Quoted 2 lines
bugreport.c +1 -0
+       strbuf_complete_line(sys_info);
 }

Martin

#137 lore raw

Re: [PATCH v5 07/15] bugreport: add curl version

Quoted 12 lines · Martin Ågren
Quoted 11 lines · On Fri, 24 Jan 2020 at 04:41, <[email protected]> wrote:
bugreport.c +10 -0
+static void get_curl_version_info(struct strbuf *curl_info)
+{
+       struct child_process cp = CHILD_PROCESS_INIT;
+
+       argv_array_push(&cp.args, "git");
+       argv_array_push(&cp.args, "remote-https");
+       argv_array_push(&cp.args, "--build-info");
+       if (capture_command(&cp, curl_info, 0))
+           strbuf_addstr(curl_info, "'git-remote-https --build-info' not supported\n");
+}
Quoted 15 lines

bugreport.c static void get_system_info(struct strbuf *sys_info) · line 48

Quoted 9 lines

static void get_system_info(struct strbuf *sys_info) {

bugreport.c +3 -0
@@ -31,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
        strbuf_addstr(sys_info, "compiler info: ");
        get_compiler_info(sys_info);
        strbuf_complete_line(sys_info);
+
+       strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+       get_curl_version_info(sys_info);

The header here looks a lot like an implementation detail of get_curl_version_info(). Or put differently, these risk getting out of sync. Maybe frame the header a bit more human readable: "curl version". But is this "curl version", or more like "git-remote-https version"? There's some discrepancy here.

Hm, I think you're saying "If we switch to future-https-lib instead of cURL for git-remote-https, then this command will be incorrectly named." Sure, I agree. It's true that with this change git-remote-https also tells us some info about itself.

Thanks.
- Emily

#138 lore raw

PATCH v5 8/15 bugreport: include user interactive shell

In reply to [email protected]

From: Emily Shaffer <[email protected]>

It's possible a user may complain about the way that Git interacts with their interactive shell, e.g. autocompletion or shell prompt. In that case, it's useful for us to know which shell they're using interactively.

---
 bugreport.c | 5 +++++
 1 file changed, 5 insertions(+)
bugreport.c +5 -0
index 73f6d39517..07b84b9c94 100644
@@ -22,6 +22,7 @@ static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
 	struct utsname uname_info;
+	char *shell = NULL;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
@@ -44,6 +45,10 @@ static void get_system_info(struct strbuf *sys_info)
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
 
+	shell = getenv("SHELL");
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    shell ? shell : "(NULL)");
+
 	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
 	get_curl_version_info(sys_info);
 	strbuf_complete_line(sys_info);
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#139 lore raw

Re: [PATCH v5 08/15] bugreport: include user interactive shell

Quoted 1 line · On Fri, 24 Jan 2020 at 04:41, <[email protected]> wrote:

+ char *shell = NULL;

(Unnecessary initialization.)

bugreport.c static void get_system_info(struct strbuf *sys_info) · line 50

Quoted 3 lines
bugreport.c +3 -0
+       shell = getenv("SHELL");
+       strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+                   shell ? shell : "(NULL)");

Thanks for avoiding a classic pitfall. :-)

"<unused>" instead of "(NULL)"? "NULL" is mostly an implementation detail.

Martin

#140 lore raw

Re: [PATCH v5 08/15] bugreport: include user interactive shell

Quoted 10 lines · Martin Ågren
Quoted 1 line · On Fri, 24 Jan 2020 at 04:41, <[email protected]> wrote:

+ char *shell = NULL;

(Unnecessary initialization.)

bugreport.c static void get_system_info(struct strbuf *sys_info) · line 50

Quoted 3 lines
bugreport.c +3 -0
+       shell = getenv("SHELL");
+       strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+                   shell ? shell : "(NULL)");

Thanks for avoiding a classic pitfall. :-)

Thank Junio. I fell right into it because it Just Works in gcc. ;)

Quoted 2 lines

"<unused>" instead of "(NULL)"? "NULL" is mostly an implementation detail.

Sure, why not.

#141 lore raw

Re: [PATCH v5 08/15] bugreport: include user interactive shell

In reply to Martin Ågren
Quoted 5 lines · Martin Ågren
Quoted 1 line · On Fri, 24 Jan 2020 at 04:41, <[email protected]> wrote:

+ char *shell = NULL;

(Unnecessary initialization.)

Quoted 8 lines

bugreport.c static void get_system_info(struct strbuf *sys_info) · line 50

Quoted 3 lines
bugreport.c +3 -0
+       shell = getenv("SHELL");
+       strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+                   shell ? shell : "(NULL)");

Thanks for avoiding a classic pitfall. :-)

"<unused>" instead of "(NULL)"? "NULL" is mostly an implementation detail.

Isn't that <unset>?

#142 lore raw

Re: [PATCH v5 08/15] bugreport: include user interactive shell

Quoted 12 lines · Junio C Hamano
Quoted 9 lines · Martin Ågren
Quoted 1 line · On Fri, 24 Jan 2020 at 04:41, <[email protected]> wrote:

+ char *shell = NULL;

(Unnecessary initialization.)

Quoted 3 lines
bugreport.c +3 -0
+       shell = getenv("SHELL");
+       strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+                   shell ? shell : "(NULL)");
Quoted 6 lines
Quoted 4 lines

Thanks for avoiding a classic pitfall. :-)

"<unused>" instead of "(NULL)"? "NULL" is mostly an implementation detail.

Isn't that <unset>?

Heh, yes, "SHELL: unused" sounds wrong. :-)

Martin

#143 lore raw

PATCH v5 9/15 bugreport: generate config safelist based on docs

In reply to [email protected]

From: Emily Shaffer <[email protected]>

Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "bugreport" macro, which is recognized by AsciiDoc and AsciiDoctor.

Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.

In order to prevent staleness of the safelist, add a mechanism to generate the safelist from annotations in the config documentation, where contributors are already used to documenting their new config keys.

Implement a new no-op "bugreport" macro for use as "bugreport:include[x]" to annotate the config keys that should be included in the automatically generated safelist. Use "exclude" for the others.

With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc seems to want something between the brackets. A bit unfortunate, but not a huge problem -- we'll just provide an "x".

"doc-diff" reports that this macro doesn't render at all. That is, these are both empty after this commit:

  cd Documentation
  ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
  ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD

Diffing the rendered HTML shows that there is some small amount of whitespace and comments added. That shouldn't be a problem.

We could perhaps let the implementation verify that the "action" is one of "include" and "exclude". For the Asciidoctor implementation that should be straightforward, but for AsciiDoc I don't immediately know how to do it. Anyway, if someone stumbles on the keyboard and writes "bugreport:icndule", they'll "only" miss out on the config key being included in the safelist. If this were a blocklist, the consequences of a misspelled target could be a lot more severe.

Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.

---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  8 +++
 Documentation/asciidoctor-extensions.rb |  7 +++
 Documentation/config/sendemail.txt      | 68 ++++++++++++-------------
 Makefile                                |  7 +++
 generate-bugreport-config-safelist.sh   | 17 +++++++
 6 files changed, 74 insertions(+), 34 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh
.gitignore +1 -0
index d89bf9e11e..bd2f49b996 100644
@@ -192,6 +192,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
Documentation/asciidoc.conf +8 -0
index 8fc4b67081..5d5359fcf9 100644
@@ -6,9 +6,13 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The bugreport macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +32,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[bugreport-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[bugreport-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
Documentation/asciidoctor-extensions.rb +7 -0
index d906a00803..750bdff9af 100644
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, action, attrs)
+        ""
       end
     end
   end
@@ -45,4 +49,7 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  # The bugreport macro does nothing as far as rendering is
+  # concerned -- we just grep for it in the sources.
+  inline_macro Git::Documentation::BugReportProcessor, :bugreport
 end
Documentation/config/sendemail.txt +34 -34

97 lines not on this page · raw message

Makefile +7 -0
index a01a050aa3..2bc9f112ea 100644
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2163,6 +2164,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
generate-bugreport-config-safelist.sh new file +17 -0
new file mode 100755
index 0000000000..44612d5538
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the bugreport-include macro
+sed -n 's/^\(.*\) \+bugreport:include.* ::$/  "\1",/p' | sort
+
+cat <<EOF
+};
+EOF
Signature · 1 line
2.25.0.341.g760bfbb309-goog
#144 lore raw

Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs

Quoted 7 lines · On Fri, 24 Jan 2020 at 04:35, <[email protected]> wrote:

Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "bugreport" macro, which is recognized by AsciiDoc and AsciiDoctor.

Quoted 7 lines

Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.

I sort of wonder whether safelist/blocklist is to prefer over whitelist/blacklist, or if it's the other way round. The former are new to me, whereas the latter are the terms I would have used. But that's just me, of course. I was a little surprised, that's all.

Quoted 8 lines

Implement a new no-op "bugreport" macro for use as "bugreport:include[x]" to annotate the config keys that should be included in the automatically generated safelist. Use "exclude" for the others.

With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc seems to want something between the brackets. A bit unfortunate, but not a huge problem -- we'll just provide an "x".

I recognize this reasoning :-) and I'm not terribly opposed to it, but after some nights' sleeping on this, I have to wonder if "annotate:bugreport[include]" wouldn't be better than "bugreport[x]" with that ugly "x". Maybe this isn't the biggest problem, but if we expect this macro to eventually sit right next to ~90% of all our config variables...

Quoted 6 lines

"doc-diff" reports that this macro doesn't render at all. That is, these are both empty after this commit:

  cd Documentation
  ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
  ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD

That was true in [1], but alas, no more. In that patch, it's sort of obvious from the diff how it adds a "class" which "end"s.

[1] https://lore.kernel.org/git/[email protected]/

Documentation/asciidoctor-extensions.rb module Git · line 46

Quoted 13 lines
Documentation/asciidoctor-extensions.rb +4 -0
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, action, attrs)
+        ""
       end
     end
   end

But this one doesn't add an "end", and Asciidoctor trips up badly.

Documentation/asciidoctor-extensions.rb end · line 54

Quoted 3 lines
Documentation/asciidoctor-extensions.rb +3 -0
+  # The bugreport macro does nothing as far as rendering is
+  # concerned -- we just grep for it in the sources.
+  inline_macro Git::Documentation::BugReportProcessor, :bugreport

(I never much liked this copy-paste comment then, and I still don't, to be honest.)

Martin

#145 lore raw

Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs

Quoted 16 lines · Martin Ågren
Quoted 7 lines · On Fri, 24 Jan 2020 at 04:35, <[email protected]> wrote:

Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "bugreport" macro, which is recognized by AsciiDoc and AsciiDoctor.

Quoted 7 lines

Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.

Quoted 4 lines

I sort of wonder whether safelist/blocklist is to prefer over whitelist/blacklist, or if it's the other way round. The former are new to me, whereas the latter are the terms I would have used. But that's just me, of course. I was a little surprised, that's all.

Eh. I think the following things are true:

- Whitelist/blacklist has a "smell" of discrimination, whether that's the true etymology or not.
- Those with experience in the field can easily understand what whitelist or blacklist means.
- Safelist/blocklist do not "smell" the same way.
- It is easy to tell what "safelist" means: "a list of stuff which is safe." No experience needed.

So, while it's new, I think it's not harmful. I see only a no-op or positive impact from using this term instead of whitelist/blacklist. Computer science seems to have quite a few terms which fall into this long-standing but potentially negative area, so I don't mind looking for alternatives where it's harmless to do so.

Quoted 10 lines
Quoted 9 lines

Implement a new no-op "bugreport" macro for use as "bugreport:include[x]" to annotate the config keys that should be included in the automatically generated safelist. Use "exclude" for the others.

With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc seems to want something between the brackets. A bit unfortunate, but not a huge problem -- we'll just provide an "x".

Quoted 6 lines

I recognize this reasoning :-) and I'm not terribly opposed to it, but after some nights' sleeping on this, I have to wonder if "annotate:bugreport[include]" wouldn't be better than "bugreport[x]" with that ugly "x". Maybe this isn't the biggest problem, but if we expect this macro to eventually sit right next to ~90% of all our config variables...

Hm. I wanted to say, "Ok, but I don't know how to do that, so can you help?" But I think that's all the more reason that I should do it ;)

Ok. I will try and change it to annotate:bugreport[include] like you suggested, and hopefully learn more about asciidoc macros :)

Quoted 13 lines
Quoted 7 lines

"doc-diff" reports that this macro doesn't render at all. That is, these are both empty after this commit:

  cd Documentation
  ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
  ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD

That was true in [1], but alas, no more. In that patch, it's sort of obvious from the diff how it adds a "class" which "end"s.

[1] https://lore.kernel.org/git/[email protected]/

Quoted 15 lines

Documentation/asciidoctor-extensions.rb module Git · line 46

Quoted 13 lines
Documentation/asciidoctor-extensions.rb +4 -0
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, action, attrs)
+        ""
       end
     end
   end

But this one doesn't add an "end", and Asciidoctor trips up badly.

Ok, I'll have a look. I'm sure I copied something badly.

Quoted 7 lines

Documentation/asciidoctor-extensions.rb end · line 54

Quoted 4 lines
Documentation/asciidoctor-extensions.rb +3 -0
+  # The bugreport macro does nothing as far as rendering is
+  # concerned -- we just grep for it in the sources.
+  inline_macro Git::Documentation::BugReportProcessor, :bugreport

(I never much liked this copy-paste comment then, and I still don't, to be honest.)

I'll see if I can find a reasonable alternative (or remove it).

- Emily

#146 lore raw

Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs

Quoted 16 lines · Emily Shaffer
Quoted 5 lines

I sort of wonder whether safelist/blocklist is to prefer over whitelist/blacklist, or if it's the other way round. The former are new to me, whereas the latter are the terms I would have used. But that's just me, of course. I was a little surprised, that's all.

Eh. I think the following things are true:

- Whitelist/blacklist has a "smell" of discrimination, whether that's the true etymology or not.
- Those with experience in the field can easily understand what whitelist or blacklist means.
- Safelist/blocklist do not "smell" the same way.
- It is easy to tell what "safelist" means: "a list of stuff which is safe." No experience needed.

Quoted 5 lines

So, while it's new, I think it's not harmful. I see only a no-op or positive impact from using this term instead of whitelist/blacklist. Computer science seems to have quite a few terms which fall into this long-standing but potentially negative area, so I don't mind looking for alternatives where it's harmless to do so.

Ok, that all makes sense.

Thanks
Martin

#147 lore raw

Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs

In reply to [email protected]
Quoted 6 lines · On Fri, 24 Jan 2020 at 04:35, <[email protected]> wrote:

Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "bugreport" macro, which is recognized by AsciiDoc and AsciiDoctor.

"which is recognized by" sounds like it's built-in. Maybe "with a new no-op 'bugreport' macro" or something like that.

Documentation/config/sendemail.txt line 59

Quoted 10 lines
Documentation/config/sendemail.txt +2 -2
-sendemail.signedoffcc (deprecated)::
+sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
        Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize bugreport:include[x] ::
        Number of messages to be sent per connection, after that a relogin
        will happen.  If the value is 0 or undefined, send all messages in
        one connection.
        See also the `--batch-size` option of linkgit:git-send-email[1].

Do we really need to list includes *and* excludes? I could see the point of adding an exclude here and there to signal that "this might look innocent enough, but trust me, we really need to exclude this" in order to avoid future commits to more or less accidentally over-include. Should we add some internal documentation and/or a remark in the commit message about this? As an example, is "sendemail.signedofcc" sensitive enough that we need to explicitly exclude it? If someone wants to come along and include it, they don't just need to argue for an inclusion, but also for lifting the exclusion. Hmm?

Martin

#148 lore raw

Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs

Quoted 10 lines · Martin Ågren
Quoted 6 lines · On Fri, 24 Jan 2020 at 04:35, <[email protected]> wrote:

Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "bugreport" macro, which is recognized by AsciiDoc and AsciiDoctor.

"which is recognized by" sounds like it's built-in. Maybe "with a new no-op 'bugreport' macro" or something like that.

Sure, I'll wordsmith this a little.

Quoted 5 lines
Quoted 5 lines
Documentation/config/sendemail.txt +1 -1
-sendemail.signedoffcc (deprecated)::
+sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
        Deprecated alias for `sendemail.signedoffbycc`.
Quoted 8 lines

Documentation/config/sendemail.txt line 59

Quoted 6 lines
Documentation/config/sendemail.txt +1 -1
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize bugreport:include[x] ::
        Number of messages to be sent per connection, after that a relogin
        will happen.  If the value is 0 or undefined, send all messages in
        one connection.
        See also the `--batch-size` option of linkgit:git-send-email[1].

Do we really need to list includes *and* excludes?

Need? Nah. The exclude markers are not even acknowledged by the current implementation.

Quoted 4 lines

I could see the point of adding an exclude here and there to signal that "this might look innocent enough, but trust me, we really need to exclude this" in order to avoid future commits to more or less accidentally over-include.

This was more of my line of thinking, along with the thought that some folks may use bug reporting internally, where they have A) more control over which extensions/tools are in use, and B) less concern about sharing things like remote URLs, branch names, etc. and may want to go for a blocklist as opposed to a safelist, to make it easier for their support folks to diagnose.

Quoted 5 lines

Should we add some internal documentation and/or a remark in the commit message about this? As an example, is "sendemail.signedofcc" sensitive enough that we need to explicitly exclude it? If someone wants to come along and include it, they don't just need to argue for an inclusion, but also for lifting the exclusion. Hmm?

Yeah, I think this is a good point. I'll try to figure out how to reword the commit message, and take another look at the sample include/exclude change I made to see where I can omit entirely.

- Emily

#149 lore raw

Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs

Quoted 3 lines · Emily Shaffer

Yeah, I think this is a good point. I'll try to figure out how to reword the commit message, and take another look at the sample include/exclude change I made to see where I can omit entirely.

Eh. I'm going to try and reword the whole commit message. I combined your scissors patch, plus something from Dscho's scissors patch, plus my original commit message, and now it's huge and hard to follow.

- Emily

#150 lore raw

PATCH v5 10/15 bugreport: add config values from safelist

In reply to [email protected]

From: Emily Shaffer <[email protected]>

Teach bugreport to gather the values of config options which are present in 'bugreport-config-safelist.h'.

Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.

Taking the build-time generated array and putting it into a set saves us time - since git_config_bugreport() is called for every option the user has configured, performing option lookup in constant time is a useful optimization.

---
 Makefile    |  2 +-
 bugreport.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 76 insertions(+), 1 deletion(-)
Makefile +1 -1
index 2bc9f112ea..eb17ece120 100644
@@ -2465,7 +2465,7 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
-git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+git-bugreport$X: bugreport-config-safelist.h bugreport.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(LIBS)
 
bugreport.c +75 -0

111 lines not on this page · raw message

Signature · 1 line
2.25.0.341.g760bfbb309-goog
#151 lore raw

Re: [PATCH v5 10/15] bugreport: add config values from safelist

Makefile endif · line 2470

Quoted 10 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:
Makefile +1 -1
@@ -2465,7 +2465,7 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
        $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
-git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+git-bugreport$X: bugreport-config-safelist.h bugreport.o GIT-LDFLAGS $(GITLIBS)
        $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
                $(LIBS)

Haven't looked at this patch at all, except I've noticed that something is up with the dependencies. I think bugreport.o needs to depend on bugreport-config-safelist.h. As it is, the latter may or may not be available as bugreport.o is built. (Reproduces fairly well for me with something like make clean && make -j16.)

I'll try to continue tomorrow. :-)

Martin