bugreport: add tool to generate debugging info
Subsystems (2)
PATCH 1/1 bugreport: add tool to generate debugging info
Make it easier for users who encounter a bug to send a report by collecting some state information, intended to be viewed by humans familiar with Git.
Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Also, teach Git to write down its own version, the version of some of its dependencies, the operating system information, the effective gitconfig, the configured hooks, and the contents of $GIT_DIR/logs. Finally, make sure Git asks the user to review the contents of the report after it's generated.
If users can send us a well-written bug report complete with system information, a remote we may be able to clone, and a reflog showing us where the problem occurred, we can reduce the number of questions and answers which must travel between the reporter and the Git contributor.
Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.
---
There are some things I'm not certain about from this patch, I'd appreciate feedback.
- Do we want to advertise the Git mailing list for bug reports?
- Which config options should we filter to avoid accidentally receiving credentials?
- At the moment, the test is trying to check "everything we thought we would populate got populated" - that seemed to me like it would hold up best to changes to the set of information being collected. But maybe there's something more robust we can do.
And of course, if there are suggestions for other things to include in the report I'm interested in adding.
An example of the output can be found here:
https://gist.github.com/nasamuffin/2e320892f5c2147e829cbcf5bd0759a2
Thanks.
- Emily
.gitignore | 1 + Documentation/git-bugreport.txt | 48 ++++++++++++++++++ Makefile | 1 + command-list.txt | 1 + git-bugreport.sh | 86 +++++++++++++++++++++++++++++++++ t/t0091-bugreport.sh | 41 ++++++++++++++++ 6 files changed, 178 insertions(+) create mode 100644 Documentation/git-bugreport.txt create mode 100755 git-bugreport.sh create mode 100755 t/t0091-bugreport.sh
.gitignore +1 -0
@@ -25,6 +25,7 @@ /git-bisect--helper /git-blame /git-branch +/git-bugreport /git-bundle /git-cat-file /git-check-attr
Documentation/git-bugreport.txt new file +48 -0
48 lines not on this page · raw message
Makefile +1 -0
@@ -606,6 +606,7 @@ TEST_PROGRAMS_NEED_X = unexport CDPATH SCRIPT_SH += git-bisect.sh +SCRIPT_SH += git-bugreport.sh SCRIPT_SH += git-difftool--helper.sh SCRIPT_SH += git-filter-branch.sh SCRIPT_SH += git-merge-octopus.sh
command-list.txt +1 -0
@@ -54,6 +54,7 @@ git-archive mainporcelain git-bisect mainporcelain info git-blame ancillaryinterrogators complete git-branch mainporcelain history +git-bugreport ancillaryinterrogators git-bundle mainporcelain git-cat-file plumbinginterrogators git-check-attr purehelpers
git-bugreport.sh new file +86 -0
86 lines not on this page · raw message
t/t0091-bugreport.sh new file +41 -0
41 lines not on this page · raw message
Signature · 1 line
2.23.0.rc1.153.gdeed80330f-goog
Quoted 3 lines · Emily Shaffer
Make it easier for users who encounter a bug to send a report by collecting some state information, intended to be viewed by humans familiar with Git.
This is an excellent idea! VFS for Git has a similar "diagnose" command that collects logs, config, and other details (like packfile sizes and loose object counts). That has been a critical tool for supporting users.
Quoted 16 lines
Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Also, teach Git to write down its own version, the version of some of its dependencies, the operating system information, the effective gitconfig, the configured hooks, and the contents of $GIT_DIR/logs. Finally, make sure Git asks the user to review the contents of the report after it's generated.
If users can send us a well-written bug report complete with system information, a remote we may be able to clone, and a reflog showing us where the problem occurred, we can reduce the number of questions and answers which must travel between the reporter and the Git contributor.
Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.
Quoted 6 lines
Signed-off-by Emily Shaffer <[email protected]>---
There are some things I'm not certain about from this patch, I'd appreciate feedback.- Do we want to advertise the Git mailing list for bug reports?
That is possible. Isn't there another mailing list for git users?
I could see a patch added on top of this for git-for-windows/git that changes the instructions to create issues on GitHub.
Quoted 2 lines
- Which config options should we filter to avoid accidentally receiving credentials?
The remote URLs are pretty sensitive. Not only do users sometimes put passwords or PATs into their URLs, the literal name of the repo could be a secret.
git-bugreport.sh
Quoted 120 lines
- At the moment, the test is trying to check "everything we thought we would populate got populated" - that seemed to me like it would hold up best to changes to the set of information being collected. But maybe there's something more robust we can do.
And of course, if there are suggestions for other things to include in the report I'm interested in adding.
An example of the output can be found here:
https://gist.github.com/nasamuffin/2e320892f5c2147e829cbcf5bd0759a2Thanks.
- Emily.gitignore | 1 + Documentation/git-bugreport.txt | 48 ++++++++++++++++++ Makefile | 1 + command-list.txt | 1 + git-bugreport.sh | 86 +++++++++++++++++++++++++++++++++ t/t0091-bugreport.sh | 41 ++++++++++++++++ 6 files changed, 178 insertions(+) create mode 100644 Documentation/git-bugreport.txt create mode 100755 git-bugreport.sh create mode 100755 t/t0091-bugreport.sh.gitignore +1
-0@@ -25,6 +25,7 @@ /git-bisect--helper /git-blame /git-branch +/git-bugreport /git-bundle /git-cat-file /git-check-attrDocumentation/git-bugreport.txt new file +48
-0@@ -0,0 +1,48 @@ +git-bugreport(1) +================ + +NAME +---- +git-bugreport - Collect information for user to file a bug report + +SYNOPSIS +-------- +[verse] +'git bugreport' [-o | --output <path>] + +DESCRIPTION +----------- +Captures information about the user's machine, Git client, and repository state, +as well as a form requesting information about the behavior the user observed, +into a single text file which the user can then share, for example to the Git +mailing list, in order to report an observed bug. + +The following information is requested from the user: + + - Reproduction steps + - Expected behavior + - Actual behavior + +The following information is captured automatically: + + - Git version (`git version --build-options`) + - Machine information (`uname -a`) + - Versions of various dependencies + - Git config contents (`git config --show-origin --list`) + - The contents of all configured git-hooks in `.git/hooks/` + - The contents of `.git/logs` + +OPTIONS +------- +-o [<path>]:: +--output [<path>]:: + Place the resulting bug report file in <path> instead of the root of the + Git repository. + +NOTE +---- +Bug reports can be sent to [email protected]. + +GIT +--- +Part of the linkgit:git[1] suiteMakefile +1
-0@@ -606,6 +606,7 @@ TEST_PROGRAMS_NEED_X = unexport CDPATH SCRIPT_SH += git-bisect.sh +SCRIPT_SH += git-bugreport.sh SCRIPT_SH += git-difftool--helper.sh SCRIPT_SH += git-filter-branch.sh SCRIPT_SH += git-merge-octopus.shcommand-list.txt +1
-0@@ -54,6 +54,7 @@ git-archive mainporcelain git-bisect mainporcelain info git-blame ancillaryinterrogators complete git-branch mainporcelain history +git-bugreport ancillaryinterrogators git-bundle mainporcelain git-cat-file plumbinginterrogators git-check-attr purehelpersgit-bugreport.sh new file +0
-0
At first I was alarmed by "What? another shell script?" but this command should prioritize flexibility and extensibility over speed. Running multiple processes shouldn't be too taxing for what we are trying to do here.
Quoted 31 lines
git-bugreport.sh new file +30
-0@@ -0,0 +1,86 @@ +#!/bin/sh + +print_filenames_and_content() { +while read -r line; do + echo "$line"; + echo "========"; + cat "$line"; + echo; +done +} + +generate_report_text() { + + # Generate a form for the user to fill out with their issue. + gettextln "Thank you for filling out a Git bug report!" + gettextln "Please answer the following questions to help us understand your issue." + echo + gettextln "What did you do before the bug happened? (Steps to reproduce your issue)" + echo + gettextln "What did you expect to happen? (Expected behavior)" + echo + gettextln "What happened instead? (Actual behavior)" + echo + gettextln "Anything else you want to add:" + echo + gettextln "Please review the rest of the bug report below." + gettextln "You can delete any lines you don't wish to send." + echo + + echo "[System Information]"
git-bugreport.sh line 40
Quoted 10 lines
git-bugreport.sh new file +10
-0+ git version --build-options + uname -a + curl-config --version + ldd --version + echo + + echo "[Git Config]" + # TODO: Pass this through grep -v to avoid users sending us their credentials. + git config --show-origin --list + echo
Config options to consider stripping out:
*url* *pass* (anything "password" but also "sendmail.smtppass")
git-bugreport.sh line 44
Quoted 3 lines
git-bugreport.sh new file +3
-0+ echo "[Configured Hooks]" + find "$GIT_DIR/hooks/" -type f | grep -v "\.sample$" | print_filenames_and_content + echo
Remove the sample hooks, but focus on the others. Will this look like garbage if a hook is a binary file?
git-bugreport.sh line 48
Quoted 4 lines
git-bugreport.sh new file +4
-0+ + echo "[Git Logs]" + find "$GIT_DIR/logs" -type f | print_filenames_and_content + echo
As mentioned before, I've sometimes found it helpful to know the data shape for the object store. Having a few extra steps such as the following could be nice:
echo "[Loose Objects]"
for objdir in $(find "$GIT_DIR/objects/??" -type d) do
echo "$objdir: $(ls $objdir | wc -l)"
done
echo
echo "[Pack Data]"
ls -l "$GIT_DIR/objects/pack"
echo
echo "[Object Info Data]"
ls -lR "$GIT_DIR/objects/info"
echo
echo "[Alternates File]"
echo "========"
cat "$GIT_DIR/objects/info/alternates"
echo
That last one will collect information on the commit-graph file, even if it is an incremental file.
Quoted 76 lines
git-bugreport.sh new file +38
-0+ +} + +USAGE="[-o | --output <path>]" + +SUBDIRECTORY_OK=t +OPTIONS_SPEC= +. git-sh-setup +. git-sh-i18n + +basedir="$PWD" +while : +do + case "$1" in + -o|--output) + shift + basedir="$1" + shift + continue + ;; + "") + break + ;; + *) + usage + ;; + esac +done + + +# Create bugreport file +BUGREPORT_FILE="$basedir/git-bugreport-$(whoami)-$(hostname)-$(date -Iminutes)" + +generate_report_text >$BUGREPORT_FILE + +git_editor $BUGREPORT_FILE + +eval_gettextln "Your new bug report is in \$BUGREPORT_FILE."t/t0091-bugreport.sh new file +31
-0@@ -0,0 +1,41 @@ +#!/bin/bash + +test_description='git bugreport' + +. ./test-lib.sh + +# Headers "[System Info]" will be followed by a non-empty line if we put some +# information there; we can make sure all our headers were followed by some +# information to check if the command was successful. +HEADER_PATTERN="^\[.*\]$" +check_all_headers_populated() { + while read -r line; do + if [$(grep $HEADER_PATTERN $line)]; then + read -r nextline + if [-z $nextline]; then + return 1; + fi + fi + done +} + +test_expect_success 'creates a report with content in the right places' ' + git bugreport && + check_all_headers_populated <git-bugreport-* && + rm git-bugreport-* +' + +test_expect_success '--output puts the report in the provided dir' ' + mkdir foo/ && + git bugreport -o foo/ && + test -f foo/git-bugreport-* &&
t/t0091-bugreport.sh line 41
Quoted 10 lines
t/t0091-bugreport.sh new file +10
-0+ rm -fr foo/ +' + +test_expect_success 'incorrect arguments abort with usage' ' + test_must_fail git bugreport --false 2>output && + grep usage output && + test ! -f git-bugreport-* +' + +test_done
I think this is a great start, and I'll take some time later to try it out.
Thanks,
-Stolee
Quoted 4 lines · Derrick Stolee
Config options to consider stripping out:
*url* *pass* (anything "password" but also "sendmail.smtppass")
Blacklisting? I wonder if users feel safer if these are limited to known-benign ones.
Quoted 6 lines
git-bugreport.sh line 44
Quoted 3 lines
git-bugreport.sh new file +3
-0+ echo "[Configured Hooks]" + find "$GIT_DIR/hooks/" -type f | grep -v "\.sample$" | print_filenames_and_content + echoRemove the sample hooks, but focus on the others. Will this look like garbage if a hook is a binary file?
This makes me feel very nervous. $GIT_DIR/hooks/ are private and people can hardcode credentials in them; $GIT_DIR/hooks/pre-foo may be written toread from $GIT_DIR/hooks/mypassword with the knowledge that there won't be any "mypassword" hook.
Quoted 9 lines · Junio C Hamano
Quoted 4 lines · Derrick Stolee
Config options to consider stripping out:
*url* *pass* (anything "password" but also "sendmail.smtppass")Blacklisting? I wonder if users feel safer if these are limited to known-benign ones.
I think a whitelist of config options to print would grow stale immediately, and the options we're missing would be very likely to be configs to turn on new experimental features - which is probably what we most want the bugreport for.
Quoted 8 lines
Quoted 7 lines
git-bugreport.sh line 44
Quoted 4 lines
git-bugreport.sh new file +3
-0+ echo "[Configured Hooks]" + find "$GIT_DIR/hooks/" -type f | grep -v "\.sample$" | print_filenames_and_content + echoRemove the sample hooks, but focus on the others. Will this look like garbage if a hook is a binary file?
Quoted 4 lines
This makes me feel very nervous. $GIT_DIR/hooks/ are private and people can hardcode credentials in them; $GIT_DIR/hooks/pre-foo may be written toread from $GIT_DIR/hooks/mypassword with the knowledge that there won't be any "mypassword" hook.
Hmm. I think the list of valid hooks isn't one that changes often, but it's also not enumerated in some machine-parseable way - it exists in Documentation/githooks.txt but that's all. I'd still be a little worried about bitrot... I think it's probably better to list the filenames in $GIT_DIR/hooks but not print their contents. I'll modify it.
- Emily
Quoted 4 lines · Emily Shaffer
I think a whitelist of config options to print would grow stale immediately, and the options we're missing would be very likely to be configs to turn on new experimental features - which is probably what we most want the bugreport for.
The implementation of your "git bugreport" command comes from the same version of Git as such an experimental feature you want feedback to is shipped with, so I am not sure where your concern about staleness comes from.
If you really care about getting quality reports, you need to earn users' trust (one way of doing so would be to maintain a good "list of benign configuration knobs whose values help diagnosing issues"), and you need to make sure we obtain relevant pieces of information. Both of these things are something you need to actively work on.
We have trained ourselves to the point that we consider it a bug if you add a new command git-bugreport without adding a corresponding pattern in the .gitignore file. And thanks to that discipline, we have been fairly good at keeping .gitignore up to date. I do not see a reason why we cannot do something similar to the registry of configuration variables that we care about, which may be shipped as part of "git bugreport" command.
Quoted 18 lines · Junio C Hamano
Quoted 4 lines · Emily Shaffer
I think a whitelist of config options to print would grow stale immediately, and the options we're missing would be very likely to be configs to turn on new experimental features - which is probably what we most want the bugreport for.
The implementation of your "git bugreport" command comes from the same version of Git as such an experimental feature you want feedback to is shipped with, so I am not sure where your concern about staleness comes from.
If you really care about getting quality reports, you need to earn users' trust (one way of doing so would be to maintain a good "list of benign configuration knobs whose values help diagnosing issues"), and you need to make sure we obtain relevant pieces of information. Both of these things are something you need to actively work on.
Quoted 7 lines
We have trained ourselves to the point that we consider it a bug if you add a new command git-bugreport without adding a corresponding pattern in the .gitignore file. And thanks to that discipline, we have been fairly good at keeping .gitignore up to date. I do not see a reason why we cannot do something similar to the registry of configuration variables that we care about, which may be shipped as part of "git bugreport" command.
I think comparing this habit to the .gitignore isn't quite fair - .gitignore tells me I forgot to add my new command binary to it, when I run git status to see what I need to add to my commit with new command.
But, I agree that it's not going to be possible to create a completely leakproof blacklisting heuristic here. So I'll use a whitelist for the next patchset.
- Emily
Quoted 4 lines · Emily Shaffer
I think comparing this habit to the .gitignore isn't quite fair - .gitignore tells me I forgot to add my new command binary to it, when I run
git statusto see what I need to add to my commit with new command.
That is why I said that we need to actively work on, if we care about getting quality reports.
I do not think it is unreasonable to expect the build procedure for "git bugreport" to involve scanning in Documentation/config/ to pick up variable names, annotated in such a way that is invisible to AsciiDoc to allow us tell which ones are sensitive and which ones are not. A test in t/ could even check if a documented configuration variable has such an annotation. A commit that adds configuration variables without documentiong them does exist, but variables without documentation are (1) bugs, and (2) are not worth serious engineering effort on until they get documented.
I am not saying that you must implement the whitelist exactly like the above. I am not even saying that this must be whitelist and not blacklist---you'd have the same issue maintaining the list anyway.
The above is merely to illustrate the reason why I think that the kind of "active work" to make sure that the list will not go stale would be feasible.
Thanks.
Quoted 10 lines · Junio C Hamano
Quoted 4 lines · Emily Shaffer
I think comparing this habit to the .gitignore isn't quite fair - .gitignore tells me I forgot to add my new command binary to it, when I run
git statusto see what I need to add to my commit with new command.That is why I said that we need to actively work on, if we care about getting quality reports.
Quoted 9 lines
I do not think it is unreasonable to expect the build procedure for "git bugreport" to involve scanning in
Documentation/config/to pick up variable names, annotated in such a way that is invisible to AsciiDoc to allow us tell which ones are sensitive and which ones are not. A test in t/ could even check if a documented configuration variable has such an annotation. A commit that adds configuration variables without documentiong them does exist, but variables without documentation are (1) bugs, and (2) are not worth serious engineering effort on until they get documented.
Interesting. I think I have an idea for a way to do this, but it ends up fairly large; I'll send proof of concept as a follow-on patch to this one. Thanks for the suggestions.
- Emily
Hi,
Quoted 11 lines · Derrick Stolee
git-bugreport.sh
Quoted 5 lines · Emily Shaffer
git-bugreport.sh new file +0
-0At first I was alarmed by "What? another shell script?" but this command should prioritize flexibility and extensibility over speed. Running multiple processes shouldn't be too taxing for what we are trying to do here.
Git for Windows sometimes receives bug reports about Bash not being able to start (usually it is a DLL base address problem, related to the way Cygwin and MSYS2 emulate fork()).
In such a case, git bugreport would only be able to offer a reason for yet another bug report instead of adding useful metadata.
Something to keep in mind when deciding how to implement this command.
Ciao,
Dscho
Quoted 16 lines · Johannes Schindelin
Hi,
Quoted 11 lines · Derrick Stolee
git-bugreport.sh
Quoted 5 lines · Emily Shaffer
git-bugreport.sh new file +0
-0At first I was alarmed by "What? another shell script?" but this command should prioritize flexibility and extensibility over speed. Running multiple processes shouldn't be too taxing for what we are trying to do here.
Quoted 3 lines
Git for Windows sometimes receives bug reports about Bash not being able to start (usually it is a DLL base address problem, related to the way Cygwin and MSYS2 emulate
fork()).
Hmm. In a case like this, then, how is someone using GfW at all? Embarrassingly, I don't actually have a way to try it out for myself. It seems there's no GUI, and users are using it through the command line in mingw, so when you say "bash doesn't start" do you mean "users can't use the mingw prompt at all"?
Quoted 8 lines
In such a case,
git bugreportwould only be able to offer a reason for yet another bug report instead of adding useful metadata.Something to keep in mind when deciding how to implement this command.
Ciao,
Dscho
Yeah, that's an interesting point, thanks for bringing it up. So, in the case when bash won't start at all, what does that failure look like? How much of Git can users still use? For example, would they be able to get far enough to run a binary git-bugreport?
- Emily
Hi Emily,
Quoted 16 lines · Emily Shaffer
Quoted 15 lines · Johannes Schindelin
Quoted 11 lines · Derrick Stolee
git-bugreport.sh
Quoted 5 lines · Emily Shaffer
git-bugreport.sh new file +0
-0At first I was alarmed by "What? another shell script?" but this command should prioritize flexibility and extensibility over speed. Running multiple processes shouldn't be too taxing for what we are trying to do here.
Quoted 5 lines
Quoted 3 lines
Git for Windows sometimes receives bug reports about Bash not being able to start (usually it is a DLL base address problem, related to the way Cygwin and MSYS2 emulate
fork()).Hmm. In a case like this, then, how is someone using GfW at all?
On Windows, there are native command-line interpreters available that are _not_ Bash: PowerShell and CMD.
These days, you can get away with using Git _without_ a working Unix shell most of the time. There are only preciously few commands left that are still written as scripts, and most of these seem to be used less often than one might think:
- submodule
- bisect
- filter-branch
- instaweb
- mergetool
- some uncommon merge strategies
- rebase -p (already deprecated)
- several CVS/Subversion/Arch/Quilt integrations
- send-email (only relevant for mailing list centric projects, again, not very common any longer)
- request-pull (I would not be surprised if this is specific for the Linux project, as if that project was even close to the main user of Git)
So as long as you don't use submodules, and as long as you are unaware of the bisect command, which would comprise the majority of Git users in my experience, you can totally use Git for Windows without using Bash or Perl, i.e. without using the Cygwin/MSYS2 runtime that seems to be one of the common causes for trouble in Git for Windows.
Quoted 4 lines
Embarrassingly, I don't actually have a way to try it out for myself. It seems there's no GUI, and users are using it through the command line in mingw, so when you say "bash doesn't start" do you mean "users can't use the mingw prompt at all"?
There is no MinGW prompt.
MinGW stands for "Minimal GNU on Windows", but it really refers to a way to compile native Win32 programs via GCC. Meaning that some of the POSIX functionality Unix/Linux developers have come to expect is unavailable. For example, fork(). Which means that there is no native port of Bash to Windows, at least not that _I_ am aware of.
Git for Windows comes with a "Git Bash", which is essentially a Bash built against the MSYS2 runtime (i.e. it is much more like a Cygwin executable than like a Win32 executable).
The common way to run Git for Windows is actually via git.exe, independent of what particular command-line interpreter you prefer.
Quoted 9 lines
Quoted 4 lines
In such a case,
git bugreportwould only be able to offer a reason for yet another bug report instead of adding useful metadata.Something to keep in mind when deciding how to implement this command.
Yeah, that's an interesting point, thanks for bringing it up. So, in the case when bash won't start at all, what does that failure look like? How much of Git can users still use? For example, would they be able to get far enough to run a binary git-bugreport?
See above.
If I were you, I would not write git bugreport as anything but a built-in, written in C. Unless there are _really_ good reasons not to [*1*]. And quite honestly, I don't see any such reasons.
Ciao,
Dscho
Footnote *1*: One good reason to use the MSYS2 Bash would be to be able to use MSYS2's POSIX emulation layer, e.g. to talk to the SSH agent (that is also an MSYS2 program, meaning that it communicates via emulated Unix sockets, which is a facility not available to native Win32 programs).
Another valid reason to use the MSYS2 Bash is to be able to talk to the (emulated) pseudo TTYs provided by the MSYS2 runtime, e.g. when running inside a MinTTY window, which is the default for Git Bash in Git for Windows.
Quoted 3 lines · Derrick Stolee
Quoted 1 line
- Do we want to advertise the Git mailing list for bug reports?
That is possible. Isn't there another mailing list for git users?
I know there's an IRC channel for Git users, I dunno about mailing list. I'm worried that places I name as points of contact will grow stale, although I suppose #git on freenode isn't really going anywhere, for example. (But as a counterexample, I hope that nobody sends something like this to #git-devel, which seems to have a lower population than this mailing list.)
Quoted 3 lines
I could see a patch added on top of this for git-for-windows/git that changes the instructions to create issues on GitHub.
Indeed; I imagine we'd probably like to patch it to ask for bugs in our bug tracker.
Quoted 6 lines
Quoted 3 lines
- Which config options should we filter to avoid accidentally receiving credentials?
The remote URLs are pretty sensitive. Not only do users sometimes put passwords or PATs into their URLs, the literal name of the repo could be a secret.
Now here's where I start to wonder. We often debug internally by asking for the remote URL and replicating the issue there, which is why I mentioned it explicitly in the commit message. But I hadn't considered folks including the password in the URL.
Well, I suppose anybody can keep a local patch to change the config filter pattern. I'll try to make it easy to spot and modify.
[snip]
Quoted 3 lines
At first I was alarmed by "What? another shell script?" but this command should prioritize flexibility and extensibility over speed. Running multiple processes shouldn't be too taxing for what we are trying to do here.
If shell scripts are entirely deprecated I can convert it, but doing it in C seemed like overkill when I really just wanted "what are all the commands we would ask the user to run and tell us the output?". I figured also that it would be a little more immune to bitrot to output the contents of porcelain commands here.
Quoted 9 lines
git-bugreport.sh line 40
Quoted 4 lines
git-bugreport.sh new file +4
-0+ echo "[Git Config]" + # TODO: Pass this through grep -v to avoid users sending us their credentials. + git config --show-origin --list + echoConfig options to consider stripping out:
*url* *pass* (anything "password" but also "sendmail.smtppass")
Done, thanks.
Quoted 7 lines
git-bugreport.sh line 44
Quoted 4 lines
git-bugreport.sh new file +3
-0+ echo "[Configured Hooks]" + find "$GIT_DIR/hooks/" -type f | grep -v "\.sample$" | print_filenames_and_content + echoRemove the sample hooks, but focus on the others. Will this look like garbage if a hook is a binary file?
Yeah, I'm sure it will. I'll add a check to print_filenames_and_content() so it can tell us if there is a hook installed there, even if we can't see the content.
Quoted 9 lines
git-bugreport.sh line 48
Quoted 5 lines
git-bugreport.sh new file +4
-0+ + echo "[Git Logs]" + find "$GIT_DIR/logs" -type f | print_filenames_and_content + echoAs mentioned before, I've sometimes found it helpful to know the data shape for the object store. Having a few extra steps such as the following could be nice:
Quoted 4 lines
echo "[Loose Objects]" for objdir in $(find "$GIT_DIR/objects/??" -type d) do echo "$objdir: $(ls $objdir | wc -l)"
echo "$objdir: $(ls $objdir | wc -l) objects
I'll add context so we don't need to have the bugreport script memorized in order to read a bugreport :)
Quoted 11 lines
done
echoecho "[Pack Data]"
ls -l "$GIT_DIR/objects/pack"
echoecho "[Object Info Data]"
ls -lR "$GIT_DIR/objects/info"
echo
Quoted 7 lines
echo "[Alternates File]"
echo "========"
cat "$GIT_DIR/objects/info/alternates"
echoThat last one will collect information on the commit-graph file, even if it is an incremental file.
Thanks Stolee, these are awesome and exactly the kind of feedback I was hoping for.
Quoted 5 lines
I think this is a great start, and I'll take some time later to try it out.
Thanks,
-Stolee
Awesome. I'm excited to hear how it goes.
- Emily
Quoted 20 lines · Emily Shaffer
Documentation/git-bugreport.txt new file +14
-0@@ -0,0 +1,48 @@ +git-bugreport(1) +================ + +NAME +---- +git-bugreport - Collect information for user to file a bug report + +SYNOPSIS +-------- +[verse] +'git bugreport' [-o | --output <path>] + +DESCRIPTION +-----------
Documentation/git-bugreport.txt line 24
Quoted 10 lines
Documentation/git-bugreport.txt new file +10
-0+Captures information about the user's machine, Git client, and repository state, +as well as a form requesting information about the behavior the user observed, +into a single text file which the user can then share, for example to the Git +mailing list, in order to report an observed bug. + +The following information is requested from the user: + + - Reproduction steps + - Expected behavior + - Actual behavior
+ - How the above two are different
It is often helpful to have users explain how the expected and actual are different in their own words.
Documentation/git-bugreport.txt line 44
Quoted 3 lines
Documentation/git-bugreport.txt new file +3
-0+NOTE +---- +Bug reports can be sent to [email protected].
I am not sure if this belongs here.
Quoted 6 lines
git-bugreport.sh new file +0
-0
git-bugreport.sh line 10
Quoted 10 lines
git-bugreport.sh new file +10
-0@@ -0,0 +1,86 @@ +#!/bin/sh + +print_filenames_and_content() { +while read -r line; do + echo "$line"; + echo "========"; + cat "$line"; + echo; +done +}
Style.
- have SP on both sides of ()
- one more HT indent for the function body
- "do" on its own line
- no unnecessary semicolons when LF would do
You probably are better off asking xargs to do this instead of relying on "read -r".
find ... | xargs -n 1 sh -c 'echo "$1" && cat "$1"' -
or something like that, perhaps.
Quoted 8 lines
git-bugreport.sh new file +8
-0+ +generate_report_text() { + + # Generate a form for the user to fill out with their issue. + gettextln "Thank you for filling out a Git bug report!" + gettextln "Please answer the following questions to help us understand your issue." + echo + gettextln "What did you do before the bug happened? (Steps to reproduce your issue)"
git-bugreport.sh line 28
Quoted 10 lines
git-bugreport.sh new file +10
-0+ echo + gettextln "What did you expect to happen? (Expected behavior)" + echo + gettextln "What happened instead? (Actual behavior)" + echo + gettextln "Anything else you want to add:" + echo + gettextln "Please review the rest of the bug report below." + gettextln "You can delete any lines you don't wish to send." + echo
Would we on the receiving end be able to tell these section headers in translated to 47 different languages? I am sure that i18n is used here to encourage non-C-locale users to file bugs in their own languages, but are we prepared to react to them?
git-bugreport.sh line 34
Quoted 5 lines
git-bugreport.sh new file +5
-0+ echo "[System Information]" + git version --build-options + uname -a + curl-config --version + ldd --version
"curl-config: command not found" may be clear enough, but would there be a case where errors from two or more consecutive commands in the above sequence would make the output confusing to the person sitting on the receiving end? Would it help, as a convention, to always ahve "echo [what am I reporting]" before each of these commands?
git-bugreport.sh line 80
Quoted 2 lines
git-bugreport.sh new file +2
-0+# Create bugreport file +BUGREPORT_FILE="$basedir/git-bugreport-$(whoami)-$(hostname)-$(date -Iminutes)"
How portable is -Iminutes?
t/t0091-bugreport.sh line 1
Quoted 7 lines
t/t0091-bugreport.sh new file +1
-0@@ -0,0 +1,41 @@ +#!/bin/bash
Make sure /bin/sh suffices to run the test script.
t/t0091-bugreport.sh line 13
Quoted 11 lines
t/t0091-bugreport.sh new file +11
-0+test_description='git bugreport' + +. ./test-lib.sh + +# Headers "[System Info]" will be followed by a non-empty line if we put some +# information there; we can make sure all our headers were followed by some +# information to check if the command was successful. +HEADER_PATTERN="^\[.*\]$" +check_all_headers_populated() { + while read -r line; do + if [$(grep $HEADER_PATTERN $line)]; then
Documentation/CodingGuidelines
I am not sure if the traits this test script checks about the contents of the output is all that interesting. Whenever we add new sections to the main command because we want other kinds of information collected, we'd update this test script because otherwise the test would fail, but would it result in quality bugreport tool, or is it just additional busywork?
If we decide later that a header and its body needs to be separated with a blank line for better readablity, the check done here would also need to be updated, but again, that does not feel like anything more than just busywork to me.
The tests for "-o" and unknown options do make sense, though.
Thanks.
Quoted 35 lines
t/t0091-bugreport.sh new file +35
-0+# Headers "[System Info]" will be followed by a non-empty line if we put some +# information there; we can make sure all our headers were followed by some +# information to check if the command was successful. +HEADER_PATTERN="^\[.*\]$" +check_all_headers_populated() { + while read -r line; do + if [$(grep $HEADER_PATTERN $line)]; then + read -r nextline + if [-z $nextline]; then + return 1; + fi + fi + done +} + +test_expect_success 'creates a report with content in the right places' ' + git bugreport && + check_all_headers_populated <git-bugreport-* && + rm git-bugreport-* +' + +test_expect_success '--output puts the report in the provided dir' ' + mkdir foo/ && + git bugreport -o foo/ && + test -f foo/git-bugreport-* && + rm -fr foo/ +' + +test_expect_success 'incorrect arguments abort with usage' ' + test_must_fail git bugreport --false 2>output && + grep usage output && + test ! -f git-bugreport-* +' + +test_done
Quoted 35 lines · Junio C Hamano
Quoted 20 lines · Emily Shaffer
Documentation/git-bugreport.txt new file +14
-0@@ -0,0 +1,48 @@ +git-bugreport(1) +================ + +NAME +---- +git-bugreport - Collect information for user to file a bug report + +SYNOPSIS +-------- +[verse] +'git bugreport' [-o | --output <path>] + +DESCRIPTION +-----------Documentation/git-bugreport.txt line 24
Quoted 10 lines
Documentation/git-bugreport.txt new file +10
-0+Captures information about the user's machine, Git client, and repository state, +as well as a form requesting information about the behavior the user observed, +into a single text file which the user can then share, for example to the Git +mailing list, in order to report an observed bug. + +The following information is requested from the user: + + - Reproduction steps + - Expected behavior + - Actual behavior+ - How the above two are different
Quoted 2 lines
It is often helpful to have users explain how the expected and actual are different in their own words.
Good point; added.
Quoted 6 lines
Documentation/git-bugreport.txt line 44
Quoted 4 lines
Documentation/git-bugreport.txt new file +3
-0+NOTE +---- +Bug reports can be sent to [email protected].I am not sure if this belongs here.
Sure, I wasn't certain either. Would you rather I remove the "what to do with this bugreport" NOTE section entirely? I worry that folks will think this generated text file is being magically reported behind the scenes, or that they won't know what to do with the file now that it's created.
I suppose I did mention in the DESCRIPTION section that the user can share the generated text file to report a bug. Maybe I should remove "for example to the Git ML" from there as well.
Quoted 25 lines
Quoted 7 lines
git-bugreport.sh new file +0
-0git-bugreport.sh line 10
Quoted 10 lines
git-bugreport.sh new file +10
-0@@ -0,0 +1,86 @@ +#!/bin/sh + +print_filenames_and_content() { +while read -r line; do + echo "$line"; + echo "========"; + cat "$line"; + echo; +done +}Style.
- have SP on both sides of ()
- one more HT indent for the function body
- "do" on its own line
- no unnecessary semicolons when LF would do
Quoted 6 lines
You probably are better off asking xargs to do this instead of relying on "read -r".
find ... | xargs -n 1 sh -c 'echo "$1" && cat "$1"' -or something like that, perhaps.
Hm. In responding to Stolee's comments, I ended up replacing "cat $line" with another function, and it seems xargs isn't happy directing to functions which haven't been exported, because it spins a new process for each argument it's passed. I had originally intended to do this with xargs but then didn't want to pack many lines into one xargs call.
The way the code is in PS1 I think it's a reasonable replacement, but the code in PS2 is less so - there's a check for whether a file given is binary or whether it exists at all (the former for hooks, which I think you were worried about, and the latter for the alternates file, which I found I do not have.)
Why is "read -r" unreliable? Is it better to export a function and use xargs, compared to using "read -r" in a loop like this? I was worried that exporting a function is similar to namespace pollution in C++, but maybe I'm mistaken.
Quoted 16 lines
Quoted 16 lines
git-bugreport.sh new file +15
-0+ +generate_report_text() { + + # Generate a form for the user to fill out with their issue. + gettextln "Thank you for filling out a Git bug report!" + gettextln "Please answer the following questions to help us understand your issue." + echo + gettextln "What did you do before the bug happened? (Steps to reproduce your issue)" + echo + gettextln "What did you expect to happen? (Expected behavior)" + echo + gettextln "What happened instead? (Actual behavior)" + echo + gettextln "Anything else you want to add:" + echo
Quoted 8 lines
git-bugreport.sh line 28
Quoted 3 lines
git-bugreport.sh new file +3
-0+ gettextln "Please review the rest of the bug report below." + gettextln "You can delete any lines you don't wish to send." + echoWould we on the receiving end be able to tell these section headers in translated to 47 different languages? I am sure that i18n is used here to encourage non-C-locale users to file bugs in their own languages, but are we prepared to react to them?
I hope that we could accept reports in many languages with online translation as a guide, but practically speaking, machine-learning-powered online translation may not be as good for technical topics as it is for conversational.
At the same time, it seems unnecessarily restrictive to ask for bugs to come in English. My heart thinks it's better to encourage other languages too.
Quoted 13 lines
git-bugreport.sh line 34
Quoted 6 lines
git-bugreport.sh new file +5
-0+ echo "[System Information]" + git version --build-options + uname -a + curl-config --version + ldd --version"curl-config: command not found" may be clear enough, but would there be a case where errors from two or more consecutive commands in the above sequence would make the output confusing to the person sitting on the receiving end? Would it help, as a convention, to always ahve "echo [what am I reporting]" before each of these commands?
Yeah, I think that's a good point.
In responding to Stolee's review, I mentioned adding additional context to some command's output so that "those reviewing the report don't need to be intimately familiar with what the bugreport script runs". But it might actually be most useful to echo the command that's being run everywhere.
I imagine it's fine to include the [] header as well, to make it easier for the user to understand what was generated.
Quoted 5 lines
git-bugreport.sh line 80
Quoted 3 lines
git-bugreport.sh new file +2
-0+# Create bugreport file +BUGREPORT_FILE="$basedir/git-bugreport-$(whoami)-$(hostname)-$(date -Iminutes)"How portable is -Iminutes?
Ah, it seems while it works in MingW and bash, it doesn't work in zsh. I'll figure something else out. Thanks.
Quoted 10 lines
t/t0091-bugreport.sh line 1
Quoted 8 lines
t/t0091-bugreport.sh new file +1
-0@@ -0,0 +1,41 @@ +#!/bin/bashMake sure
/bin/shsuffices to run the test script.
Will do. Thanks.
Quoted 15 lines
t/t0091-bugreport.sh line 13
Quoted 12 lines
t/t0091-bugreport.sh new file +11
-0+test_description='git bugreport' + +. ./test-lib.sh + +# Headers "[System Info]" will be followed by a non-empty line if we put some +# information there; we can make sure all our headers were followed by some +# information to check if the command was successful. +HEADER_PATTERN="^\[.*\]$" +check_all_headers_populated() { + while read -r line; do + if [$(grep $HEADER_PATTERN $line)]; then
Documentation/CodingGuidelines
Quoted 6 lines
I am not sure if the traits this test script checks about the contents of the output is all that interesting. Whenever we add new sections to the main command because we want other kinds of information collected, we'd update this test script because otherwise the test would fail, but would it result in quality bugreport tool, or is it just additional busywork?
I'm a little confused about this - I checked for a general [...] header standing alone on a line specifically so that we wouldn't need to update the test script when adding a new section. What do you see that will grow stale?
Quoted 5 lines
If we decide later that a header and its body needs to be separated with a blank line for better readablity, the check done here would also need to be updated, but again, that does not feel like anything more than just busywork to me.
Sure, I agree with that.
So, what's your suggestion? Not to check the output at all? (This may actually be fine; it occurred to me while reading your review that if a user is filing a bug report about something, one of the diagnostic commands in bugreport might be what's broken for them. So perhaps it should be tolerant to missing information...)
Quoted 4 lines
The tests for "-o" and unknown options do make sense, though.
Thanks.
Thanks for reviewing.
- Emily
Quoted 8 lines · Emily Shaffer
Quoted 5 lines
Documentation/git-bugreport.txt line 44
Quoted 3 lines
Documentation/git-bugreport.txt new file +3
-0+NOTE +---- +Bug reports can be sent to [email protected].I am not sure if this belongs here.
Sure, I wasn't certain either. Would you rather I remove the "what to do with this bugreport" NOTE section entirely?
Not really. You are invoking an editor to let the user edit the pre-populated report, and I would imagine that a comment in that file would be the best place to give instructions, not a manpage for "git bugreport" command.
Quoted 5 lines
So, what's your suggestion? Not to check the output at all? (This may actually be fine; it occurred to me while reading your review that if a user is filing a bug report about something, one of the diagnostic commands in bugreport might be what's broken for them. So perhaps it should be tolerant to missing information...)
Right.
Quoted 8 lines · Junio C Hamano
Quoted 6 lines · Emily Shaffer
Quoted 5 lines
Documentation/git-bugreport.txt line 44
Quoted 3 lines
Documentation/git-bugreport.txt new file +3
-0+NOTE +---- +Bug reports can be sent to [email protected].I am not sure if this belongs here.
Quoted 7 lines
Quoted 2 lines
Sure, I wasn't certain either. Would you rather I remove the "what to do with this bugreport" NOTE section entirely?
Not really. You are invoking an editor to let the user edit the pre-populated report, and I would imagine that a comment in that file would be the best place to give instructions, not a manpage for "git bugreport" command.
Oh, I see! In that case, do you still want the Git mailing list address shown in the report text?
Quoted 8 lines
Quoted 6 lines
So, what's your suggestion? Not to check the output at all? (This may actually be fine; it occurred to me while reading your review that if a user is filing a bug report about something, one of the diagnostic commands in bugreport might be what's broken for them. So perhaps it should be tolerant to missing information...)
Right.
Ok, will do.
v2 cover add git-bugreport tool
In reply to Emily ShafferHaving read Dscho's response after already writing this reroll, I think it is a compelling case to rewrite as a C builtin, although I'm worried a little about the best way to continue to call porcelain commands to protect against bitrot.
Regardless, I think the proof of concept in patch 2/2 still deserves review; likely a reimplementation in C would see git-bugreport-config-whitelist generated in a similar way, but as a header instead of a textfile. So I'm still very interested in comments on the direction and the Makefile changes there.
The patch 1/2 changes are primarily based on Stolee and Junio's suggestions; thanks both. There's some rewording of manpage and commit message, as well as fixes when I realized this command did not work as expected from a worktree created with git worktree add. I also moved the reflog contents to the very bottom of the output as they're likely harmless, but very verbose; I didn't want the user to stop reading and never examine the shorter and scarier generated output, like the config.
Thanks for thoughts.
- Emily
Emily Shaffer (2):
bugreport: add tool to generate debugging info
bugreport: generate config whitelist based on docs
.gitignore | 2 + Documentation/config/sendemail.txt | 68 ++++++------ Documentation/git-bugreport.txt | 51 +++++++++ Makefile | 10 +- bugreport-generate-config-whitelist.sh | 4 + command-list.txt | 1 + git-bugreport.sh | 139 +++++++++++++++++++++++++ git-sh-setup.sh | 2 + t/t0091-bugreport.sh | 25 +++++ 9 files changed, 267 insertions(+), 35 deletions(-) create mode 100644 Documentation/git-bugreport.txt create mode 100755 bugreport-generate-config-whitelist.sh create mode 100755 git-bugreport.sh create mode 100755 t/t0091-bugreport.sh
Signature · 1 line
2.23.0.rc1.153.gdeed80330f-goog
PATCH v2 1/2 bugreport: add tool to generate debugging info
Make it easier for users who encounter a bug to send a report by collecting some state information, intended to be viewed by humans familiar with Git.
Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Also, teach Git to write down its own version, the version of some of its dependencies, the operating system information, select entries from the gitconfig, the configured hooks, and the contents of $GIT_DIR/logs. Finally, make sure Git asks the user to review the contents of the report after it's generated.
If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.
Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.
--- .gitignore | 1 + Documentation/git-bugreport.txt | 51 ++++++++++++ Makefile | 1 + command-list.txt | 1 + git-bugreport-config-whitelist | 22 +++++ git-bugreport.sh | 139 ++++++++++++++++++++++++++++++++ git-sh-setup.sh | 2 + t/t0091-bugreport.sh | 25 ++++++ 8 files changed, 242 insertions(+) create mode 100644 Documentation/git-bugreport.txt create mode 100644 git-bugreport-config-whitelist create mode 100755 git-bugreport.sh create mode 100755 t/t0091-bugreport.sh
.gitignore +1 -0
@@ -25,6 +25,7 @@ /git-bisect--helper /git-blame /git-branch +/git-bugreport /git-bundle /git-cat-file /git-check-attr
Documentation/git-bugreport.txt new file +51 -0
51 lines not on this page · raw message
Makefile +1 -0
@@ -606,6 +606,7 @@ TEST_PROGRAMS_NEED_X = unexport CDPATH SCRIPT_SH += git-bisect.sh +SCRIPT_SH += git-bugreport.sh SCRIPT_SH += git-difftool--helper.sh SCRIPT_SH += git-filter-branch.sh SCRIPT_SH += git-merge-octopus.sh
command-list.txt +1 -0
@@ -54,6 +54,7 @@ git-archive mainporcelain git-bisect mainporcelain info git-blame ancillaryinterrogators complete git-branch mainporcelain history +git-bugreport ancillaryinterrogators git-bundle mainporcelain git-cat-file plumbinginterrogators git-check-attr purehelpers
git-bugreport-config-whitelist new file +22 -0
@@ -0,0 +1,22 @@ +http.version +protocol.version +protocol.persistent-https.allow +protocol.rpc.allow +protocol.sso.allow +submodule.repolike +trace2.eventtarget +trace2.configparams +color.ui +core.pager +sendemail.smtpencryption +sendemail.smtpserver +sendemail.smtpserverport +sendemail.smtpsslcertpath +credential.helper +merge.tool +grep.linenumber +rerere.enabled +core.repositoryformatversion +core.filemode +core.bare +core.logallrefupdates
git-bugreport.sh new file +139 -0
139 lines not on this page · raw message
git-sh-setup.sh +2 -0
@@ -379,6 +379,8 @@ git_dir_init () { exit 1 } : "${GIT_OBJECT_DIRECTORY="$(git rev-parse --git-path objects)"}" + : "${GIT_LOGS_DIRECTORY="$(git rev-parse --git-path logs)"}" + : "${GIT_HOOKS_DIRECTORY="$(git rev-parse --git-path hooks)"}" } if test -z "$NONGIT_OK"
t/t0091-bugreport.sh new file +25 -0
@@ -0,0 +1,25 @@ +#!/bin/sh + +test_description='git bugreport' + +. ./test-lib.sh + +test_expect_success 'creates a report without crashing' ' + git bugreport && + rm git-bugreport-* +' + +test_expect_success '--output puts the report in the provided dir' ' + mkdir foo/ && + git bugreport -o foo/ && + test -f foo/git-bugreport-* && + rm -fr foo/ +' + +test_expect_success 'incorrect arguments abort with usage' ' + test_must_fail git bugreport --false 2>output && + grep usage output && + test ! -f git-bugreport-* +' + +test_done
Signature · 1 line
2.23.0.rc1.153.gdeed80330f-goog
PATCH v2 2/2 bugreport: generate config whitelist based on docs
In reply to Emily ShafferAdd a new step to the build to generate a whitelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the whitelist by annotating their documentation in Documentation/config with the line "// bugreport-include".
Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blacklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a whitelist is more reliable.
In order to prevent staleness of the whitelist, add a mechanism to generate the whitelist from annotations in the config documentation, where contributors are already used to documenting their new config keys.
Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.
--- .gitignore | 1 + Documentation/config/sendemail.txt | 68 +++++++++++++------------- Makefile | 9 +++- bugreport-generate-config-whitelist.sh | 4 ++ git-bugreport-config-whitelist | 22 --------- 5 files changed, 47 insertions(+), 57 deletions(-) create mode 100755 bugreport-generate-config-whitelist.sh delete mode 100644 git-bugreport-config-whitelist
.gitignore +1 -0
@@ -26,6 +26,7 @@ /git-blame /git-branch /git-bugreport +/git-bugreport-config-whitelist /git-bundle /git-cat-file /git-check-attr
Documentation/config/sendemail.txt +34 -34
97 lines not on this page · raw message
Makefile +8 -1
@@ -639,6 +639,10 @@ SCRIPT_PYTHON += git-p4.py SCRIPT_SH_GEN = $(patsubst %.sh,%,$(SCRIPT_SH)) SCRIPT_PERL_GEN = $(patsubst %.perl,%,$(SCRIPT_PERL)) SCRIPT_PYTHON_GEN = $(patsubst %.py,%,$(SCRIPT_PYTHON)) +SCRIPT_DEPENDENCIES = git-bugreport-config-whitelist + +$(SCRIPT_DEPENDENCIES): Documentation/config/*.txt + sh bugreport-generate-config-whitelist.sh # Individual rules to allow e.g. # "make -C ../.. SCRIPT_PERL=contrib/foo/bar.perl build-perl-script"
@@ -656,17 +660,20 @@ install-perl-script: $(SCRIPT_PERL_GEN) install-python-script: $(SCRIPT_PYTHON_GEN) $(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' -.PHONY: clean-perl-script clean-sh-script clean-python-script +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies clean-sh-script: $(RM) $(SCRIPT_SH_GEN) clean-perl-script: $(RM) $(SCRIPT_PERL_GEN) clean-python-script: $(RM) $(SCRIPT_PYTHON_GEN) +clean-script-dependencies: + $(RM) $(SCRIPT_DEPENDENCIES) SCRIPTS = $(SCRIPT_SH_GEN) \ $(SCRIPT_PERL_GEN) \ $(SCRIPT_PYTHON_GEN) \ + $(SCRIPT_DEPENDENCIES) \ git-instaweb ETAGS_TARGET = TAGS
bugreport-generate-config-whitelist.sh new file +4 -0
@@ -0,0 +1,4 @@ +#!/bin/sh + +grep -RhPo ".*(?=:: \/\/ bugreport-include)" Documentation/config \ + >git-bugreport-config-whitelist
git-bugreport-config-whitelist deleted +0 -22
@@ -1,22 +0,0 @@ -http.version -protocol.version -protocol.persistent-https.allow -protocol.rpc.allow -protocol.sso.allow -submodule.repolike -trace2.eventtarget -trace2.configparams -color.ui -core.pager -sendemail.smtpencryption -sendemail.smtpserver -sendemail.smtpserverport -sendemail.smtpsslcertpath -credential.helper -merge.tool -grep.linenumber -rerere.enabled -core.repositoryformatversion -core.filemode -core.bare -core.logallrefupdates
Signature · 1 line
2.23.0.rc1.153.gdeed80330f-goog
Re: [PATCH v2 2/2] bugreport: generate config whitelist based on docs
Quoted 6 lines · Emily Shaffer
Add a new step to the build to generate a whitelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the whitelist by annotating their documentation in
Documentation/configwith the line "// bugreport-include".
These "// bugreport-include" show up in the rendered manpages, both with AsciiDoc and Asciidoctor. :-(
Documentation/config/sendemail.txt line 2
Quoted 6 lines
Documentation/config/sendemail.txt +1
-1@@ -1,63 +1,63 @@ -sendemail.identity:: +sendemail.identity:: // bugreport-exclude A configuration identity. When given, causes values in the
If I put each comment on a line of its own (after the config option, but I suppose before would work the same way), Asciidoctor truly ignores them and everything's fine. And AsciiDoc renders this one and others like it ok.
Documentation/config/sendemail.txt line 5
Quoted 6 lines
Documentation/config/sendemail.txt +3
-3-sendemail.aliasesFile:: -sendemail.aliasFileType:: -sendemail.annotate:: +sendemail.aliasesFile:: // bugreport-exclude +sendemail.aliasFileType:: // bugreport-exclude +sendemail.annotate:: // bugreport-include
However, AsciiDoc (version 8.6.10) seems to effectively replace those comments with an empty line during processing, and it makes quite the difference here. Instead of these appearing in a compact comma-separated list, they are treated as individual items in the description list with no supporting content.
FWIW, I like the idea of annotating things here to make it harder to forget this whitelisting when adding a new config item.
Below is what I came up with as an alternative approach. Feel free to steal, squash and/or ignore as you see fit.
BTW, I'm not sure the grep invocation is portable (-R ? -h ? -P ? -o ?). We should probably also do the usual "create a candidate output file, then move it into place" dance for robustness.
I do think we should test the generated whitelist in some minimal way, e.g., to check that it does contain something which objectively belongs in the whitelist and -- more importantly IMHO -- does *not* contain something that shouldn't be there, such as sendemail.smtpPass.
Martin
-- >8 --
Subject: [PATCH] Use a bugreport:include/exclude macro instead
Implement a no-op "bugreport" macro for use as "bugreport:include[x]" to annotate the config keys that should be included in the automatically generated whitelist. Use "exclude" for the others.
With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc seems to want something between the brackets. A bit unfortunate, but not a huge problem -- we'll just provide an "x".
"doc-diff" reports that this macro doesn't render at all. That is, these are both empty after this commit:
cd Documentation ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD ./doc-diff --asciidoc :/"bugreport: add tool" HEAD
Diffing the rendered HTML shows that there is some small amount of whitespace and comments added. That shouldn't be a problem.
We could perhaps let the implementation verify that the "action" is one of "include" and "exclude". For the Asciidoctor implementation that should be straightforward, but for AsciiDoc I don't immediately know how to do it. Anyway, if someone stumbles on the keyboard and writes "bugreport:icndule", they'll "only" miss out on the config key being included in the whitelist. If this were a blacklist, the consequences of a misspelled target could be a lot more severe.
Disclaimer: This is the outcome of a combination of copy-paste and trial and error -- better solutions might be possible...
---
I suppose this could also be "annotate:bugreport[ignore]" to avoid the "[x]" and to place all of this under a more general "annotate" macro.
Documentation/asciidoc.conf | 8 +++ Documentation/asciidoctor-extensions.rb | 8 +++ Documentation/config/sendemail.txt | 68 ++++++++++++------------- bugreport-generate-config-whitelist.sh | 2 +- 4 files changed, 51 insertions(+), 35 deletions(-)
Documentation/asciidoc.conf +8 -0
@@ -6,9 +6,13 @@ # # Show Git link as: <command>(<section>); if section is defined, else just show # the command. +# +# The bugreport macro does nothing as far as rendering is +# concerned -- we just grep for it in the sources. [macros] (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]= +(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]= [attributes] asterisk=*
@@ -28,6 +32,8 @@ ifdef::backend-docbook[] {0#<citerefentry>} {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>} {0#</citerefentry>} +[bugreport-inlinemacro] +{0#} endif::backend-docbook[] ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[] git-relative-html-prefix= [linkgit-inlinemacro] <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a> +[bugreport-inlinemacro] +<!-- --> endif::backend-xhtml11[]
Documentation/asciidoctor-extensions.rb +8 -0
@@ -20,9 +20,17 @@ module Git end end end + class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor + def process(parent, action, attrs) + "" + end + end end end Asciidoctor::Extensions.register do inline_macro Git::Documentation::LinkGitProcessor, :linkgit + # The bugreport macro does nothing as far as rendering is + # concerned -- we just grep for it in the sources. + inline_macro Git::Documentation::BugReportProcessor, :bugreport end
Documentation/config/sendemail.txt +34 -34
97 lines not on this page · raw message
bugreport-generate-config-whitelist.sh +1 -1
@@ -1,4 +1,4 @@ #!/bin/sh -grep -RhPo ".*(?=:: \/\/ bugreport-include)" Documentation/config \ +grep -RhPo ".*(?= +bugreport:include.* ::)" Documentation/config \ >git-bugreport-config-whitelist
Signature · 1 line
2.23.0
Re: [PATCH v2 2/2] bugreport: generate config whitelist based on docs
Quoted 10 lines · Martin Ågren
Quoted 6 lines · Emily Shaffer
Add a new step to the build to generate a whitelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the whitelist by annotating their documentation in
Documentation/configwith the line "// bugreport-include".These "// bugreport-include" show up in the rendered manpages, both with AsciiDoc and Asciidoctor. :-(
Hmm, I don't see it in the troff or the HTML with asciidoc using make
- but I do see with asciidoctor, or asciidoc -d html5. Nice catch, thanks.
Quoted 26 lines
Documentation/config/sendemail.txt line 2
Quoted 7 lines
Documentation/config/sendemail.txt +1
-1@@ -1,63 +1,63 @@ -sendemail.identity:: +sendemail.identity:: // bugreport-exclude A configuration identity. When given, causes values in theIf I put each comment on a line of its own (after the config option, but I suppose before would work the same way), Asciidoctor truly ignores them and everything's fine. And AsciiDoc renders this one and others like it ok.
Documentation/config/sendemail.txt line 5
Quoted 6 lines
Documentation/config/sendemail.txt +3
-3-sendemail.aliasesFile:: -sendemail.aliasFileType:: -sendemail.annotate:: +sendemail.aliasesFile:: // bugreport-exclude +sendemail.aliasFileType:: // bugreport-exclude +sendemail.annotate:: // bugreport-includeHowever, AsciiDoc (version 8.6.10) seems to effectively replace those comments with an empty line during processing, and it makes quite the difference here. Instead of these appearing in a compact comma-separated list, they are treated as individual items in the description list with no supporting content.
Quoted 5 lines
FWIW, I like the idea of annotating things here to make it harder to forget this whitelisting when adding a new config item.
Below is what I came up with as an alternative approach. Feel free to steal, squash and/or ignore as you see fit.
This is cool - I'll add this to the commit chain and build on top of it. Thanks!
Quoted 2 lines
BTW, I'm not sure the grep invocation is portable (-R ? -h ? -P ? -o ?).
Yeah, I'll see what I can do about that (recursive, no filename, Perl-compatible regex, match only) - I can probably replace this with something like find | xargs pgrep - I'll keep digging. Thanks.
Quoted 7 lines
We should probably also do the usual "create a candidate output file, then move it into place" dance for robustness.
I do think we should test the generated whitelist in some minimal way, e.g., to check that it does contain something which objectively belongs in the whitelist and -- more importantly IMHO -- does *not* contain something that shouldn't be there, such as sendemail.smtpPass.
Good point, I'll add a test for this.
Thanks very much for this!
- Emily
v3 cover add git-bugreport tool
In reply to Emily ShafferThanks for the patience with the long wait, all. Here's an attempt at the rewrite in C; I think it does verbatim what the sh version did except that this doesn't print the reflog - Jonathan Nieder was good enough to point out to me that folks probably don't want to share their commit subjects all willy-nilly if they're working on something secretive.
For the most part I hoped to do this stuff stupidly - that is to say, independent of the libraries within Git - in an effort to both avoid bitrot, and see what is (as opposed to what the libraries think should be).
The one common patch with v2 is "generate config whitelist based on docs", which I have not changed at all.
I look forward to everyone's suggestions.
- Emily
Emily Shaffer (9):
bugreport: add tool to generate debugging info
bugreport: generate config whitelist based on docs
bugreport: add version and system information
bugreport: add config values from whitelist
bugreport: collect list of populated hooks
bugreport: count loose objects
bugreport: add packed object summary
bugreport: list contents of $OBJDIR/info
bugreport: print contents of alternates file
.gitignore | 1 + Documentation/config/sendemail.txt | 68 +++--- Makefile | 11 +- bugreport-generate-config-whitelist.sh | 4 + bugreport.c | 314 +++++++++++++++++++++++++ bugreport.h | 44 ++++ builtin.h | 1 + builtin/bugreport.c | 90 +++++++ git.c | 1 + 9 files changed, 499 insertions(+), 35 deletions(-) create mode 100755 bugreport-generate-config-whitelist.sh create mode 100644 bugreport.c create mode 100644 bugreport.h create mode 100644 builtin/bugreport.c
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
PATCH v3 1/9 bugreport: add tool to generate debugging info
Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.
If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.
Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.
--- Makefile | 1 + builtin.h | 1 + builtin/bugreport.c | 50 +++++++++++++++++++++++++++++++++++++++++++++ git.c | 1 + 4 files changed, 53 insertions(+) create mode 100644 builtin/bugreport.c
Makefile +1 -0
@@ -1039,6 +1039,7 @@ BUILTIN_OBJS += builtin/archive.o BUILTIN_OBJS += builtin/bisect--helper.o BUILTIN_OBJS += builtin/blame.o BUILTIN_OBJS += builtin/branch.o +BUILTIN_OBJS += builtin/bugreport.o BUILTIN_OBJS += builtin/bundle.o BUILTIN_OBJS += builtin/cat-file.o BUILTIN_OBJS += builtin/check-attr.o
builtin.h +1 -0
@@ -135,6 +135,7 @@ int cmd_archive(int argc, const char **argv, const char *prefix);int cmd_bisect__helper (int argc,const char **argv,const char *prefix);int cmd_blame (int argc,const char **argv,const char *prefix);int cmd_branch (int argc,const char **argv,const char *prefix); +int cmd_bugreport (int argc,const char **argv,const char *prefix);int cmd_bundle (int argc,const char **argv,const char *prefix);int cmd_cat_file (int argc,const char **argv,const char *prefix);int cmd_checkout (int argc,const char **argv,const char *prefix);
builtin/bugreport.c new file +50 -0
50 lines not on this page · raw message
git.c +1 -0
@@ -473,6 +473,7 @@ static struct cmd_struct commands[] = { { "bisect--helper", cmd_bisect__helper, RUN_SETUP }, { "blame", cmd_blame, RUN_SETUP }, { "branch", cmd_branch, RUN_SETUP | DELAY_PAGER_CONFIG }, + { "bugreport", cmd_bugreport, RUN_SETUP }, { "bundle", cmd_bundle, RUN_SETUP_GENTLY | NO_PARSEOPT }, { "cat-file", cmd_cat_file, RUN_SETUP }, { "check-attr", cmd_check_attr, RUN_SETUP },
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Quoted 52 lines · Emily Shaffer
Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.
If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.
Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.
Signed-off-by Emily Shaffer <[email protected]>--- Makefile | 1 + builtin.h | 1 + builtin/bugreport.c | 50 +++++++++++++++++++++++++++++++++++++++++++++ git.c | 1 + 4 files changed, 53 insertions(+) create mode 100644 builtin/bugreport.cMakefile +1
-0@@ -1039,6 +1039,7 @@ BUILTIN_OBJS += builtin/archive.o BUILTIN_OBJS += builtin/bisect--helper.o BUILTIN_OBJS += builtin/blame.o BUILTIN_OBJS += builtin/branch.o +BUILTIN_OBJS += builtin/bugreport.o BUILTIN_OBJS += builtin/bundle.o BUILTIN_OBJS += builtin/cat-file.o BUILTIN_OBJS += builtin/check-attr.obuiltin.h +1
-0@@ -135,6 +135,7 @@ int cmd_archive(int argc, const char **argv, const char *prefix);int cmd_bisect__helper (int argc,const char **argv,const char *prefix);int cmd_blame (int argc,const char **argv,const char *prefix);int cmd_branch (int argc,const char **argv,const char *prefix); +int cmd_bugreport (int argc,const char **argv,const char *prefix);int cmd_bundle (int argc,const char **argv,const char *prefix);int cmd_cat_file (int argc,const char **argv,const char *prefix);int cmd_checkout (int argc,const char **argv,const char *prefix);builtin/bugreport.c new file +0
-0
builtin/bugreport.c line 6
Quoted 6 lines
builtin/bugreport.c new file +6
-0@@ -0,0 +1,50 @@ +#include "builtin.h" +#include "stdio.h" +#include "strbuf.h" +#include "time.h" + +int get_bug_template (struct strbuf *template)
Compilation fails here for me with:
builtin/bugreport.c:6:5: error: no previous prototype
for ‘get_bug_template’ [-Werror=missing-prototypes]
Can you make this function static?
Quoted 58 lines
builtin/bugreport.c new file +44
-0+{ +const char template_text[] = +"Thank you for filling out a Git bug report!\n" +"Please answer the following questions to help us understand your issue.\n" +"\n" +"What did you do before the bug happened? (Steps to reproduce your issue)\n" +"\n" +"What did you expect to happen? (Expected behavior)\n" +"\n" +"What happened instead? (Actual behavior)\n" +"\n" +"What's different between what you expected and what actually happened?\n" +"\n" +"Anything else you want to add:\n" +"\n" +"Please review the rest of the bug report below.\n" +"You can delete any lines you don't wish to send.\n" ; + +strbuf_reset (template); +strbuf_add (template, template_text,strlen (template_text)); +return 0 ; +} + +int cmd_bugreport (int argc,const char **argv,const char *prefix) +{ +struct strbuf buffer =STRBUF_INIT ; +struct strbuf report_path =STRBUF_INIT ; +FILE *report; +time_t now =time (NULL ); + +strbuf_addstr (&report_path,"git-bugreport-" ); +strbuf_addftime (&report_path,"%F" ,gmtime (&now),0 ,0 ); +strbuf_addstr (&report_path,".txt" ); + + report =fopen_for_writing (report_path.buf); + +get_bug_template (&buffer); +strbuf_write (&buffer, report); + +fclose (report); + +launch_editor (report_path.buf,NULL ,NULL ); +return 0 ; +}git.c +1
-0@@ -473,6 +473,7 @@ static struct cmd_struct commands[] = { { "bisect--helper", cmd_bisect__helper, RUN_SETUP }, { "blame", cmd_blame, RUN_SETUP }, { "branch", cmd_branch, RUN_SETUP | DELAY_PAGER_CONFIG }, + { "bugreport", cmd_bugreport, RUN_SETUP }, { "bundle", cmd_bundle, RUN_SETUP_GENTLY | NO_PARSEOPT }, { "cat-file", cmd_cat_file, RUN_SETUP }, { "check-attr", cmd_check_attr, RUN_SETUP },Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Can you also add /git-bugreport to .gitignore?
Quoted 3 lines · Emily Shaffer
Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.
It makes sense, but I do not think of any good reason why this should be implemented as a builtin. I'd expect it would probably need to collect more info on the running environment than otherwise necessary for the regular Git operation, and perhaps you'd want to even link with libraries that are not needed for the regular Git operation to achieve that.
Can you make it a standalone binary instead to avoid bloat of the main "git" binary?
Thanks.
Quoted 13 lines · Junio C Hamano
Quoted 3 lines · Emily Shaffer
Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.
It makes sense, but I do not think of any good reason why this should be implemented as a builtin. I'd expect it would probably need to collect more info on the running environment than otherwise necessary for the regular Git operation, and perhaps you'd want to even link with libraries that are not needed for the regular Git operation to achieve that.
Quoted 2 lines
Can you make it a standalone binary instead to avoid bloat of the main "git" binary?
Sure. This would fix some other issues (needing to link against curl to get the curl version, which is exactly what you implied). I wasn't certain which circumstances a standalone binary was preferred, but I agree with your reasoning here for sure.
- Emily
Hi,
Quoted 14 lines · Emily Shaffer
Quoted 13 lines · Junio C Hamano
Quoted 3 lines · Emily Shaffer
Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.
It makes sense, but I do not think of any good reason why this should be implemented as a builtin. I'd expect it would probably need to collect more info on the running environment than otherwise necessary for the regular Git operation, and perhaps you'd want to even link with libraries that are not needed for the regular Git operation to achieve that.
Quoted 7 lines
Quoted 2 lines
Can you make it a standalone binary instead to avoid bloat of the main "git" binary?
Sure. This would fix some other issues (needing to link against curl to get the curl version, which is exactly what you implied). I wasn't certain which circumstances a standalone binary was preferred, but I agree with your reasoning here for sure.
FWIW I disagree with the idea that a tiny built-in command like bugreport would "bloat" the main git binary.
In contrast, I think that stand-alone commands _do_ bloat. Look here:
$ ls -lh git-daemon.exe git-credential-store.exe -rwxr-xr-x 1 me 4096 1.8M Nov 6 13:43 git-credential-store.exe* -rwxr-xr-x 1 me 4096 1.8M Nov 6 13:43 git-daemon.exe*
In other words, even a super simple stand-alone like credential-store (the credential-store.c file has only 198 lines!) weighs in with almost two megabytes.
So I fear that the claim that a stand-alone command would add less bloat than a built-in one, especially for a relatively small thing like bugreport has it exactly backwards.
Ciao,
Dscho
Quoted 4 lines · Johannes Schindelin
FWIW I disagree with the idea that a tiny built-in command like
bugreportwould "bloat" the maingitbinary.In contrast, I think that stand-alone commands _do_ bloat. Look here:
I probably should have clarified that "bloat" in the context of this discussion is not about on-disk space. It is about resources needed to run "git status/diff/etc" that are everyday local commands that are expected to be lightweight, i.e. the same criteria applied when making the networking part (which the user expects to be coffee time) separate from them.
Hi Junio,
Quoted 5 lines · Junio C Hamano
Quoted 3 lines · Johannes Schindelin
FWIW I disagree with the idea that a tiny built-in command like
bugreportwould "bloat" the maingitbinary.
Quoted 8 lines
Quoted 1 line
In contrast, I think that stand-alone commands _do_ bloat. Look here:
I probably should have clarified that "bloat" in the context of this discussion is not about on-disk space. It is about resources needed to run "git status/diff/etc" that are everyday local commands that are expected to be lightweight, i.e. the same criteria applied when making the networking part (which the user expects to be coffee time) separate from them.
I guess I still do not understand.
Are you suggesting that bugreport would be unwelcome as a built-in? If so, I would really like to know why. Because I think it would make for a very fine built-in. I interact with users all the time, and a good tool to provide good information to accompany a bug report is definitely something that would improve the current situation.
Ciao,
Dscho
Hi Emily,
Quoted 2 lines · Emily Shaffer
[...] some other issues (needing to link against curl to get the curl version, which is exactly what you implied) [...]
I did suggest on IRC to teach git-remote-https an option where it prints the cURL version (and build options) and exits.
This would have the further advantage of making sure that the correct information is included. If you have two separate binaries that both link to cURL, they could still be linked statically, to different versions of cURL (this could happen e.g. if you have a git-remote-https from a different build in your path).
In short: I still think that it would make for a much better idea to query the git-remote-https executable for the version information than to link bugreport to libcurl.
Ciao,
Dscho
Quoted 2 lines · Johannes Schindelin
I did suggest on IRC to teach
git-remote-httpsan option where it prints the cURL version (and build options) and exits.
I like that. You ask the exact binary what (it thinks) it uses, so that there won't be skew between the view by "git remote-http" and "git bugreport" on the world.
Quoted 7 lines · Johannes Schindelin
Hi Emily,
Quoted 2 lines · Emily Shaffer
[...] some other issues (needing to link against curl to get the curl version, which is exactly what you implied) [...]
Quoted 8 lines
I did suggest on IRC to teach
git-remote-httpsan option where it prints the cURL version (and build options) and exits.This would have the further advantage of making sure that the correct information is included. If you have two separate binaries that both link to cURL, they could still be linked statically, to different versions of cURL (this could happen e.g. if you have a
git-remote-httpsfrom a different build in your path).
Yeah, it's a good point and an angle I hadn't thought of. Thanks.
Quoted 3 lines
In short: I still think that it would make for a much better idea to query the
git-remote-httpsexecutable for the version information than to linkbugreportto libcurl.
Will do - the git-bugreport standalone will invoke the git-remote-https standalone to ask for version info.
- Emily
Quoted 5 lines · Emily Shaffer
Quoted 2 lines
Can you make it a standalone binary instead to avoid bloat of the main "git" binary?
Sure. This would fix some other issues (needing to link against curl to get the curl version, which is exactly what you implied).
Hmph, I actually was not thinking about the cURL library.
I imagined that your tool may need to learn more about the system in order to make the report and for that there may be libraries that makes it easy than say reading directly from the /proc filesystem etc. that you may end up using. Unlike cURL, such a library would have no use in the rest of Git anywhere.
PATCH v3 2/9 bugreport: generate config whitelist based on docs
In reply to Emily ShafferAdd a new step to the build to generate a whitelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the whitelist by annotating their documentation in Documentation/config with the line "// bugreport-include".
Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blacklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a whitelist is more reliable.
In order to prevent staleness of the whitelist, add a mechanism to generate the whitelist from annotations in the config documentation, where contributors are already used to documenting their new config keys.
Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.
--- .gitignore | 1 + Documentation/config/sendemail.txt | 68 +++++++++++++------------- Makefile | 9 +++- bugreport-generate-config-whitelist.sh | 4 ++ 4 files changed, 47 insertions(+), 35 deletions(-) create mode 100755 bugreport-generate-config-whitelist.sh
.gitignore +1 -0
@@ -25,6 +25,7 @@ /git-bisect--helper /git-blame /git-branch +/git-bugreport /git-bundle /git-cat-file /git-check-attr
Documentation/config/sendemail.txt +34 -34
97 lines not on this page · raw message
Makefile +8 -1
@@ -634,6 +634,10 @@ SCRIPT_PYTHON += git-p4.py SCRIPT_SH_GEN = $(patsubst %.sh,%,$(SCRIPT_SH)) SCRIPT_PERL_GEN = $(patsubst %.perl,%,$(SCRIPT_PERL)) SCRIPT_PYTHON_GEN = $(patsubst %.py,%,$(SCRIPT_PYTHON)) +SCRIPT_DEPENDENCIES = git-bugreport-config-whitelist + +$(SCRIPT_DEPENDENCIES): Documentation/config/*.txt + sh bugreport-generate-config-whitelist.sh # Individual rules to allow e.g. # "make -C ../.. SCRIPT_PERL=contrib/foo/bar.perl build-perl-script"
@@ -651,17 +655,20 @@ install-perl-script: $(SCRIPT_PERL_GEN) install-python-script: $(SCRIPT_PYTHON_GEN) $(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' -.PHONY: clean-perl-script clean-sh-script clean-python-script +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies clean-sh-script: $(RM) $(SCRIPT_SH_GEN) clean-perl-script: $(RM) $(SCRIPT_PERL_GEN) clean-python-script: $(RM) $(SCRIPT_PYTHON_GEN) +clean-script-dependencies: + $(RM) $(SCRIPT_DEPENDENCIES) SCRIPTS = $(SCRIPT_SH_GEN) \ $(SCRIPT_PERL_GEN) \ $(SCRIPT_PYTHON_GEN) \ + $(SCRIPT_DEPENDENCIES) \ git-instaweb ETAGS_TARGET = TAGS
bugreport-generate-config-whitelist.sh new file +4 -0
@@ -0,0 +1,4 @@ +#!/bin/sh + +grep -RhPo ".*(?=:: \/\/ bugreport-include)" Documentation/config \ + >git-bugreport-config-whitelist
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 2/9] bugreport: generate config whitelist based on docs
Hi Emily,
bugreport-generate-config-whitelist.sh line 3
Quoted 9 lines · Emily Shaffer
bugreport-generate-config-whitelist.sh new file +3
-0@@ -0,0 +1,4 @@ +#!/bin/sh + +grep -RhPo ".*(?=:: \/\/ bugreport-include)" Documentation/config \
I am rather certain that -P is not supported by BSD grep (see
https://man.openbsd.org/grep).
Why not something portable, e.g.
find Documentation/config -type f -exec cat {} \; |
sed -n 's/^\(.*\):: \/\/ bugreport-include$/\1/p' \?
Ciao,
Dscho
Quoted 5 lines
bugreport-generate-config-whitelist.sh new file +1
-1+ >git-bugreport-config-whitelist --2.24.0.rc0.303.g954a862665-goog
PATCH v3 3/9 bugreport: add version and system information
In reply to Emily ShafferTeach bugreport how to collect the Git, curl, and ldd versions, as well as the uname string.
Learning the uname and versions in the user's environment will enable us to reproduce behavior reported by the user; the versions will also give us a good starting place while bisecting a newly introduced bug.
Put this functionality in a library, so that other commands can easily include this debug info if necessary.
--- Makefile | 1 + bugreport.c | 55 +++++++++++++++++++++++++++++++++++++++++++++ bugreport.h | 7 ++++++ builtin/bugreport.c | 13 +++++++++++ 4 files changed, 76 insertions(+) create mode 100644 bugreport.c create mode 100644 bugreport.h
Makefile +1 -0
@@ -844,6 +844,7 @@ LIB_OBJS += bisect.o LIB_OBJS += blame.o LIB_OBJS += blob.o LIB_OBJS += branch.o +LIB_OBJS += bugreport.o LIB_OBJS += bulk-checkin.o LIB_OBJS += bundle.o LIB_OBJS += cache-tree.o
bugreport.c new file +55 -0
55 lines not on this page · raw message
bugreport.h new file +7 -0
@@ -0,0 +1,7 @@ +#include "strbuf.h" + +/** +* Adds the Git version, `uname -a`, and `curl-config --version` to sys_info. +* The previous contents of sys_info will be discarded. +*/ +void get_system_info (struct strbuf *sys_info);
builtin/bugreport.c +13 -0
@@ -1,4 +1,5 @@#include "builtin.h" +#include "bugreport.h" #include "stdio.h" #include "strbuf.h" #include "time.h"
@@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)return 0 ; } +void add_header (FILE *report,const char *title) +{ +struct strbuf buffer =STRBUF_INIT ; +strbuf_addf (&buffer,"\n\n[%s]\n" , title); +strbuf_write (&buffer, report); +} +int cmd_bugreport (int argc,const char **argv,const char *prefix) {struct strbuf buffer =STRBUF_INIT ;
@@ -43,6 +51,11 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_bug_template (&buffer);strbuf_write (&buffer, report); +// add other contents +add_header (report,"System Info" ); +get_system_info (&buffer); +strbuf_write (&buffer, report); +fclose (report);launch_editor (report_path.buf,NULL ,NULL );
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 3/9] bugreport: add version and system information
Hi Emily,
Quoted 11 lines · Emily Shaffer
bugreport.c new file +5
-0@@ -0,0 +1,55 @@ +#include "cache.h" + +#include "bugreport.h" +#include "help.h" +#include "run-command.h"
bugreport.c line 15
Quoted 10 lines
bugreport.c new file +10
-0+#include "version.h" + +void get_system_info (struct strbuf *sys_info) +{ +struct child_process cp =CHILD_PROCESS_INIT ; +struct strbuf std_out =STRBUF_INIT ; + +strbuf_reset (sys_info); + +// get git version from native cmd
Please use C-style comments instead of C++ ones.
bugreport.c line 23
Quoted 8 lines
bugreport.c new file +8
-0+strbuf_addstr (sys_info,"git version: " ); +strbuf_addstr (sys_info, git_version_string); +strbuf_complete_line (sys_info); + +// system call for other version info +argv_array_push (&cp.args,"uname" ); +argv_array_push (&cp.args,"-a" ); +capture_command (&cp, &std_out,0 );
Mmmkay. I am not too much of a fan of relying on the uname executable, as it can very well be a 32-bit uname.exe on Windows, which obviously would _not_ report the architecture of the machine, but something misleading.
Why not use the uname() function (that we even define in compat/mingw.c)?
Also, why not include the same information as git version --build-options, most importantly GIT_HOST_CPU?
In any case, if you are capturing the output of uname -a, you definitely will want to pass the RUN_COMMAND_STDOUT_TO_STDERR flag (in case, say, the MSYS2 uname.exe fails with those dreaded Cygwin error messages like "*** fatal error - add_item ("\??\D:\git\installation\Git", "/", ...) failed, errno 1").
bugreport.c line 35
Quoted 12 lines
bugreport.c new file +12
-0+ +strbuf_addstr (sys_info,"uname -a: " ); +strbuf_addbuf (sys_info, &std_out); +strbuf_complete_line (sys_info); + +argv_array_clear (&cp.args); +strbuf_reset (&std_out); + + +argv_array_push (&cp.args,"curl-config" ); +argv_array_push (&cp.args,"--version" ); +capture_command (&cp, &std_out,0 );
This will be almost certainly be incorrect, as most _regular_ users won't have curl-config. I know, it is easy to forget that most Git users are no hard-core C developers ;-)
A better alternative would be to use curl_version(), guarded, of course, by #ifndef NO_CURL...
bugreport.c line 47
Quoted 12 lines
bugreport.c new file +12
-0+ +strbuf_addstr (sys_info,"curl-config --version: " ); +strbuf_addbuf (sys_info, &std_out); +strbuf_complete_line (sys_info); + +argv_array_clear (&cp.args); +strbuf_reset (&std_out); + + +argv_array_push (&cp.args,"ldd" ); +argv_array_push (&cp.args,"--version" ); +capture_command (&cp, &std_out,0 );
Again, this command will only be present in few setups. I am not actually sure that the output of this is interesting to begin with.
What I _do_ think is that a much more interesting piece of information would be the exact GLIBC version, the OS name and the OS version.
Quoted 32 lines
bugreport.c new file +8
-0+ +strbuf_addstr (sys_info,"ldd --version: " ); +strbuf_addbuf (sys_info, &std_out); +strbuf_complete_line (sys_info); + +argv_array_clear (&cp.args); +strbuf_reset (&std_out); +}bugreport.h new file +7
-0@@ -0,0 +1,7 @@ +#include "strbuf.h" + +/** +* Adds the Git version, `uname -a`, and `curl-config --version` to sys_info. +* The previous contents of sys_info will be discarded. +*/ +void get_system_info (struct strbuf *sys_info);builtin/bugreport.c +1
-0@@ -1,4 +1,5 @@#include "builtin.h" +#include "bugreport.h" #include "stdio.h" #include "strbuf.h" #include "time.h"
builtin/bugreport.c int get_bug_template(struct strbuf *template) · line 35
Quoted 8 lines
builtin/bugreport.c +5
-0@@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)return 0 ; } +void add_header (FILE *report,const char *title) +{ +struct strbuf buffer =STRBUF_INIT ; +strbuf_addf (&buffer,"\n\n[%s]\n" , title); +strbuf_write (&buffer, report);
This leaks buffer. Why not write into report via fprintf() directly?
Ciao,
Dscho
Quoted 21 lines
builtin/bugreport.c +7
-1+} +int cmd_bugreport (int argc,const char **argv,const char *prefix) {struct strbuf buffer =STRBUF_INIT ;@@ -43,6 +51,11 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_bug_template (&buffer);strbuf_write (&buffer, report); +// add other contents +add_header (report,"System Info" ); +get_system_info (&buffer); +strbuf_write (&buffer, report); +fclose (report);launch_editor (report_path.buf,NULL ,NULL ); --2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 3/9] bugreport: add version and system information
Quoted 42 lines · Johannes Schindelin
Hi Emily,
Quoted 11 lines · Emily Shaffer
bugreport.c new file +5
-0@@ -0,0 +1,55 @@ +#include "cache.h" + +#include "bugreport.h" +#include "help.h" +#include "run-command.h" bugreport.c line 15
Quoted 10 lines
bugreport.c new file +10
-0+#include "version.h" + +void get_system_info (struct strbuf *sys_info) +{ +struct child_process cp =CHILD_PROCESS_INIT ; +struct strbuf std_out =STRBUF_INIT ; + +strbuf_reset (sys_info); + +// get git version from native cmd Please use C-style comments instead of C++ ones.
bugreport.c line 23
Quoted 8 lines
bugreport.c new file +8
-0+strbuf_addstr (sys_info,"git version: " ); +strbuf_addstr (sys_info, git_version_string); +strbuf_complete_line (sys_info); + +// system call for other version info +argv_array_push (&cp.args,"uname" ); +argv_array_push (&cp.args,"-a" ); +capture_command (&cp, &std_out,0 );Mmmkay. I am not too much of a fan of relying on the
unameexecutable, as it can very well be a 32-bituname.exeon Windows, which obviously would _not_ report the architecture of the machine, but something misleading.
Quoted 2 lines
Why not use the
uname()function (that we even define incompat/mingw.c)?
Very glad to have your review and point this kind of thing out. :) It simplifies the code, too. I wasn't sure about these system calls, so I appreciate you suggesting alternatives.
Quoted 10 lines
Also, why not include the same information as
git version --build-options, most importantlyGIT_HOST_CPU?In any case, if you are capturing the output of
uname -a, you definitely will want to pass theRUN_COMMAND_STDOUT_TO_STDERRflag (in case, say, the MSYS2uname.exefails with those dreaded Cygwin error messages like "*** fatal error -add_item("\??\D:\git\installation\Git", "/", ...) failed, errno 1").
Quoted 16 lines
bugreport.c line 35
Quoted 12 lines
bugreport.c new file +12
-0+ +strbuf_addstr (sys_info,"uname -a: " ); +strbuf_addbuf (sys_info, &std_out); +strbuf_complete_line (sys_info); + +argv_array_clear (&cp.args); +strbuf_reset (&std_out); + + +argv_array_push (&cp.args,"curl-config" ); +argv_array_push (&cp.args,"--version" ); +capture_command (&cp, &std_out,0 );This will be almost certainly be incorrect, as most _regular_ users won't have
curl-config. I know, it is easy to forget that most Git users are no hard-core C developers ;-)
Heresy! :)
Quoted 4 lines
A better alternative would be to use
curl_version(), guarded, of course, by#ifndef NO_CURL...
Quoted 15 lines
bugreport.c line 47
Quoted 12 lines
bugreport.c new file +12
-0+ +strbuf_addstr (sys_info,"curl-config --version: " ); +strbuf_addbuf (sys_info, &std_out); +strbuf_complete_line (sys_info); + +argv_array_clear (&cp.args); +strbuf_reset (&std_out); + + +argv_array_push (&cp.args,"ldd" ); +argv_array_push (&cp.args,"--version" ); +capture_command (&cp, &std_out,0 );Again, this command will only be present in few setups. I am not actually sure that the output of this is interesting to begin with.
It was a suggestion, I believe, from Jonathan Nieder.
Quoted 3 lines
What I _do_ think is that a much more interesting piece of information would be the exact GLIBC version, the OS name and the OS version.
The glibc version is easy; I've done that. It certainly looks nicer than the ldd call.
I guess I may be missing something, because as I start to look into how to the OS info, I fall down a hole of many, many preprocessor defines to check. If that's the approach you want me to take, just say the word, but it will be ugly :) I suppose I had hoped the uname info would give us a close enough idea that full OS info isn't necessary.
Quoted 36 lines
Quoted 36 lines
bugreport.c new file +8
-0+ +strbuf_addstr (sys_info,"ldd --version: " ); +strbuf_addbuf (sys_info, &std_out); +strbuf_complete_line (sys_info); + +argv_array_clear (&cp.args); +strbuf_reset (&std_out); +}bugreport.h new file +7
-0@@ -0,0 +1,7 @@ +#include "strbuf.h" + +/** +* Adds the Git version, `uname -a`, and `curl-config --version` to sys_info. +* The previous contents of sys_info will be discarded. +*/ +void get_system_info (struct strbuf *sys_info);builtin/bugreport.c +1
-0@@ -1,4 +1,5 @@#include "builtin.h" +#include "bugreport.h" #include "stdio.h" #include "strbuf.h" #include "time.h" @@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)return 0 ; }
Quoted 8 lines
builtin/bugreport.c int get_bug_template(struct strbuf *template) · line 35
Quoted 5 lines
builtin/bugreport.c +5
-0+void add_header (FILE *report,const char *title) +{ +struct strbuf buffer =STRBUF_INIT ; +strbuf_addf (&buffer,"\n\n[%s]\n" , title); +strbuf_write (&buffer, report);This leaks
buffer. Why not write intoreportviafprintf()directly?
Rather, to match the style of the rest of the builtin, modified get_header to add the header to a passed-in strbuf instead of modifying the file directly.
Quoted 20 lines
Ciao,
DschoQuoted 16 lines
builtin/bugreport.c +7
-0+} +int cmd_bugreport (int argc,const char **argv,const char *prefix) {struct strbuf buffer =STRBUF_INIT ;@@ -43,6 +51,11 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_bug_template (&buffer);strbuf_write (&buffer, report); +// add other contents +add_header (report,"System Info" ); +get_system_info (&buffer); +strbuf_write (&buffer, report); +fclose (report);
Quoted 5 lines
Quoted 5 lines
builtin/bugreport.c +0
-1launch_editor (report_path.buf,NULL ,NULL ); --2.24.0.rc0.303.g954a862665-goog
Based on Dscho's comments, each piece of system info is gathered differently enough that I will do each in an independent commit now.
Re: [PATCH v3 3/9] bugreport: add version and system information
Hi Emily,
Quoted 25 lines · Emily Shaffer
Quoted 14 lines · Johannes Schindelin
Quoted 12 lines · Emily Shaffer
bugreport.c new file +6
-0+strbuf_addstr (sys_info,"curl-config --version: " ); +strbuf_addbuf (sys_info, &std_out); +strbuf_complete_line (sys_info); + +argv_array_clear (&cp.args); +strbuf_reset (&std_out);Quoted 8 lines
bugreport.c line 47
Quoted 5 lines
bugreport.c new file +5
-0+ + +argv_array_push (&cp.args,"ldd" ); +argv_array_push (&cp.args,"--version" ); +capture_command (&cp, &std_out,0 );Again, this command will only be present in few setups. I am not actually sure that the output of this is interesting to begin with.
Quoted 1 line
It was a suggestion, I believe, from Jonathan Nieder.
Yes, I guess Jonathan builds their Git locally, too.
It _is_ easy to forget that most users find this too involved to even try.
Nothing like reading through a bug tracker quite frequently to learn about the actual troubles actual users have :-)
Quoted 11 lines
Quoted 2 lines
What I _do_ think is that a much more interesting piece of information would be the exact GLIBC version, the OS name and the OS version.
The glibc version is easy; I've done that. It certainly looks nicer than the ldd call.
I guess I may be missing something, because as I start to look into how to the OS info, I fall down a hole of many, many preprocessor defines to check. If that's the approach you want me to take, just say the word, but it will be ugly :) I suppose I had hoped the uname info would give us a close enough idea that full OS info isn't necessary.
We could go down the pre-processor route, but that would give us the OS name and version at build time, not at run time. I think we will be mostly interested in the latter, though.
We might need to enhance our uname() emulation in compat/mingw.c, but I think we already have enough information there.
When it comes to glibc, I think gnu_get_libc_version() would get us what we want. A trickier thing might be to determine whether we're actually linking against glibc; I do not want to break musl builds again, I already did that inadvertently when requiring REG_STARTEND back in the days.
Quoted 23 lines
Quoted 14 lines
Quoted 14 lines
builtin/bugreport.c +1
-0@@ -1,4 +1,5 @@#include "builtin.h" +#include "bugreport.h" #include "stdio.h" #include "strbuf.h" #include "time.h" @@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)return 0 ; }Quoted 8 lines
builtin/bugreport.c int get_bug_template(struct strbuf *template) · line 35
Quoted 5 lines
builtin/bugreport.c +5
-0+void add_header (FILE *report,const char *title) +{ +struct strbuf buffer =STRBUF_INIT ; +strbuf_addf (&buffer,"\n\n[%s]\n" , title); +strbuf_write (&buffer, report);This leaks
buffer. Why not write intoreportviafprintf()directly?
Quoted 3 lines
Rather, to match the style of the rest of the builtin, modified
get_headerto add the header to a passed-in strbuf instead of modifying the file directly.
Hmm. It makes the code less elegant in my opinion. I would rather either render the entire bug report into a single strbuf and then write it via write_in_full(), or use fprintf() directly.
Ciao,
Dscho
Re: [PATCH v3 3/9] bugreport: add version and system information
Quoted 23 lines · Johannes Schindelin
Quoted 23 lines
Quoted 22 lines
Quoted 11 lines
builtin/bugreport.c +1
-0@@ -1,4 +1,5 @@#include "builtin.h" +#include "bugreport.h" #include "stdio.h" #include "strbuf.h" #include "time.h" builtin/bugreport.c int get_bug_template(struct strbuf *template) · line 35
Quoted 8 lines
builtin/bugreport.c +5
-0@@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)return 0 ; } +void add_header (FILE *report,const char *title) +{ +struct strbuf buffer =STRBUF_INIT ; +strbuf_addf (&buffer,"\n\n[%s]\n" , title); +strbuf_write (&buffer, report);This leaks
buffer. Why not write intoreportviafprintf()directly?
Quoted 7 lines
Quoted 3 lines
Rather, to match the style of the rest of the builtin, modified
get_headerto add the header to a passed-in strbuf instead of modifying the file directly.Hmm. It makes the code less elegant in my opinion. I would rather either render the entire bug report into a single
strbufand then write it viawrite_in_full(), or usefprintf()directly.
Yeah, that sounds good. I will do that. :)
Re: [PATCH v3 3/9] bugreport: add version and system information
In reply to Emily ShafferOn 2019.10.24 19:51, Emily Shaffer wrote:
[snip]
Quoted 11 lines
builtin/bugreport.c +1
-0@@ -1,4 +1,5 @@#include "builtin.h" +#include "bugreport.h" #include "stdio.h" #include "strbuf.h" #include "time.h"
builtin/bugreport.c int get_bug_template(struct strbuf *template) · line 31
Quoted 4 lines
builtin/bugreport.c +1
-0@@ -27,6 +28,13 @@ int get_bug_template(struct strbuf *template)return 0 ; } +void add_header (FILE *report,const char *title)
I get the same compilation error here, can you make add_header() static as well?
PATCH v3 4/9 bugreport: add config values from whitelist
In reply to Emily ShafferTeach bugreport to gather the values of config options which are present in 'git-bugreport-config-whitelist'.
Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.
Reading the whitelist into memory and sorting it saves us time - since git_config_bugreport() is called for every option the user has configured, limiting the file IO to one open/read/close and performing option lookup in sublinear time is a useful optimization.
--- bugreport.c | 50 +++++++++++++++++++++++++++++++++++++++++++++ bugreport.h | 7 +++++++ builtin/bugreport.c | 4 ++++ 3 files changed, 61 insertions(+)
bugreport.c +50 -0
@@ -1,10 +1,24 @@#include "cache.h" #include "bugreport.h" +#include "config.h" +#include "exec-cmd.h" #include "help.h" #include "run-command.h" #include "version.h" +/** +* A sorted list of config options which we will add to the bugreport. Managed +* by 'gather_whitelist(...)'. +*/ +struct string_list whitelist =STRING_LIST_INIT_DUP ; +struct strbuf configs_and_values =STRBUF_INIT ; + +// git version --build-options +// uname -a +// curl-config --version +// ldd --version +// echo $SHELL void get_system_info (struct strbuf *sys_info) {struct child_process cp =CHILD_PROCESS_INIT ;
@@ -53,3 +67,39 @@ void get_system_info(struct strbuf *sys_info)argv_array_clear (&cp.args);strbuf_reset (&std_out); } + +void gather_whitelist (struct strbuf *path) +{ +struct strbuf tmp =STRBUF_INIT ; +strbuf_read_file (&tmp, path->buf,0 ); +string_list_init (&whitelist,1 ); +string_list_split (&whitelist, tmp.buf,'\n' ,-1 ); +string_list_sort (&whitelist); +} + +int git_config_bugreport (const char *var,const char *value,void *cb) +{ +if (string_list_has_string (&whitelist, var)) { +strbuf_addf (&configs_and_values, +"%s : %s\n" , + var, value); + } + +return 0 ; +} + +void get_whitelisted_config (struct strbuf *config_info) +{ +struct strbuf path =STRBUF_INIT ; + +strbuf_addstr (&path,git_exec_path ()); +strbuf_addstr (&path,"/git-bugreport-config-whitelist" ); + +gather_whitelist (&path); +strbuf_init (&configs_and_values, whitelist.nr); + +git_config (git_config_bugreport,NULL ); + +strbuf_reset (config_info); +strbuf_addbuf (config_info, &configs_and_values); +}
bugreport.h +7 -0
@@ -5,3 +5,10 @@* The previous contents of sys_info will be discarded. */ void get_system_info (struct strbuf *sys_info); + +/** +* Adds the values of the config items listed in +* 'git-bugreport-config-whitelist' to config_info. The previous contents of +* config_info will be discarded. +*/ +void get_whitelisted_config (struct strbuf *sys_info);
builtin/bugreport.c +4 -0
@@ -56,6 +56,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_system_info (&buffer);strbuf_write (&buffer, report); +add_header (report,"Whitelisted Config" ); +get_whitelisted_config (&buffer); +strbuf_write (&buffer, report); +fclose (report);launch_editor (report_path.buf,NULL ,NULL );
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 4/9] bugreport: add config values from whitelist
Hi Emily,
Quoted 7 lines · Emily Shaffer
Teach bugreport to gather the values of config options which are present in 'git-bugreport-config-whitelist'.
Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.
Should we still have the // bugreport-exclude comments, then?
Quoted 5 lines
Reading the whitelist into memory and sorting it saves us time - since
git_config_bugreport()is called for every option the user has configured, limiting the file IO to one open/read/close and performing option lookup in sublinear time is a useful optimization.
Maybe we even want a hashmap? That would reduce the time complexity even further.
Quoted 16 lines
bugreport.c +4
-0@@ -1,10 +1,24 @@#include "cache.h" #include "bugreport.h" +#include "config.h" +#include "exec-cmd.h" #include "help.h" #include "run-command.h" #include "version.h" +/** +* A sorted list of config options which we will add to the bugreport. Managed
bugreport.c line 21
Quoted 10 lines
bugreport.c +10
-0+* by 'gather_whitelist(...)'. +*/ +struct string_list whitelist =STRING_LIST_INIT_DUP ; +struct strbuf configs_and_values =STRBUF_INIT ; + +// git version --build-options +// uname -a +// curl-config --version +// ldd --version +// echo $SHELL
These comments probably want to move to a single, C style comment, and they probably want to be introduced together with get_system_info().
I also have to admit that I might have missed where $SHELL was added to the output...
Quoted 13 lines
bugreport.c +6
-0void get_system_info (struct strbuf *sys_info) {struct child_process cp =CHILD_PROCESS_INIT ;@@ -53,3 +67,39 @@ void get_system_info(struct strbuf *sys_info)argv_array_clear (&cp.args);strbuf_reset (&std_out); } + +void gather_whitelist (struct strbuf *path) +{ +struct strbuf tmp =STRBUF_INIT ; +strbuf_read_file (&tmp, path->buf,0 ); +string_list_init (&whitelist,1 );
bugreport.c void get_system_info(struct strbuf *sys_info) · line 85
Quoted 10 lines
bugreport.c +10
-0+string_list_split (&whitelist, tmp.buf,'\n' ,-1 ); +string_list_sort (&whitelist); +} + +int git_config_bugreport (const char *var,const char *value,void *cb) +{ +if (string_list_has_string (&whitelist, var)) { +strbuf_addf (&configs_and_values, +"%s : %s\n" , + var, value);
A quite useful piece of information would be the config source. Not sure whether we can do that outside of config.c yet...
bugreport.c void get_system_info(struct strbuf *sys_info) · line 96
Quoted 11 lines
bugreport.c +11
-0+ } + +return 0 ; +} + +void get_whitelisted_config (struct strbuf *config_info) +{ +struct strbuf path =STRBUF_INIT ; + +strbuf_addstr (&path,git_exec_path ()); +strbuf_addstr (&path,"/git-bugreport-config-whitelist" );
Hmm. I would have expected this patch to come directly after the patch 2/9 that generates that white-list, and I would also have expected that to be pre-sorted, and compiled in.
Do you want users to _edit_ the file in the exec path? In general, that path will be write-protected, though. A better alternative would probably be to compile in a hard-coded list, and to allow including more values e.g. by offering command-line options to specify config setting patterns. But if we allow patterns, we might actually want to have those exclusions to prevent sensitive data from being included.
Quoted 14 lines
bugreport.c +9
-0+ +gather_whitelist (&path); +strbuf_init (&configs_and_values, whitelist.nr); + +git_config (git_config_bugreport,NULL ); + +strbuf_reset (config_info); +strbuf_addbuf (config_info, &configs_and_values); +}bugreport.h +0
-0
bugreport.h line 9
Quoted 5 lines
bugreport.h +2
-0@@ -5,3 +5,10 @@* The previous contents of sys_info will be discarded. */ void get_system_info (struct strbuf *sys_info); + +/**
I also frequently use JavaDoc-style /**, but I am not sure that this is actually desired in Git's source code ;-)
Quoted 10 lines
bugreport.h +5
-0+* Adds the values of the config items listed in +* 'git-bugreport-config-whitelist' to config_info. The previous contents of +* config_info will be discarded. +*/ +void get_whitelisted_config (struct strbuf *sys_info);builtin/bugreport.c +0
-0
builtin/bugreport.c int cmd_bugreport(int argc, const char **argv, const char *prefix) · line 59
Quoted 4 lines
builtin/bugreport.c +1
-0@@ -56,6 +56,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_system_info (&buffer);strbuf_write (&buffer, report); +add_header (report,"Whitelisted Config" );
Quite honestly, I would like to avoid the term "whitelist" for good. How about "Selected config settings" instead?
Thanks,
Dscho
Quoted 10 lines
builtin/bugreport.c +3
-1+get_whitelisted_config (&buffer); +strbuf_write (&buffer, report); +fclose (report);launch_editor (report_path.buf,NULL ,NULL ); --2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 4/9] bugreport: add config values from whitelist
Quoted 1 line · Johannes Schindelin
Hi Emily,
Sorry for the delay in replying. This work has been backburnered and this mail slipped through the cracks.
Quoted 6 lines · Emily Shaffer
Quoted 4 lines
Teach bugreport to gather the values of config options which are present in 'git-bugreport-config-whitelist'.
Quoted 6 lines
Quoted 4 lines
Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.
Should we still have the
// bugreport-excludecomments, then?
They were optional (useless) before too. I can remove them if you want; I suppose I like the idea of having precedent if someone wants to build their own internal version with opt-out configs rather than opt-in. I can remove them if we want; it doesn't matter very much to me either way.
Quoted 9 lines
Quoted 6 lines
Reading the whitelist into memory and sorting it saves us time - since
git_config_bugreport()is called for every option the user has configured, limiting the file IO to one open/read/close and performing option lookup in sublinear time is a useful optimization.Maybe we even want a hashmap? That would reduce the time complexity even further.
Sure, we can do it. I'll make that change.
Quoted 15 lines
Quoted 15 lines
bugreport.c +2
-0@@ -1,10 +1,24 @@#include "cache.h" #include "bugreport.h" +#include "config.h" +#include "exec-cmd.h" #include "help.h" #include "run-command.h" #include "version.h"
Quoted 15 lines
bugreport.c line 21
Quoted 12 lines
bugreport.c +12
-0+/** +* A sorted list of config options which we will add to the bugreport. Managed +* by 'gather_whitelist(...)'. +*/ +struct string_list whitelist =STRING_LIST_INIT_DUP ; +struct strbuf configs_and_values =STRBUF_INIT ; + +// git version --build-options +// uname -a +// curl-config --version +// ldd --version +// echo $SHELL These comments probably want to move to a single, C style comment, and they probably want to be introduced together with
get_system_info().
Yeah, it's stale and has been removed now. It was less commentary and more todo list for author ;)
Quoted 3 lines
I also have to admit that I might have missed where
$SHELLwas added to the output...
I skipped it entirely since bugreport doesn't run in shell anymore. If you have advice for gathering the user's shell I can try to add it; is there such a difference between, say, a Debian user using bash and a Debian user using zsh? I suppose it could be useful if someone has an issue with GIT_PS1, or with autocompletion. I'll look into gathering it.
Quoted 19 lines
Quoted 19 lines
bugreport.c +11
-0void get_system_info (struct strbuf *sys_info) {struct child_process cp =CHILD_PROCESS_INIT ;@@ -53,3 +67,39 @@ void get_system_info(struct strbuf *sys_info)argv_array_clear (&cp.args);strbuf_reset (&std_out); } + +void gather_whitelist (struct strbuf *path) +{ +struct strbuf tmp =STRBUF_INIT ; +strbuf_read_file (&tmp, path->buf,0 ); +string_list_init (&whitelist,1 ); +string_list_split (&whitelist, tmp.buf,'\n' ,-1 ); +string_list_sort (&whitelist); +} + +int git_config_bugreport (const char *var,const char *value,void *cb)
Quoted 8 lines
bugreport.c void get_system_info(struct strbuf *sys_info) · line 85
Quoted 5 lines
bugreport.c +5
-0+{ +if (string_list_has_string (&whitelist, var)) { +strbuf_addf (&configs_and_values, +"%s : %s\n" , + var, value);A quite useful piece of information would be the config source. Not sure whether we can do that outside of
config.cyet...
It's possible. I can add it.
Quoted 17 lines
bugreport.c void get_system_info(struct strbuf *sys_info) · line 96
Quoted 12 lines
bugreport.c +11
-0+ } + +return 0 ; +} + +void get_whitelisted_config (struct strbuf *config_info) +{ +struct strbuf path =STRBUF_INIT ; + +strbuf_addstr (&path,git_exec_path ()); +strbuf_addstr (&path,"/git-bugreport-config-whitelist" );Hmm. I would have expected this patch to come directly after the patch 2/9 that generates that white-list, and I would also have expected that to be pre-sorted, and compiled in.
Quoted 6 lines
Do you want users to _edit_ the file in the exec path? In general, that path will be write-protected, though. A better alternative would probably be to compile in a hard-coded list, and to allow including more values e.g. by offering command-line options to specify config setting patterns. But if we allow patterns, we might actually want to have those exclusions to prevent sensitive data from being included.
Hm, interesting. Do we have precedent for compiling in a header generated during the build process? I think I saw one when I was adding this script - I'll take a look.
Quoted 37 lines
Quoted 15 lines
bugreport.c +9
-0+ +gather_whitelist (&path); +strbuf_init (&configs_and_values, whitelist.nr); + +git_config (git_config_bugreport,NULL ); + +strbuf_reset (config_info); +strbuf_addbuf (config_info, &configs_and_values); +}bugreport.h +0
-0bugreport.h line 9
Quoted 5 lines
bugreport.h +2
-0@@ -5,3 +5,10 @@* The previous contents of sys_info will be discarded. */ void get_system_info (struct strbuf *sys_info); + +/** I also frequently use JavaDoc-style
/**, but I am not sure that this is actually desired in Git's source code ;-)Quoted 13 lines
bugreport.h +5
-0+* Adds the values of the config items listed in +* 'git-bugreport-config-whitelist' to config_info. The previous contents of +* config_info will be discarded. +*/ +void get_whitelisted_config (struct strbuf *sys_info);builtin/bugreport.c +0
-0@@ -56,6 +56,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_system_info (&buffer);strbuf_write (&buffer, report);
Quoted 4 lines
builtin/bugreport.c int cmd_bugreport(int argc, const char **argv, const char *prefix) · line 59
Quoted 1 line
builtin/bugreport.c +1
-0+add_header (report,"Whitelisted Config" );Quite honestly, I would like to avoid the term "whitelist" for good. How about "Selected config settings" instead?
Will do - thanks for the callout.
Quoted 14 lines
Thanks,
DschoQuoted 10 lines
builtin/bugreport.c +3
-1+get_whitelisted_config (&buffer); +strbuf_write (&buffer, report); +fclose (report);launch_editor (report_path.buf,NULL ,NULL ); --2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 4/9] bugreport: add config values from whitelist
Hi Emily,
Quoted 13 lines · Emily Shaffer
Quoted 11 lines · Johannes Schindelin
Quoted 7 lines · Emily Shaffer
Teach bugreport to gather the values of config options which are present in 'git-bugreport-config-whitelist'.
Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.
Quoted 7 lines
Quoted 1 line
Should we still have the
// bugreport-excludecomments, then?They were optional (useless) before too. I can remove them if you want; I suppose I like the idea of having precedent if someone wants to build their own internal version with opt-out configs rather than opt-in. I can remove them if we want; it doesn't matter very much to me either way.
How are you guaranteeing that this information does not become stale, like, immediately?
If you _still_ insist on keeping those comments even after answering this question, at least please turn them into C-style comments: we have no C++-style comments in Git and want to keep it this way.
Quoted 25 lines
Quoted 18 lines
bugreport.c void get_system_info(struct strbuf *sys_info) · line 96
Quoted 13 lines
bugreport.c +11
-0+ } + +return 0 ; +} + +void get_whitelisted_config (struct strbuf *config_info) +{ +struct strbuf path =STRBUF_INIT ; + +strbuf_addstr (&path,git_exec_path ()); +strbuf_addstr (&path,"/git-bugreport-config-whitelist" );Hmm. I would have expected this patch to come directly after the patch 2/9 that generates that white-list, and I would also have expected that to be pre-sorted, and compiled in.
Quoted 6 lines
Do you want users to _edit_ the file in the exec path? In general, that path will be write-protected, though. A better alternative would probably be to compile in a hard-coded list, and to allow including more values e.g. by offering command-line options to specify config setting patterns. But if we allow patterns, we might actually want to have those exclusions to prevent sensitive data from being included.
Quoted 3 lines
Hm, interesting. Do we have precedent for compiling in a header generated during the build process? I think I saw one when I was adding this script - I'll take a look.
Yes, we do. The entire command-list.h is generated during the build. Look for GENERATED_H in the Makefile.
Ciao,
Dscho
Re: [PATCH v3 4/9] bugreport: add config values from whitelist
In reply to Emily ShafferOn 2019.10.24 19:51, Emily Shaffer wrote:
[snip]
Quoted 30 lines
bugreport.c +14
-0@@ -1,10 +1,24 @@#include "cache.h" #include "bugreport.h" +#include "config.h" +#include "exec-cmd.h" #include "help.h" #include "run-command.h" #include "version.h" +/** +* A sorted list of config options which we will add to the bugreport. Managed +* by 'gather_whitelist(...)'. +*/ +struct string_list whitelist =STRING_LIST_INIT_DUP ; +struct strbuf configs_and_values =STRBUF_INIT ; + +// git version --build-options +// uname -a +// curl-config --version +// ldd --version +// echo $SHELL void get_system_info (struct strbuf *sys_info) {struct child_process cp =CHILD_PROCESS_INIT ;
bugreport.c void get_system_info(struct strbuf *sys_info) · line 71
Quoted 5 lines
bugreport.c +2
-0@@ -53,3 +67,39 @@ void get_system_info(struct strbuf *sys_info)argv_array_clear (&cp.args);strbuf_reset (&std_out); } + +void gather_whitelist (struct strbuf *path)
This and git_config_bugreport() below should both be static as well. Rather than repeating advice on the later patches, I'll just note that any new functions that don't show up in the corresponding .h file should be marked static.
Re: [PATCH v3 4/9] bugreport: add config values from whitelist
Quoted 4 lines · Josh Steadmon
This and
git_config_bugreport()below should both be static as well. Rather than repeating advice on the later patches, I'll just note that any new functions that don't show up in the corresponding .h file should be marked static.
Good advice.
More importantly, given that "git bugreport" itself has no service of itself to offer other existing parts of Git (it is just a "gather various pieces of information from different places, and then produce a text file output" application), I do not see much point in it having its own header file that others would #include (i.e. the include file is to define services that are offered by it). If there are common enough service routines invented to support the need of bugreport.c (e.g. perhaps it wants to give more info than what is currently available via the existing API on the contents of in-core index), by definition of being them common enough, they should be added to the header that can be used by both bugreport.c and other existing users of the same subsystem (e.g. if it is about in-core index, perhaps cache.h).
It makes perfect sense for bugreport.c to #include header files for the Git internals to collect pieces of information from inside Git, though.
Re: [PATCH v3 4/9] bugreport: add config values from whitelist
Quoted 9 lines · Junio C Hamano
Quoted 4 lines · Josh Steadmon
This and
git_config_bugreport()below should both be static as well. Rather than repeating advice on the later patches, I'll just note that any new functions that don't show up in the corresponding .h file should be marked static.Good advice.
Quoted 13 lines
More importantly, given that "git bugreport" itself has no service of itself to offer other existing parts of Git (it is just a "gather various pieces of information from different places, and then produce a text file output" application), I do not see much point in it having its own header file that others would #include (i.e. the include file is to define services that are offered by it). If there are common enough service routines invented to support the need of
bugreport.c(e.g. perhaps it wants to give more info than what is currently available via the existing API on the contents of in-core index), by definition of being them common enough, they should be added to the header that can be used by bothbugreport.cand other existing users of the same subsystem (e.g. if it is about in-core index, perhapscache.h).
Are you asking me to have only builtin/bugreport.c and implement each function there, and eliminate both <basedir>/bugreport.[ch]?
It may be true that deciding to put this functionality into a library "in case someone needs it later" was premature, so I can do that - I just want to make sure I understand you.
- Emily
PATCH v3 5/9 bugreport: collect list of populated hooks
In reply to Emily ShafferOccasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong.
Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot. To reduce the amount of code humans need to read, we turn the list into a string_list and iterate over it (as we are calling the same find_hook operation on each string). However, since bugreport should primarily be called by the user, the performance loss from massaging the string seems acceptable.
--- bugreport.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ bugreport.h | 6 ++++++ builtin/bugreport.c | 4 ++++ 3 files changed, 54 insertions(+)
bugreport.c +44 -0
@@ -103,3 +103,47 @@ void get_whitelisted_config(struct strbuf *config_info)strbuf_reset (config_info);strbuf_addbuf (config_info, &configs_and_values); } + +void get_populated_hooks (struct strbuf *hook_info) +{ +/* +* Doesn't look like there is a list of all possible hooks; so below is +* a transcription of `git help hook`. +*/ +const char *hooks ="applypatch-msg," +"pre-applypatch," +"post-applypatch," +"pre-commit," +"pre-merge-commit," +"prepare-commit-msg," +"commit-msg," +"post-commit," +"pre-rebase," +"post-checkout," +"post-merge," +"pre-push," +"pre-receive," +"update," +"post-receive," +"post-update," +"push-to-checkout," +"pre-auto-gc," +"post-rewrite," +"sendemail-validate," +"fsmonitor-watchman," +"p4-pre-submit," +"post-index-changex" ; +struct string_list hooks_list =STRING_LIST_INIT_DUP ; +struct string_list_item *iter =NULL ; + +strbuf_reset (hook_info); + +string_list_split (&hooks_list, hooks,',' ,-1 ); + +for_each_string_list_item (iter, &hooks_list) { +if (find_hook (iter->string)) { +strbuf_addstr (hook_info, iter->string); +strbuf_complete_line (hook_info); + } + } +}
bugreport.h +6 -0
@@ -12,3 +12,9 @@ void get_system_info(struct strbuf *sys_info);* config_info will be discarded. */ void get_whitelisted_config (struct strbuf *sys_info); + +/** +* Adds the paths to all configured hooks (but not their contents). The previous +* contents of hook_info will be discarded. +*/ +void get_populated_hooks (struct strbuf *hook_info);
builtin/bugreport.c +4 -0
@@ -60,6 +60,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_whitelisted_config (&buffer);strbuf_write (&buffer, report); +add_header (report,"Populated Hooks" ); +get_populated_hooks (&buffer); +strbuf_write (&buffer, report); +fclose (report);launch_editor (report_path.buf,NULL ,NULL );
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 5/9] bugreport: collect list of populated hooks
Hi Emily,
Quoted 7 lines · Emily Shaffer
Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong.
Quoted 7 lines
Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot. To reduce the amount of code humans need to read, we turn the list into a
string_listand iterate over it (as we are calling the samefind_hookoperation on each string). However, since bugreport should primarily be called by the user, the performance loss from massaging the string seems acceptable.
Good idea!
Quoted 36 lines
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ bugreport.h | 6 ++++++ builtin/bugreport.c | 4 ++++ 3 files changed, 54 insertions(+)bugreport.c +20
-0@@ -103,3 +103,47 @@ void get_whitelisted_config(struct strbuf *config_info)strbuf_reset (config_info);strbuf_addbuf (config_info, &configs_and_values); } + +void get_populated_hooks (struct strbuf *hook_info) +{ +/* +* Doesn't look like there is a list of all possible hooks; so below is +* a transcription of `git help hook`. +*/ +const char *hooks ="applypatch-msg," +"pre-applypatch," +"post-applypatch," +"pre-commit," +"pre-merge-commit," +"prepare-commit-msg," +"commit-msg," +"post-commit," +"pre-rebase," +"post-checkout," +"post-merge," +"pre-push," +"pre-receive,"
bugreport.c void get_whitelisted_config(struct strbuf *config_info) · line 135
Quoted 10 lines
bugreport.c +10
-0+"update," +"post-receive," +"post-update," +"push-to-checkout," +"pre-auto-gc," +"post-rewrite," +"sendemail-validate," +"fsmonitor-watchman," +"p4-pre-submit," +"post-index-changex" ;
Well, this is disappointing: I tried to come up with a scripted way to extract the commit names from our source code, and I failed! This is where I gave up:
git grep run_hook | sed -n 's/.*run_hook[^(]*([^,]*, *\([^,]*\).*/\1/p' | sed -e '/^name$/d' -e '/^const char \*name$/d' \ -e 's/push_to_checkout_hook/"push-to-checkout"/' | sort | uniq
This prints only the following hook names:
"applypatch-msg" "post-applypatch" "post-checkout" "post-index-change" "post-merge" "pre-applypatch" "pre-auto-gc" "pre-rebase" "prepare-commit-msg" "push-to-checkout"
But at least it was easy to script the extracting from the Documentation:
sed -n '/^[a-z]/{N;/\n~~~/{s/\n.*//;p}}' \
Documentation/githooks.txtbugreport.c void get_whitelisted_config(struct strbuf *config_info) · line 143
Quoted 8 lines
bugreport.c +8
-0+struct string_list hooks_list =STRING_LIST_INIT_DUP ; +struct string_list_item *iter =NULL ; + +strbuf_reset (hook_info); + +string_list_split (&hooks_list, hooks,',' ,-1 ); + +for_each_string_list_item (iter, &hooks_list) {
This should definitely be done at compile time, I think. We should be able to generate an appropriate header via something like this:
cat >hook-names.h <<-EOF
static const char *hook_names[] = {
$(sed -n '/^[a-z]/{N;/\n~~~/{s/\n.*/",/;s/^/ "/;p}}' \
Documentation/githooks.txt)
};
EOFThen you would use a simple for() loop using ARRAY_SIZE() to iterate over the hook names.
Quoted 25 lines
bugreport.c +6
-0+if (find_hook (iter->string)) { +strbuf_addstr (hook_info, iter->string); +strbuf_complete_line (hook_info); + } + } +}bugreport.h +6
-0@@ -12,3 +12,9 @@ void get_system_info(struct strbuf *sys_info);* config_info will be discarded. */ void get_whitelisted_config (struct strbuf *sys_info); + +/** +* Adds the paths to all configured hooks (but not their contents). The previous +* contents of hook_info will be discarded. +*/ +void get_populated_hooks (struct strbuf *hook_info);builtin/bugreport.c +0
-0
builtin/bugreport.c int cmd_bugreport(int argc, const char **argv, const char *prefix) · line 63
Quoted 4 lines
builtin/bugreport.c +1
-0@@ -60,6 +60,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_whitelisted_config (&buffer);strbuf_write (&buffer, report); +add_header (report,"Populated Hooks" );
Wait! I should have stumbled over this in an earlier patch. The add_header() function should not take a FILE * parameter at all, but instead an struct strbuf * one!
Ciao,
Dscho
Quoted 10 lines
builtin/bugreport.c +3
-1+get_populated_hooks (&buffer); +strbuf_write (&buffer, report); +fclose (report);launch_editor (report_path.buf,NULL ,NULL ); --2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 5/9] bugreport: collect list of populated hooks
Quoted 92 lines · Johannes Schindelin
Hi Emily,
Quoted 7 lines · Emily Shaffer
Occasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong.
Quoted 7 lines
Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot. To reduce the amount of code humans need to read, we turn the list into a
string_listand iterate over it (as we are calling the samefind_hookoperation on each string). However, since bugreport should primarily be called by the user, the performance loss from massaging the string seems acceptable.Good idea!
Quoted 36 lines
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ bugreport.h | 6 ++++++ builtin/bugreport.c | 4 ++++ 3 files changed, 54 insertions(+)bugreport.c +20
-0@@ -103,3 +103,47 @@ void get_whitelisted_config(struct strbuf *config_info)strbuf_reset (config_info);strbuf_addbuf (config_info, &configs_and_values); } + +void get_populated_hooks (struct strbuf *hook_info) +{ +/* +* Doesn't look like there is a list of all possible hooks; so below is +* a transcription of `git help hook`. +*/ +const char *hooks ="applypatch-msg," +"pre-applypatch," +"post-applypatch," +"pre-commit," +"pre-merge-commit," +"prepare-commit-msg," +"commit-msg," +"post-commit," +"pre-rebase," +"post-checkout," +"post-merge," +"pre-push," +"pre-receive," bugreport.c void get_whitelisted_config(struct strbuf *config_info) · line 135
Quoted 10 lines
bugreport.c +10
-0+"update," +"post-receive," +"post-update," +"push-to-checkout," +"pre-auto-gc," +"post-rewrite," +"sendemail-validate," +"fsmonitor-watchman," +"p4-pre-submit," +"post-index-changex" ;Well, this is disappointing: I tried to come up with a scripted way to extract the commit names from our source code, and I failed! This is where I gave up:
git grep run_hook | sed -n 's/.*run_hook[^(]*([^,]*, *\([^,]*\).*/\1/p' | sed -e '/^name$/d' -e '/^const char \*name$/d' \ -e 's/push_to_checkout_hook/"push-to-checkout"/' | sort | uniqThis prints only the following hook names:
"applypatch-msg" "post-applypatch" "post-checkout" "post-index-change" "post-merge" "pre-applypatch" "pre-auto-gc" "pre-rebase" "prepare-commit-msg" "push-to-checkout"
Quoted 6 lines
But at least it was easy to script the extracting from the Documentation:
sed -n '/^[a-z]/{N;/\n~~~/{s/\n.*//;p}}' \ Documentation/githooks.txt
To be totally frank, I'm not keen on spending a lot of time with scripting this. My parallel work with config-based hooks will also involve an in-code source of truth for available hooknames; I'd like to punt on some scripting, putting it in the makefile, etc blah since I know I'm planning to fix this particular annoyance at the source - and since it looks like bugreport will stay in the review phase for at least a little longer.
Quoted 69 lines
bugreport.c void get_whitelisted_config(struct strbuf *config_info) · line 143
Quoted 8 lines
bugreport.c +8
-0+struct string_list hooks_list =STRING_LIST_INIT_DUP ; +struct string_list_item *iter =NULL ; + +strbuf_reset (hook_info); + +string_list_split (&hooks_list, hooks,',' ,-1 ); + +for_each_string_list_item (iter, &hooks_list) {This should definitely be done at compile time, I think. We should be able to generate an appropriate header via something like this:
cat >hook-names.h <<-EOF static const char *hook_names[] = { $(sed -n '/^[a-z]/{N;/\n~~~/{s/\n.*/",/;s/^/ "/;p}}' \ Documentation/githooks.txt) }; EOFThen you would use a simple
for()loop usingARRAY_SIZE()to iterate over the hook names.Quoted 25 lines
bugreport.c +6
-0+if (find_hook (iter->string)) { +strbuf_addstr (hook_info, iter->string); +strbuf_complete_line (hook_info); + } + } +}bugreport.h +6
-0@@ -12,3 +12,9 @@ void get_system_info(struct strbuf *sys_info);* config_info will be discarded. */ void get_whitelisted_config (struct strbuf *sys_info); + +/** +* Adds the paths to all configured hooks (but not their contents). The previous +* contents of hook_info will be discarded. +*/ +void get_populated_hooks (struct strbuf *hook_info);builtin/bugreport.c +0
-0builtin/bugreport.c int cmd_bugreport(int argc, const char **argv, const char *prefix) · line 63
Quoted 4 lines
builtin/bugreport.c +1
-0@@ -60,6 +60,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_whitelisted_config (&buffer);strbuf_write (&buffer, report); +add_header (report,"Populated Hooks" );Wait! I should have stumbled over this in an earlier patch. The
add_header()function should not take aFILE *parameter at all, but instead anstruct strbuf *one!Ciao,
DschoQuoted 10 lines
builtin/bugreport.c +3
-1+get_populated_hooks (&buffer); +strbuf_write (&buffer, report); +fclose (report);launch_editor (report_path.buf,NULL ,NULL ); --2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 5/9] bugreport: collect list of populated hooks
Hi Emily,
Quoted 74 lines · Emily Shaffer
Quoted 65 lines · Johannes Schindelin
Quoted 28 lines · Emily Shaffer
bugreport.c +20
-0@@ -103,3 +103,47 @@ void get_whitelisted_config(struct strbuf *config_info)strbuf_reset (config_info);strbuf_addbuf (config_info, &configs_and_values); } + +void get_populated_hooks (struct strbuf *hook_info) +{ +/* +* Doesn't look like there is a list of all possible hooks; so below is +* a transcription of `git help hook`. +*/ +const char *hooks ="applypatch-msg," +"pre-applypatch," +"post-applypatch," +"pre-commit," +"pre-merge-commit," +"prepare-commit-msg," +"commit-msg," +"post-commit," +"pre-rebase," +"post-checkout," +"post-merge," +"pre-push," +"pre-receive," bugreport.c void get_whitelisted_config(struct strbuf *config_info) · line 135
Quoted 10 lines
bugreport.c +10
-0+"update," +"post-receive," +"post-update," +"push-to-checkout," +"pre-auto-gc," +"post-rewrite," +"sendemail-validate," +"fsmonitor-watchman," +"p4-pre-submit," +"post-index-changex" ;Well, this is disappointing: I tried to come up with a scripted way to extract the commit names from our source code, and I failed! This is where I gave up:
git grep run_hook | sed -n 's/.*run_hook[^(]*([^,]*, *\([^,]*\).*/\1/p' | sed -e '/^name$/d' -e '/^const char \*name$/d' \ -e 's/push_to_checkout_hook/"push-to-checkout"/' | sort | uniqThis prints only the following hook names:
"applypatch-msg" "post-applypatch" "post-checkout" "post-index-change" "post-merge" "pre-applypatch" "pre-auto-gc" "pre-rebase" "prepare-commit-msg" "push-to-checkout"Quoted 6 lines
But at least it was easy to script the extracting from the Documentation:
sed -n '/^[a-z]/{N;/\n~~~/{s/\n.*//;p}}' \ Documentation/githooks.txt
Quoted 7 lines
To be totally frank, I'm not keen on spending a lot of time with scripting this. My parallel work with config-based hooks will also involve an in-code source of truth for available hooknames; I'd like to punt on some scripting, putting it in the makefile, etc blah since I know I'm planning to fix this particular annoyance at the source - and since it looks like bugreport will stay in the review phase for at least a little longer.
Fair enough, especially if it addresses my complaint about the scriptability.
Ciao,
Dscho
PATCH v3 6/9 bugreport: count loose objects
In reply to Emily ShafferThe number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.
Rather than directly invoking 'git-count-objects', which may sometimes fail unexpectedly on Git for Windows, manually count the contents of .git/objects. Additionally, since we may wish to inspect other directories' contents for bugreport in the future, put the directory listing into a helper function.
--- bugreport.c | 72 +++++++++++++++++++++++++++++++++++++++++++++ bugreport.h | 6 ++++ builtin/bugreport.c | 4 +++ 3 files changed, 82 insertions(+)
bugreport.c +72 -0
@@ -5,8 +5,11 @@#include "exec-cmd.h" #include "help.h" #include "run-command.h" +#include "strbuf.h" #include "version.h" +#include "dirent.h" +/** * A sorted list of config options which we will add to the bugreport. Managed * by 'gather_whitelist(...)'.
@@ -147,3 +150,72 @@ void get_populated_hooks(struct strbuf *hook_info) } } } + +/** +* Fill 'contents' with the contents of the dir at 'dirpath'. +* If 'filter' is nonzero, the contents are filtered on d_type as 'type' - see +* 'man readdir'. opendir() doesn't take string length as an arg, so don't +* bother passing it in. +*/ +void list_contents_of_dir (struct string_list *contents,struct strbuf *dirpath, +int filter,unsigned char type) +{ +struct dirent *dir =NULL ; +DIR *dh =NULL ; + + dh =opendir (dirpath->buf); +while (dh && (dir =readdir (dh))) { +if (!filter || type == dir->d_type) { +string_list_append (contents, dir->d_name); + } + } +} + + +void get_object_counts (struct strbuf *obj_info) +{ +struct child_process cp =CHILD_PROCESS_INIT ; +struct strbuf std_out =STRBUF_INIT ; + +argv_array_push (&cp.args,"count-objects" ); +argv_array_push (&cp.args,"-vH" ); + cp.git_cmd =1 ; +capture_command (&cp, &std_out,0 ); + +strbuf_reset (obj_info); +strbuf_addstr (obj_info,"git-count-objects -vH:\n" ); +strbuf_addbuf (obj_info, &std_out); +} + +void get_loose_object_summary (struct strbuf *obj_info) +{ +struct strbuf dirpath =STRBUF_INIT ; +struct string_list subdirs =STRING_LIST_INIT_DUP ; +struct string_list_item *subdir; + +strbuf_reset (obj_info); + +strbuf_addstr (&dirpath,get_object_directory ()); +strbuf_complete (&dirpath,'/' ); + +list_contents_of_dir (&subdirs, &dirpath,1 ,DT_DIR ); + +for_each_string_list_item (subdir, &subdirs) + { +struct strbuf subdir_buf =STRBUF_INIT ; +struct string_list objects =STRING_LIST_INIT_DUP ; + +/* +* Only interested in loose objects - so dirs named with the +* first byte of the object ID +*/ +if (strlen (subdir->string) !=2 || !strcmp (subdir->string,".." )) +continue ; + +strbuf_addbuf (&subdir_buf, &dirpath); +strbuf_addstr (&subdir_buf, subdir->string); +list_contents_of_dir (&objects, &subdir_buf,0 ,0 ); +strbuf_addf (obj_info,"%s: %d objects\n" , subdir->string, + objects.nr); + } +}
bugreport.h +6 -0
@@ -18,3 +18,9 @@ void get_whitelisted_config(struct strbuf *sys_info);* contents of hook_info will be discarded. */ void get_populated_hooks (struct strbuf *hook_info); + +/** +* Adds the output of `git count-object -vH`. The previous contents of hook_info +* will be discarded. +*/ +void get_loose_object_summary (struct strbuf *obj_info);
builtin/bugreport.c +4 -0
@@ -64,6 +64,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_populated_hooks (&buffer);strbuf_write (&buffer, report); +add_header (report,"Object Counts" ); +get_loose_object_summary (&buffer); +strbuf_write (&buffer, report); +fclose (report);launch_editor (report_path.buf,NULL ,NULL );
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 6/9] bugreport: count loose objects
Hi Emily,
Quoted 9 lines · Emily Shaffer
The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.
Rather than directly invoking 'git-count-objects', which may sometimes fail unexpectedly on Git for Windows, manually count the contents of .git/objects. Additionally, since we may wish to inspect other directories' contents for bugreport in the future, put the directory listing into a helper function.
Thank you, much appreciated!
I guess the next step is to count the number of packs, and the number of submodules ;-)
Quoted 13 lines
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 72 +++++++++++++++++++++++++++++++++++++++++++++ bugreport.h | 6 ++++ builtin/bugreport.c | 4 +++ 3 files changed, 82 insertions(+)bugreport.c +0
-0
bugreport.c line 8
Quoted 4 lines
bugreport.c +1
-0@@ -5,8 +5,11 @@#include "exec-cmd.h" #include "help.h" #include "run-command.h" +#include "strbuf.h"
Why not append this to the end of the #include list, as is common in Git's commit history?
bugreport.c line 11
Quoted 3 lines
bugreport.c +1
-0#include "version.h" +#include "dirent.h"
This header (although with pointy brackets instead of double quotes) is already included in git-compat-util.h
bugreport.c void get_populated_hooks(struct strbuf *hook_info) · line 155
Quoted 11 lines
bugreport.c +4
-0+/** * A sorted list of config options which we will add to the bugreport. Managed * by 'gather_whitelist(...)'. @@ -147,3 +150,72 @@ void get_populated_hooks(struct strbuf *hook_info) } } } + +/** +* Fill 'contents' with the contents of the dir at 'dirpath'.
Since you start this comment in JavaDoc style, there should be an almost empty line after this one ("almost" because it still contains the asterisk, of course).
bugreport.c void get_populated_hooks(struct strbuf *hook_info) · line 160
Quoted 5 lines
bugreport.c +5
-0+* If 'filter' is nonzero, the contents are filtered on d_type as 'type' - see +* 'man readdir'. opendir() doesn't take string length as an arg, so don't +* bother passing it in. +*/ +void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath,
Shouldn't this be static?
Quoted 5 lines
bugreport.c +5
-0+ int filter, unsigned char type) +{ +struct dirent *dir =NULL ; +DIR *dh =NULL ; +
bugreport.c void get_populated_hooks(struct strbuf *hook_info) · line 175
Quoted 10 lines
bugreport.c +10
-0+ dh =opendir (dirpath->buf); +while (dh && (dir =readdir (dh))) { +if (!filter || type == dir->d_type) { +string_list_append (contents, dir->d_name); + } + } +} + + +void get_object_counts (struct strbuf *obj_info)
Oops. This function is no longer used.
Quoted 34 lines
bugreport.c +34
-0+{ +struct child_process cp =CHILD_PROCESS_INIT ; +struct strbuf std_out =STRBUF_INIT ; + +argv_array_push (&cp.args,"count-objects" ); +argv_array_push (&cp.args,"-vH" ); + cp.git_cmd =1 ; +capture_command (&cp, &std_out,0 ); + +strbuf_reset (obj_info); +strbuf_addstr (obj_info,"git-count-objects -vH:\n" ); +strbuf_addbuf (obj_info, &std_out); +} + +void get_loose_object_summary (struct strbuf *obj_info) +{ +struct strbuf dirpath =STRBUF_INIT ; +struct string_list subdirs =STRING_LIST_INIT_DUP ; +struct string_list_item *subdir; + +strbuf_reset (obj_info); + +strbuf_addstr (&dirpath,get_object_directory ()); +strbuf_complete (&dirpath,'/' ); + +list_contents_of_dir (&subdirs, &dirpath,1 ,DT_DIR ); + +for_each_string_list_item (subdir, &subdirs) + { +struct strbuf subdir_buf =STRBUF_INIT ; +struct string_list objects =STRING_LIST_INIT_DUP ; + +/* +* Only interested in loose objects - so dirs named with the
bugreport.c void get_populated_hooks(struct strbuf *hook_info) · line 219
Quoted 10 lines
bugreport.c +10
-0+* first byte of the object ID +*/ +if (strlen (subdir->string) !=2 || !strcmp (subdir->string,".." )) +continue ; + +strbuf_addbuf (&subdir_buf, &dirpath); +strbuf_addstr (&subdir_buf, subdir->string); +list_contents_of_dir (&objects, &subdir_buf,0 ,0 ); +strbuf_addf (obj_info,"%s: %d objects\n" , subdir->string, + objects.nr);
Hmm. Not only does this leak objects, it also throws away the contents that we so painfully constructed.
Wouldn't it make more sense to do something like this instead?
static int is_hex(const char *string, size_t count)
{
for (; count; string++, count--)
if (hexval(*string) < 0)
return 0;
return 1;
}static ssize_t count_loose_objects(struct strbuf *objects_path)
{
ssize_t ret = 0;
size_t len;
struct dirent *d;
DIR *dir, *subdir;
dir = opendir(objects_path->buf);
if (!dir)
return -1;
strbuf_complete(objects_path, '/');
len = objects_path->len;
while ((d = readdir(dir))) {
if (d->d_type != DT_DIR)
continue;
strbuf_setlen(objects_path, len);
strbuf_addstr(objects_path, d->d_name);
subdir = opendir(objects_path->buf);
if (!subdir)
continue;
while ((d = readdir(subdir)))
if (d->dt_type == DT_REG &&
is_hex(dir->d_name, the_repository->hash_algo->hexsz))
ret++;
closedir(subdir);
}
closedir(dir);
strbuf_reset(objects_path, len);
return ret;
}Ciao,
Dscho
Quoted 35 lines
bugreport.c +2
-0+ } +}bugreport.h +6
-0@@ -18,3 +18,9 @@ void get_whitelisted_config(struct strbuf *sys_info);* contents of hook_info will be discarded. */ void get_populated_hooks (struct strbuf *hook_info); + +/** +* Adds the output of `git count-object -vH`. The previous contents of hook_info +* will be discarded. +*/ +void get_loose_object_summary (struct strbuf *obj_info);builtin/bugreport.c +4
-1@@ -64,6 +64,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_populated_hooks (&buffer);strbuf_write (&buffer, report); +add_header (report,"Object Counts" ); +get_loose_object_summary (&buffer); +strbuf_write (&buffer, report); +fclose (report);launch_editor (report_path.buf,NULL ,NULL ); --2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 6/9] bugreport: count loose objects
Quoted 139 lines · Johannes Schindelin
Hi Emily,
Quoted 9 lines · Emily Shaffer
The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.
Rather than directly invoking 'git-count-objects', which may sometimes fail unexpectedly on Git for Windows, manually count the contents of .git/objects. Additionally, since we may wish to inspect other directories' contents for bugreport in the future, put the directory listing into a helper function.
Thank you, much appreciated!
I guess the next step is to count the number of packs, and the number of submodules ;-)
Quoted 13 lines
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 72 +++++++++++++++++++++++++++++++++++++++++++++ bugreport.h | 6 ++++ builtin/bugreport.c | 4 +++ 3 files changed, 82 insertions(+)bugreport.c +0
-0bugreport.c line 8
Quoted 4 lines
bugreport.c +1
-0@@ -5,8 +5,11 @@#include "exec-cmd.h" #include "help.h" #include "run-command.h" +#include "strbuf.h" Why not append this to the end of the
#includelist, as is common in Git's commit history?bugreport.c line 11
Quoted 3 lines
bugreport.c +1
-0#include "version.h" +#include "dirent.h" This header (although with pointy brackets instead of double quotes) is already included in
git-compat-util.hbugreport.c void get_populated_hooks(struct strbuf *hook_info) · line 155
Quoted 11 lines
bugreport.c +4
-0+/** * A sorted list of config options which we will add to the bugreport. Managed * by 'gather_whitelist(...)'. @@ -147,3 +150,72 @@ void get_populated_hooks(struct strbuf *hook_info) } } } + +/** +* Fill 'contents' with the contents of the dir at 'dirpath'. Since you start this comment in JavaDoc style, there should be an almost empty line after this one ("almost" because it still contains the asterisk, of course).
bugreport.c void get_populated_hooks(struct strbuf *hook_info) · line 160
Quoted 5 lines
bugreport.c +5
-0+* If 'filter' is nonzero, the contents are filtered on d_type as 'type' - see +* 'man readdir'. opendir() doesn't take string length as an arg, so don't +* bother passing it in. +*/ +void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath,Shouldn't this be
static?Quoted 5 lines
bugreport.c +5
-0+ int filter, unsigned char type) +{ +struct dirent *dir =NULL ; +DIR *dh =NULL ; +bugreport.c void get_populated_hooks(struct strbuf *hook_info) · line 175
Quoted 10 lines
bugreport.c +10
-0+ dh =opendir (dirpath->buf); +while (dh && (dir =readdir (dh))) { +if (!filter || type == dir->d_type) { +string_list_append (contents, dir->d_name); + } + } +} + + +void get_object_counts (struct strbuf *obj_info)Oops. This function is no longer used.
Quoted 34 lines
bugreport.c +34
-0+{ +struct child_process cp =CHILD_PROCESS_INIT ; +struct strbuf std_out =STRBUF_INIT ; + +argv_array_push (&cp.args,"count-objects" ); +argv_array_push (&cp.args,"-vH" ); + cp.git_cmd =1 ; +capture_command (&cp, &std_out,0 ); + +strbuf_reset (obj_info); +strbuf_addstr (obj_info,"git-count-objects -vH:\n" ); +strbuf_addbuf (obj_info, &std_out); +} + +void get_loose_object_summary (struct strbuf *obj_info) +{ +struct strbuf dirpath =STRBUF_INIT ; +struct string_list subdirs =STRING_LIST_INIT_DUP ; +struct string_list_item *subdir; + +strbuf_reset (obj_info); + +strbuf_addstr (&dirpath,get_object_directory ()); +strbuf_complete (&dirpath,'/' ); + +list_contents_of_dir (&subdirs, &dirpath,1 ,DT_DIR ); + +for_each_string_list_item (subdir, &subdirs) + { +struct strbuf subdir_buf =STRBUF_INIT ; +struct string_list objects =STRING_LIST_INIT_DUP ; + +/* +* Only interested in loose objects - so dirs named with the bugreport.c void get_populated_hooks(struct strbuf *hook_info) · line 219
Quoted 10 lines
bugreport.c +10
-0+* first byte of the object ID +*/ +if (strlen (subdir->string) !=2 || !strcmp (subdir->string,".." )) +continue ; + +strbuf_addbuf (&subdir_buf, &dirpath); +strbuf_addstr (&subdir_buf, subdir->string); +list_contents_of_dir (&objects, &subdir_buf,0 ,0 ); +strbuf_addf (obj_info,"%s: %d objects\n" , subdir->string, + objects.nr);Hmm. Not only does this leak
objects, it also throws away the contents that we so painfully constructed.Wouldn't it make more sense to do something like this instead?
Quoted 7 lines
static int
is_hex(const char *string, size_t count){ for (; count; string++, count--) if (hexval(*string) < 0) return 0; return 1; }
True if the string matches [0-9a-fA-F]*, false otherwise.
Quoted 13 lines
static
ssize_tcount_loose_objects(struct strbuf *objects_path){ ssize_t ret = 0; size_t len; struct dirent *d; DIR *dir, *subdir; dir = opendir(objects_path->buf); if (!dir) return -1; strbuf_complete(objects_path, '/');
starting with .git/objects/...
Quoted 2 lines
len = objects_path->len;
while ((d = readdir(dir))) {For all contents of dir...
Quoted 2 lines
if (d->d_type != DT_DIR) continue;
..which are directories...
Quoted 3 lines
strbuf_setlen(objects_path, len); strbuf_addstr(objects_path, d->d_name); subdir = opendir(objects_path->buf);
show all the contents of that subdirectory.
Quoted 3 lines
if (!subdir) continue; while ((d = readdir(subdir)))
for each regular file there,
Quoted 1 line
if (d->dt_type == DT_REG &&
if it's a regular file,
Quoted 1 line
is_hex(dir->d_name, the_repository->hash_algo->hexsz))
and the directory is named like an object prefix,
Quoted 1 line
ret++;
increase the total count of numbers of loose objects.
Quoted 6 lines
closedir(subdir); } closedir(dir); strbuf_reset(objects_path, len); return ret; }
The foremost difference here is that the loose object count previously was not given in total - instead, it was divided up by object prefix. I can't speak to whether that's actually very important to know about, but the original request from stolee was to have the summary by dirname. That's certainly still possible with a light modification to this code.
(Suggestion from Stolee, some months ago earlier in this thread:)
As mentioned before, I've sometimes found it helpful to know the data shape for the object store. Having a few extra steps such as the following could be nice:
echo "[Loose Objects]"
for objdir in $(find "$GIT_DIR/objects/??" -type d)
do
echo "$objdir: $(ls $objdir | wc -l)"
done
echo...
Checking that the directory we're about to inspect is hexval rather than that it's only two characters long is also an interesting point. I'd probably rather do both, though, since I think we both missed futureproofing by a little bit.
Dropping the many string_list is fine by me - call it object-oriented habits dying hard.
I worry somewhat on delving into every directory and only afterwards checking whether the directory is one we care about, but that's an easy modification too to your basic suggestion ("don't use a string list for that, silly").
Thanks. I'll make the changes with the modifications I mentioned and add your Helped-by line on this commit.
- Emily
Quoted 39 lines
Ciao,
DschoQuoted 35 lines
bugreport.c +2
-0+ } +}bugreport.h +6
-0@@ -18,3 +18,9 @@ void get_whitelisted_config(struct strbuf *sys_info);* contents of hook_info will be discarded. */ void get_populated_hooks (struct strbuf *hook_info); + +/** +* Adds the output of `git count-object -vH`. The previous contents of hook_info +* will be discarded. +*/ +void get_loose_object_summary (struct strbuf *obj_info);builtin/bugreport.c +4
-1@@ -64,6 +64,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_populated_hooks (&buffer);strbuf_write (&buffer, report); +add_header (report,"Object Counts" ); +get_loose_object_summary (&buffer); +strbuf_write (&buffer, report); +fclose (report);launch_editor (report_path.buf,NULL ,NULL ); --2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 6/9] bugreport: count loose objects
In reply to Emily ShafferOn 2019.10.24 19:51, Emily Shaffer wrote:
[snip]
Quoted 5 lines
bugreport.h +0
-0
bugreport.h void get_whitelisted_config(struct strbuf *sys_info); · line 26
Quoted 9 lines
bugreport.h +6
-0@@ -18,3 +18,9 @@ void get_whitelisted_config(struct strbuf *sys_info);* contents of hook_info will be discarded. */ void get_populated_hooks (struct strbuf *hook_info); + +/** +* Adds the output of `git count-object -vH`. The previous contents of hook_info +* will be discarded. +*/ +void get_loose_object_summary (struct strbuf *obj_info);
Looks like a copy/paste typo here, shouldn't "hook_info" be "obj_info" in the comment?
PATCH v3 7/9 bugreport: add packed object summary
In reply to Emily ShafferAlongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.
--- bugreport.c | 21 +++++++++++++++++++++ bugreport.h | 6 ++++++ builtin/bugreport.c | 4 ++++ 3 files changed, 31 insertions(+)
bugreport.c +21 -0
@@ -219,3 +219,24 @@ void get_loose_object_summary(struct strbuf *obj_info) objects.nr); } } + +void get_packed_object_summary (struct strbuf *obj_info) +{ +struct strbuf dirpath =STRBUF_INIT ; +struct string_list contents =STRING_LIST_INIT_DUP ; +struct string_list_item *entry; + +strbuf_reset (obj_info); + +strbuf_addstr (&dirpath,get_object_directory ()); +strbuf_complete (&dirpath,'/' ); +strbuf_addstr (&dirpath,"pack/" ); +list_contents_of_dir (&contents, &dirpath,0 ,0 ); + +// list full contents of $GIT_OBJECT_DIRECTORY/pack/ +for_each_string_list_item (entry, &contents) { +strbuf_addbuf (obj_info, &dirpath); +strbuf_addstr (obj_info, entry->string); +strbuf_complete_line (obj_info); + } +}
bugreport.h +6 -0
@@ -24,3 +24,9 @@ void get_populated_hooks(struct strbuf *hook_info);* will be discarded. */ void get_loose_object_summary (struct strbuf *obj_info); + +/** +* Adds a list of the contents of '.git/objects/pack'. The previous contents of +* hook_info will be discarded. +*/ +void get_packed_object_summary (struct strbuf *obj_info);
builtin/bugreport.c +4 -0
@@ -68,6 +68,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_loose_object_summary (&buffer);strbuf_write (&buffer, report); +add_header (report,"Packed Object Summary" ); +get_packed_object_summary (&buffer); +strbuf_write (&buffer, report); +fclose (report);launch_editor (report_path.buf,NULL ,NULL );
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 7/9] bugreport: add packed object summary
Hi Emily,
Quoted 3 lines · Emily Shaffer
Alongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.
Yes, I was write! Packs are next ;-)
Quoted 27 lines
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 21 +++++++++++++++++++++ bugreport.h | 6 ++++++ builtin/bugreport.c | 4 ++++ 3 files changed, 31 insertions(+)bugreport.c +11
-0@@ -219,3 +219,24 @@ void get_loose_object_summary(struct strbuf *obj_info) objects.nr); } } + +void get_packed_object_summary (struct strbuf *obj_info) +{ +struct strbuf dirpath =STRBUF_INIT ; +struct string_list contents =STRING_LIST_INIT_DUP ; +struct string_list_item *entry; + +strbuf_reset (obj_info); + +strbuf_addstr (&dirpath,get_object_directory ()); +strbuf_complete (&dirpath,'/' );
bugreport.c void get_loose_object_summary(struct strbuf *obj_info) · line 242
Quoted 10 lines
bugreport.c +10
-0+strbuf_addstr (&dirpath,"pack/" ); +list_contents_of_dir (&contents, &dirpath,0 ,0 ); + +// list full contents of $GIT_OBJECT_DIRECTORY/pack/ +for_each_string_list_item (entry, &contents) { +strbuf_addbuf (obj_info, &dirpath); +strbuf_addstr (obj_info, entry->string); +strbuf_complete_line (obj_info); + } +}
Okay, but I think that you will want to discern between regular .pack files, .idx files and tmp_* files.
Quoted 19 lines
bugreport.h +6
-0@@ -24,3 +24,9 @@ void get_populated_hooks(struct strbuf *hook_info);* will be discarded. */ void get_loose_object_summary (struct strbuf *obj_info); + +/** +* Adds a list of the contents of '.git/objects/pack'. The previous contents of +* hook_info will be discarded. +*/ +void get_packed_object_summary (struct strbuf *obj_info);builtin/bugreport.c +0
-0
builtin/bugreport.c int cmd_bugreport(int argc, const char **argv, const char *prefix) · line 74
Quoted 7 lines
builtin/bugreport.c +4
-0@@ -68,6 +68,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_loose_object_summary (&buffer);strbuf_write (&buffer, report); +add_header (report,"Packed Object Summary" ); +get_packed_object_summary (&buffer); +strbuf_write (&buffer, report); +
Hmm. At this point, I am unclear whether you want to write into an strbuf, or directly into a FILE *? I would rather have only one, not a mix.
Ciao,
Dscho
Quoted 7 lines
builtin/bugreport.c +0
-1fclose (report);launch_editor (report_path.buf,NULL ,NULL ); --2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 7/9] bugreport: add packed object summary
Quoted 42 lines · Johannes Schindelin
Hi Emily,
Quoted 3 lines · Emily Shaffer
Alongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.
Yes, I was write! Packs are next ;-)
Quoted 32 lines
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 21 +++++++++++++++++++++ bugreport.h | 6 ++++++ builtin/bugreport.c | 4 ++++ 3 files changed, 31 insertions(+)bugreport.c +16
-0@@ -219,3 +219,24 @@ void get_loose_object_summary(struct strbuf *obj_info) objects.nr); } } + +void get_packed_object_summary (struct strbuf *obj_info) +{ +struct strbuf dirpath =STRBUF_INIT ; +struct string_list contents =STRING_LIST_INIT_DUP ; +struct string_list_item *entry; + +strbuf_reset (obj_info); + +strbuf_addstr (&dirpath,get_object_directory ()); +strbuf_complete (&dirpath,'/' ); +strbuf_addstr (&dirpath,"pack/" ); +list_contents_of_dir (&contents, &dirpath,0 ,0 ); + +// list full contents of $GIT_OBJECT_DIRECTORY/pack/ +for_each_string_list_item (entry, &contents) {
Quoted 8 lines
bugreport.c void get_loose_object_summary(struct strbuf *obj_info) · line 242
Quoted 5 lines
bugreport.c +5
-0+strbuf_addbuf (obj_info, &dirpath); +strbuf_addstr (obj_info, entry->string); +strbuf_complete_line (obj_info); + } +}Okay, but I think that you will want to discern between regular
.packfiles,.idxfiles andtmp_*files.
Discern in what way? How would you like to see them treated separately? They're all being listed here, not just counted, so it seems to me like I can read the generated bugreport and see which files are index, pack, or temporary here.
Quoted 42 lines
Quoted 20 lines
bugreport.h +6
-0@@ -24,3 +24,9 @@ void get_populated_hooks(struct strbuf *hook_info);* will be discarded. */ void get_loose_object_summary (struct strbuf *obj_info); + +/** +* Adds a list of the contents of '.git/objects/pack'. The previous contents of +* hook_info will be discarded. +*/ +void get_packed_object_summary (struct strbuf *obj_info);builtin/bugreport.c +0
-0builtin/bugreport.c int cmd_bugreport(int argc, const char **argv, const char *prefix) · line 74
Quoted 7 lines
builtin/bugreport.c +4
-0@@ -68,6 +68,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_loose_object_summary (&buffer);strbuf_write (&buffer, report); +add_header (report,"Packed Object Summary" ); +get_packed_object_summary (&buffer); +strbuf_write (&buffer, report); +Hmm. At this point, I am unclear whether you want to write into an
strbuf, or directly into aFILE *? I would rather have only one, not a mix.Ciao,
DschoQuoted 7 lines
builtin/bugreport.c +0
-1fclose (report);launch_editor (report_path.buf,NULL ,NULL ); --2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 7/9] bugreport: add packed object summary
Hi Emily,
Quoted 52 lines · Emily Shaffer
Quoted 42 lines · Johannes Schindelin
Hi Emily,
Quoted 3 lines · Emily Shaffer
Alongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.
Yes, I was write! Packs are next ;-)
Quoted 32 lines
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 21 +++++++++++++++++++++ bugreport.h | 6 ++++++ builtin/bugreport.c | 4 ++++ 3 files changed, 31 insertions(+)bugreport.c +16
-0@@ -219,3 +219,24 @@ void get_loose_object_summary(struct strbuf *obj_info) objects.nr); } } + +void get_packed_object_summary (struct strbuf *obj_info) +{ +struct strbuf dirpath =STRBUF_INIT ; +struct string_list contents =STRING_LIST_INIT_DUP ; +struct string_list_item *entry; + +strbuf_reset (obj_info); + +strbuf_addstr (&dirpath,get_object_directory ()); +strbuf_complete (&dirpath,'/' ); +strbuf_addstr (&dirpath,"pack/" ); +list_contents_of_dir (&contents, &dirpath,0 ,0 ); + +// list full contents of $GIT_OBJECT_DIRECTORY/pack/ +for_each_string_list_item (entry, &contents) {Quoted 8 lines
bugreport.c void get_loose_object_summary(struct strbuf *obj_info) · line 242
Quoted 5 lines
bugreport.c +5
-0+strbuf_addbuf (obj_info, &dirpath); +strbuf_addstr (obj_info, entry->string); +strbuf_complete_line (obj_info); + } +}Okay, but I think that you will want to discern between regular
.packfiles,.idxfiles andtmp_*files.
Quoted 4 lines
Discern in what way? How would you like to see them treated separately? They're all being listed here, not just counted, so it seems to me like I can read the generated bugreport and see which files are index, pack, or temporary here.
I take your word for it (sorry, it's been half an eternity since I wrapped my head around the diff, I forgotten pretty much all about it ;-))
Ciao,
Dscho
Quoted 44 lines
Quoted 43 lines
Quoted 21 lines
bugreport.h +6
-0@@ -24,3 +24,9 @@ void get_populated_hooks(struct strbuf *hook_info);* will be discarded. */ void get_loose_object_summary (struct strbuf *obj_info); + +/** +* Adds a list of the contents of '.git/objects/pack'. The previous contents of +* hook_info will be discarded. +*/ +void get_packed_object_summary (struct strbuf *obj_info);builtin/bugreport.c +0
-0builtin/bugreport.c int cmd_bugreport(int argc, const char **argv, const char *prefix) · line 74
Quoted 7 lines
builtin/bugreport.c +4
-0@@ -68,6 +68,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_loose_object_summary (&buffer);strbuf_write (&buffer, report); +add_header (report,"Packed Object Summary" ); +get_packed_object_summary (&buffer); +strbuf_write (&buffer, report); +Hmm. At this point, I am unclear whether you want to write into an
strbuf, or directly into aFILE *? I would rather have only one, not a mix.Ciao,
DschoQuoted 7 lines
builtin/bugreport.c +0
-1fclose (report);launch_editor (report_path.buf,NULL ,NULL ); --2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 7/9] bugreport: add packed object summary
Quoted 56 lines · Johannes Schindelin
Hi Emily,
Quoted 52 lines · Emily Shaffer
Quoted 50 lines · Johannes Schindelin
Hi Emily,
Quoted 3 lines · Emily Shaffer
Alongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.
Yes, I was write! Packs are next ;-)
Quoted 27 lines
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 21 +++++++++++++++++++++ bugreport.h | 6 ++++++ builtin/bugreport.c | 4 ++++ 3 files changed, 31 insertions(+)bugreport.c +11
-0@@ -219,3 +219,24 @@ void get_loose_object_summary(struct strbuf *obj_info) objects.nr); } } + +void get_packed_object_summary (struct strbuf *obj_info) +{ +struct strbuf dirpath =STRBUF_INIT ; +struct string_list contents =STRING_LIST_INIT_DUP ; +struct string_list_item *entry; + +strbuf_reset (obj_info); + +strbuf_addstr (&dirpath,get_object_directory ()); +strbuf_complete (&dirpath,'/' );bugreport.c void get_loose_object_summary(struct strbuf *obj_info) · line 242
Quoted 10 lines
bugreport.c +10
-0+strbuf_addstr (&dirpath,"pack/" ); +list_contents_of_dir (&contents, &dirpath,0 ,0 ); + +// list full contents of $GIT_OBJECT_DIRECTORY/pack/ +for_each_string_list_item (entry, &contents) { +strbuf_addbuf (obj_info, &dirpath); +strbuf_addstr (obj_info, entry->string); +strbuf_complete_line (obj_info); + } +}Okay, but I think that you will want to discern between regular
.packfiles,.idxfiles andtmp_*files.
Quoted 7 lines
Quoted 4 lines
Discern in what way? How would you like to see them treated separately? They're all being listed here, not just counted, so it seems to me like I can read the generated bugreport and see which files are index, pack, or temporary here.
I take your word for it (sorry, it's been half an eternity since I wrapped my head around the diff, I forgotten pretty much all about it ;-))
That makes two of us. :)
Quoted 48 lines
Ciao,
DschoQuoted 44 lines
Quoted 43 lines
Quoted 21 lines
bugreport.h +6
-0@@ -24,3 +24,9 @@ void get_populated_hooks(struct strbuf *hook_info);* will be discarded. */ void get_loose_object_summary (struct strbuf *obj_info); + +/** +* Adds a list of the contents of '.git/objects/pack'. The previous contents of +* hook_info will be discarded. +*/ +void get_packed_object_summary (struct strbuf *obj_info);builtin/bugreport.c +0
-0builtin/bugreport.c int cmd_bugreport(int argc, const char **argv, const char *prefix) · line 74
Quoted 7 lines
builtin/bugreport.c +4
-0@@ -68,6 +68,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_loose_object_summary (&buffer);strbuf_write (&buffer, report); +add_header (report,"Packed Object Summary" ); +get_packed_object_summary (&buffer); +strbuf_write (&buffer, report); +Hmm. At this point, I am unclear whether you want to write into an
strbuf, or directly into aFILE *? I would rather have only one, not a mix.Ciao,
DschoQuoted 7 lines
builtin/bugreport.c +0
-1fclose (report);launch_editor (report_path.buf,NULL ,NULL ); --2.24.0.rc0.303.g954a862665-goog
PATCH v3 8/9 bugreport: list contents of $OBJDIR/info
In reply to Emily ShafferMiscellaneous information used about the object store can end up in .git/objects/info; this can help us understand what may be going on with the object store when the user is reporting a bug. Otherwise, it could be difficult to track down what is going wrong with an object which isn't kept locally to .git/objects/ or .git/objects/pack. Having some understanding of where the user's objects may be kept can save us some hops during the bug reporting process.
--- bugreport.c | 58 +++++++++++++++++++++++++++++++++++++++++++++ bugreport.h | 6 +++++ builtin/bugreport.c | 7 ++++++ 3 files changed, 71 insertions(+)
bugreport.c +58 -0
@@ -6,6 +6,7 @@#include "help.h" #include "run-command.h" #include "strbuf.h" +#include "string-list.h" #include "version.h" #include "dirent.h"
@@ -171,6 +172,41 @@ void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath, } } +/** +* Fills 'contents' with a list of all directories within the provided +* directory, recursing into each directory. +*/ +void list_contents_of_dir_recursively (struct string_list *contents, +struct strbuf *dirpath) +{ +struct string_list current_contents =STRING_LIST_INIT_DUP ; +struct string_list current_subdirs =STRING_LIST_INIT_DUP ; +struct string_list_item *it; +struct strbuf buf =STRBUF_INIT ; + +list_contents_of_dir (¤t_contents, dirpath,0 ,0 ); +for_each_string_list_item (it, ¤t_contents) { +strbuf_reset (&buf); +strbuf_addbuf (&buf, dirpath); +strbuf_complete (&buf,'/' ); +strbuf_addstr (&buf, it->string); + +string_list_append (contents, buf.buf); + } + +list_contents_of_dir (¤t_subdirs, dirpath,1 ,DT_DIR ); +for_each_string_list_item (it, ¤t_subdirs) { +if (strcmp (it->string,"." ) ==0 + ||strcmp (it->string,".." ) ==0 ) +continue ; +strbuf_reset (&buf); +strbuf_addbuf (&buf, dirpath); +strbuf_complete (&buf,'/' ); +strbuf_addstr (&buf, it->string); + +list_contents_of_dir_recursively (contents, &buf); + } +}void get_object_counts (struct strbuf *obj_info) {
@@ -240,3 +276,25 @@ void get_packed_object_summary(struct strbuf *obj_info)strbuf_complete_line (obj_info); } } + +void get_object_info_summary (struct strbuf *obj_info) +{ +// strbuf += GITDIR/info/: +// recursively list contents of $GIT_OBJECT_DIRECTORY/info +struct strbuf dirpath =STRBUF_INIT ; +struct string_list contents =STRING_LIST_INIT_DUP ; +struct string_list_item *entry; + +strbuf_reset (obj_info); + +strbuf_addstr (&dirpath,get_object_directory ()); +strbuf_complete (&dirpath,'/' ); +strbuf_addstr (&dirpath,"info/" ); + +list_contents_of_dir_recursively (&contents, &dirpath); + +for_each_string_list_item (entry, &contents) { +strbuf_addstr (obj_info, entry->string); +strbuf_complete_line (obj_info); + } +}
bugreport.h +6 -0
@@ -30,3 +30,9 @@ void get_loose_object_summary(struct strbuf *obj_info);* hook_info will be discarded. */ void get_packed_object_summary (struct strbuf *obj_info); + +/** +* Adds a list of all contents (recursively) of '.git/objects/info'. The +* previous contents of hook_info will be discarded. +*/ +void get_object_info_summary (struct strbuf *obj_info);
builtin/bugreport.c +7 -0
@@ -72,6 +72,13 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_packed_object_summary (&buffer);strbuf_write (&buffer, report); +add_header (report,"Object Info Data" ); +get_object_info_summary (&buffer); +strbuf_write (&buffer, report); + +// Close file +// open file in editor +launch_editor (report_path,NULL ,NULL );fclose (report);launch_editor (report_path.buf,NULL ,NULL );
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 8/9] bugreport: list contents of $OBJDIR/info
Hi Emily,
Quoted 7 lines · Emily Shaffer
Miscellaneous information used about the object store can end up in .git/objects/info; this can help us understand what may be going on with the object store when the user is reporting a bug. Otherwise, it could be difficult to track down what is going wrong with an object which isn't kept locally to .git/objects/ or .git/objects/pack. Having some understanding of where the user's objects may be kept can save us some hops during the bug reporting process.
Makes sense.
Quoted 23 lines
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 58 +++++++++++++++++++++++++++++++++++++++++++++ bugreport.h | 6 +++++ builtin/bugreport.c | 7 ++++++ 3 files changed, 71 insertions(+)bugreport.c +1
-0@@ -6,6 +6,7 @@#include "help.h" #include "run-command.h" #include "strbuf.h" +#include "string-list.h" #include "version.h" #include "dirent.h" @@ -171,6 +172,41 @@ void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath, } }
bugreport.c void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath, · line 183
Quoted 10 lines
bugreport.c +9
-0+/** +* Fills 'contents' with a list of all directories within the provided +* directory, recursing into each directory. +*/ +void list_contents_of_dir_recursively (struct string_list *contents, +struct strbuf *dirpath) +{ +struct string_list current_contents =STRING_LIST_INIT_DUP ; +struct string_list current_subdirs =STRING_LIST_INIT_DUP ;
Rather than populating those string lists, why not write into the contents directly, _while_ traversing the directories?
But if you really, really want to use string lists (hey, I'm a fan, I introduced them to Git's source code, after all), then please make sure to release them in the end, too.
bugreport.c void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath, · line 192
Quoted 9 lines
bugreport.c +9
-0+struct string_list_item *it; +struct strbuf buf =STRBUF_INIT ; + +list_contents_of_dir (¤t_contents, dirpath,0 ,0 ); +for_each_string_list_item (it, ¤t_contents) { +strbuf_reset (&buf); +strbuf_addbuf (&buf, dirpath); +strbuf_complete (&buf,'/' ); +strbuf_addstr (&buf, it->string);
You don't need a separate strbuf for that: just complete dirpath with a trailing slash, remember the length, then in each iteration reset the length and append the current subdirectory name.
Quoted 35 lines
bugreport.c +28
-0+ +string_list_append (contents, buf.buf); + } + +list_contents_of_dir (¤t_subdirs, dirpath,1 ,DT_DIR ); +for_each_string_list_item (it, ¤t_subdirs) { +if (strcmp (it->string,"." ) ==0 + ||strcmp (it->string,".." ) ==0 ) +continue ; +strbuf_reset (&buf); +strbuf_addbuf (&buf, dirpath); +strbuf_complete (&buf,'/' ); +strbuf_addstr (&buf, it->string); + +list_contents_of_dir_recursively (contents, &buf); + } +}void get_object_counts (struct strbuf *obj_info) {@@ -240,3 +276,25 @@ void get_packed_object_summary(struct strbuf *obj_info)strbuf_complete_line (obj_info); } } + +void get_object_info_summary (struct strbuf *obj_info) +{ +// strbuf += GITDIR/info/: +// recursively list contents of $GIT_OBJECT_DIRECTORY/info +struct strbuf dirpath =STRBUF_INIT ; +struct string_list contents =STRING_LIST_INIT_DUP ; +struct string_list_item *entry; + +strbuf_reset (obj_info); +
bugreport.c void get_packed_object_summary(struct strbuf *obj_info) · line 299
Quoted 10 lines
bugreport.c +10
-0+strbuf_addstr (&dirpath,get_object_directory ()); +strbuf_complete (&dirpath,'/' ); +strbuf_addstr (&dirpath,"info/" ); + +list_contents_of_dir_recursively (&contents, &dirpath); + +for_each_string_list_item (entry, &contents) { +strbuf_addstr (obj_info, entry->string); +strbuf_complete_line (obj_info); + }
As mentioned earlier, I would rather write things into obj_info directly, without detouring to a string list.
Or, if you decide that you want to write to a FILE * anyway, I'd rather avoid the detour to the strbuf altogether.
Ciao,
Dscho
Quoted 37 lines
bugreport.c +1
-0+}bugreport.h +6
-0@@ -30,3 +30,9 @@ void get_loose_object_summary(struct strbuf *obj_info);* hook_info will be discarded. */ void get_packed_object_summary (struct strbuf *obj_info); + +/** +* Adds a list of all contents (recursively) of '.git/objects/info'. The +* previous contents of hook_info will be discarded. +*/ +void get_object_info_summary (struct strbuf *obj_info);builtin/bugreport.c +7
-1@@ -72,6 +72,13 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_packed_object_summary (&buffer);strbuf_write (&buffer, report); +add_header (report,"Object Info Data" ); +get_object_info_summary (&buffer); +strbuf_write (&buffer, report); + +// Close file +// open file in editor +launch_editor (report_path,NULL ,NULL );fclose (report);launch_editor (report_path.buf,NULL ,NULL ); --2.24.0.rc0.303.g954a862665-goog
PATCH v3 9/9 bugreport: print contents of alternates file
In reply to Emily ShafferIn some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead.
--- bugreport.c | 14 ++++++++++++++ bugreport.h | 6 ++++++ builtin/bugreport.c | 4 ++++ 3 files changed, 24 insertions(+)
bugreport.c +14 -0
@@ -298,3 +298,17 @@ void get_object_info_summary(struct strbuf *obj_info)strbuf_complete_line (obj_info); } } + +void get_alternates_file (struct strbuf *alternates_info) +{ +struct strbuf alternates_path =STRBUF_INIT ; + +strbuf_addstr (&alternates_path,get_object_directory ()); +strbuf_complete (&alternates_path,'/' ); +strbuf_addstr (&alternates_path,"info/alternates" ); + +strbuf_reset (alternates_info); +strbuf_addbuf (alternates_info, &alternates_path); +strbuf_complete_line (alternates_info); +strbuf_read_file (alternates_info, alternates_path.buf,0 ); +}
bugreport.h +6 -0
@@ -36,3 +36,9 @@ void get_packed_object_summary(struct strbuf *obj_info);* previous contents of hook_info will be discarded. */ void get_object_info_summary (struct strbuf *obj_info); + +/** +* Adds the contents of '.git/info/alternates'. The previous contents of +* alternates_info will be discarded. +*/ +void get_alternates_file (struct strbuf *alt_info);
builtin/bugreport.c +4 -0
@@ -76,6 +76,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_object_info_summary (&buffer);strbuf_write (&buffer, report); +add_header (report,"Alternates File" ); +get_alternates_file (&buffer); +strbuf_write (&buffer, report); +// Close file // open file in editor launch_editor (report_path,NULL ,NULL );
Signature · 1 line
2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 9/9] bugreport: print contents of alternates file
Hi Emily,
Quoted 4 lines · Emily Shaffer
In some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead.
Doesn't this open the possibility of leaking project's (possibly NDA'ed) names?
I could imagine that we might rather want to count the alternates, and maybe separate into those alternates that actually exist and alternates that do not exist (which would produce a warning that the user might have trained themselves to ignore).
Ciao,
Dscho
Quoted 63 lines
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 14 ++++++++++++++ bugreport.h | 6 ++++++ builtin/bugreport.c | 4 ++++ 3 files changed, 24 insertions(+)bugreport.c +14
-0@@ -298,3 +298,17 @@ void get_object_info_summary(struct strbuf *obj_info)strbuf_complete_line (obj_info); } } + +void get_alternates_file (struct strbuf *alternates_info) +{ +struct strbuf alternates_path =STRBUF_INIT ; + +strbuf_addstr (&alternates_path,get_object_directory ()); +strbuf_complete (&alternates_path,'/' ); +strbuf_addstr (&alternates_path,"info/alternates" ); + +strbuf_reset (alternates_info); +strbuf_addbuf (alternates_info, &alternates_path); +strbuf_complete_line (alternates_info); +strbuf_read_file (alternates_info, alternates_path.buf,0 ); +}bugreport.h +6
-0@@ -36,3 +36,9 @@ void get_packed_object_summary(struct strbuf *obj_info);* previous contents of hook_info will be discarded. */ void get_object_info_summary (struct strbuf *obj_info); + +/** +* Adds the contents of '.git/info/alternates'. The previous contents of +* alternates_info will be discarded. +*/ +void get_alternates_file (struct strbuf *alt_info);builtin/bugreport.c +4
-1@@ -76,6 +76,10 @@ int cmd_bugreport(int argc, const char **argv, const char *prefix)get_object_info_summary (&buffer);strbuf_write (&buffer, report); +add_header (report,"Alternates File" ); +get_alternates_file (&buffer); +strbuf_write (&buffer, report); +// Close file // open file in editor launch_editor (report_path,NULL ,NULL ); --2.24.0.rc0.303.g954a862665-goog
Re: [PATCH v3 9/9] bugreport: print contents of alternates file
Quoted 9 lines · Johannes Schindelin
Hi Emily,
Quoted 4 lines · Emily Shaffer
In some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead.
Quoted 6 lines
Doesn't this open the possibility of leaking project's (possibly NDA'ed) names?
I could imagine that we might rather want to count the alternates, and maybe separate into those alternates that actually exist and alternates that do not exist (which would produce a warning that the user might have trained themselves to ignore).
Sounds reasonable. Will do.
- Emily
Re: [PATCH v3 0/9] add git-bugreport tool
In reply to Emily ShafferQuoted 6 lines · Emily Shaffer
Thanks for the patience with the long wait, all. Here's an attempt at the rewrite in C; I think it does verbatim what the sh version did except that this doesn't print the reflog - Jonathan Nieder was good enough to point out to me that folks probably don't want to share their commit subjects all willy-nilly if they're working on something secretive.
Is the goal to give a tool the end users can type "git bugreport<RET>" and automatically send the result to this mailing list (or some bug tracker)? Or is this only about producing a pre-filled bug report template to be slurped into their MUA and then further manually edited before sending it out?
It probably is controversial if we exposed contents of hooks scripts (I can imagine some people may trigger external process by pinging a URL that includes credential material out of laziness), so the presence test you have is probably a good stopping point. I do not know how much it helps to know which hooks exist in order to diagnose an anomaly without knowing what their contents are, but it is a start.
By the way, I doubt that it is the best use of developer time to write these in C---taking various pieces of information from different places to prepare a text file sounds more suited to scripting languages, especially while we are still learning what kind of information we want to collect and how we want to present the final result (iow, for the first handful of years after deploying this command).
Thanks.
Re: [PATCH v3 0/9] add git-bugreport tool
Hi Junio,
Quoted 23 lines · Junio C Hamano
Quoted 6 lines · Emily Shaffer
Thanks for the patience with the long wait, all. Here's an attempt at the rewrite in C; I think it does verbatim what the sh version did except that this doesn't print the reflog - Jonathan Nieder was good enough to point out to me that folks probably don't want to share their commit subjects all willy-nilly if they're working on something secretive.
Is the goal to give a tool the end users can type "git bugreport<RET>" and automatically send the result to this mailing list (or some bug tracker)? Or is this only about producing a pre-filled bug report template to be slurped into their MUA and then further manually edited before sending it out?
It probably is controversial if we exposed contents of hooks scripts (I can imagine some people may trigger external process by pinging a URL that includes credential material out of laziness), so the presence test you have is probably a good stopping point. I do not know how much it helps to know which hooks exist in order to diagnose an anomaly without knowing what their contents are, but it is a start.
Quoted 7 lines
By the way, I doubt that it is the best use of developer time to write these in C---taking various pieces of information from different places to prepare a text file sounds more suited to scripting languages, especially while we are still learning what kind of information we want to collect and how we want to present the final result (iow, for the first handful of years after deploying this command).
If you want to limit Git to Linux, then relying on a specific scripting language like Unix shell scripting sounds okay.
If you target more platforms, then a range of scripting languages sounds a lot more sensible, my top choice would be node.js.
I understand that that is not an option here, and there is a strong preference for Unix shell scripting in Git. This is a bit unfortunate:
A good chunk of the bug reports I see in Git for Windows are _about_ problems with Unix shell scripting. There are fork problems, problems where add_item failed (which is one of the least helpful error messages I saw in my entire life), there are problems with anti-malware causing segmentation faults in the Unix shell script interpreter (the MSYS2 Bash), and there are even (believe it or not) problems with _display drivers_. Yes, that is right, display drivers can interfere with Unix shell script execution on Windows. (This is really rare, but hey, bug reports are rare in Git for Windows to begin with...)
All of these problems _prevent_ Unix shell scripts from being interpreted as expected on Windows. Which would ridicule the idea of reporting any of these bugs using a tool that is implemented in Unix shell.
And what is worse than a bug in an application? A bug in the application's very own bug reporting tool.
I pointed this out to Emily, and that is the reason why this amount of work was put in to _improve_ the tool, precisely by _not_ implementing it as a Unix shell script, but instead in plain C (which is a _lot_ more robust).
Ciao,
Dscho
v4 cover add git-bugreport tool
In reply to Emily ShafferThis topic branch depends on the patch mailed in lore.kernel.org/git/[email protected] in order to display scopes for configs gathered during "bugreport: add config values from safelist".
Quite a few changes since v3. I'll try to summarize the big picture stuff, plus highlight new changes and difference in patches inline in the commit list. I'll also stick an interdiff since v3 at the bottom of this cover letter.
Some overall changes:
- The entire command now exists in one file, bugreport.c, which is compiled as a standalone.
- static-ness of local functions has been cleaned up.
- strbuf lifetime management has been cleaned up.
- Mention of "whitelist" and "blacklist" has been changed to "safelist" and "blocklist" while dealing with Git-approved bugreportable configs. Thanks Dscho for pointing this out; "safelist" makes sense to me as these are configs we have deemed safe for public inspection.
- Inquiry into both Git's $SHELL_PATH and the user's $SHELL is new to v4.
- The 'man git bugreport' page which was lost from the script version of git-bugreport has been recovered, as have the "tests" (which only test whether it runs and doesn't crash).
Emily Shaffer (15):
bugreport: add tool to generate debugging info
Moved from library + builtin to a standalone git-bugreport binary.
help: move list_config_help to builtin/help
Resolves a circular dependency where using help.o required all builtins to exist. This also moves generation of config-list.h into its own script (it was previously roommates with command-list.h).
The following 4 commits were previously a single commit which used run-command.h to call different processes; this has been removed except for the cURL version.
bugreport: gather git version and build info
Additionally capture the "--build-info" output from 'git version' where it was not previously captured.
help: add shell-path to --build-options
Adding $SHELL_PATH to 'git --version --build-options' gets it in the bugreport for free. (This is the shell Git uses, determined at build time.)
bugreport: add uname info
Capture uname from the library instead of invoking a different process.
bugreport: add glibc version
Capture glibc version info from the library instead of invoking the 'ldd' process.
bugreport: add curl version
Capture the cURL version from git-http-fetch, and add a -V flag to that binary for this purpose.
bugreport: include user interactive shell
A new feature; check what $SHELL the user is using (at runtime)
bugreport: generate config safelist based on docs
This commit squashes in suggestions both from Johannes Schindelin and Martin Ågren, hopefully resulting in Asciidoc(tor) not showing the safelist macros, and a more portable safelist generation script. It also puts the safelist, sorted, directly into a header instead of into a text file, saving us a file read at runtime.
bugreport: add config values from safelist
At Dscho's suggestion, now we put these into a map (based on khash and oidset.h's example) for constant-time lookup during the config parse.
bugreport: collect list of populated hooks
Only a minor change - now this works if 'git bugreport' is invoked outside of a Git repo.
bugreport: count loose objects
bugreport: add packed object summary
bugreport: list contents of $OBJDIR/info
For the above 3, the entire 'string_list' has been refactored out; now the obj_info buffer is written to directly during the directory traversal. Also paid more attention to when to free strbuf used for paths.
bugreport: summarize contents of alternates file
Rather than printing the contents of the alternates file, now the number of specified alternate stores are listed, and they are tested to check whether they exist. This way we stop looking at user filepaths (sensitive) and start learning whether they have a broken alternate (new and useful info).
.gitignore | 3 + Documentation/asciidoc.conf | 8 + Documentation/asciidoctor-extensions.rb | 7 + Documentation/config/sendemail.txt | 68 ++-- Documentation/git-bugreport.txt | 43 +++ Documentation/git-http-fetch.txt | 5 + Makefile | 31 +- bugreport.c | 434 ++++++++++++++++++++++++ builtin/help.c | 86 +++++ generate-bugreport-config-safelist.sh | 22 ++ generate-cmdlist.sh | 19 -- generate-configlist.sh | 24 ++ help.c | 131 ++----- help.h | 2 +- http-fetch.c | 13 +- t/t0091-bugreport.sh | 41 +++ 16 files changed, 774 insertions(+), 163 deletions(-) create mode 100644 Documentation/git-bugreport.txt create mode 100644 bugreport.c create mode 100755 generate-bugreport-config-safelist.sh create mode 100755 generate-configlist.sh create mode 100755 t/t0091-bugreport.sh
.gitignore +2 -0
@@ -189,7 +189,9 @@ /gitweb/gitweb.cgi /gitweb/static/gitweb.js /gitweb/static/gitweb.min.* +/config-list.h /command-list.h +/bugreport-config-safelist.h *.tar.gz *.dsc *.deb
Documentation/asciidoc.conf +8 -0
@@ -6,9 +6,13 @@ # # Show Git link as: <command>(<section>); if section is defined, else just show # the command. +# +# The bugreport macro does nothing as far as rendering is +# concerned -- we just grep for it in the sources. [macros] (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]= +(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]= [attributes] asterisk=*
@@ -28,6 +32,8 @@ ifdef::backend-docbook[] {0#<citerefentry>} {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>} {0#</citerefentry>} +[bugreport-inlinemacro] +{0#} endif::backend-docbook[] ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[] git-relative-html-prefix= [linkgit-inlinemacro] <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a> +[bugreport-inlinemacro] +<!-- --> endif::backend-xhtml11[]
Documentation/asciidoctor-extensions.rb +7 -0
@@ -37,6 +37,10 @@ module Git output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>") end output + + class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor + def process(parent, action, attrs) + "" end end end
@@ -45,4 +49,7 @@ end Asciidoctor::Extensions.register do inline_macro Git::Documentation::LinkGitProcessor, :linkgit postprocessor Git::Documentation::DocumentPostProcessor + # The bugreport macro does nothing as far as rendering is + # concerned -- we just grep for it in the sources. + inline_macro Git::Documentation::BugReportProcessor, :bugreport end
Documentation/config/sendemail.txt +34 -34
97 lines not on this page · raw message
Documentation/git-bugreport.txt new file +43 -0
43 lines not on this page · raw message
Documentation/git-http-fetch.txt +5 -0
@@ -10,6 +10,7 @@ SYNOPSIS -------- [verse] 'git http-fetch' [-c] [-t] [-a] [-d] [-v] [-w filename] [--recover] [--stdin] <commit> <url> +'git http-fetch' [-V] DESCRIPTION -----------
@@ -30,6 +31,10 @@ commit-id:: -v:: Report what is downloaded. +-V:: + Report information about the version of git-http-fetch, including the + versions of its dependencies. + -w <filename>:: Writes the commit-id into the filename under $GIT_DIR/refs/<filename> on the local end after the transfer is complete.
Makefile +25 -13
@@ -634,10 +634,6 @@ SCRIPT_PYTHON += git-p4.py SCRIPT_SH_GEN = $(patsubst %.sh,%,$(SCRIPT_SH)) SCRIPT_PERL_GEN = $(patsubst %.perl,%,$(SCRIPT_PERL)) SCRIPT_PYTHON_GEN = $(patsubst %.py,%,$(SCRIPT_PYTHON)) -SCRIPT_DEPENDENCIES = git-bugreport-config-whitelist - -$(SCRIPT_DEPENDENCIES): Documentation/config/*.txt - sh bugreport-generate-config-whitelist.sh # Individual rules to allow e.g. # "make -C ../.. SCRIPT_PERL=contrib/foo/bar.perl build-perl-script"
@@ -662,13 +658,10 @@ clean-perl-script: $(RM) $(SCRIPT_PERL_GEN) clean-python-script: $(RM) $(SCRIPT_PYTHON_GEN) -clean-script-dependencies: - $(RM) $(SCRIPT_DEPENDENCIES) SCRIPTS = $(SCRIPT_SH_GEN) \ $(SCRIPT_PERL_GEN) \ $(SCRIPT_PYTHON_GEN) \ - $(SCRIPT_DEPENDENCIES) \ git-instaweb ETAGS_TARGET = TAGS
@@ -688,6 +681,7 @@ EXTRA_PROGRAMS = # ... and all the rest that could be moved out of bindir to gitexecdir PROGRAMS += $(EXTRA_PROGRAMS) +PROGRAM_OBJS += bugreport.o PROGRAM_OBJS += credential-store.o PROGRAM_OBJS += daemon.o PROGRAM_OBJS += fast-import.o
@@ -821,7 +815,9 @@ LIB_FILE = libgit.a XDIFF_LIB = xdiff/lib.a VCSSVN_LIB = vcs-svn/lib.a +GENERATED_H += config-list.h GENERATED_H += command-list.h +GENERATED_H += bugreport-config-safelist.h LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \ $(FIND) . \
@@ -846,7 +842,6 @@ LIB_OBJS += bisect.o LIB_OBJS += blame.o LIB_OBJS += blob.o LIB_OBJS += branch.o -LIB_OBJS += bugreport.o LIB_OBJS += bulk-checkin.o LIB_OBJS += bundle.o LIB_OBJS += cache-tree.o
@@ -1049,7 +1044,6 @@ BUILTIN_OBJS += builtin/archive.o BUILTIN_OBJS += builtin/bisect--helper.o BUILTIN_OBJS += builtin/blame.o BUILTIN_OBJS += builtin/branch.o -BUILTIN_OBJS += builtin/bugreport.o BUILTIN_OBJS += builtin/bundle.o BUILTIN_OBJS += builtin/cat-file.o BUILTIN_OBJS += builtin/check-attr.o
@@ -2135,7 +2129,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS) help.sp help.s help.o: command-list.h -builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX +builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \ '-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \ '-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2155,6 +2149,12 @@ $(BUILT_INS): git$X ln -s $< $@ 2>/dev/null || \ cp $< $@ +config-list.h: generate-configlist.sh + +config-list.h: + $(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \ + >$@+ && mv $@+ $@ + command-list.h: generate-cmdlist.sh command-list.txt command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2162,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc $(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \ command-list.txt >$@+ && mv $@+ $@ +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh + +bugreport-config-safelist.h: Documentation/config/*.txt + $(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \ + >$@+ && mv $@+ $@ + SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\ $(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\ $(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
@@ -2457,6 +2463,10 @@ endif git-%$X: %.o GIT-LDFLAGS $(GITLIBS) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS) +git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS) + $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \ + $(LIBS) + git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \ $(IMAP_SEND_LDFLAGS) $(LIBS)
@@ -2788,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE .PHONY: sparse $(SP_OBJ) sparse: $(SP_OBJ) -GEN_HDRS := command-list.h unicode-width.h +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/% ifndef GCRYPT_SHA256 EXCEPT_HDRS += sha256/gcrypt.h
@@ -2811,7 +2821,7 @@ hdr-check: $(HCO) style: git clang-format --style file --diff --extensions c,h -check: command-list.h +check: config-list.h command-list.h @if sparse; \ then \ echo >&2 "Use 'make sparse' instead"; \
@@ -3114,7 +3124,9 @@ clean: profile-clean coverage-clean cocciclean $(RM) $(HCC) $(RM) -r bin-wrappers $(dep_dirs) $(RM) -r po/build/ - $(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope* + $(RM) *.pyc *.pyo */*.pyc */*.pyo + $(RM) config-list.h command-list.h bugreport-config-safelist.h + $(RM) $(ETAGS_TARGET) tags cscope* $(RM) -r $(GIT_TARNAME) .doc-tmp-dir $(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz $(RM) $(htmldocs).tar.gz $(manpages).tar.gz
bugreport-generate-config-whitelist.sh deleted +0 -4
@@ -1,4 +0,0 @@ -#!/bin/sh - -grep -RhPo ".*(?=:: \/\/ bugreport-include)" Documentation/config \ - >git-bugreport-config-whitelist
bugreport.c +312 -192
597 lines not on this page · raw message
bugreport.h deleted +0 -44
44 lines not on this page · raw message
builtin.h +0 -1
@@ -135,7 +135,6 @@ int cmd_archive(int argc, const char **argv, const char *prefix);int cmd_bisect__helper (int argc,const char **argv,const char *prefix);int cmd_blame (int argc,const char **argv,const char *prefix);int cmd_branch (int argc,const char **argv,const char *prefix); -int cmd_bugreport (int argc,const char **argv,const char *prefix);int cmd_bundle (int argc,const char **argv,const char *prefix);int cmd_cat_file (int argc,const char **argv,const char *prefix);int cmd_checkout (int argc,const char **argv,const char *prefix);
builtin/bugreport.c deleted +0 -90
90 lines not on this page · raw message
builtin/help.c +86 -0
98 lines not on this page · raw message
config.c +17 -0
@@ -3312,6 +3312,23 @@ enum config_scope current_config_scope(void)return current_parsing_scope; } +const char *config_scope_to_string (enum config_scope scope) +{ +switch (scope) { +case CONFIG_SCOPE_SYSTEM : +return "system" ; +case CONFIG_SCOPE_GLOBAL : +return "global" ; +case CONFIG_SCOPE_REPO : +return "repo" ; +case CONFIG_SCOPE_CMDLINE : +return "cmdline" ; +case CONFIG_SCOPE_UNKNOWN : +default : +return "unknown" ; + } +} +int lookup_config (const char **mapping,int nr_mapping,const char *var) {int i;
config.h +1 -0
@@ -303,6 +303,7 @@ enum config_scope { };enum config_scope current_config_scope (void ); +const char *config_scope_to_string (enum config_scope );const char *current_config_origin_type (void );const char *current_config_name (void );
generate-bugreport-config-safelist.sh new file +22 -0
@@ -0,0 +1,22 @@ +#!/bin/sh + +cat <<EOF +/* Automatically generated by bugreport-generate-config-safelist.sh */ + + +static const char *bugreport_config_safelist[] = { +EOF + +# cat all regular files in Documentation/config +find Documentation/config -type f -exec cat {} \; | +# print the command name which matches the bugreport-include macro +sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' | +sort | +while read line +do + echo " \"$line\"," +done + +cat <<EOF +}; +EOF
generate-cmdlist.sh +0 -19
@@ -76,23 +76,6 @@ print_command_list () { echo "};" } -print_config_list () { - cat <<EOF -static const char *config_name_list[] = { -EOF - grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt | - sed '/deprecated/d; s/::$//; s/, */\n/g' | - sort | - while read line - do - echo " \"$line\"," - done - cat <<EOF - NULL, -}; -EOF -} - exclude_programs= while test "--exclude-program" = "$1" do
@@ -113,5 +96,3 @@ echo define_category_names "$1" echo print_command_list "$1" -echo -print_config_list
generate-configlist.sh new file +24 -0
@@ -0,0 +1,24 @@ +#!/bin/sh + +echo "/* Automatically generated by generate-configlist.sh */" +echo + +print_config_list () { + cat <<EOF +static const char *config_name_list[] = { +EOF + grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt | + sed '/deprecated/d; s/::$//; s/, */\n/g' | + sort | + while read line + do + echo " \"$line\"," + done + cat <<EOF + NULL, +}; +EOF +} + +echo +print_config_list
git.c +0 -1
@@ -473,7 +473,6 @@ static struct cmd_struct commands[] = { { "bisect--helper", cmd_bisect__helper, RUN_SETUP }, { "blame", cmd_blame, RUN_SETUP }, { "branch", cmd_branch, RUN_SETUP | DELAY_PAGER_CONFIG }, - { "bugreport", cmd_bugreport, RUN_SETUP }, { "bundle", cmd_bundle, RUN_SETUP_GENTLY | NO_PARSEOPT }, { "cat-file", cmd_cat_file, RUN_SETUP }, { "check-attr", cmd_check_attr, RUN_SETUP },
help.c +28 -103
152 lines not on this page · raw message
help.h +1 -1
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)void list_common_cmds_help (void );void list_all_cmds_help (void );void list_common_guides_help (void ); -void list_config_help (int for_human);void list_all_main_cmds (struct string_list *list);void list_all_other_cmds (struct string_list *list);
@@ -38,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);void exclude_cmds (struct cmdnames *cmds,struct cmdnames *excludes);int is_in_cmdlist (struct cmdnames *cmds,const char *name);void list_commands (unsigned int colopts,struct cmdnames *main_cmds,struct cmdnames *other_cmds); +void list_version_info (struct strbuf *buf,int build_options);/* * call this to die(), when it is suspected that the user mistyped a
http-fetch.c +12 -1
@@ -3,9 +3,18 @@#include "exec-cmd.h" #include "http.h" #include "walker.h" +#include "version.h" static const char http_fetch_usage[] ="git http-fetch " -"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url" ; +"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url" ; + +void NORETURN version_info() +{ +printf ("git-http-fetch version: %s\n" , git_version_string); +printf ("built from commit: %s\n" , git_built_from_commit_string); +printf ("curl version: %s\n" ,curl_version ()); +exit (0 ); +}int cmd_main (int argc,const char **argv) {
@@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv) }else if (argv[arg][1 ] =='a' ) { }else if (argv[arg][1 ] =='v' ) { get_verbosely =1 ; + }else if (argv[arg][1 ] =='V' ) { +version_info (); }else if (argv[arg][1 ] =='w' ) { write_ref = &argv[arg +1 ]; arg++;
t/t0091-bugreport.sh new file +41 -0
41 lines not on this page · raw message
Signature · 1 line
2.24.1.735.g03f4e72817-goog
PATCH v4 1/15 bugreport: add tool to generate debugging info
Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.
If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.
Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.
--- .gitignore | 1 + Documentation/git-bugreport.txt | 43 ++++++++++++++++++++ Makefile | 5 +++ bugreport.c | 69 +++++++++++++++++++++++++++++++++ t/t0091-bugreport.sh | 41 ++++++++++++++++++++ 5 files changed, 159 insertions(+) create mode 100644 Documentation/git-bugreport.txt create mode 100644 bugreport.c create mode 100755 t/t0091-bugreport.sh
.gitignore +1 -0
@@ -25,6 +25,7 @@ /git-bisect--helper /git-blame /git-branch +/git-bugreport /git-bundle /git-cat-file /git-check-attr
Documentation/git-bugreport.txt new file +43 -0
43 lines not on this page · raw message
Makefile +5 -0
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS = # ... and all the rest that could be moved out of bindir to gitexecdir PROGRAMS += $(EXTRA_PROGRAMS) +PROGRAM_OBJS += bugreport.o PROGRAM_OBJS += credential-store.o PROGRAM_OBJS += daemon.o PROGRAM_OBJS += fast-import.o
@@ -2448,6 +2449,10 @@ endif git-%$X: %.o GIT-LDFLAGS $(GITLIBS) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS) +git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS) + $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \ + $(LIBS) + git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \ $(IMAP_SEND_LDFLAGS) $(LIBS)
bugreport.c new file +69 -0
69 lines not on this page · raw message
t/t0091-bugreport.sh new file +41 -0
41 lines not on this page · raw message
Signature · 1 line
2.24.1.735.g03f4e72817-goog
PATCH v4 2/15 help: move list_config_help to builtin/help
In reply to Emily ShafferStarting in 3ac68a93fd2, help.o began to depend on builtin/branch.o, builtin/clean.o, and builtin/config.o. This meant that help.o was unusable outside of the context of the main Git executable.
To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present).
When command-list.h is included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule.
--- .gitignore | 1 + Makefile | 16 ++++++-- builtin/help.c | 86 ++++++++++++++++++++++++++++++++++++++++++ generate-cmdlist.sh | 19 ---------- generate-configlist.sh | 24 ++++++++++++ help.c | 85 ----------------------------------------- help.h | 1 - 7 files changed, 123 insertions(+), 109 deletions(-) create mode 100755 generate-configlist.sh
.gitignore +1 -0
@@ -189,6 +189,7 @@ /gitweb/gitweb.cgi /gitweb/static/gitweb.js /gitweb/static/gitweb.min.* +/config-list.h /command-list.h *.tar.gz *.dsc
Makefile +12 -4
@@ -815,6 +815,7 @@ LIB_FILE = libgit.a XDIFF_LIB = xdiff/lib.a VCSSVN_LIB = vcs-svn/lib.a +GENERATED_H += config-list.h GENERATED_H += command-list.h LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2127,7 +2128,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS) help.sp help.s help.o: command-list.h -builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX +builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \ '-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \ '-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2147,6 +2148,12 @@ $(BUILT_INS): git$X ln -s $< $@ 2>/dev/null || \ cp $< $@ +config-list.h: generate-configlist.sh + +config-list.h: + $(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \ + >$@+ && mv $@+ $@ + command-list.h: generate-cmdlist.sh command-list.txt command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2784,7 +2791,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE .PHONY: sparse $(SP_OBJ) sparse: $(SP_OBJ) -GEN_HDRS := command-list.h unicode-width.h +GEN_HDRS := config-list.h command-list.h unicode-width.h EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/% ifndef GCRYPT_SHA256 EXCEPT_HDRS += sha256/gcrypt.h
@@ -2807,7 +2814,7 @@ hdr-check: $(HCO) style: git clang-format --style file --diff --extensions c,h -check: command-list.h +check: config-list.h command-list.h @if sparse; \ then \ echo >&2 "Use 'make sparse' instead"; \
@@ -3110,7 +3117,8 @@ clean: profile-clean coverage-clean cocciclean $(RM) $(HCC) $(RM) -r bin-wrappers $(dep_dirs) $(RM) -r po/build/ - $(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope* + $(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h + $(RM) $(ETAGS_TARGET) tags cscope* $(RM) -r $(GIT_TARNAME) .doc-tmp-dir $(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz $(RM) $(htmldocs).tar.gz $(manpages).tar.gz
builtin/help.c +86 -0
98 lines not on this page · raw message
generate-cmdlist.sh +0 -19
@@ -76,23 +76,6 @@ print_command_list () { echo "};" } -print_config_list () { - cat <<EOF -static const char *config_name_list[] = { -EOF - grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt | - sed '/deprecated/d; s/::$//; s/, */\n/g' | - sort | - while read line - do - echo " \"$line\"," - done - cat <<EOF - NULL, -}; -EOF -} - exclude_programs= while test "--exclude-program" = "$1" do
@@ -113,5 +96,3 @@ echo define_category_names "$1" echo print_command_list "$1" -echo -print_config_list
generate-configlist.sh new file +24 -0
@@ -0,0 +1,24 @@ +#!/bin/sh + +echo "/* Automatically generated by generate-configlist.sh */" +echo + +print_config_list () { + cat <<EOF +static const char *config_name_list[] = { +EOF + grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt | + sed '/deprecated/d; s/::$//; s/, */\n/g' | + sort | + while read line + do + echo " \"$line\"," + done + cat <<EOF + NULL, +}; +EOF +} + +echo +print_config_list
help.c +0 -85
91 lines not on this page · raw message
help.h +0 -1
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)void list_common_cmds_help (void );void list_all_cmds_help (void );void list_common_guides_help (void ); -void list_config_help (int for_human);void list_all_main_cmds (struct string_list *list);void list_all_other_cmds (struct string_list *list);
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Re: [PATCH v4 02/15] help: move list_config_help to builtin/help
Quoted 8 lines · Emily Shaffer
Starting in
3ac68a93fd2, help.o began to depend onbuiltin/branch.o,builtin/clean.o, andbuiltin/config.o. This meant that help.o was unusable outside of the context of the main Git executable.To make help.o usable by other commands again, move
list_config_help()intobuiltin/help.c(where it makes sense to assume other builtin libraries are present).
Quoted 8 lines
When
command-list.his included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule.Signed-off-by Emily Shaffer <[email protected]>---
OK. Looks like a clean-up that is very worthy even without the remainder of the topic.
Thanks.
Quoted 360 lines
.gitignore | 1 + Makefile | 16 ++++++-- builtin/help.c | 86 ++++++++++++++++++++++++++++++++++++++++++ generate-cmdlist.sh | 19 ---------- generate-configlist.sh | 24 ++++++++++++ help.c | 85 ----------------------------------------- help.h | 1 - 7 files changed, 123 insertions(+), 109 deletions(-) create mode 100755 generate-configlist.sh.gitignore +1
-0@@ -189,6 +189,7 @@ /gitweb/gitweb.cgi /gitweb/static/gitweb.js /gitweb/static/gitweb.min.* +/config-list.h /command-list.h *.tar.gz *.dscMakefile +12
-4@@ -815,6 +815,7 @@ LIB_FILE = libgit.a XDIFF_LIB = xdiff/lib.a VCSSVN_LIB = vcs-svn/lib.a +GENERATED_H += config-list.h GENERATED_H += command-list.h LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \@@ -2127,7 +2128,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS) help.sp help.s help.o: command-list.h -builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX +builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \ '-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \ '-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \@@ -2147,6 +2148,12 @@ $(BUILT_INS): git$X ln -s $< $@ 2>/dev/null || \ cp $< $@ +config-list.h: generate-configlist.sh + +config-list.h: + $(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \ + >$@+ && mv $@+ $@ + command-list.h: generate-cmdlist.sh command-list.txt command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt@@ -2784,7 +2791,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE .PHONY: sparse $(SP_OBJ) sparse: $(SP_OBJ) -GEN_HDRS := command-list.h unicode-width.h +GEN_HDRS := config-list.h command-list.h unicode-width.h EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/% ifndef GCRYPT_SHA256 EXCEPT_HDRS += sha256/gcrypt.h@@ -2807,7 +2814,7 @@ hdr-check: $(HCO) style: git clang-format --style file --diff --extensions c,h -check: command-list.h +check: config-list.h command-list.h @if sparse; \ then \ echo >&2 "Use 'make sparse' instead"; \@@ -3110,7 +3117,8 @@ clean: profile-clean coverage-clean cocciclean $(RM) $(HCC) $(RM) -r bin-wrappers $(dep_dirs) $(RM) -r po/build/ - $(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope* + $(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h + $(RM) $(ETAGS_TARGET) tags cscope* $(RM) -r $(GIT_TARNAME) .doc-tmp-dir $(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz $(RM) $(htmldocs).tar.gz $(manpages).tar.gzbuiltin/help.c +86
-0@@ -8,6 +8,7 @@#include "parse-options.h" #include "run-command.h" #include "column.h" +#include "config-list.h" #include "help.h" #include "alias.h" @@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {NULL }; +struct slot_expansion { +const char *prefix; +const char *placeholder; +void (*fn)(struct string_list *list,const char *prefix); +int found; +}; + +static void list_config_help (int for_human) +{ +struct slot_expansion slot_expansions[] = { + {"advice" ,"*" , list_config_advices }, + {"color.branch" ,"<slot>" , list_config_color_branch_slots }, + {"color.decorate" ,"<slot>" , list_config_color_decorate_slots }, + {"color.diff" ,"<slot>" , list_config_color_diff_slots }, + {"color.grep" ,"<slot>" , list_config_color_grep_slots }, + {"color.interactive" ,"<slot>" , list_config_color_interactive_slots }, + {"color.remote" ,"<slot>" , list_config_color_sideband_slots }, + {"color.status" ,"<slot>" , list_config_color_status_slots }, + {"fsck" ,"<msg-id>" , list_config_fsck_msg_ids }, + {"receive.fsck" ,"<msg-id>" , list_config_fsck_msg_ids }, + {NULL ,NULL ,NULL } + }; +const char **p; +struct slot_expansion *e; +struct string_list keys =STRING_LIST_INIT_DUP ; +int i; + +for (p = config_name_list; *p; p++) { +const char *var = *p; +struct strbuf sb =STRBUF_INIT ; + +for (e = slot_expansions; e->prefix; e++) { + +strbuf_reset (&sb); +strbuf_addf (&sb,"%s.%s" , e->prefix, e->placeholder); +if (!strcasecmp (var, sb.buf)) { + e->fn (&keys, e->prefix); + e->found++; +break ; + } + } +strbuf_release (&sb); +if (!e->prefix) +string_list_append (&keys, var); + } + +for (e = slot_expansions; e->prefix; e++) +if (!e->found) +BUG ("slot_expansion %s.%s is not used" , + e->prefix, e->placeholder); + +string_list_sort (&keys); +for (i =0 ; i < keys.nr; i++) { +const char *var = keys.items[i].string; +const char *wildcard, *tag, *cut; + +if (for_human) { +puts (var); +continue ; + } + + wildcard =strchr (var,'*' ); + tag =strchr (var,'<' ); + +if (!wildcard && !tag) { +puts (var); +continue ; + } + +if (wildcard && !tag) + cut = wildcard; +else if (!wildcard && tag) + cut = tag; +else + cut = wildcard < tag ? wildcard : tag; + +/* +* We may produce duplicates, but that's up to +* git-completion.bash to handle +*/ +printf ("%.*s\n" , (int )(cut - var), var); + } +string_list_clear (&keys,0 ); +} +static enum help_format parse_help_format (const char *format) {if (!strcmp (format,"man" ))generate-cmdlist.sh +0
-19@@ -76,23 +76,6 @@ print_command_list () { echo "};" } -print_config_list () { - cat <<EOF -static const char *config_name_list[] = { -EOF - grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt | - sed '/deprecated/d; s/::$//; s/, */\n/g' | - sort | - while read line - do - echo " \"$line\"," - done - cat <<EOF - NULL, -}; -EOF -} - exclude_programs= while test "--exclude-program" = "$1" do@@ -113,5 +96,3 @@ echo define_category_names "$1" echo print_command_list "$1" -echo -print_config_listgenerate-configlist.sh new file +24
-0@@ -0,0 +1,24 @@ +#!/bin/sh + +echo "/* Automatically generated by generate-configlist.sh */" +echo + +print_config_list () { + cat <<EOF +static const char *config_name_list[] = { +EOF + grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt | + sed '/deprecated/d; s/::$//; s/, */\n/g' | + sort | + while read line + do + echo " \"$line\"," + done + cat <<EOF + NULL, +}; +EOF +} + +echo +print_config_listhelp.c +0
-85@@ -407,91 +407,6 @@ void list_common_guides_help(void)putchar ('\n' ); } -struct slot_expansion { -const char *prefix; -const char *placeholder; -void (*fn)(struct string_list *list,const char *prefix); -int found; -}; - -void list_config_help (int for_human) -{ -struct slot_expansion slot_expansions[] = { - {"advice" ,"*" , list_config_advices }, - {"color.branch" ,"<slot>" , list_config_color_branch_slots }, - {"color.decorate" ,"<slot>" , list_config_color_decorate_slots }, - {"color.diff" ,"<slot>" , list_config_color_diff_slots }, - {"color.grep" ,"<slot>" , list_config_color_grep_slots }, - {"color.interactive" ,"<slot>" , list_config_color_interactive_slots }, - {"color.remote" ,"<slot>" , list_config_color_sideband_slots }, - {"color.status" ,"<slot>" , list_config_color_status_slots }, - {"fsck" ,"<msg-id>" , list_config_fsck_msg_ids }, - {"receive.fsck" ,"<msg-id>" , list_config_fsck_msg_ids }, - {NULL ,NULL ,NULL } - }; -const char **p; -struct slot_expansion *e; -struct string_list keys =STRING_LIST_INIT_DUP ; -int i; - -for (p = config_name_list; *p; p++) { -const char *var = *p; -struct strbuf sb =STRBUF_INIT ; - -for (e = slot_expansions; e->prefix; e++) { - -strbuf_reset (&sb); -strbuf_addf (&sb,"%s.%s" , e->prefix, e->placeholder); -if (!strcasecmp (var, sb.buf)) { - e->fn (&keys, e->prefix); - e->found++; -break ; - } - } -strbuf_release (&sb); -if (!e->prefix) -string_list_append (&keys, var); - } - -for (e = slot_expansions; e->prefix; e++) -if (!e->found) -BUG ("slot_expansion %s.%s is not used" , - e->prefix, e->placeholder); - -string_list_sort (&keys); -for (i =0 ; i < keys.nr; i++) { -const char *var = keys.items[i].string; -const char *wildcard, *tag, *cut; - -if (for_human) { -puts (var); -continue ; - } - - wildcard =strchr (var,'*' ); - tag =strchr (var,'<' ); - -if (!wildcard && !tag) { -puts (var); -continue ; - } - -if (wildcard && !tag) - cut = wildcard; -else if (!wildcard && tag) - cut = tag; -else - cut = wildcard < tag ? wildcard : tag; - -/* -* We may produce duplicates, but that's up to -* git-completion.bash to handle -*/ -printf ("%.*s\n" , (int )(cut - var), var); - } -string_list_clear (&keys,0 ); -} -static int get_alias (const char *var,const char *value,void *data) {struct string_list *list = data;help.h +0
-1@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)void list_common_cmds_help (void );void list_all_cmds_help (void );void list_common_guides_help (void ); -void list_config_help (int for_human);void list_all_main_cmds (struct string_list *list);void list_all_other_cmds (struct string_list *list);
Re: [PATCH v4 02/15] help: move list_config_help to builtin/help
Quoted 16 lines · Junio C Hamano
Quoted 14 lines · Emily Shaffer
Starting in
3ac68a93fd2, help.o began to depend onbuiltin/branch.o,builtin/clean.o, andbuiltin/config.o. This meant that help.o was unusable outside of the context of the main Git executable.To make help.o usable by other commands again, move
list_config_help()intobuiltin/help.c(where it makes sense to assume other builtin libraries are present).When
command-list.his included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule.
Quoted 7 lines
Quoted 2 lines
Signed-off-by Emily Shaffer <[email protected]>---
OK. Looks like a clean-up that is very worthy even without the remainder of the topic.
Thanks.
Ok. Are you going to take it on its own? If so, I'll be relying on two branches in pu/ - what's the best way to let you know I need them both if/when I send another version of this chain?
Quoted 361 lines
Quoted 361 lines
.gitignore | 1 + Makefile | 16 ++++++-- builtin/help.c | 86 ++++++++++++++++++++++++++++++++++++++++++ generate-cmdlist.sh | 19 ---------- generate-configlist.sh | 24 ++++++++++++ help.c | 85 ----------------------------------------- help.h | 1 - 7 files changed, 123 insertions(+), 109 deletions(-) create mode 100755 generate-configlist.sh.gitignore +1
-0@@ -189,6 +189,7 @@ /gitweb/gitweb.cgi /gitweb/static/gitweb.js /gitweb/static/gitweb.min.* +/config-list.h /command-list.h *.tar.gz *.dscMakefile +12
-4@@ -815,6 +815,7 @@ LIB_FILE = libgit.a XDIFF_LIB = xdiff/lib.a VCSSVN_LIB = vcs-svn/lib.a +GENERATED_H += config-list.h GENERATED_H += command-list.h LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \@@ -2127,7 +2128,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS) help.sp help.s help.o: command-list.h -builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX +builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \ '-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \ '-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \@@ -2147,6 +2148,12 @@ $(BUILT_INS): git$X ln -s $< $@ 2>/dev/null || \ cp $< $@ +config-list.h: generate-configlist.sh + +config-list.h: + $(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \ + >$@+ && mv $@+ $@ + command-list.h: generate-cmdlist.sh command-list.txt command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt@@ -2784,7 +2791,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE .PHONY: sparse $(SP_OBJ) sparse: $(SP_OBJ) -GEN_HDRS := command-list.h unicode-width.h +GEN_HDRS := config-list.h command-list.h unicode-width.h EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/% ifndef GCRYPT_SHA256 EXCEPT_HDRS += sha256/gcrypt.h@@ -2807,7 +2814,7 @@ hdr-check: $(HCO) style: git clang-format --style file --diff --extensions c,h -check: command-list.h +check: config-list.h command-list.h @if sparse; \ then \ echo >&2 "Use 'make sparse' instead"; \@@ -3110,7 +3117,8 @@ clean: profile-clean coverage-clean cocciclean $(RM) $(HCC) $(RM) -r bin-wrappers $(dep_dirs) $(RM) -r po/build/ - $(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope* + $(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h + $(RM) $(ETAGS_TARGET) tags cscope* $(RM) -r $(GIT_TARNAME) .doc-tmp-dir $(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz $(RM) $(htmldocs).tar.gz $(manpages).tar.gzbuiltin/help.c +86
-0@@ -8,6 +8,7 @@#include "parse-options.h" #include "run-command.h" #include "column.h" +#include "config-list.h" #include "help.h" #include "alias.h" @@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {NULL }; +struct slot_expansion { +const char *prefix; +const char *placeholder; +void (*fn)(struct string_list *list,const char *prefix); +int found; +}; + +static void list_config_help (int for_human) +{ +struct slot_expansion slot_expansions[] = { + {"advice" ,"*" , list_config_advices }, + {"color.branch" ,"<slot>" , list_config_color_branch_slots }, + {"color.decorate" ,"<slot>" , list_config_color_decorate_slots }, + {"color.diff" ,"<slot>" , list_config_color_diff_slots }, + {"color.grep" ,"<slot>" , list_config_color_grep_slots }, + {"color.interactive" ,"<slot>" , list_config_color_interactive_slots }, + {"color.remote" ,"<slot>" , list_config_color_sideband_slots }, + {"color.status" ,"<slot>" , list_config_color_status_slots }, + {"fsck" ,"<msg-id>" , list_config_fsck_msg_ids }, + {"receive.fsck" ,"<msg-id>" , list_config_fsck_msg_ids }, + {NULL ,NULL ,NULL } + }; +const char **p; +struct slot_expansion *e; +struct string_list keys =STRING_LIST_INIT_DUP ; +int i; + +for (p = config_name_list; *p; p++) { +const char *var = *p; +struct strbuf sb =STRBUF_INIT ; + +for (e = slot_expansions; e->prefix; e++) { + +strbuf_reset (&sb); +strbuf_addf (&sb,"%s.%s" , e->prefix, e->placeholder); +if (!strcasecmp (var, sb.buf)) { + e->fn (&keys, e->prefix); + e->found++; +break ; + } + } +strbuf_release (&sb); +if (!e->prefix) +string_list_append (&keys, var); + } + +for (e = slot_expansions; e->prefix; e++) +if (!e->found) +BUG ("slot_expansion %s.%s is not used" , + e->prefix, e->placeholder); + +string_list_sort (&keys); +for (i =0 ; i < keys.nr; i++) { +const char *var = keys.items[i].string; +const char *wildcard, *tag, *cut; + +if (for_human) { +puts (var); +continue ; + } + + wildcard =strchr (var,'*' ); + tag =strchr (var,'<' ); + +if (!wildcard && !tag) { +puts (var); +continue ; + } + +if (wildcard && !tag) + cut = wildcard; +else if (!wildcard && tag) + cut = tag; +else + cut = wildcard < tag ? wildcard : tag; + +/* +* We may produce duplicates, but that's up to +* git-completion.bash to handle +*/ +printf ("%.*s\n" , (int )(cut - var), var); + } +string_list_clear (&keys,0 ); +} +static enum help_format parse_help_format (const char *format) {if (!strcmp (format,"man" ))generate-cmdlist.sh +0
-19@@ -76,23 +76,6 @@ print_command_list () { echo "};" } -print_config_list () { - cat <<EOF -static const char *config_name_list[] = { -EOF - grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt | - sed '/deprecated/d; s/::$//; s/, */\n/g' | - sort | - while read line - do - echo " \"$line\"," - done - cat <<EOF - NULL, -}; -EOF -} - exclude_programs= while test "--exclude-program" = "$1" do@@ -113,5 +96,3 @@ echo define_category_names "$1" echo print_command_list "$1" -echo -print_config_listgenerate-configlist.sh new file +24
-0@@ -0,0 +1,24 @@ +#!/bin/sh + +echo "/* Automatically generated by generate-configlist.sh */" +echo + +print_config_list () { + cat <<EOF +static const char *config_name_list[] = { +EOF + grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt | + sed '/deprecated/d; s/::$//; s/, */\n/g' | + sort | + while read line + do + echo " \"$line\"," + done + cat <<EOF + NULL, +}; +EOF +} + +echo +print_config_listhelp.c +0
-85@@ -407,91 +407,6 @@ void list_common_guides_help(void)putchar ('\n' ); } -struct slot_expansion { -const char *prefix; -const char *placeholder; -void (*fn)(struct string_list *list,const char *prefix); -int found; -}; - -void list_config_help (int for_human) -{ -struct slot_expansion slot_expansions[] = { - {"advice" ,"*" , list_config_advices }, - {"color.branch" ,"<slot>" , list_config_color_branch_slots }, - {"color.decorate" ,"<slot>" , list_config_color_decorate_slots }, - {"color.diff" ,"<slot>" , list_config_color_diff_slots }, - {"color.grep" ,"<slot>" , list_config_color_grep_slots }, - {"color.interactive" ,"<slot>" , list_config_color_interactive_slots }, - {"color.remote" ,"<slot>" , list_config_color_sideband_slots }, - {"color.status" ,"<slot>" , list_config_color_status_slots }, - {"fsck" ,"<msg-id>" , list_config_fsck_msg_ids }, - {"receive.fsck" ,"<msg-id>" , list_config_fsck_msg_ids }, - {NULL ,NULL ,NULL } - }; -const char **p; -struct slot_expansion *e; -struct string_list keys =STRING_LIST_INIT_DUP ; -int i; - -for (p = config_name_list; *p; p++) { -const char *var = *p; -struct strbuf sb =STRBUF_INIT ; - -for (e = slot_expansions; e->prefix; e++) { - -strbuf_reset (&sb); -strbuf_addf (&sb,"%s.%s" , e->prefix, e->placeholder); -if (!strcasecmp (var, sb.buf)) { - e->fn (&keys, e->prefix); - e->found++; -break ; - } - } -strbuf_release (&sb); -if (!e->prefix) -string_list_append (&keys, var); - } - -for (e = slot_expansions; e->prefix; e++) -if (!e->found) -BUG ("slot_expansion %s.%s is not used" , - e->prefix, e->placeholder); - -string_list_sort (&keys); -for (i =0 ; i < keys.nr; i++) { -const char *var = keys.items[i].string; -const char *wildcard, *tag, *cut; - -if (for_human) { -puts (var); -continue ; - } - - wildcard =strchr (var,'*' ); - tag =strchr (var,'<' ); - -if (!wildcard && !tag) { -puts (var); -continue ; - } - -if (wildcard && !tag) - cut = wildcard; -else if (!wildcard && tag) - cut = tag; -else - cut = wildcard < tag ? wildcard : tag; - -/* -* We may produce duplicates, but that's up to -* git-completion.bash to handle -*/ -printf ("%.*s\n" , (int )(cut - var), var); - } -string_list_clear (&keys,0 ); -} -static int get_alias (const char *var,const char *value,void *data) {struct string_list *list = data;help.h +0
-1@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)void list_common_cmds_help (void );void list_all_cmds_help (void );void list_common_guides_help (void ); -void list_config_help (int for_human);void list_all_main_cmds (struct string_list *list);void list_all_other_cmds (struct string_list *list);
Re: [PATCH v4 02/15] help: move list_config_help to builtin/help
Quoted 6 lines · Emily Shaffer
Quoted 4 lines
OK. Looks like a clean-up that is very worthy even without the remainder of the topic.
Thanks.
Ok. Are you going to take it on its own?
Not really. Unless you abandon the rest of the topic, in which case it may be worth resurrecting this one on its own, that is. But that is not what you plan to do anyway, so I expect to see it as part of an updated series ;-)
Re: [PATCH v4 02/15] help: move list_config_help to builtin/help
Quoted 5 lines · Junio C Hamano
Quoted 3 lines · Emily Shaffer
Quoted 3 lines
OK. Looks like a clean-up that is very worthy even without the remainder of the topic.
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
Thanks.
Ok. Are you going to take it on its own?
Not really. Unless you abandon the rest of the topic, in which case it may be worth resurrecting this one on its own, that is. But that is not what you plan to do anyway, so I expect to see it as part of an updated series ;-)
Ok! That's fine and makes my life easier. Just making sure. :)
- Emily
PATCH v4 3/15 bugreport: gather git version and build info
In reply to Emily ShafferKnowing which version of Git a user has and how it was built allows us to more precisely pin down the circumstances when a certain issue occurs, so teach bugreport how to tell us the same output as 'git version --build-options'.
It's not ideal to directly call 'git version --build-options' because that output goes to stdout. Instead, wrap the version string in a helper within help.[ch] library, and call that helper from within the bugreport library.
--- bugreport.c | 23 ++++++++++++++++++++++- help.c | 45 +++++++++++++++++++++++++++------------------ help.h | 1 + 3 files changed, 50 insertions(+), 19 deletions(-)
bugreport.c +22 -1
@@ -1,8 +1,20 @@ -#include "builtin.h" +#include "cache.h" #include "parse-options.h" #include "stdio.h" #include "strbuf.h" #include "time.h" +#include "help.h" + +static void get_system_info (struct strbuf *sys_info) +{ +struct strbuf version_info =STRBUF_INIT ; + +/* get git version from native cmd */ +strbuf_addstr (sys_info,"git version:\n" ); +list_version_info (&version_info,1 ); +strbuf_addbuf (sys_info, &version_info); +strbuf_complete_line (sys_info); +}static const char *const bugreport_usage[] = {N_ ("git bugreport [-o|--output <file>]" ),
@@ -32,6 +44,11 @@ static int get_bug_template(struct strbuf *template)return 0 ; } +static void get_header (struct strbuf *buf,const char *title) +{ +strbuf_addf (buf,"\n\n[%s]\n" , title); +} +int cmd_main (int argc,const char **argv) {struct strbuf buffer =STRBUF_INIT ;
@@ -60,6 +77,10 @@ int cmd_main(int argc, const char **argv)get_bug_template (&buffer); +// add other contents +get_header (&buffer,"System Info" ); +get_system_info (&buffer); + report =fopen_for_writing (report_path.buf);strbuf_write (&buffer, report);fclose (report);
help.c +27 -18
60 lines not on this page · raw message
help.h +1 -0
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);void exclude_cmds (struct cmdnames *cmds,struct cmdnames *excludes);int is_in_cmdlist (struct cmdnames *cmds,const char *name);void list_commands (unsigned int colopts,struct cmdnames *main_cmds,struct cmdnames *other_cmds); +void list_version_info (struct strbuf *buf,int build_options);/* * call this to die(), when it is suspected that the user mistyped a
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Re: [PATCH v4 03/15] bugreport: gather git version and build info
Quoted 9 lines · Emily Shaffer
Knowing which version of Git a user has and how it was built allows us to more precisely pin down the circumstances when a certain issue occurs, so teach bugreport how to tell us the same output as 'git version --build-options'.
It's not ideal to directly call 'git version --build-options' because that output goes to stdout. Instead, wrap the version string in a helper within help.[ch] library, and call that helper from within the bugreport library.
Move to strbuf() from stdio makes sense.
bugreport.c int cmd_main(int argc, const char **argv) · line 80
Quoted 1 line
bugreport.c +1
-0+// add other contents
Style.
help.h void add_cmdname(struct cmdnames *cmds, const char *name, int len); · line 38
Quoted 7 lines
help.h +0
-0@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);void exclude_cmds (struct cmdnames *cmds,struct cmdnames *excludes);int is_in_cmdlist (struct cmdnames *cmds,const char *name);
Many new helpers are called get_frotz(), but only one among these is called list_version_info(). I do not think the naming of the get_*() ones, that are private to the bugreport tool, matters that much, but unlike "list_commands()" whose purpose is to list the commands ;-), the new function does not list versions---it just gives information about a single version which is the one that is being run, so perhaps it is a misnomer.
help.h void add_cmdname(struct cmdnames *cmds, const char *name, int len); · line 40
Quoted 2 lines
help.h +1
-0void list_commands (unsigned int colopts,struct cmdnames *main_cmds,struct cmdnames *other_cmds); +void list_version_info (struct strbuf *buf,int build_options);
It is not clear to the readers build_options is a boolean that tells the function to include (or not to include) build options. Perhaps rename it to "int show_build_options" or something?
Re: [PATCH v4 03/15] bugreport: gather git version and build info
Quoted 12 lines · Junio C Hamano
Quoted 9 lines · Emily Shaffer
Knowing which version of Git a user has and how it was built allows us to more precisely pin down the circumstances when a certain issue occurs, so teach bugreport how to tell us the same output as 'git version --build-options'.
It's not ideal to directly call 'git version --build-options' because that output goes to stdout. Instead, wrap the version string in a helper within help.[ch] library, and call that helper from within the bugreport library.
Quoted 5 lines
Move to
strbuf()from stdio makes sense.bugreport.c int cmd_main(int argc, const char **argv) · line 80
Quoted 1 line
bugreport.c +1
-0+// add other contents Style.
Sure, dropped this entirely. I think with the helpers the code is self-documenting there.
Quoted 17 lines
help.h void add_cmdname(struct cmdnames *cmds, const char *name, int len); · line 38
Quoted 8 lines
help.h +0
-0@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);void exclude_cmds (struct cmdnames *cmds,struct cmdnames *excludes);int is_in_cmdlist (struct cmdnames *cmds,const char *name);Many new helpers are called
get_frotz(), but only one among these is calledlist_version_info(). I do not think the naming of the get_*() ones, that are private to the bugreport tool, matters that much, but unlike "list_commands()" whose purpose is to list the commands ;-), the new function does not list versions---it just gives information about a single version which is the one that is being run, so perhaps it is a misnomer.
Hm, sure. I renamed it to get_version_info(); I had named it list_* because all the other helpers in help.h are named list_*, and it does print more than one piece of info. But I do see your point (all the info is about the same version) so I've renamed it.
Quoted 7 lines
help.h void add_cmdname(struct cmdnames *cmds, const char *name, int len); · line 40
Quoted 3 lines
help.h +1
-0void list_commands (unsigned int colopts,struct cmdnames *main_cmds,struct cmdnames *other_cmds); +void list_version_info (struct strbuf *buf,int build_options);It is not clear to the readers
build_optionsis a boolean that tells the function to include (or not to include) build options. Perhaps rename it to "intshow_build_options" or something?
Agree, done.
Re: [PATCH v4 03/15] bugreport: gather git version and build info
In reply to Emily ShafferHi Emily,
Quoted 16 lines · Emily Shaffer
help.c +8
-0@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)exit (1 ); } +void list_version_info (struct strbuf *buf,int build_options) +{ +strbuf_reset (buf); +/* +* The format of this string should be kept stable for compatibility +* with external projects that rely on the output of "git version". +* +* Always show the version, even if other options are given.
help.c const char *help_unknown_cmd(const char *cmd) · line 642
Quoted 10 lines
help.c +10
-0+*/ +strbuf_addf (buf,"git version %s\n" , git_version_string); + +if (build_options) { +strbuf_addf (buf,"cpu: %s\n" ,GIT_HOST_CPU ); +if (git_built_from_commit_string[0 ]) +strbuf_addf (buf,"built from commit: %s\n" , + git_built_from_commit_string); +else +strbuf_addf (buf,"no commit associated with this build\n" );
The "StaticAnalysis" job of the Azure Pipeline is not happy with this, claiming that this should be an strbuf_addstr() call instead. For details, see:
Ciao,
Dscho
Quoted 57 lines
help.c +9
-18+strbuf_addf (buf,"sizeof-long: %d\n" , (int )sizeof (long )); +strbuf_addf (buf,"sizeof-size_t: %d\n" , (int )sizeof (size_t )); +/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */ + } +} +int cmd_version (int argc,const char **argv,const char *prefix) { +struct strbuf buf =STRBUF_INIT ;int build_options =0 ;const char *const usage[] = {N_ ("git version [<options>]" ),@@ -637,25 +662,9 @@ int cmd_version(int argc, const char **argv, const char *prefix) argc =parse_options (argc, argv, prefix, options, usage,0 ); -/* -* The format of this string should be kept stable for compatibility -* with external projects that rely on the output of "git version". -* -* Always show the version, even if other options are given. -*/ -printf ("git version %s\n" , git_version_string); +list_version_info (&buf, build_options); +printf ("%s" , buf.buf); -if (build_options) { -printf ("cpu: %s\n" ,GIT_HOST_CPU ); -if (git_built_from_commit_string[0 ]) -printf ("built from commit: %s\n" , - git_built_from_commit_string); -else -printf ("no commit associated with this build\n" ); -printf ("sizeof-long: %d\n" , (int )sizeof (long )); -printf ("sizeof-size_t: %d\n" , (int )sizeof (size_t )); -/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */ - }return 0 ; }help.h +1
-1@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);void exclude_cmds (struct cmdnames *cmds,struct cmdnames *excludes);int is_in_cmdlist (struct cmdnames *cmds,const char *name);void list_commands (unsigned int colopts,struct cmdnames *main_cmds,struct cmdnames *other_cmds); +void list_version_info (struct strbuf *buf,int build_options);/* * call this to die(), when it is suspected that the user mistyped a -- 2.24.1.735.g03f4e72817-goog
Re: [PATCH v4 03/15] bugreport: gather git version and build info
Quoted 10 lines · Johannes Schindelin
help.c const char *help_unknown_cmd(const char *cmd) · line 642
Quoted 7 lines
help.c +7
-0+if (build_options) { +strbuf_addf (buf,"cpu: %s\n" ,GIT_HOST_CPU ); +if (git_built_from_commit_string[0 ]) +strbuf_addf (buf,"built from commit: %s\n" , + git_built_from_commit_string); +else +strbuf_addf (buf,"no commit associated with this build\n" );The "StaticAnalysis" job of the Azure Pipeline is not happy with this, claiming that this should be an
strbuf_addstr()call instead.
You mean the "else" clause, right? That feels similar to say
printf("Hello world\n");should better be written as
fputs("Hello world\n", stdout);which I do not agree with at all. IOW, I view the distinction more like "once it is written one way or the other way, it is not worth spending bits and braincycles to see if it is worth changing it" kind of minor stylistic preference.
Re: [PATCH v4 03/15] bugreport: gather git version and build info
Quoted 17 lines · Junio C Hamano
Quoted 10 lines · Johannes Schindelin
help.c const char *help_unknown_cmd(const char *cmd) · line 642
Quoted 7 lines
help.c +7
-0+if (build_options) { +strbuf_addf (buf,"cpu: %s\n" ,GIT_HOST_CPU ); +if (git_built_from_commit_string[0 ]) +strbuf_addf (buf,"built from commit: %s\n" , + git_built_from_commit_string); +else +strbuf_addf (buf,"no commit associated with this build\n" );The "StaticAnalysis" job of the Azure Pipeline is not happy with this, claiming that this should be an
strbuf_addstr()call instead.You mean the "else" clause, right? That feels similar to say
printf("Hello world\n");
Quoted 8 lines
should better be written as
fputs("Hello world\n", stdout);which I do not agree with at all. IOW, I view the distinction more like "once it is written one way or the other way, it is not worth spending bits and braincycles to see if it is worth changing it" kind of minor stylistic preference.
I think I side with Junio here, although it's true that when strbuf_addstr() exists it doesn't make that much sense to use strbuf_addf(). Since there's some other comments, though, I'll change this too to make your CI shut up. :)
- Emily
PATCH v4 4/15 help: add shell-path to --build-options
In reply to Emily ShafferIt may be useful to know which shell Git was built to try to point to, in the event that shell-based Git commands are failing. $SHELL_PATH is set during the build and used to launch the manpage viewer, as well as by git-compat-util.h, and it's used during tests. 'git version --build-options' is encouraged for use in bug reports, so it makes sense to include this information there.
--- help.c | 1 + 1 file changed, 1 insertion(+)
help.c +1 -0
@@ -642,6 +642,7 @@ void list_version_info(struct strbuf *buf, int build_options)strbuf_addf (buf,"no commit associated with this build\n" );strbuf_addf (buf,"sizeof-long: %d\n" , (int )sizeof (long ));strbuf_addf (buf,"sizeof-size_t: %d\n" , (int )sizeof (size_t )); +strbuf_addf (buf,"shell-path: %s\n" ,SHELL_PATH );/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */ } }
Signature · 1 line
2.24.1.735.g03f4e72817-goog
PATCH v4 5/15 bugreport: add uname info
In reply to Emily ShafferThe contents of uname() can give us some insight into what sort of system the user is running on, and help us replicate their setup if need be. The domainname field is not guaranteed to be available, so don't collect it.
--- bugreport.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
bugreport.c +13 -0
@@ -8,12 +8,25 @@static void get_system_info (struct strbuf *sys_info) {struct strbuf version_info =STRBUF_INIT ; +struct utsname uname_info;/* get git version from native cmd */ strbuf_addstr (sys_info,"git version:\n" );list_version_info (&version_info,1 );strbuf_addbuf (sys_info, &version_info);strbuf_complete_line (sys_info); + +/* system call for other version info */ +strbuf_addstr (sys_info,"uname -a: " ); +if (uname (&uname_info)) +strbuf_addf (sys_info,"uname() failed with code %d\n" , errno); +else +strbuf_addf (sys_info,"%s %s %s %s %s\n" , + uname_info.sysname, + uname_info.nodename, + uname_info.release, + uname_info.version, + uname_info.machine); }static const char *const bugreport_usage[] = {
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Re: [PATCH v4 05/15] bugreport: add uname info
Quoted 4 lines · Emily Shaffer
The contents of
uname()can give us some insight into what sort of system the user is running on, and help us replicate their setup if need be. The domainname field is not guaranteed to be available, so don't collect it.
It was surprising to me that we do use "struct utsname" somewhere in the system ;-) ... so at least we know this is just as portable as the remainder of Git, which is good.
Quoted 35 lines
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)bugreport.c +13
-0@@ -8,12 +8,25 @@static void get_system_info (struct strbuf *sys_info) {struct strbuf version_info =STRBUF_INIT ; +struct utsname uname_info;/* get git version from native cmd */ strbuf_addstr (sys_info,"git version:\n" );list_version_info (&version_info,1 );strbuf_addbuf (sys_info, &version_info);strbuf_complete_line (sys_info); + +/* system call for other version info */ +strbuf_addstr (sys_info,"uname -a: " ); +if (uname (&uname_info)) +strbuf_addf (sys_info,"uname() failed with code %d\n" , errno); +else +strbuf_addf (sys_info,"%s %s %s %s %s\n" , + uname_info.sysname, + uname_info.nodename, + uname_info.release, + uname_info.version, + uname_info.machine); }static const char *const bugreport_usage[] = {
Re: [PATCH v4 05/15] bugreport: add uname info
In reply to Emily ShafferQuoted 4 lines · Emily Shaffer
The contents of
uname()can give us some insight into what sort of system the user is running on, and help us replicate their setup if need be. The domainname field is not guaranteed to be available, so don't collect it.
The manpage on Linux says that it's a GNU extension; on Mac OS it isn't mentioned. So it would be more accurate to say that it's known not to be available on many systems rather than just not being guaranteed.
Besides that I think in some cases it may be considered sensitive information, so another reason to not include it. Perhaps even worthy of being mentioned as the primary reason.
bugreport.c line 26
Quoted 1 line
bugreport.c +1
-0+ uname_info.nodename,
I think in some cases this could also be considered as sensitive information, and is unlikely to help in diagnosing problems. So I'd move to exclude this as well.
PATCH v4 6/15 bugreport: add glibc version
In reply to Emily ShafferTo help pinpoint the source of a regression, it is useful to know the version of libc which the user's Git client was built with. Let's include it alongside the other versioning information, which is used to identify how the client was built.
--- bugreport.c | 5 +++++ 1 file changed, 5 insertions(+)
bugreport.c +5 -0
@@ -4,6 +4,7 @@#include "strbuf.h" #include "time.h" #include "help.h" +#include <gnu/libc-version.h> static void get_system_info (struct strbuf *sys_info) {
@@ -27,6 +28,10 @@ static void get_system_info(struct strbuf *sys_info) uname_info.release, uname_info.version, uname_info.machine); + +strbuf_addstr (sys_info,"glibc version: " ); +strbuf_addstr (sys_info,gnu_get_libc_version ()); +strbuf_complete_line (sys_info); }static const char *const bugreport_usage[] = {
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Re: [PATCH v4 06/15] bugreport: add glibc version
Quoted 9 lines · Emily Shaffer
To help pinpoint the source of a regression, it is useful to know the version of libc which the user's Git client was built with. Let's include it alongside the other versioning information, which is used to identify how the client was built.
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 5 +++++ 1 file changed, 5 insertions(+)
This is the first iffy one in the series. It is unreasonable to assume that we can dictate that Git can be built only on glibc systems, no?
Making this conditional, perhaps make "bugreport.c" depend on "extern void get_compiler_info(struct strbuf *sys_info)" and require the function to be defined in compat/; the glibc variant that uses gnu_get_libc_version() would become just one of them.
Quoted 23 lines
bugreport.c +5
-0@@ -4,6 +4,7 @@#include "strbuf.h" #include "time.h" #include "help.h" +#include <gnu/libc-version.h> static void get_system_info (struct strbuf *sys_info) {@@ -27,6 +28,10 @@ static void get_system_info(struct strbuf *sys_info) uname_info.release, uname_info.version, uname_info.machine); + +strbuf_addstr (sys_info,"glibc version: " ); +strbuf_addstr (sys_info,gnu_get_libc_version ()); +strbuf_complete_line (sys_info); }static const char *const bugreport_usage[] = {
Re: [PATCH v4 06/15] bugreport: add glibc version
Quoted 12 lines · Junio C Hamano
Quoted 9 lines · Emily Shaffer
To help pinpoint the source of a regression, it is useful to know the version of libc which the user's Git client was built with. Let's include it alongside the other versioning information, which is used to identify how the client was built.
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 5 +++++ 1 file changed, 5 insertions(+)
Quoted 8 lines
This is the first iffy one in the series. It is unreasonable to assume that we can dictate that Git can be built only on glibc systems, no?
Making this conditional, perhaps make "
bugreport.c" depend on "extern voidget_compiler_info(struct strbuf *sys_info)" and require the function to be defined in compat/; the glibc variant that usesgnu_get_libc_version()would become just one of them.
Interesting. This sounds like a good approach - thanks, I'll look into it.
- Emily
PATCH v4 7/15 bugreport: add curl version
In reply to Emily ShafferIt's possible for git-http* to be built separately from git; in that case we want to know what version of cURL is used by git-http*, not necessarily which version was present at git-bugreport's build time. So instead, ask git-http-fetch for the version information it knows about.
git-http-fetch was chosen as git-http-backend was described as a server-side implementation, and as an accidental fetch in case of problems was considered less harmful than an accidental push.
Since it could have been built at a different time, also report the version and built-from commit of git-http-fetch alongside the cURL info.
--- Documentation/git-http-fetch.txt | 5 +++++ bugreport.c | 16 ++++++++++++++++ http-fetch.c | 13 ++++++++++++- 3 files changed, 33 insertions(+), 1 deletion(-)
Documentation/git-http-fetch.txt +5 -0
@@ -10,6 +10,7 @@ SYNOPSIS -------- [verse] 'git http-fetch' [-c] [-t] [-a] [-d] [-v] [-w filename] [--recover] [--stdin] <commit> <url> +'git http-fetch' [-V] DESCRIPTION -----------
@@ -30,6 +31,10 @@ commit-id:: -v:: Report what is downloaded. +-V:: + Report information about the version of git-http-fetch, including the + versions of its dependencies. + -w <filename>:: Writes the commit-id into the filename under $GIT_DIR/refs/<filename> on the local end after the transfer is complete.
bugreport.c +16 -0
@@ -5,6 +5,18 @@#include "time.h" #include "help.h" #include <gnu/libc-version.h> +#include "run-command.h" + +static void get_http_version_info (struct strbuf *http_info) +{ +struct child_process cp =CHILD_PROCESS_INIT ; + +argv_array_push (&cp.args,"git" ); +argv_array_push (&cp.args,"http-fetch" ); +argv_array_push (&cp.args,"-V" ); +if (capture_command (&cp, http_info,0 )) +strbuf_addstr (http_info,"'git-http-fetch -V' not supported\n" ); +}static void get_system_info (struct strbuf *sys_info) {
@@ -32,6 +44,10 @@ static void get_system_info(struct strbuf *sys_info)strbuf_addstr (sys_info,"glibc version: " );strbuf_addstr (sys_info,gnu_get_libc_version ());strbuf_complete_line (sys_info); + +strbuf_addstr (sys_info,"git-http-fetch -V:\n" ); +get_http_version_info (sys_info); +strbuf_complete_line (sys_info); }static const char *const bugreport_usage[] = {
http-fetch.c +12 -1
@@ -3,9 +3,18 @@#include "exec-cmd.h" #include "http.h" #include "walker.h" +#include "version.h" static const char http_fetch_usage[] ="git http-fetch " -"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url" ; +"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url" ; + +void NORETURN version_info() +{ +printf ("git-http-fetch version: %s\n" , git_version_string); +printf ("built from commit: %s\n" , git_built_from_commit_string); +printf ("curl version: %s\n" ,curl_version ()); +exit (0 ); +}int cmd_main (int argc,const char **argv) {
@@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv) }else if (argv[arg][1 ] =='a' ) { }else if (argv[arg][1 ] =='v' ) { get_verbosely =1 ; + }else if (argv[arg][1 ] =='V' ) { +version_info (); }else if (argv[arg][1 ] =='w' ) { write_ref = &argv[arg +1 ]; arg++;
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Re: [PATCH v4 07/15] bugreport: add curl version
Quoted 6 lines · Emily Shaffer
It's possible for git-http* to be built separately from git; in that case we want to know what version of cURL is used by git-http*, not necessarily which version was present at git-bugreport's build time. So instead, ask git-http-fetch for the version information it knows about.
Quoted 6 lines
git-http-fetch was chosen as git-http-backend was described as a server-side implementation, and as an accidental fetch in case of problems was considered less harmful than an accidental push.
Since it could have been built at a different time, also report the version and built-from commit of git-http-fetch alongside the cURL info.
One possible issue I have is that I was hoping that eventually we can discard "git http-fetch" altogether sometime in the future. Does anybody still use the dumb HTTP transport seriously?
And the first move in that direction would be to allow the system be built without http-fetch, even if git-remote-curl (and its aliases) would still be built to access smart-http transports.
So, I am not sure. This is just the matter of adding an out-of-line hidden option used only for environment inspection, so if it can be done to git-remote-curl, that would probably be much more future proof.
Quoted 9 lines
bugreport.c +1
-0@@ -5,6 +5,18 @@#include "time.h" #include "help.h" #include <gnu/libc-version.h> +#include "run-command.h"
bugreport.c line 18
Quoted 10 lines
bugreport.c +10
-0+ +static void get_http_version_info (struct strbuf *http_info) +{ +struct child_process cp =CHILD_PROCESS_INIT ; + +argv_array_push (&cp.args,"git" ); +argv_array_push (&cp.args,"http-fetch" ); +argv_array_push (&cp.args,"-V" ); +if (capture_command (&cp, http_info,0 )) +strbuf_addstr (http_info,"'git-http-fetch -V' not supported\n" );
OK. We probably can also take the compile-time "NO_CURL" into account, so that we can tell a misconfigured installation that wanted to have CURL but failed to install a usable http-fetch and an installation that deliberately omitted anything cURL?
Quoted 18 lines
bugreport.c +4
-0static void get_system_info (struct strbuf *sys_info) {@@ -32,6 +44,10 @@ static void get_system_info(struct strbuf *sys_info)strbuf_addstr (sys_info,"glibc version: " );strbuf_addstr (sys_info,gnu_get_libc_version ());strbuf_complete_line (sys_info); + +strbuf_addstr (sys_info,"git-http-fetch -V:\n" ); +get_http_version_info (sys_info); +strbuf_complete_line (sys_info); }static const char *const bugreport_usage[] = {http-fetch.c +0
-0
http-fetch.c line 11
Quoted 10 lines
http-fetch.c +4
-1@@ -3,9 +3,18 @@#include "exec-cmd.h" #include "http.h" #include "walker.h" +#include "version.h" static const char http_fetch_usage[] ="git http-fetch " -"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url" ; +"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url" ; + +void NORETURN version_info()
void NORETURN version_info(void)
Quoted 18 lines
http-fetch.c +8
-0+{ +printf ("git-http-fetch version: %s\n" , git_version_string); +printf ("built from commit: %s\n" , git_built_from_commit_string); +printf ("curl version: %s\n" ,curl_version ()); +exit (0 ); +}int cmd_main (int argc,const char **argv) {@@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv) }else if (argv[arg][1 ] =='a' ) { }else if (argv[arg][1 ] =='v' ) { get_verbosely =1 ; + }else if (argv[arg][1 ] =='V' ) { +version_info (); }else if (argv[arg][1 ] =='w' ) { write_ref = &argv[arg +1 ]; arg++;
Re: [PATCH v4 07/15] bugreport: add curl version
Quoted 12 lines · Junio C Hamano
Quoted 10 lines · Emily Shaffer
It's possible for git-http* to be built separately from git; in that case we want to know what version of cURL is used by git-http*, not necessarily which version was present at git-bugreport's build time. So instead, ask git-http-fetch for the version information it knows about.
git-http-fetch was chosen as git-http-backend was described as a server-side implementation, and as an accidental fetch in case of problems was considered less harmful than an accidental push.
Quoted 6 lines
Quoted 2 lines
Since it could have been built at a different time, also report the version and built-from commit of git-http-fetch alongside the cURL info.
One possible issue I have is that I was hoping that eventually we can discard "git http-fetch" altogether sometime in the future. Does anybody still use the dumb HTTP transport seriously?
Oh, interesting. I was about to say, "I still use it to fetch, when I don't really care" - but that isn't even true, as I fetch via https (and have so much muscle memory to type https that I don't even notice).`
Quoted 9 lines
And the first move in that direction would be to allow the system be built without http-fetch, even if git-remote-curl (and its aliases) would still be built to access smart-http transports.
So, I am not sure. This is just the matter of adding an out-of-line hidden option used only for environment inspection, so if it can be done to git-remote-curl, that would probably be much more future proof.
Ok. I'll move it.
Quoted 17 lines
Quoted 17 lines
bugreport.c +8
-0@@ -5,6 +5,18 @@#include "time.h" #include "help.h" #include <gnu/libc-version.h> +#include "run-command.h" + +static void get_http_version_info (struct strbuf *http_info) +{ +struct child_process cp =CHILD_PROCESS_INIT ; + +argv_array_push (&cp.args,"git" ); +argv_array_push (&cp.args,"http-fetch" );
Quoted 8 lines
bugreport.c line 18
Quoted 3 lines
bugreport.c +3
-0+argv_array_push (&cp.args,"-V" ); +if (capture_command (&cp, http_info,0 )) +strbuf_addstr (http_info,"'git-http-fetch -V' not supported\n" );OK. We probably can also take the compile-time "
NO_CURL" into account, so that we can tell a misconfigured installation that wanted to have CURL but failed to install a usable http-fetch and an installation that deliberately omitted anything cURL?
Oh, interesting idea! I'll add that.
Quoted 51 lines
Quoted 19 lines
bugreport.c +4
-0static void get_system_info (struct strbuf *sys_info) {@@ -32,6 +44,10 @@ static void get_system_info(struct strbuf *sys_info)strbuf_addstr (sys_info,"glibc version: " );strbuf_addstr (sys_info,gnu_get_libc_version ());strbuf_complete_line (sys_info); + +strbuf_addstr (sys_info,"git-http-fetch -V:\n" ); +get_http_version_info (sys_info); +strbuf_complete_line (sys_info); }static const char *const bugreport_usage[] = {http-fetch.c +0
-0http-fetch.c line 11
Quoted 10 lines
http-fetch.c +4
-1@@ -3,9 +3,18 @@#include "exec-cmd.h" #include "http.h" #include "walker.h" +#include "version.h" static const char http_fetch_usage[] ="git http-fetch " -"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url" ; +"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url" ; + +void NORETURN version_info()void NORETURN
version_info(void)Quoted 18 lines
http-fetch.c +8
-0+{ +printf ("git-http-fetch version: %s\n" , git_version_string); +printf ("built from commit: %s\n" , git_built_from_commit_string); +printf ("curl version: %s\n" ,curl_version ()); +exit (0 ); +}int cmd_main (int argc,const char **argv) {@@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv) }else if (argv[arg][1 ] =='a' ) { }else if (argv[arg][1 ] =='v' ) { get_verbosely =1 ; + }else if (argv[arg][1 ] =='V' ) { +version_info (); }else if (argv[arg][1 ] =='w' ) { write_ref = &argv[arg +1 ]; arg++;
Re: [PATCH v4 07/15] bugreport: add curl version
In reply to Emily ShafferHi Emily,
Quoted 5 lines · Emily Shaffer
http-fetch.c +0
-0
http-fetch.c line 11
Quoted 10 lines
http-fetch.c +4
-1@@ -3,9 +3,18 @@#include "exec-cmd.h" #include "http.h" #include "walker.h" +#include "version.h" static const char http_fetch_usage[] ="git http-fetch " -"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url" ; +"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url" ; + +void NORETURN version_info()
Pretty much all the builds in
https://dev.azure.com/gitgitgadget/git/_build/results?buildId=23830&view=logs&jobId=253e5128-1058-5bd4-fdf1-9b556d3207f8&j=fd490c07-0b22-5182-fac9-6d67fe1e939b&t=ce91d5d6-0c55-50f5-8ab9-6695c03ab102
fail because this function definition needs (void) instead of ().
Ciao,
Dscho
Quoted 23 lines
http-fetch.c +8
-1+{ +printf ("git-http-fetch version: %s\n" , git_version_string); +printf ("built from commit: %s\n" , git_built_from_commit_string); +printf ("curl version: %s\n" ,curl_version ()); +exit (0 ); +}int cmd_main (int argc,const char **argv) {@@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv) }else if (argv[arg][1 ] =='a' ) { }else if (argv[arg][1 ] =='v' ) { get_verbosely =1 ; + }else if (argv[arg][1 ] =='V' ) { +version_info (); }else if (argv[arg][1 ] =='w' ) { write_ref = &argv[arg +1 ]; arg++; --2.24.1.735.g03f4e72817-goog
PATCH v4 8/15 bugreport: include user interactive shell
In reply to Emily ShafferIt's possible a user may complain about the way that Git interacts with their interactive shell, e.g. autocompletion or shell prompt. In that case, it's useful for us to know which shell they're using interactively.
$SHELL isn't set by every shell, but getenv() returns NULL in the event the variable isn't found, so we'll see a line like "$SHELL:
(null)" to tell us that variable wasn't set.
--- bugreport.c | 3 +++ 1 file changed, 3 insertions(+)
bugreport.c +3 -0
@@ -45,6 +45,9 @@ static void get_system_info(struct strbuf *sys_info)strbuf_addstr (sys_info,gnu_get_libc_version ());strbuf_complete_line (sys_info); +strbuf_addf (sys_info,"$SHELL (typically, interactive shell): %s\n" , +getenv ("SHELL" )); +strbuf_addstr (sys_info,"git-http-fetch -V:\n" );get_http_version_info (sys_info);strbuf_complete_line (sys_info);
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Re: [PATCH v4 08/15] bugreport: include user interactive shell
Quoted 8 lines · Emily Shaffer
It's possible a user may complain about the way that Git interacts with their interactive shell, e.g. autocompletion or shell prompt. In that case, it's useful for us to know which shell they're using interactively.
$SHELL isn't set by every shell, but
getenv()returns NULL in the event the variable isn't found, so we'll see a line like "$SHELL:
(null)" to tell us that variable wasn't set.
Do not depend on vsnprintf() from glibc that accepts NULL and shows "(null)". You can segfault on systems without glibc that way.
Quoted 19 lines
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 3 +++ 1 file changed, 3 insertions(+)bugreport.c +3
-0@@ -45,6 +45,9 @@ static void get_system_info(struct strbuf *sys_info)strbuf_addstr (sys_info,gnu_get_libc_version ());strbuf_complete_line (sys_info); +strbuf_addf (sys_info,"$SHELL (typically, interactive shell): %s\n" , +getenv ("SHELL" )); +strbuf_addstr (sys_info,"git-http-fetch -V:\n" );get_http_version_info (sys_info);strbuf_complete_line (sys_info);
PATCH v4 9/15 bugreport: generate config safelist based on docs
In reply to Emily ShafferAdd a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "bugreport" macro, which is recognized by AsciiDoc and AsciiDoctor.
Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.
In order to prevent staleness of the safelist, add a mechanism to generate the safelist from annotations in the config documentation, where contributors are already used to documenting their new config keys.
Implement a new no-op "bugreport" macro for use as "bugreport:include[x]" to annotate the config keys that should be included in the automatically generated safelist. Use "exclude" for the others.
With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc seems to want something between the brackets. A bit unfortunate, but not a huge problem -- we'll just provide an "x".
"doc-diff" reports that this macro doesn't render at all. That is, these are both empty after this commit:
cd Documentation ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD ./doc-diff --asciidoc :/"bugreport: add tool" HEAD
Diffing the rendered HTML shows that there is some small amount of whitespace and comments added. That shouldn't be a problem.
We could perhaps let the implementation verify that the "action" is one of "include" and "exclude". For the Asciidoctor implementation that should be straightforward, but for AsciiDoc I don't immediately know how to do it. Anyway, if someone stumbles on the keyboard and writes "bugreport:icndule", they'll "only" miss out on the config key being included in the safelist. If this were a blocklist, the consequences of a misspelled target could be a lot more severe.
Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.
--- .gitignore | 1 + Documentation/asciidoc.conf | 8 +++ Documentation/asciidoctor-extensions.rb | 7 +++ Documentation/config/sendemail.txt | 68 ++++++++++++------------- Makefile | 14 +++-- generate-bugreport-config-safelist.sh | 22 ++++++++ 6 files changed, 83 insertions(+), 37 deletions(-) create mode 100755 generate-bugreport-config-safelist.sh
.gitignore +1 -0
@@ -191,6 +191,7 @@ /gitweb/static/gitweb.min.* /config-list.h /command-list.h +/bugreport-config-safelist.h *.tar.gz *.dsc *.deb
Documentation/asciidoc.conf +8 -0
@@ -6,9 +6,13 @@ # # Show Git link as: <command>(<section>); if section is defined, else just show # the command. +# +# The bugreport macro does nothing as far as rendering is +# concerned -- we just grep for it in the sources. [macros] (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]= +(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]= [attributes] asterisk=*
@@ -28,6 +32,8 @@ ifdef::backend-docbook[] {0#<citerefentry>} {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>} {0#</citerefentry>} +[bugreport-inlinemacro] +{0#} endif::backend-docbook[] ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[] git-relative-html-prefix= [linkgit-inlinemacro] <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a> +[bugreport-inlinemacro] +<!-- --> endif::backend-xhtml11[]
Documentation/asciidoctor-extensions.rb +7 -0
@@ -37,6 +37,10 @@ module Git output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>") end output + + class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor + def process(parent, action, attrs) + "" end end end
@@ -45,4 +49,7 @@ end Asciidoctor::Extensions.register do inline_macro Git::Documentation::LinkGitProcessor, :linkgit postprocessor Git::Documentation::DocumentPostProcessor + # The bugreport macro does nothing as far as rendering is + # concerned -- we just grep for it in the sources. + inline_macro Git::Documentation::BugReportProcessor, :bugreport end
Documentation/config/sendemail.txt +34 -34
97 lines not on this page · raw message
Makefile +11 -3
@@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN) install-python-script: $(SCRIPT_PYTHON_GEN) $(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' -.PHONY: clean-perl-script clean-sh-script clean-python-script +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies clean-sh-script: $(RM) $(SCRIPT_SH_GEN) clean-perl-script:
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a GENERATED_H += config-list.h GENERATED_H += command-list.h +GENERATED_H += bugreport-config-safelist.h LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \ $(FIND) . \
@@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc $(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \ command-list.txt >$@+ && mv $@+ $@ +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh + +bugreport-config-safelist.h: Documentation/config/*.txt + $(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \ + >$@+ && mv $@+ $@ + SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\ $(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\ $(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
@@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE .PHONY: sparse $(SP_OBJ) sparse: $(SP_OBJ) -GEN_HDRS := config-list.h command-list.h unicode-width.h +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/% ifndef GCRYPT_SHA256 EXCEPT_HDRS += sha256/gcrypt.h
@@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean $(RM) $(HCC) $(RM) -r bin-wrappers $(dep_dirs) $(RM) -r po/build/ - $(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h + $(RM) *.pyc *.pyo */*.pyc */*.pyo + $(RM) config-list.h command-list.h bugreport-config-safelist.h $(RM) $(ETAGS_TARGET) tags cscope* $(RM) -r $(GIT_TARNAME) .doc-tmp-dir $(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
generate-bugreport-config-safelist.sh new file +22 -0
@@ -0,0 +1,22 @@ +#!/bin/sh + +cat <<EOF +/* Automatically generated by bugreport-generate-config-safelist.sh */ + + +static const char *bugreport_config_safelist[] = { +EOF + +# cat all regular files in Documentation/config +find Documentation/config -type f -exec cat {} \; | +# print the command name which matches the bugreport-include macro +sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' | +sort | +while read line +do + echo " \"$line\"," +done + +cat <<EOF +}; +EOF
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
Quoted 14 lines · Emily Shaffer
Makefile +1
-1@@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN) install-python-script: $(SCRIPT_PYTHON_GEN) $(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' -.PHONY: clean-perl-script clean-sh-script clean-python-script +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies clean-sh-script: $(RM) $(SCRIPT_SH_GEN) clean-perl-script:
Makefile VCSSVN_LIB = vcs-svn/lib.a · line 820
Quoted 4 lines
Makefile +1
-0@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a GENERATED_H += config-list.h GENERATED_H += command-list.h +GENERATED_H += bugreport-config-safelist.h
OK.
Makefile command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc · line 2169
Quoted 11 lines
Makefile +5
-0LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \ $(FIND) . \@@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc $(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \ command-list.txt >$@+ && mv $@+ $@ +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh + +bugreport-config-safelist.h: Documentation/config/*.txt + $(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \ + >$@+ && mv $@+ $@
OK.
Makefile command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc · line 2173
Quoted 3 lines
Makefile +0
-0SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\ $(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\ $(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
But bugreport.o needs this *.h file generated before a compiler attempts to produce it out of bugreport.c; that dependency is missing from this update to the Makefile.
Quoted 10 lines
Makefile +1
-1@@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE .PHONY: sparse $(SP_OBJ) sparse: $(SP_OBJ) -GEN_HDRS := config-list.h command-list.h unicode-width.h +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/% ifndef GCRYPT_SHA256 EXCEPT_HDRS += sha256/gcrypt.h
Makefile clean: profile-clean coverage-clean cocciclean · line 3128
Quoted 6 lines
Makefile +2
-1@@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean $(RM) $(HCC) $(RM) -r bin-wrappers $(dep_dirs) $(RM) -r po/build/ - $(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h + $(RM) *.pyc *.pyo */*.pyc */*.pyo + $(RM) config-list.h command-list.h bugreport-config-safelist.h
It is kind of sad that GEN_HDRS defines the list of build artifact that we should be able to clean, and then we manually list them to be removed here independently. Can we fix it up too?
We probably clean up the build/update procedure around unicode-width.h before we can do so, probably. It may be generatable using contrib/ script, but as far as our normal build is concerned, it is a tracked source and not part of the build byproducts, so we probably would want to remove it from GEN_HDRS. When that happens, we can $(RM) all of the $(GEN_HDRS) in the "clean" target.
Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
Quoted 37 lines · Junio C Hamano
Quoted 14 lines · Emily Shaffer
Makefile +1
-1@@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN) install-python-script: $(SCRIPT_PYTHON_GEN) $(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' -.PHONY: clean-perl-script clean-sh-script clean-python-script +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies clean-sh-script: $(RM) $(SCRIPT_SH_GEN) clean-perl-script:Makefile VCSSVN_LIB = vcs-svn/lib.a · line 820
Quoted 4 lines
Makefile +1
-0@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a GENERATED_H += config-list.h GENERATED_H += command-list.h +GENERATED_H += bugreport-config-safelist.hOK.
Makefile command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc · line 2169
Quoted 11 lines
Makefile +5
-0LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \ $(FIND) . \@@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc $(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \ command-list.txt >$@+ && mv $@+ $@ +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh + +bugreport-config-safelist.h: Documentation/config/*.txt + $(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \ + >$@+ && mv $@+ $@OK.
Quoted 7 lines
Makefile command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc · line 2173
Quoted 3 lines
Makefile +0
-0SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\ $(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\ $(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\But bugreport.o needs this *.h file generated before a compiler attempts to produce it out of
bugreport.c; that dependency is missing from this update to theMakefile.
Hm. Somewhere I misformatted my commits, then. My goal was to add the new header here, but not add it as a dependency to anybody; then, in the next commit, add it as a dependency to git-bugreport and start to use it (commit 1: make the thing; commit 2: use the thing). But now when I look at the next commit, I don't see a Makefile change. So I missed something while I was shuffling around fixups. Thanks for noticing.
Quoted 5 lines
Quoted 5 lines
Makefile +0
-0@@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE .PHONY: sparse $(SP_OBJ) sparse: $(SP_OBJ)
Quoted 16 lines
Makefile clean: profile-clean coverage-clean cocciclean · line 3128
Quoted 12 lines
Makefile +3
-2-GEN_HDRS := config-list.h command-list.h unicode-width.h +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/% ifndef GCRYPT_SHA256 EXCEPT_HDRS += sha256/gcrypt.h@@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean $(RM) $(HCC) $(RM) -r bin-wrappers $(dep_dirs) $(RM) -r po/build/ - $(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h + $(RM) *.pyc *.pyo */*.pyc */*.pyo + $(RM) config-list.h command-list.h bugreport-config-safelist.hIt is kind of sad that
GEN_HDRSdefines the list of build artifact that we should be able to clean, and then we manually list them to be removed here independently. Can we fix it up too?
Yeah, I can do that. I thought so too when I was updating this.
I *think* what happened is that when I split config-list away (in the earleir commit in this chain) I didn't notice that command-list.h was already in GEN_HDRS, and instead just grepped Makefile for "command-list.h" and added config-list.h next to it. So I'll try to fix it much earlier, in that commit, and then simply add to the appropriate variables in this commit.
Quoted 7 lines
We probably clean up the build/update procedure around
unicode-width.hbefore we can do so, probably. It may be generatable using contrib/ script, but as far as our normal build is concerned, it is a tracked source and not part of the build byproducts, so we probably would want to remove it fromGEN_HDRS. When that happens, we can $(RM) all of the $(GEN_HDRS) in the "clean" target.
Noted. Thanks.
- Emily
Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
Quoted 85 lines · Emily Shaffer
Quoted 37 lines · Junio C Hamano
Quoted 14 lines · Emily Shaffer
Makefile +1
-1@@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN) install-python-script: $(SCRIPT_PYTHON_GEN) $(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' -.PHONY: clean-perl-script clean-sh-script clean-python-script +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies clean-sh-script: $(RM) $(SCRIPT_SH_GEN) clean-perl-script:Makefile VCSSVN_LIB = vcs-svn/lib.a · line 820
Quoted 4 lines
Makefile +1
-0@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a GENERATED_H += config-list.h GENERATED_H += command-list.h +GENERATED_H += bugreport-config-safelist.hOK.
Makefile command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc · line 2169
Quoted 11 lines
Makefile +5
-0LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \ $(FIND) . \@@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc $(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \ command-list.txt >$@+ && mv $@+ $@ +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh + +bugreport-config-safelist.h: Documentation/config/*.txt + $(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \ + >$@+ && mv $@+ $@OK.
Quoted 7 lines
Makefile command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc · line 2173
Quoted 3 lines
Makefile +0
-0SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\ $(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\ $(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\But bugreport.o needs this *.h file generated before a compiler attempts to produce it out of
bugreport.c; that dependency is missing from this update to theMakefile.Hm. Somewhere I misformatted my commits, then. My goal was to add the new header here, but not add it as a dependency to anybody; then, in the next commit, add it as a dependency to git-bugreport and start to use it (commit 1: make the thing; commit 2: use the thing). But now when I look at the next commit, I don't see a
Makefilechange. So I missed something while I was shuffling around fixups. Thanks for noticing.Quoted 5 lines
Quoted 5 lines
Makefile +0
-0@@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE .PHONY: sparse $(SP_OBJ) sparse: $(SP_OBJ)Quoted 16 lines
Makefile clean: profile-clean coverage-clean cocciclean · line 3128
Quoted 12 lines
Makefile +3
-2-GEN_HDRS := config-list.h command-list.h unicode-width.h +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/% ifndef GCRYPT_SHA256 EXCEPT_HDRS += sha256/gcrypt.h@@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean $(RM) $(HCC) $(RM) -r bin-wrappers $(dep_dirs) $(RM) -r po/build/ - $(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h + $(RM) *.pyc *.pyo */*.pyc */*.pyo + $(RM) config-list.h command-list.h bugreport-config-safelist.hIt is kind of sad that
GEN_HDRSdefines the list of build artifact that we should be able to clean, and then we manually list them to be removed here independently. Can we fix it up too?Yeah, I can do that. I thought so too when I was updating this.
I *think* what happened is that when I split config-list away (in the earleir commit in this chain) I didn't notice that
command-list.hwas already inGEN_HDRS, and instead just greppedMakefilefor "command-list.h" and addedconfig-list.hnext to it. So I'll try to fix it much earlier, in that commit, and then simply add to the appropriate variables in this commit.Quoted 1 line
Quoted 8 lines
Quoted 6 lines
We probably clean up the build/update procedure around
unicode-width.hbefore we can do so, probably. It may be generatable using contrib/ script, but as far as our normal build is concerned, it is a tracked source and not part of the build byproducts, so we probably would want to remove it fromGEN_HDRS. When that happens, we can $(RM) all of the $(GEN_HDRS) in the "clean" target.Noted. Thanks.
Ah, after I sent this mail I saw your patch. For those playing along at home, https://lore.kernel.org/[email protected]
Will review.
- Emily
Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
In reply to Emily ShafferHi Emily,
Quoted 9 lines · Emily Shaffer
generate-bugreport-config-safelist.sh new file +3
-0@@ -0,0 +1,22 @@ +#!/bin/sh + +cat <<EOF
generate-bugreport-config-safelist.sh line 13
Quoted 10 lines
generate-bugreport-config-safelist.sh new file +10
-0+/* Automatically generated by bugreport-generate-config-safelist.sh */ + + +static const char *bugreport_config_safelist[] = { +EOF + +# cat all regular files in Documentation/config +find Documentation/config -type f -exec cat {} \; | +# print the command name which matches the bugreport-include macro +sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |
If you use / "\1",/ as replacement, you can totally avoid that ugly while loop (that is unfortunately quite slow in MSYS2/Cygwin). You can still pipe the result into sort just the same.
Ciao,
Dscho
Quoted 13 lines
generate-bugreport-config-safelist.sh new file +9
-1+sort | +while read line +do + echo " \"$line\"," +done + +cat <<EOF +}; +EOF --2.24.1.735.g03f4e72817-goog
Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
Quoted 19 lines · Johannes Schindelin
Hi Emily,
Quoted 15 lines · Emily Shaffer
generate-bugreport-config-safelist.sh new file +9
-0@@ -0,0 +1,22 @@ +#!/bin/sh + +cat <<EOF +/* Automatically generated by bugreport-generate-config-safelist.sh */ + + +static const char *bugreport_config_safelist[] = { +EOF +
Quoted 8 lines
generate-bugreport-config-safelist.sh line 13
Quoted 4 lines
generate-bugreport-config-safelist.sh new file +4
-0+# cat all regular files in Documentation/config +find Documentation/config -type f -exec cat {} \; | +# print the command name which matches the bugreport-include macro +sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |If you use
/ "\1",/as replacement, you can totally avoid that uglywhileloop (that is unfortunately quite slow in MSYS2/Cygwin). You can still pipe the result intosortjust the same.
Oooof. How embarrassing - I will do that.
Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
In reply to Emily ShafferHi Emily,
Documentation/asciidoctor-extensions.rb module Git · line 43
Quoted 12 lines · Emily Shaffer
Documentation/asciidoctor-extensions.rb +4
-0@@ -37,6 +37,10 @@ module Git output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>") end output + + class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor + def process(parent, action, attrs) + ""
The Azure Pipeline fails on this hunk, saying:
/usr/bin/asciidoctor: /home/vsts/work/1/s/Documentation/asciidoctor-extensions.rb:41: class definition in method body (SyntaxError) /home/vsts/work/1/s/Documentation/asciidoctor-extensions.rb:55: syntax error, unexpected end-of-input, expecting keyword_end
Ciao,
Dscho
Quoted 199 lines
Documentation/asciidoctor-extensions.rb +3
-0end end end@@ -45,4 +49,7 @@ end Asciidoctor::Extensions.register do inline_macro Git::Documentation::LinkGitProcessor, :linkgit postprocessor Git::Documentation::DocumentPostProcessor + # The bugreport macro does nothing as far as rendering is + # concerned -- we just grep for it in the sources. + inline_macro Git::Documentation::BugReportProcessor, :bugreport endDocumentation/config/sendemail.txt +34
-34@@ -1,63 +1,63 @@ -sendemail.identity:: +sendemail.identity bugreport:exclude[x] :: A configuration identity. When given, causes values in the 'sendemail.<identity>' subsection to take precedence over values in the 'sendemail' section. The default identity is the value of `sendemail.identity`. -sendemail.smtpEncryption:: +sendemail.smtpEncryption bugreport:include[x] :: See linkgit:git-send-email[1] for description. Note that this setting is not subject to the 'identity' mechanism. -sendemail.smtpssl (deprecated):: +sendemail.smtpssl (deprecated) bugreport:exclude[x] :: Deprecated alias for 'sendemail.smtpEncryption = ssl'. -sendemail.smtpsslcertpath:: +sendemail.smtpsslcertpath bugreport:exclude[x] :: Path to ca-certificates (either a directory or a single file). Set it to an empty string to disable certificate verification. -sendemail.<identity>.*:: +sendemail.<identity>.* bugreport:exclude[x] :: Identity-specific versions of the 'sendemail.*' parameters found below, taking precedence over those when this identity is selected, through either the command-line or `sendemail.identity`. -sendemail.aliasesFile:: -sendemail.aliasFileType:: -sendemail.annotate:: -sendemail.bcc:: -sendemail.cc:: -sendemail.ccCmd:: -sendemail.chainReplyTo:: -sendemail.confirm:: -sendemail.envelopeSender:: -sendemail.from:: -sendemail.multiEdit:: -sendemail.signedoffbycc:: -sendemail.smtpPass:: -sendemail.suppresscc:: -sendemail.suppressFrom:: -sendemail.to:: -sendemail.tocmd:: -sendemail.smtpDomain:: -sendemail.smtpServer:: -sendemail.smtpServerPort:: -sendemail.smtpServerOption:: -sendemail.smtpUser:: -sendemail.thread:: -sendemail.transferEncoding:: -sendemail.validate:: -sendemail.xmailer:: +sendemail.aliasesFile bugreport:exclude[x] :: +sendemail.aliasFileType bugreport:exclude[x] :: +sendemail.annotate bugreport:include[x] :: +sendemail.bcc bugreport:include[x] :: +sendemail.cc bugreport:include[x] :: +sendemail.ccCmd bugreport:include[x] :: +sendemail.chainReplyTo bugreport:include[x] :: +sendemail.confirm bugreport:include[x] :: +sendemail.envelopeSender bugreport:include[x] :: +sendemail.from bugreport:include[x] :: +sendemail.multiEdit bugreport:include[x] :: +sendemail.signedoffbycc bugreport:include[x] :: +sendemail.smtpPass bugreport:exclude[x] :: +sendemail.suppresscc bugreport:include[x] :: +sendemail.suppressFrom bugreport:include[x] :: +sendemail.to bugreport:include[x] :: +sendemail.tocmd bugreport:include[x] :: +sendemail.smtpDomain bugreport:include[x] :: +sendemail.smtpServer bugreport:include[x] :: +sendemail.smtpServerPort bugreport:include[x] :: +sendemail.smtpServerOption bugreport:include[x] :: +sendemail.smtpUser bugreport:exclude[x] :: +sendemail.thread bugreport:include[x] :: +sendemail.transferEncoding bugreport:include[x] :: +sendemail.validate bugreport:include[x] :: +sendemail.xmailer bugreport:include[x] :: See linkgit:git-send-email[1] for description. -sendemail.signedoffcc (deprecated):: +sendemail.signedoffcc (deprecated) bugreport:exclude[x] :: Deprecated alias for `sendemail.signedoffbycc`. -sendemail.smtpBatchSize:: +sendemail.smtpBatchSize bugreport:include[x] :: Number of messages to be sent per connection, after that a relogin will happen. If the value is 0 or undefined, send all messages in one connection. See also the `--batch-size` option of linkgit:git-send-email[1]. -sendemail.smtpReloginDelay:: +sendemail.smtpReloginDelay bugreport:include[x] :: Seconds wait before reconnecting to smtp server. See also the `--relogin-delay` option of linkgit:git-send-email[1].Makefile +11
-3@@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN) install-python-script: $(SCRIPT_PYTHON_GEN) $(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' -.PHONY: clean-perl-script clean-sh-script clean-python-script +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies clean-sh-script: $(RM) $(SCRIPT_SH_GEN) clean-perl-script:@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a GENERATED_H += config-list.h GENERATED_H += command-list.h +GENERATED_H += bugreport-config-safelist.h LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \ $(FIND) . \@@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc $(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \ command-list.txt >$@+ && mv $@+ $@ +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh + +bugreport-config-safelist.h: Documentation/config/*.txt + $(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \ + >$@+ && mv $@+ $@ + SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\ $(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\ $(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\@@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE .PHONY: sparse $(SP_OBJ) sparse: $(SP_OBJ) -GEN_HDRS := config-list.h command-list.h unicode-width.h +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/% ifndef GCRYPT_SHA256 EXCEPT_HDRS += sha256/gcrypt.h@@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean $(RM) $(HCC) $(RM) -r bin-wrappers $(dep_dirs) $(RM) -r po/build/ - $(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h + $(RM) *.pyc *.pyo */*.pyc */*.pyo + $(RM) config-list.h command-list.h bugreport-config-safelist.h $(RM) $(ETAGS_TARGET) tags cscope* $(RM) -r $(GIT_TARNAME) .doc-tmp-dir $(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gzgenerate-bugreport-config-safelist.sh new file +22
-1@@ -0,0 +1,22 @@ +#!/bin/sh + +cat <<EOF +/* Automatically generated by bugreport-generate-config-safelist.sh */ + + +static const char *bugreport_config_safelist[] = { +EOF + +# cat all regular files in Documentation/config +find Documentation/config -type f -exec cat {} \; | +# print the command name which matches the bugreport-include macro +sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' | +sort | +while read line +do + echo " \"$line\"," +done + +cat <<EOF +}; +EOF --2.24.1.735.g03f4e72817-goog
PATCH v4 10/15 bugreport: add config values from safelist
In reply to Emily ShafferTeach bugreport to gather the values of config options which are present in 'bugreport-config-safelist.h'.
Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.
Taking the build-time generated array and putting it into a set saves us time - since git_config_bugreport() is called for every option the user has configured, performing option lookup in constant time is a useful optimization.
--- bugreport.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 71 insertions(+), 1 deletion(-)
bugreport.c +71 -1
@@ -6,6 +6,9 @@#include "help.h" #include <gnu/libc-version.h> #include "run-command.h" +#include "config.h" +#include "bugreport-config-safelist.h" +#include "khash.h" static void get_http_version_info (struct strbuf *http_info) {
@@ -18,6 +21,41 @@ static void get_http_version_info(struct strbuf *http_info)strbuf_addstr (http_info,"'git-http-fetch -V' not supported\n" ); } +KHASH_INIT(cfg_set, const char*, int, 0, kh_str_hash_func, kh_str_hash_equal); + +struct cfgset { +kh_cfg_set_t set; +}; + +struct cfgset safelist; + +static void cfgset_init (struct cfgset *set,size_t initial_size) +{ +memset (&set->set,0 ,sizeof (set->set)); +if (initial_size) +kh_resize_cfg_set (&set->set, initial_size); +} + +static int cfgset_insert (struct cfgset *set,const char *cfg_key) +{ +int added; +kh_put_cfg_set (&set->set, cfg_key, &added); +printf ("ESS: added %s\n" , cfg_key); +return !added; +} + +static int cfgset_contains (struct cfgset *set,const char *cfg_key) +{ +khiter_t pos =kh_get_cfg_set (&set->set, cfg_key); +return pos !=kh_end (&set->set); +} + +static void cfgset_clear (struct cfgset *set) +{ +kh_release_cfg_set (&set->set); +cfgset_init (set,0 ); +} +static void get_system_info (struct strbuf *sys_info) {struct strbuf version_info =STRBUF_INIT ;
@@ -53,6 +91,36 @@ static void get_system_info(struct strbuf *sys_info)strbuf_complete_line (sys_info); } +static void gather_safelist () +{ +int index; +int safelist_len =sizeof (bugreport_config_safelist) /sizeof (const char *); +cfgset_init (&safelist, safelist_len); +for (index =0 ; index < safelist_len; index++) +cfgset_insert (&safelist, bugreport_config_safelist[index]); + +} + +static int git_config_bugreport (const char *var,const char *value,void *cb) +{ +struct strbuf *config_info = (struct strbuf *)cb; + +if (cfgset_contains (&safelist, var)) +strbuf_addf (config_info, +"%s (%s) : %s\n" , + var,config_scope_to_string (current_config_scope ()), + value); + +return 0 ; +} + +static void get_safelisted_config (struct strbuf *config_info) +{ +gather_safelist (); +git_config (git_config_bugreport, config_info); +cfgset_clear (&safelist); +} +static const char *const bugreport_usage[] = {N_ ("git bugreport [-o|--output <file>]" ),NULL
@@ -114,10 +182,12 @@ int cmd_main(int argc, const char **argv)get_bug_template (&buffer); -// add other contents get_header (&buffer,"System Info" );get_system_info (&buffer); +get_header (&buffer,"Safelisted Config Info" ); +get_safelisted_config (&buffer); + report =fopen_for_writing (report_path.buf);strbuf_write (&buffer, report);fclose (report);
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Re: [PATCH v4 10/15] bugreport: add config values from safelist
Quoted 8 lines · Emily Shaffer
Teach bugreport to gather the values of config options which are present in '
bugreport-config-safelist.h'.Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.
Quoted 4 lines
Taking the build-time generated array and putting it into a set saves us time - since
git_config_bugreport()is called for every option the user has configured, performing option lookup in constant time is a useful optimization.
Interesting. I actually was expecting the look-up to go the other way around. We know the safe keys, so iterate over them and grab their values, if defined. No need for hashes or anything, but just a simple linear list of safe stuff.
But that is too simple-minded. If we wanted to safelist foo.*.bar, where '*' can be anything, walking on the list of safe variables would not work. We must have a hash table that knows "foo.*.bar" is allowed, and while walking all the configuration keys, when we see foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if it is whitelisted, or something like that.
But then I am not sure if this implementation does something like this for three-level names. If not, I do not see much point in use of the hash there either.
Puzzled.
Quoted 113 lines
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 71 insertions(+), 1 deletion(-)bugreport.c +71
-1@@ -6,6 +6,9 @@#include "help.h" #include <gnu/libc-version.h> #include "run-command.h" +#include "config.h" +#include "bugreport-config-safelist.h" +#include "khash.h" static void get_http_version_info (struct strbuf *http_info) {@@ -18,6 +21,41 @@ static void get_http_version_info(struct strbuf *http_info)strbuf_addstr (http_info,"'git-http-fetch -V' not supported\n" ); } +KHASH_INIT(cfg_set, const char*, int, 0, kh_str_hash_func, kh_str_hash_equal); + +struct cfgset { +kh_cfg_set_t set; +}; + +struct cfgset safelist; + +static void cfgset_init (struct cfgset *set,size_t initial_size) +{ +memset (&set->set,0 ,sizeof (set->set)); +if (initial_size) +kh_resize_cfg_set (&set->set, initial_size); +} + +static int cfgset_insert (struct cfgset *set,const char *cfg_key) +{ +int added; +kh_put_cfg_set (&set->set, cfg_key, &added); +printf ("ESS: added %s\n" , cfg_key); +return !added; +} + +static int cfgset_contains (struct cfgset *set,const char *cfg_key) +{ +khiter_t pos =kh_get_cfg_set (&set->set, cfg_key); +return pos !=kh_end (&set->set); +} + +static void cfgset_clear (struct cfgset *set) +{ +kh_release_cfg_set (&set->set); +cfgset_init (set,0 ); +} +static void get_system_info (struct strbuf *sys_info) {struct strbuf version_info =STRBUF_INIT ;@@ -53,6 +91,36 @@ static void get_system_info(struct strbuf *sys_info)strbuf_complete_line (sys_info); } +static void gather_safelist () +{ +int index; +int safelist_len =sizeof (bugreport_config_safelist) /sizeof (const char *); +cfgset_init (&safelist, safelist_len); +for (index =0 ; index < safelist_len; index++) +cfgset_insert (&safelist, bugreport_config_safelist[index]); + +} + +static int git_config_bugreport (const char *var,const char *value,void *cb) +{ +struct strbuf *config_info = (struct strbuf *)cb; + +if (cfgset_contains (&safelist, var)) +strbuf_addf (config_info, +"%s (%s) : %s\n" , + var,config_scope_to_string (current_config_scope ()), + value); + +return 0 ; +} + +static void get_safelisted_config (struct strbuf *config_info) +{ +gather_safelist (); +git_config (git_config_bugreport, config_info); +cfgset_clear (&safelist); +} +static const char *const bugreport_usage[] = {N_ ("git bugreport [-o|--output <file>]" ),NULL @@ -114,10 +182,12 @@ int cmd_main(int argc, const char **argv)get_bug_template (&buffer); -// add other contents get_header (&buffer,"System Info" );get_system_info (&buffer); +get_header (&buffer,"Safelisted Config Info" ); +get_safelisted_config (&buffer); + report =fopen_for_writing (report_path.buf);strbuf_write (&buffer, report);fclose (report);
Re: [PATCH v4 10/15] bugreport: add config values from safelist
Quoted 15 lines · Junio C Hamano
Quoted 8 lines · Emily Shaffer
Teach bugreport to gather the values of config options which are present in '
bugreport-config-safelist.h'.Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.
Quoted 4 lines
Taking the build-time generated array and putting it into a set saves us time - since
git_config_bugreport()is called for every option the user has configured, performing option lookup in constant time is a useful optimization.
Quoted 4 lines
Interesting. I actually was expecting the look-up to go the other way around. We know the safe keys, so iterate over them and grab their values, if defined. No need for hashes or anything, but just a simple linear list of safe stuff.
Hm. Without looking at the code, I said to myself, "repo_config_get_value() will open all the available config files to find the resolved value, so I don't want to do n*4 file open/closes, I only want to do 4 total."
Now I look at the code and see that the configs are already being read into a hashset before now. So you're right that it doesn't make sense for me to do it this way....
Quoted 7 lines
But that is too simple-minded. If we wanted to safelist foo.*.bar, where '*' can be anything, walking on the list of safe variables would not work. We must have a hash table that knows "foo.*.bar" is allowed, and while walking all the configuration keys, when we see foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if it is whitelisted, or something like that.
...unless we want to use wildcards like you suggest.
But I'm not sure it's a good idea. I envision someone writing another Git add-on, which offers someone to specify "user.password" and automagically do some Bitbucket interaction, or something. (An extreme and misguided example, but I hope the point remains.) Then if I say, "all the user.* configs I can see in 'git help config' are safe, so I will just safelist user.*," I'm not accounting for this other tool's known configs which are a superset of what Git knows and expects.
The point of this safelist-generation exercise is to avoid accidentally printing some sensitive user config, and the point of using a safelist instead of a blocklist is to avoid printing a third-party config we didn't know about (or messing up our pattern matching). So I suggest we avoid pattern matching entirely.
Quoted 6 lines
But then I am not sure if this implementation does something like this for three-level names. If not, I do not see much point in use of the hash there either.
Puzzled.
I'd prefer to solve it by iterating over the compile-time-generated safelist using get_config_value_multi(), which I clock at O(n) for n = safelist length: performing n calls to an O(1) hashset lookup. That saves us the O(n*log(n)) hashset population, and the implementation of Yet Another Set in the source.
Thanks.
- Emily
Re: [PATCH v4 10/15] bugreport: add config values from safelist
Quoted 11 lines · Emily Shaffer
Quoted 6 lines
But that is too simple-minded. If we wanted to safelist foo.*.bar, where '*' can be anything, walking on the list of safe variables would not work. We must have a hash table that knows "foo.*.bar" is allowed, and while walking all the configuration keys, when we see foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if it is whitelisted, or something like that.
...unless we want to use wildcards like you suggest.
But I'm not sure it's a good idea. I envision someone writing another Git add-on, which offers someone to specify "user.password" ...
Wildcarding the leaf level of two (or for that matter three) level names like "user.*" in your example is of course a nonsense way to use the safelist. But think about three-level names where the second level is designed to be used for user-supplied token to give things of similar nature a name the user can use to distinguish things. If remote.origin.url is worth reporting, wouldn't remote.upstream.url also be? Shouldn't remote.*.url be the way to say "the URL field for each and every remote is a safe thing to report" in the safelist?
Re: [PATCH v4 10/15] bugreport: add config values from safelist
Quoted 14 lines · Junio C Hamano
Quoted 11 lines · Emily Shaffer
Quoted 6 lines
But that is too simple-minded. If we wanted to safelist foo.*.bar, where '*' can be anything, walking on the list of safe variables would not work. We must have a hash table that knows "foo.*.bar" is allowed, and while walking all the configuration keys, when we see foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if it is whitelisted, or something like that.
...unless we want to use wildcards like you suggest.
But I'm not sure it's a good idea. I envision someone writing another Git add-on, which offers someone to specify "user.password" ...
Quoted 9 lines
Wildcarding the leaf level of two (or for that matter three) level names like "user.*" in your example is of course a nonsense way to use the safelist. But think about three-level names where the second level is designed to be used for user-supplied token to give things of similar nature a name the user can use to distinguish things. If remote.origin.url is worth reporting, wouldn't remote.upstream.url also be? Shouldn't remote.*.url be the way to say "the URL field for each and every remote is a safe thing to report" in the safelist?
Bah, somehow this mail slipped through the cracks for me.
When you put it that way, I see how it can be useful. Although I admit to fatigue on this patchset and now I'm hesitant about feature creep. :)
I'm going to send the reroll for v5 as I have it, not having seen this mail, and think more about how I'd approach wildcarding in the middle like this. I want to take a look at the other configs which use a middle level as a user token and try to reason about whether we ought to be collecting them. I'll hope to report back sooner (or later).
- Emily
Re: [PATCH v4 10/15] bugreport: add config values from safelist
In reply to Emily ShafferHi Emily,
bugreport.c line 11
Quoted 11 lines · Emily Shaffer
bugreport.c +3
-0@@ -6,6 +6,9 @@#include "help.h" #include <gnu/libc-version.h> #include "run-command.h" +#include "config.h" +#include "bugreport-config-safelist.h" +#include "khash.h"
Seems that this patch makes things fail in the CI build
(https://dev.azure.com/gitgitgadget/git/_build/results?buildId=23830&view=results):
-- snipsnap --
bugreport.c:10:10: fatal error: 'bugreport-config-safelist.h' file not found
#include "bugreport-config-safelist.h"
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~1 error generated.
Makefile:2395: recipe for target 'bugreport.o' failed
make: *** [bugreport.o] Error 1
make: *** Waiting for unfinished jobs....
PATCH v4 11/15 bugreport: collect list of populated hooks
In reply to Emily ShafferOccasionally a failure a user is seeing may be related to a specific hook which is being run, perhaps without the user realizing. While the contents of hooks can be sensitive - containing user data or process information specific to the user's organization - simply knowing that a hook is being run at a certain stage can help us to understand whether something is going wrong.
Without a definitive list of hook names within the code, we compile our own list from the documentation. This is likely prone to bitrot. To reduce the amount of code humans need to read, we turn the list into a string_list and iterate over it (as we are calling the same find_hook operation on each string). However, since bugreport should primarily be called by the user, the performance loss from massaging the string seems acceptable.
--- bugreport.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+)
bugreport.c +55 -0
@@ -9,6 +9,7 @@#include "config.h" #include "bugreport-config-safelist.h" #include "khash.h" +#include "run-command.h" static void get_http_version_info (struct strbuf *http_info) {
@@ -121,6 +122,57 @@ static void get_safelisted_config(struct strbuf *config_info)cfgset_clear (&safelist); } +static void get_populated_hooks (struct strbuf *hook_info) +{ +/* +* Doesn't look like there is a list of all possible hooks; so below is +* a transcription of `git help hook`. +*/ +const char *hooks ="applypatch-msg," +"pre-applypatch," +"post-applypatch," +"pre-commit," +"pre-merge-commit," +"prepare-commit-msg," +"commit-msg," +"post-commit," +"pre-rebase," +"post-checkout," +"post-merge," +"pre-push," +"pre-receive," +"update," +"post-receive," +"post-update," +"push-to-checkout," +"pre-auto-gc," +"post-rewrite," +"sendemail-validate," +"fsmonitor-watchman," +"p4-pre-submit," +"post-index-changex" ; +struct string_list hooks_list =STRING_LIST_INIT_DUP ; +struct string_list_item *iter =NULL ; +int nongit_ok; + +setup_git_directory_gently (&nongit_ok); + +if (nongit_ok) { +strbuf_addstr (hook_info, +"not run from a git repository - no hooks to show\n" ); +return ; + } + +string_list_split (&hooks_list, hooks,',' ,-1 ); + +for_each_string_list_item (iter, &hooks_list) { +if (find_hook (iter->string)) { +strbuf_addstr (hook_info, iter->string); +strbuf_complete_line (hook_info); + } + } +} +static const char *const bugreport_usage[] = {N_ ("git bugreport [-o|--output <file>]" ),NULL
@@ -188,6 +240,9 @@ int cmd_main(int argc, const char **argv)get_header (&buffer,"Safelisted Config Info" );get_safelisted_config (&buffer); +get_header (&buffer,"Configured Hooks" ); +get_populated_hooks (&buffer); + report =fopen_for_writing (report_path.buf);strbuf_write (&buffer, report);fclose (report);
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Re: [PATCH v4 11/15] bugreport: collect list of populated hooks
bugreport.c static void get_safelisted_config(struct strbuf *config_info) · line 130
Quoted 4 lines · Emily Shaffer
bugreport.c +4
-0+/* +* Doesn't look like there is a list of all possible hooks; so below is +* a transcription of `git help hook`. +*/
We probably would want to employ technique similar to what you used for the list of safe configuration we saw in an earlier patch in the series. What's not even documented is not worth reporting ;-)
Quoted 58 lines
bugreport.c +48
-0+const char *hooks ="applypatch-msg," +"pre-applypatch," +"post-applypatch," +"pre-commit," +"pre-merge-commit," +"prepare-commit-msg," +"commit-msg," +"post-commit," +"pre-rebase," +"post-checkout," +"post-merge," +"pre-push," +"pre-receive," +"update," +"post-receive," +"post-update," +"push-to-checkout," +"pre-auto-gc," +"post-rewrite," +"sendemail-validate," +"fsmonitor-watchman," +"p4-pre-submit," +"post-index-changex" ; +struct string_list hooks_list =STRING_LIST_INIT_DUP ; +struct string_list_item *iter =NULL ; +int nongit_ok; + +setup_git_directory_gently (&nongit_ok); + +if (nongit_ok) { +strbuf_addstr (hook_info, +"not run from a git repository - no hooks to show\n" ); +return ; + } + +string_list_split (&hooks_list, hooks,',' ,-1 ); + +for_each_string_list_item (iter, &hooks_list) { +if (find_hook (iter->string)) { +strbuf_addstr (hook_info, iter->string); +strbuf_complete_line (hook_info); + } + } +} +static const char *const bugreport_usage[] = {N_ ("git bugreport [-o|--output <file>]" ),NULL @@ -188,6 +240,9 @@ int cmd_main(int argc, const char **argv)get_header (&buffer,"Safelisted Config Info" );get_safelisted_config (&buffer); +get_header (&buffer,"Configured Hooks" ); +get_populated_hooks (&buffer); + report =fopen_for_writing (report_path.buf);strbuf_write (&buffer, report);fclose (report);
Re: [PATCH v4 11/15] bugreport: collect list of populated hooks
Quoted 10 lines · Junio C Hamano
bugreport.c static void get_safelisted_config(struct strbuf *config_info) · line 130
Quoted 4 lines · Emily Shaffer
bugreport.c +4
-0+/* +* Doesn't look like there is a list of all possible hooks; so below is +* a transcription of `git help hook`. +*/ We probably would want to employ technique similar to what you used for the list of safe configuration we saw in an earlier patch in the series. What's not even documented is not worth reporting ;-)
I somewhat slightly would rather not, since I hope to include a list of the available hooks in my work with git-hook:
https://lore.kernel.org/[email protected]
It's true that the way this patch is written now is prone to bitrot. It's also true that I could win the lottery tomorrow and never finish my hopes of git-hook implementation. And it's finally true that I don't want to write a fourth build-time-generated header which scrapes docs, especially if I also plan to delete it two months from now.
I wonder whether a better approach might be to drop hook support entirely here, and add it later on when this particular bit of tech debt (no canonical list of hook names) is solved, one way or another.
Another approach may be to just list all contents of $(core.hookdir)/ which doesn't end in ".sample", which is what the bash first attempt of bugreport did. This does mean that those who have something like:
.git/hooks/
pre-commit
pre-commit.d/
pre-commit:
sh pre-commit.d/a
sh pre-commit.d/b
will have their multihook config exposed, although the contents of those hooks won't be.
I'm interested to know everyone else's opinion, because mine is biased by my hope that I can solve all the world's problems with git-hook ;)
- Emily
PATCH v4 12/15 bugreport: count loose objects
In reply to Emily ShafferThe number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.
Rather than directly invoking 'git-count-objects', which may sometimes fail unexpectedly on Git for Windows, manually count the contents of .git/objects. Additionally, since we may wish to inspect other directories' contents for bugreport in the future, put the directory listing into a helper function.
--- bugreport.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+)
bugreport.c +64 -0
@@ -173,6 +173,67 @@ static void get_populated_hooks(struct strbuf *hook_info) } } +static int is_hex (const char *string,size_t count) +{ +for (; count; string++, count--) { +if (!isxdigit (*string)) +return 0 ; + } +return 1 ; +} + +static void get_loose_object_summary (struct strbuf *obj_info) { +struct dirent *d =NULL ; +DIR *dir, *subdir =NULL ; +size_t dir_len; +struct strbuf dirpath =STRBUF_INIT ; + +strbuf_addstr (&dirpath,get_object_directory ()); +strbuf_complete (&dirpath,'/' ); + + dir =opendir (dirpath.buf); +if (!dir) { +strbuf_addf (obj_info,"could not open object directory '%s'\n" , + dirpath.buf); +strbuf_release (&dirpath); +return ; + } + + dir_len = dirpath.len; + +while ((d =readdir (dir))) { +int object_count =0 ; +char subdir_name[3 ]; + +if (d->d_type !=DT_DIR ) +continue ; + +if ((strlen (d->d_name) !=2 ) || (!is_hex (d->d_name,2 ))) +continue ; + +/* copy directory name + \0 */ +memcpy (subdir_name, d->d_name,3 ); + +strbuf_setlen (&dirpath, dir_len); +strbuf_addstr (&dirpath, d->d_name); + + subdir =opendir (dirpath.buf); +if (!subdir) +continue ; +while ((d =readdir (subdir))) +if (d->d_type ==DT_REG ) + object_count++; + +closedir (subdir); + +strbuf_addf (obj_info,"%s: %d\n" , subdir_name, object_count); + } + + +closedir (dir); +strbuf_release (&dirpath); +} +static const char *const bugreport_usage[] = {N_ ("git bugreport [-o|--output <file>]" ),NULL
@@ -243,6 +304,9 @@ int cmd_main(int argc, const char **argv)get_header (&buffer,"Configured Hooks" );get_populated_hooks (&buffer); +get_header (&buffer,"Loose Object Counts" ); +get_loose_object_summary (&buffer); + report =fopen_for_writing (report_path.buf);strbuf_write (&buffer, report);fclose (report);
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Re: [PATCH v4 12/15] bugreport: count loose objects
Quoted 7 lines · Emily Shaffer
The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.
Rather than directly invoking 'git-count-objects', which may sometimes fail unexpectedly on Git for Windows, manually count the contents of .git/objects.
Is for_each_loose_object() or for_each_loose_file_in_objdir() not sufficient? We really do *not* want a redundant implementation of something we already use elsewhere in the system, especially for the purpose of this program, because you would end up reporting what you computed in a way that may be quite different from what the rest of the system that is actually used by the end users computes.
Re: [PATCH v4 12/15] bugreport: count loose objects
Quoted 10 lines · Junio C Hamano
Quoted 7 lines · Emily Shaffer
The number of unpacked objects in a user's repository may help us understand the root of the problem they're seeing, especially if a command is running unusually slowly.
Rather than directly invoking 'git-count-objects', which may sometimes fail unexpectedly on Git for Windows, manually count the contents of .git/objects.
Quoted 7 lines
Is
for_each_loose_object()orfor_each_loose_file_in_objdir()not sufficient? We really do *not* want a redundant implementation of something we already use elsewhere in the system, especially for the purpose of this program, because you would end up reporting what you computed in a way that may be quite different from what the rest of the system that is actually used by the end users computes.
Oh, thanks for the pointer. That looks sufficient - I will investigate the differences between the two and stop filesystem browsing by hand in this case.
- Emily
PATCH v4 13/15 bugreport: add packed object summary
In reply to Emily ShafferAlongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.
--- bugreport.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+)
bugreport.c +31 -0
@@ -234,6 +234,34 @@ static void get_loose_object_summary(struct strbuf *obj_info) {strbuf_release (&dirpath); } +static void get_packed_object_summary (struct strbuf *obj_info) +{ +struct strbuf dirpath =STRBUF_INIT ; +struct dirent *d; +DIR *dir =NULL ; + +strbuf_addstr (&dirpath,get_object_directory ()); +strbuf_complete (&dirpath,'/' ); +strbuf_addstr (&dirpath,"pack/" ); + + dir =opendir (dirpath.buf); +if (!dir) { +strbuf_addf (obj_info,"could not open packed object directory '%s'\n" , + dirpath.buf); +strbuf_release (&dirpath); +return ; + } + +while ((d =readdir (dir))) { +strbuf_addbuf (obj_info, &dirpath); +strbuf_addstr (obj_info, d->d_name); +strbuf_complete_line (obj_info); + } + +closedir (dir); +strbuf_release (&dirpath); +} +static const char *const bugreport_usage[] = {N_ ("git bugreport [-o|--output <file>]" ),NULL
@@ -307,6 +335,9 @@ int cmd_main(int argc, const char **argv)get_header (&buffer,"Loose Object Counts" );get_loose_object_summary (&buffer); +get_header (&buffer,"Packed Object Summary" ); +get_packed_object_summary (&buffer); + report =fopen_for_writing (report_path.buf);strbuf_write (&buffer, report);fclose (report);
Signature · 1 line
2.24.1.735.g03f4e72817-goog
Re: [PATCH v4 13/15] bugreport: add packed object summary
Quoted 3 lines · Emily Shaffer
Alongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.
Not exactly the same comment, but is in the same spirit, as the previous step applies to this one.
Would it be too much work to libify the bulk of cmd_count_objects() that computes numbers, making the cmd_count_objects() into a thin wrapper that calls the libified "counter/collector" function and prints the resulting numbers received from it?
That way, the get_packed_object_summary() can be another consumer of the same "counter/collector" function, no?
Quoted 54 lines
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+)bugreport.c +31
-0@@ -234,6 +234,34 @@ static void get_loose_object_summary(struct strbuf *obj_info) {strbuf_release (&dirpath); } +static void get_packed_object_summary (struct strbuf *obj_info) +{ +struct strbuf dirpath =STRBUF_INIT ; +struct dirent *d; +DIR *dir =NULL ; + +strbuf_addstr (&dirpath,get_object_directory ()); +strbuf_complete (&dirpath,'/' ); +strbuf_addstr (&dirpath,"pack/" ); + + dir =opendir (dirpath.buf); +if (!dir) { +strbuf_addf (obj_info,"could not open packed object directory '%s'\n" , + dirpath.buf); +strbuf_release (&dirpath); +return ; + } + +while ((d =readdir (dir))) { +strbuf_addbuf (obj_info, &dirpath); +strbuf_addstr (obj_info, d->d_name); +strbuf_complete_line (obj_info); + } + +closedir (dir); +strbuf_release (&dirpath); +} +static const char *const bugreport_usage[] = {N_ ("git bugreport [-o|--output <file>]" ),NULL @@ -307,6 +335,9 @@ int cmd_main(int argc, const char **argv)get_header (&buffer,"Loose Object Counts" );get_loose_object_summary (&buffer); +get_header (&buffer,"Packed Object Summary" ); +get_packed_object_summary (&buffer); + report =fopen_for_writing (report_path.buf);strbuf_write (&buffer, report);fclose (report);
Re: [PATCH v4 13/15] bugreport: add packed object summary
Quoted 9 lines · Junio C Hamano
Quoted 3 lines · Emily Shaffer
Alongside the list of loose objects, it's useful to see the list of object packs as well. It can help us to examine what Git did and did not pack.
Not exactly the same comment, but is in the same spirit, as the previous step applies to this one.
Quoted 7 lines
Would it be too much work to libify the bulk of
cmd_count_objects()that computes numbers, making thecmd_count_objects()into a thin wrapper that calls the libified "counter/collector" function and prints the resulting numbers received from it?That way, the
get_packed_object_summary()can be another consumer of the same "counter/collector" function, no?
That sounds like a good approach. Will do.
Quoted 55 lines
Quoted 55 lines
Signed-off-by Emily Shaffer <[email protected]>--- bugreport.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+)bugreport.c +31
-0@@ -234,6 +234,34 @@ static void get_loose_object_summary(struct strbuf *obj_info) {strbuf_release (&dirpath); } +static void get_packed_object_summary (struct strbuf *obj_info) +{ +struct strbuf dirpath =STRBUF_INIT ; +struct dirent *d; +DIR *dir =NULL ; + +strbuf_addstr (&dirpath,get_object_directory ()); +strbuf_complete (&dirpath,'/' ); +strbuf_addstr (&dirpath,"pack/" ); + + dir =opendir (dirpath.buf); +if (!dir) { +strbuf_addf (obj_info,"could not open packed object directory '%s'\n" , + dirpath.buf); +strbuf_release (&dirpath); +return ; + } + +while ((d =readdir (dir))) { +strbuf_addbuf (obj_info, &dirpath); +strbuf_addstr (obj_info, d->d_name); +strbuf_complete_line (obj_info); + } + +closedir (dir); +strbuf_release (&dirpath); +} +static const char *const bugreport_usage[] = {N_ ("git bugreport [-o|--output <file>]" ),NULL @@ -307,6 +335,9 @@ int cmd_main(int argc, const char **argv)get_header (&buffer,"Loose Object Counts" );get_loose_object_summary (&buffer); +get_header (&buffer,"Packed Object Summary" ); +get_packed_object_summary (&buffer); + report =fopen_for_writing (report_path.buf);strbuf_write (&buffer, report);fclose (report);
PATCH v4 14/15 bugreport: list contents of $OBJDIR/info
In reply to Emily ShafferMiscellaneous information used about the object store can end up in .git/objects/info; this can help us understand what may be going on with the object store when the user is reporting a bug. Otherwise, it could be difficult to track down what is going wrong with an object which isn't kept locally to .git/objects/ or .git/objects/pack. Having some understanding of where the user's objects may be kept can save us some hops during the bug reporting process.
--- bugreport.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+)
bugreport.c +48 -0
@@ -262,6 +262,51 @@ static void get_packed_object_summary(struct strbuf *obj_info)strbuf_release (&dirpath); } +static void list_contents_of_dir_recursively (struct strbuf *contents, +struct strbuf *dirpath) +{ +struct dirent *d; +DIR *dir; +size_t path_len; + + dir =opendir (dirpath->buf); +if (!dir) +return ; + +strbuf_complete (dirpath,'/' ); + path_len = dirpath->len; + +while ((d =readdir (dir))) { +if (!strcmp (d->d_name,"." ) || !strcmp (d->d_name,".." )) +continue ; + +strbuf_addbuf (contents, dirpath); +strbuf_addstr (contents, d->d_name); +strbuf_complete_line (contents); + +if (d->d_type ==DT_DIR ) { +strbuf_addstr (dirpath, d->d_name); +list_contents_of_dir_recursively (contents, dirpath); + } +strbuf_setlen (dirpath, path_len); + } + +closedir (dir); +} + +static void get_object_info_summary (struct strbuf *obj_info) +{ +struct strbuf dirpath =STRBUF_INIT ; + +strbuf_addstr (&dirpath,get_object_directory ()); +strbuf_complete (&dirpath,'/' ); +strbuf_addstr (&dirpath,"info/" ); + +list_contents_of_dir_recursively (obj_info, &dirpath); + +strbuf_release (&dirpath); +} +static const char *const bugreport_usage[] = {N_ ("git bugreport [-o|--output <file>]" ),NULL
@@ -338,6 +383,9 @@ int cmd_main(int argc, const char **argv)get_header (&buffer,"Packed Object Summary" );get_packed_object_summary (&buffer); +get_header (&buffer,"Object Info Summary" ); +get_object_info_summary (&buffer); + report =fopen_for_writing (report_path.buf);strbuf_write (&buffer, report);fclose (report);
Signature · 1 line
2.24.1.735.g03f4e72817-goog
PATCH v4 15/15 bugreport: summarize contents of alternates file
In reply to Emily ShafferIn some cases, it could be that the user is having a problem with an object which isn't present in their normal object directory. We can get a hint that that might be the case by examining the list of alternates where their object may be stored instead. Since paths to alternates may be sensitive, we'll instead count how many alternates have been specified and note how many of them exist or are broken.
--- bugreport.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+)
bugreport.c +39 -0
@@ -307,6 +307,42 @@ static void get_object_info_summary(struct strbuf *obj_info)strbuf_release (&dirpath); } +static void get_alternates_summary (struct strbuf *alternates_info) +{ +struct strbuf alternates_path =STRBUF_INIT ; +struct strbuf alternate =STRBUF_INIT ; +FILE *file; +size_t exists =0 , broken =0 ; + +strbuf_addstr (&alternates_path,get_object_directory ()); +strbuf_complete (&alternates_path,'/' ); +strbuf_addstr (&alternates_path,"info/alternates" ); + + file =fopen (alternates_path.buf,"r" ); +if (!file) { +strbuf_addstr (alternates_info,"No alternates file found.\n" ); +strbuf_release (&alternates_path); +return ; + } + +while (strbuf_getline (&alternate, file) !=EOF ) { +if (!access (alternate.buf,F_OK )) + exists++; +else + broken++; + } + +strbuf_addf (alternates_info, +"%zd alternates found (%zd working, %zd broken)\n" , + exists + broken, + exists, + broken); + +fclose (file); +strbuf_release (&alternate); +strbuf_release (&alternates_path); +} +static const char *const bugreport_usage[] = {N_ ("git bugreport [-o|--output <file>]" ),NULL
@@ -386,6 +422,9 @@ int cmd_main(int argc, const char **argv)get_header (&buffer,"Object Info Summary" );get_object_info_summary (&buffer); +get_header (&buffer,"Alternates" ); +get_alternates_summary (&buffer); + report =fopen_for_writing (report_path.buf);strbuf_write (&buffer, report);fclose (report);
Signature · 1 line
2.24.1.735.g03f4e72817-goog
v5 cover add git-bugreport tool
In reply to Emily ShafferFrom: Emily Shaffer <[email protected]>
This topic branch depends on the patch mailed in lore.kernel.org/git/[email protected] in order to display scopes for configs gathered during "bugreport: add config values from safelist".
I'll summarize v4-v5. Since v4 has languished for some time, I don't think an interdiff is too helpful, so I won't include one. Bonus, the code is much simplified from some suggestions from Junio on how to inspect objects, so I hope it's easy to review anyways.
Throughout, did some style changes away from C++ bad habits. Otherwise, as listed:
Emily Shaffer (15):
bugreport: add tool to generate debugging info
(no change)
help: move list_config_help to builtin/help
Changed to agree with f3719846134
bugreport: gather git version and build info
Style only
help: add shell-path to --build-options
(no change) bugreport: add uname info
Removed nodename for privacy reasons
bugreport: add compiler info
Moved glibc tattling into compat/.
I appreciate a close look at this one - I think I understood the right way to go about a compat/ util but it's the first one I've done.
bugreport: add curl version
Moved curl tattling from git-http-fetch into git-remote-curl
bugreport: include user interactive shell
Stop depending on a compiler quirk to save us from segfault
bugreport: generate config safelist based on docs
Changed to agree with f3719846134 Cleaned up script a little
bugreport: add config values from safelist
Made git-bugreport dependent on generated safelist header
bugreport: collect list of populated hooks
No change.
Per https://lore.kernel.org/git/[email protected],
should we even keep this patch?
bugreport: count loose objects
Use helpers from object-store.h instead of manually walking the filesystem.
bugreport: add packed object summary
Use helpers from object-store.h instead of manually walking the filesystem.
bugreport: list contents of $OBJDIR/info
(no change)
bugreport: summarize contents of alternates file
Rephrase commit message to explain why I can't use the helpers in object-store.h.
Thanks.
- Emily
.gitignore | 3 + Documentation/asciidoc.conf | 8 + Documentation/asciidoctor-extensions.rb | 7 + Documentation/config/sendemail.txt | 68 ++-- Documentation/git-bugreport.txt | 43 +++ Makefile | 25 +- bugreport.c | 427 ++++++++++++++++++++++++ builtin/help.c | 86 +++++ compat/compiler.h | 24 ++ generate-bugreport-config-safelist.sh | 22 ++ generate-cmdlist.sh | 19 -- generate-configlist.sh | 24 ++ help.c | 133 ++------ help.h | 2 +- remote-curl.c | 8 + t/t0091-bugreport.sh | 41 +++ 16 files changed, 780 insertions(+), 160 deletions(-) create mode 100644 Documentation/git-bugreport.txt create mode 100644 bugreport.c create mode 100644 compat/compiler.h create mode 100755 generate-bugreport-config-safelist.sh create mode 100755 generate-configlist.sh create mode 100755 t/t0091-bugreport.sh
Signature · 1 line
2.25.0.341.g760bfbb309-goog
PATCH v5 1/15 bugreport: add tool to generate debugging info
From: Emily Shaffer <[email protected]>
Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.
If users can send us a well-written bug report which contains diagnostic information we would otherwise need to ask the user for, we can reduce the number of question-and-answer round trips between the reporter and the Git contributor.
Users may also wish to send a report like this to their local "Git expert" if they have put their repository into a state they are confused by.
--- .gitignore | 1 + Documentation/git-bugreport.txt | 43 ++++++++++++++++++++ Makefile | 5 +++ bugreport.c | 69 +++++++++++++++++++++++++++++++++ t/t0091-bugreport.sh | 41 ++++++++++++++++++++ 5 files changed, 159 insertions(+) create mode 100644 Documentation/git-bugreport.txt create mode 100644 bugreport.c create mode 100755 t/t0091-bugreport.sh
.gitignore +1 -0
@@ -25,6 +25,7 @@ /git-bisect--helper /git-blame /git-branch +/git-bugreport /git-bundle /git-cat-file /git-check-attr
Documentation/git-bugreport.txt new file +43 -0
43 lines not on this page · raw message
Makefile +5 -0
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS = # ... and all the rest that could be moved out of bindir to gitexecdir PROGRAMS += $(EXTRA_PROGRAMS) +PROGRAM_OBJS += bugreport.o PROGRAM_OBJS += credential-store.o PROGRAM_OBJS += daemon.o PROGRAM_OBJS += fast-import.o
@@ -2450,6 +2451,10 @@ endif git-%$X: %.o GIT-LDFLAGS $(GITLIBS) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS) +git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS) + $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \ + $(LIBS) + git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \ $(IMAP_SEND_LDFLAGS) $(LIBS)
bugreport.c new file +69 -0
69 lines not on this page · raw message
t/t0091-bugreport.sh new file +41 -0
41 lines not on this page · raw message
Signature · 1 line
2.25.0.341.g760bfbb309-goog
Quoted 6 lines · On Fri, 24 Jan 2020 at 05:56, <[email protected]> wrote:
From: Emily Shaffer <[email protected]>
Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.
("Later" meaning "later in this series", or "any day now"? ;-) )
Quoted 4 lines
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [-o | --output <path>]
Hmm. Should that be "[(-o | --output) <path>]"?
Documentation/git-bugreport.txt line 18
Quoted 6 lines
Documentation/git-bugreport.txt new file +6
-0+DESCRIPTION +----------- +Captures information about the user's machine, Git client, and repository state, +as well as a form requesting information about the behavior the user observed, +into a single text file which the user can then share, for example to the Git +mailing list, in order to report an observed bug.
Nice description. Got it.
Documentation/git-bugreport.txt line 32
Quoted 13 lines
Documentation/git-bugreport.txt new file +13
-0+The following information is requested from the user: + + - Reproduction steps + - Expected behavior + - Actual behavior + +The following information is captured automatically: + + - Git version (`git version --build-options`) + - Machine information (`uname -a`) + - Versions of various dependencies + - Git config contents (`git config --show-origin --list`) + - The names of all configured git-hooks in `.git/hooks/`
I would have expected these points to appear later, both to make it clear what this does commit does (and not), and to highlight what user-visible (documentation-worthy) changes later commits bring along.
Quoted 4 lines
+OPTIONS
+------- +-o [<path>]:: +--output [<path>]::
Drop the "[" and "]"? If you give -o, you'd better give a path as well?
Quoted 1 line
+ Place the resulting bug report file in <path> instead of the root of the
<path>
Quoted 2 lines
+"Please review the rest of the bug report below.\n" +"You can delete any lines you don't wish to send.\n");
"send" sounds like we're *just* about to send this report somewhere, but it's "only" going to be written to the disk. Maybe "share", instead?
bugreport.c line 54
Quoted 4 lines
bugreport.c new file +4
-0+if (option_output) { +strbuf_addstr (&report_path, option_output); +strbuf_complete (&report_path,'/' ); + }
I thought I'd use -o to indicate the filename, but it turns out it's the *directory* where the file (of some semi-random, generated name) will end up. Re-reading the docs further up, I can see how this is consistent. I sort of wonder if this should be --output*-directory* for symmetry with git format-patch.
bugreport.c line 60
Quoted 5 lines
bugreport.c new file +5
-0+strbuf_addstr (&report_path,"git-bugreport-" ); +strbuf_addftime (&report_path,"%F" ,gmtime (&now),0 ,0 ); +strbuf_addstr (&report_path,".txt" ); + +
(Double blank line?)
bugreport.c line 63
Quoted 3 lines
bugreport.c new file +3
-0+get_bug_template (&buffer); + + report =fopen_for_writing (report_path.buf);
Report might be NULL here.
If there's already such a file, we overwrite. Should we generate the filename using not just today's date (two bug reports in a day wouldn't be unheard of?) but also something like hh:mm:ss?
bugreport.c line 65
Quoted 2 lines
bugreport.c new file +2
-0+strbuf_write (&buffer, report); +fclose (report);
Maybe clear the strbuf around here...
bugreport.c line 68
Quoted 2 lines
bugreport.c new file +2
-0+launch_editor (report_path.buf,NULL ,NULL ); +return 0 ;
... and/or UNLEAK it here, together with report_path.
Maybe "return -launch_editor(...)"?
Quoted 1 line
+#!/bin/bash
Use /bin/sh instead?
t/t0091-bugreport.sh line 13
Quoted 7 lines
t/t0091-bugreport.sh new file +7
-0+# Headers "[System Info]" will be followed by a non-empty line if we put some +# information there; we can make sure all our headers were followed by some +# information to check if the command was successful. +HEADER_PATTERN="^\[.*\]$" +check_all_headers_populated() { + while read -r line; do + if [$(grep $HEADER_PATTERN $line)]; then
I think this is a bash-ism.
t/t0091-bugreport.sh line 15
Quoted 2 lines
t/t0091-bugreport.sh new file +2
-0+ read -r nextline + if [-z $nextline]; then
Likewise.
Quoted 4 lines
t/t0091-bugreport.sh new file +4
-0+ return 1; + fi + fi + done
t/t0091-bugreport.sh line 29
Quoted 10 lines
t/t0091-bugreport.sh new file +10
-0+} + +test_expect_success 'creates a report with content in the right places' ' + git bugreport && + check_all_headers_populated <git-bugreport-* && + rm git-bugreport-* +' + +test_expect_success '--output puts the report in the provided dir' ' + mkdir foo/ &&
If foo isn't there, do we not create it? Apparently not -- in my testing, we segfault. (We don't check for NULL after opening the file.)
t/t0091-bugreport.sh line 31
Quoted 2 lines
t/t0091-bugreport.sh new file +2
-0+ git bugreport -o foo/ && + test -f foo/git-bugreport-* &&
test_path_is_file
t/t0091-bugreport.sh line 38
Quoted 7 lines
t/t0091-bugreport.sh new file +7
-0+ rm -fr foo/ +' + +test_expect_success 'incorrect arguments abort with usage' ' + test_must_fail git bugreport --false 2>output && + grep usage output && + test ! -f git-bugreport-*
test_path_is_missing
Martin
Quoted 8 lines · Martin Ågren
Quoted 6 lines · On Fri, 24 Jan 2020 at 05:56, <[email protected]> wrote:
From: Emily Shaffer <[email protected]>
Teach Git how to prompt the user for a good bug report: reproduction steps, expected behavior, and actual behavior. Later, Git can learn how to collect some diagnostic information from the repository.
Quoted 8 lines
("Later" meaning "later in this series", or "any day now"? ;-) )
Quoted 4 lines
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [-o | --output <path>]Hmm. Should that be "[(-o | --output) <path>]"?
Done.
Quoted 18 lines
Documentation/git-bugreport.txt line 18
Quoted 6 lines
Documentation/git-bugreport.txt new file +6
-0+DESCRIPTION +----------- +Captures information about the user's machine, Git client, and repository state, +as well as a form requesting information about the behavior the user observed, +into a single text file which the user can then share, for example to the Git +mailing list, in order to report an observed bug.Nice description. Got it.
Quoted 9 lines
Documentation/git-bugreport.txt new file +9
-0+The following information is requested from the user: + + - Reproduction steps + - Expected behavior + - Actual behavior + +The following information is captured automatically: + + - Git version (`git version --build-options`)
Quoted 8 lines
Quoted 4 lines
+ - Machine information (
uname -a)
+ - Versions of various dependencies
+ - Git config contents (git config --show-origin --list)
+ - The names of all configured git-hooks in.git/hooks/I would have expected these points to appear later, both to make it clear what this does commit does (and not), and to highlight what user-visible (documentation-worthy) changes later commits bring along.
Sure, agreed.
Quoted 7 lines
Quoted 5 lines
+OPTIONS
+------- +-o [<path>]:: +--output [<path>]::Drop the "[" and "]"? If you give -o, you'd better give a path as well?
Done.
Quoted 3 lines
Quoted 1 line
+ Place the resulting bug report file in <path> instead of the root of the
<path>
Done.
Quoted 5 lines
Quoted 2 lines
+"Please review the rest of the bug report below.\n" +"You can delete any lines you don't wish to send.\n");
"send" sounds like we're *just* about to send this report somewhere, but it's "only" going to be written to the disk. Maybe "share", instead?
Nice turn of phrase, done.
Quoted 10 lines
bugreport.c line 54
Quoted 4 lines
bugreport.c new file +4
-0+if (option_output) { +strbuf_addstr (&report_path, option_output); +strbuf_complete (&report_path,'/' ); + }I thought I'd use
-oto indicate the filename, but it turns out it's the *directory* where the file (of some semi-random, generated name) will end up. Re-reading the docs further up, I can see how this is consistent. I sort of wonder if this should be--output*-directory*for symmetry withgit format-patch.
Sure.
Quoted 7 lines
bugreport.c line 60
Quoted 5 lines
bugreport.c new file +5
-0+strbuf_addstr (&report_path,"git-bugreport-" ); +strbuf_addftime (&report_path,"%F" ,gmtime (&now),0 ,0 ); +strbuf_addstr (&report_path,".txt" ); + +(Double blank line?)
Done.
Quoted 5 lines
bugreport.c line 63
Quoted 3 lines
bugreport.c new file +3
-0+get_bug_template (&buffer); + + report =fopen_for_writing (report_path.buf);Report might be NULL here.
Nice.
Quoted 3 lines
If there's already such a file, we overwrite. Should we generate the filename using not just today's date (two bug reports in a day wouldn't be unheard of?) but also something like hh:mm:ss?
Sure. For the sake of brevity I'll probably neglect seconds; I hope someone is spending more than 1 minute filling in the provided form.
I'm a little worried about including : in a filename, so I went for 'git-bugreport-YYYY-MM-DD-HHMM' (24-hour).
As I started to write a test to ensure duplicate filenames were handled well, Jonathan Tan pointed out that it would be easy to add an arg like --suffix to allow specifying a custom strftime string. That would allow users to easily create a file named git-bugreport-fetch-failing.txt or git-bugreport-March-19.txt or whatever they want; it also makes testing easy. So I'll add this for the next rollup.
Quoted 4 lines
bugreport.c line 65
Quoted 3 lines
bugreport.c new file +2
-0+strbuf_write (&buffer, report); +fclose (report);
Quoted 8 lines
Maybe clear the strbuf around here...
bugreport.c line 68
Quoted 2 lines
bugreport.c new file +2
-0+launch_editor (report_path.buf,NULL ,NULL ); +return 0 ;... and/or UNLEAK it here, together with
report_path.Maybe "return -launch_editor(...)"?
Hm, sure. I see that builtin/tag.c does mark strbufs this way, so I don't see a problem using UNLEAK and tail-calling launch_editor().
As a final bonus, I also added a line to report to stderr the name of the file that was created. I noticed it's sort of unclear what the command actually did otherwise.
Quoted 3 lines
Quoted 1 line
+#!/bin/bash
Use
/bin/shinstead?
Yeah, doing so immediately pointed out the bashisms you mentioned, plus some more. :facepalm:
Quoted 25 lines
t/t0091-bugreport.sh line 13
Quoted 8 lines
t/t0091-bugreport.sh new file +7
-0+# Headers "[System Info]" will be followed by a non-empty line if we put some +# information there; we can make sure all our headers were followed by some +# information to check if the command was successful. +HEADER_PATTERN="^\[.*\]$" +check_all_headers_populated() { + while read -r line; do + if [$(grep $HEADER_PATTERN $line)]; thenI think this is a bash-ism.
t/t0091-bugreport.sh line 15
Quoted 2 lines
t/t0091-bugreport.sh new file +2
-0+ read -r nextline + if [-z $nextline]; thenLikewise.
Quoted 9 lines
t/t0091-bugreport.sh new file +9
-0+ return 1; + fi + fi + done +} + +test_expect_success 'creates a report with content in the right places' ' + git bugreport && + check_all_headers_populated <git-bugreport-* &&
Quoted 8 lines
t/t0091-bugreport.sh line 29
Quoted 5 lines
t/t0091-bugreport.sh new file +5
-0+ rm git-bugreport-* +' + +test_expect_success '--output puts the report in the provided dir' ' + mkdir foo/ &&If foo isn't there, do we not create it? Apparently not -- in my testing, we segfault. (We don't check for NULL after opening the file.)
Yeah, at the moment I just added a die() if we can't open the provided path. I think other utilties can create the path (e.g. git-format-patch) but where it makes sense to do so, I'd prefer to keep bugreport very simple.
I'll die instead of overwriting, too; you're right that spending quite a while on a bug report and then accidentally writing over it with a blank one would be a very bad user experience.
Quoted 5 lines
t/t0091-bugreport.sh line 31
Quoted 3 lines
t/t0091-bugreport.sh new file +2
-0+ git bugreport -o foo/ && + test -f foo/git-bugreport-* &&
test_path_is_file
Sure.
Quoted 9 lines
t/t0091-bugreport.sh line 38
Quoted 7 lines
t/t0091-bugreport.sh new file +7
-0+ rm -fr foo/ +' + +test_expect_success 'incorrect arguments abort with usage' ' + test_must_fail git bugreport --false 2>output && + grep usage output && + test ! -f git-bugreport-*
test_path_is_missing
OK.
Thanks very much, Martin, for the thorough review. This is incredibly helpful.
- Emily
PATCH v5 2/15 help: move list_config_help to builtin/help
In reply to [email protected]From: Emily Shaffer <[email protected]>
Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o, builtin/clean.o, and builtin/config.o. This meant that help.o was unusable outside of the context of the main Git executable.
To make help.o usable by other commands again, move list_config_help() into builtin/help.c (where it makes sense to assume other builtin libraries are present).
When command-list.h is included but a member is not used, we start to hear a compiler warning. Since the config list is generated in a fairly different way than the command list, and since commands and config options are semantically different, move the config list into its own header and move the generator into its own script and build rule.
--- .gitignore | 1 + Makefile | 13 +++++-- builtin/help.c | 86 ++++++++++++++++++++++++++++++++++++++++++ generate-cmdlist.sh | 19 ---------- generate-configlist.sh | 24 ++++++++++++ help.c | 85 ----------------------------------------- help.h | 1 - 7 files changed, 121 insertions(+), 108 deletions(-) create mode 100755 generate-configlist.sh
.gitignore +1 -0
@@ -190,6 +190,7 @@ /gitweb/gitweb.cgi /gitweb/static/gitweb.js /gitweb/static/gitweb.min.* +/config-list.h /command-list.h *.tar.gz *.dsc
Makefile +10 -3
@@ -815,6 +815,7 @@ LIB_FILE = libgit.a XDIFF_LIB = xdiff/lib.a VCSSVN_LIB = vcs-svn/lib.a +GENERATED_H += config-list.h GENERATED_H += command-list.h LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2129,7 +2130,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS) help.sp help.s help.o: command-list.h -builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX +builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \ '-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \ '-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2149,6 +2150,12 @@ $(BUILT_INS): git$X ln -s $< $@ 2>/dev/null || \ cp $< $@ +config-list.h: generate-configlist.sh + +config-list.h: + $(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \ + >$@+ && mv $@+ $@ + command-list.h: generate-cmdlist.sh command-list.txt command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2786,7 +2793,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE .PHONY: sparse $(SP_OBJ) sparse: $(SP_OBJ) -EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/% +EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/% ifndef GCRYPT_SHA256 EXCEPT_HDRS += sha256/gcrypt.h endif
@@ -2808,7 +2815,7 @@ hdr-check: $(HCO) style: git clang-format --style file --diff --extensions c,h -check: command-list.h +check: config-list.h command-list.h @if sparse; \ then \ echo >&2 "Use 'make sparse' instead"; \
builtin/help.c +86 -0
98 lines not on this page · raw message
generate-cmdlist.sh +0 -19
@@ -76,23 +76,6 @@ print_command_list () { echo "};" } -print_config_list () { - cat <<EOF -static const char *config_name_list[] = { -EOF - grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt | - sed '/deprecated/d; s/::$//; s/, */\n/g' | - sort | - while read line - do - echo " \"$line\"," - done - cat <<EOF - NULL, -}; -EOF -} - exclude_programs= while test "--exclude-program" = "$1" do
@@ -113,5 +96,3 @@ echo define_category_names "$1" echo print_command_list "$1" -echo -print_config_list
generate-configlist.sh new file +24 -0
@@ -0,0 +1,24 @@ +#!/bin/sh + +echo "/* Automatically generated by generate-configlist.sh */" +echo + +print_config_list () { + cat <<EOF +static const char *config_name_list[] = { +EOF + grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt | + sed '/deprecated/d; s/::$//; s/, */\n/g' | + sort | + while read line + do + echo " \"$line\"," + done + cat <<EOF + NULL, +}; +EOF +} + +echo +print_config_list
help.c +0 -85
91 lines not on this page · raw message
help.h +0 -1
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)void list_common_cmds_help (void );void list_all_cmds_help (void );void list_common_guides_help (void ); -void list_config_help (int for_human);void list_all_main_cmds (struct string_list *list);void list_all_other_cmds (struct string_list *list);
Signature · 1 line
2.25.0.341.g760bfbb309-goog
Re: [PATCH v5 02/15] help: move list_config_help to builtin/help
Quoted 3 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:
To make help.o usable by other commands again, move
list_config_help()intobuiltin/help.c(where it makes sense to assume other builtin libraries are present).
Nit: I think this could be patch 01/15. It shuffles things around -- *then*, starting with patch 02/15, we can add exciting stuff.
Martin
Re: [PATCH v5 02/15] help: move list_config_help to builtin/help
Quoted 7 lines · Martin Ågren
Quoted 3 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:
To make help.o usable by other commands again, move
list_config_help()intobuiltin/help.c(where it makes sense to assume other builtin libraries are present).Nit: I think this could be patch 01/15. It shuffles things around -- *then*, starting with patch 02/15, we can add exciting stuff.
Sounds reasonable, here and 04/15. Will do.
PATCH v5 3/15 bugreport: gather git version and build info
In reply to [email protected]From: Emily Shaffer <[email protected]>
Knowing which version of Git a user has and how it was built allows us to more precisely pin down the circumstances when a certain issue occurs, so teach bugreport how to tell us the same output as 'git version --build-options'.
It's not ideal to directly call 'git version --build-options' because that output goes to stdout. Instead, wrap the version string in a helper within help.[ch] library, and call that helper from within the bugreport library.
--- bugreport.c | 22 +++++++++++++++++++++- help.c | 47 +++++++++++++++++++++++++++++------------------ help.h | 1 + 3 files changed, 51 insertions(+), 19 deletions(-)
bugreport.c +21 -1
@@ -1,8 +1,20 @@ -#include "builtin.h" +#include "cache.h" #include "parse-options.h" #include "stdio.h" #include "strbuf.h" #include "time.h" +#include "help.h" + +static void get_system_info (struct strbuf *sys_info) +{ +struct strbuf version_info =STRBUF_INIT ; + +/* get git version from native cmd */ +strbuf_addstr (sys_info,"git version:\n" ); +get_version_info (&version_info,1 ); +strbuf_addbuf (sys_info, &version_info); +strbuf_complete_line (sys_info); +}static const char *const bugreport_usage[] = {N_ ("git bugreport [-o|--output <file>]" ),
@@ -32,6 +44,11 @@ static int get_bug_template(struct strbuf *template)return 0 ; } +static void get_header (struct strbuf *buf,const char *title) +{ +strbuf_addf (buf,"\n\n[%s]\n" , title); +} +int cmd_main (int argc,const char **argv) {struct strbuf buffer =STRBUF_INIT ;
@@ -60,6 +77,9 @@ int cmd_main(int argc, const char **argv)get_bug_template (&buffer); +get_header (&buffer,"System Info" ); +get_system_info (&buffer); + report =fopen_for_writing (report_path.buf);strbuf_write (&buffer, report);fclose (report);
help.c +29 -18
62 lines not on this page · raw message
help.h +1 -0
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);void exclude_cmds (struct cmdnames *cmds,struct cmdnames *excludes);int is_in_cmdlist (struct cmdnames *cmds,const char *name);void list_commands (unsigned int colopts,struct cmdnames *main_cmds,struct cmdnames *other_cmds); +void get_version_info (struct strbuf *buf,int show_build_options);/* * call this to die(), when it is suspected that the user mistyped a
Signature · 1 line
2.25.0.341.g760bfbb309-goog
Re: [PATCH v5 03/15] bugreport: gather git version and build info
bugreport.c line 16
Quoted 9 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:
bugreport.c +9
-0+static void get_system_info (struct strbuf *sys_info) +{ +struct strbuf version_info =STRBUF_INIT ; + +/* get git version from native cmd */ +strbuf_addstr (sys_info,"git version:\n" ); +get_version_info (&version_info,1 ); +strbuf_addbuf (sys_info, &version_info); +strbuf_complete_line (sys_info);
This leaks version_info.
Quoted 1 line
+}
Martin
Re: [PATCH v5 03/15] bugreport: gather git version and build info
Quoted 12 lines · Martin Ågren
bugreport.c line 16
Quoted 9 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:
bugreport.c +9
-0+static void get_system_info (struct strbuf *sys_info) +{ +struct strbuf version_info =STRBUF_INIT ; + +/* get git version from native cmd */ +strbuf_addstr (sys_info,"git version:\n" ); +get_version_info (&version_info,1 ); +strbuf_addbuf (sys_info, &version_info); +strbuf_complete_line (sys_info);This leaks
version_info.
So it does.
I think the easiest thing is to drop the 'strbuf_reset()' at the top of help.c:get_version_info(), and just pass in sys_info from bugreport.c.
- Emily
PATCH v5 4/15 help: add shell-path to --build-options
In reply to [email protected]From: Emily Shaffer <[email protected]>
It may be useful to know which shell Git was built to try to point to, in the event that shell-based Git commands are failing. $SHELL_PATH is set during the build and used to launch the manpage viewer, as well as by git-compat-util.h, and it's used during tests. 'git version --build-options' is encouraged for use in bug reports, so it makes sense to include this information there.
--- help.c | 1 + 1 file changed, 1 insertion(+)
help.c +1 -0
@@ -642,6 +642,7 @@ void get_version_info(struct strbuf *buf, int show_build_options)strbuf_addstr (buf,"no commit associated with this build\n" );strbuf_addf (buf,"sizeof-long: %d\n" , (int )sizeof (long ));strbuf_addf (buf,"sizeof-size_t: %d\n" , (int )sizeof (size_t )); +strbuf_addf (buf,"shell-path: %s\n" ,SHELL_PATH );/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */ } }
Signature · 1 line
2.25.0.341.g760bfbb309-goog
Re: [PATCH v5 04/15] help: add shell-path to --build-options
Quoted 6 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:
It may be useful to know which shell Git was built to try to point to, in the event that shell-based Git commands are failing. $SHELL_PATH is set during the build and used to launch the manpage viewer, as well as by
git-compat-util.h, and it's used during tests. 'git version --build-options' is encouraged for use in bug reports, so it makes sense to include this information there.
Hmm, similar nit as for patch 02/15: This could go in as, e.g., 02/15 and your 01/15 would be 03/15: This patch would be useful even without git bugreport.
Martin
PATCH v5 5/15 bugreport: add uname info
In reply to [email protected]From: Emily Shaffer <[email protected]>
The contents of uname() can give us some insight into what sort of system the user is running on, and help us replicate their setup if need be. The domainname field is not guaranteed to be available, so don't collect it.
--- bugreport.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
bugreport.c +12 -0
@@ -8,12 +8,24 @@static void get_system_info (struct strbuf *sys_info) {struct strbuf version_info =STRBUF_INIT ; +struct utsname uname_info;/* get git version from native cmd */ strbuf_addstr (sys_info,"git version:\n" );get_version_info (&version_info,1 );strbuf_addbuf (sys_info, &version_info);strbuf_complete_line (sys_info); + +/* system call for other version info */ +strbuf_addstr (sys_info,"uname -a: " ); +if (uname (&uname_info)) +strbuf_addf (sys_info,"uname() failed with code %d\n" , errno); +else +strbuf_addf (sys_info,"%s %s %s %s\n" , + uname_info.sysname, + uname_info.release, + uname_info.version, + uname_info.machine); }static const char *const bugreport_usage[] = {
Signature · 1 line
2.25.0.341.g760bfbb309-goog
PATCH v5 6/15 bugreport: add compiler info
In reply to [email protected]From: Emily Shaffer <[email protected]>
To help pinpoint the source of a regression, it is useful to know some info about the compiler which the user's Git client was built with. By adding a generic get_compiler_info() in 'compat/' we can choose which relevant information to share per compiler; to get started, let's demonstrate the version of glibc if the user built with 'gcc'.
--- bugreport.c | 5 +++++ compat/compiler.h | 24 ++++++++++++++++++++++++ 2 files changed, 29 insertions(+) create mode 100644 compat/compiler.h
bugreport.c +5 -0
@@ -4,6 +4,7 @@#include "strbuf.h" #include "time.h" #include "help.h" +#include "compat/compiler.h" static void get_system_info (struct strbuf *sys_info) {
@@ -26,6 +27,10 @@ static void get_system_info(struct strbuf *sys_info) uname_info.release, uname_info.version, uname_info.machine); + +strbuf_addstr (sys_info,"compiler info: " ); +get_compiler_info (sys_info); +strbuf_complete_line (sys_info); }static const char *const bugreport_usage[] = {
compat/compiler.h new file +24 -0
@@ -0,0 +1,24 @@ +#ifndef COMPILER_H +#define COMPILER_H + +#include "git-compat-util.h" +#include "strbuf.h" + +#ifdef __GLIBC__ +#include <gnu/libc-version.h> + +static inline void get_compiler_info (struct strbuf *info) +{ +strbuf_addf (info,"glibc: %s" ,gnu_get_libc_version ()); +} + +#else + +static inline void get_compiler_info (struct strbuf *info) +{ +strbuf_addstr (info,"get_compiler_info() not implemented" ); +} + +#endif + +#endif /* COMPILER_H */
Signature · 1 line
2.25.0.341.g760bfbb309-goog
Re: [PATCH v5 06/15] bugreport: add compiler info
compat/compiler.h line 19
Quoted 5 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:
compat/compiler.h new file +5
-0+#else + +static inline void get_compiler_info (struct strbuf *info) +{ +strbuf_addstr (info,"get_compiler_info() not implemented" );
Maybe "no compiler info available" (or s/ available//, or s/available/reported/), or something else more human-readable?
compat/compiler.h line 24
Quoted 5 lines
compat/compiler.h new file +5
-0+} + +#endif + +#endif /* COMPILER_H */
Martin
Re: [PATCH v5 06/15] bugreport: add compiler info
Quoted 9 lines · Martin Ågren
compat/compiler.h line 19
Quoted 5 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:
compat/compiler.h new file +5
-0+#else + +static inline void get_compiler_info (struct strbuf *info) +{ +strbuf_addstr (info,"get_compiler_info() not implemented" );Maybe "no compiler info available" (or s/ available//, or s/available/reported/), or something else more human-readable?
Hm. I envisioned the target audience for this as "Git developers" (and that's why I didn't mark the unimplemented string for translation); Git developers know exactly what "get_compiler_info() not implemented" means. But I suppose it's just as easy to grep for one string as the other.
I am hesitant to say "no info available" or "no info reported" when I'm certain there is some environment variable set at build time which contains the info we'd want to show here; it really is just a question of some Git developer having put in the time to implement this function.
So, I think I will leave it. Thanks for your suggestions.
- Emily
Re: [PATCH v5 06/15] bugreport: add compiler info
Quoted 18 lines · Emily Shaffer
Quoted 9 lines · Martin Ågren
compat/compiler.h line 19
Quoted 5 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:
compat/compiler.h new file +5
-0+#else + +static inline void get_compiler_info (struct strbuf *info) +{ +strbuf_addstr (info,"get_compiler_info() not implemented" );Maybe "no compiler info available" (or s/ available//, or s/available/reported/), or something else more human-readable?
Hm. I envisioned the target audience for this as "Git developers" (and that's why I didn't mark the unimplemented string for translation); Git developers know exactly what "
get_compiler_info()not implemented" means. But I suppose it's just as easy to grep for one string as the other.
Quoted 6 lines
I am hesitant to say "no info available" or "no info reported" when I'm certain there is some environment variable set at build time which contains the info we'd want to show here; it really is just a question of some Git developer having put in the time to implement this function.
So, I think I will leave it. Thanks for your suggestions.
Ok, that makes sense.
Martin
PATCH v5 7/15 bugreport: add curl version
In reply to [email protected]From: Emily Shaffer <[email protected]>
It's possible for git-remote-curl to be built separately from git; in that case we want to know what version of cURL is used by git-remote-curl, not necessarily which version was present at git-bugreport's build time. So instead, ask git-remote-curl for the version information it knows about.
For longevity purposes, invoke the alias "git-remote-https" instead of "git-remote-http".
Since it could have been built at a different time, also report the version and built-from commit of git-remote-curl alongside the cURL info.
--- bugreport.c | 16 ++++++++++++++++ remote-curl.c | 8 ++++++++ 2 files changed, 24 insertions(+)
bugreport.c +16 -0
@@ -5,6 +5,18 @@#include "time.h" #include "help.h" #include "compat/compiler.h" +#include "run-command.h" + +static void get_curl_version_info (struct strbuf *curl_info) +{ +struct child_process cp =CHILD_PROCESS_INIT ; + +argv_array_push (&cp.args,"git" ); +argv_array_push (&cp.args,"remote-https" ); +argv_array_push (&cp.args,"--build-info" ); +if (capture_command (&cp, curl_info,0 )) +strbuf_addstr (curl_info,"'git-remote-https --build-info' not supported\n" ); +}static void get_system_info (struct strbuf *sys_info) {
@@ -31,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)strbuf_addstr (sys_info,"compiler info: " );get_compiler_info (sys_info);strbuf_complete_line (sys_info); + +strbuf_addstr (sys_info,"git-remote-https --build-info:\n" ); +get_curl_version_info (sys_info); +strbuf_complete_line (sys_info); }static const char *const bugreport_usage[] = {
remote-curl.c +8 -0
@@ -17,6 +17,7 @@#include "protocol.h" #include "quote.h" #include "transport.h" +#include "version.h" static struct remote *remote;/* always ends with a trailing slash */
@@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)string_list_init (&options.deepen_not,1 );string_list_init (&options.push_options,1 ); +if (!strcmp ("--build-info" , argv[1 ])) { +printf ("git-http-fetch version: %s\n" , git_version_string); +printf ("built from commit: %s\n" , git_built_from_commit_string); +printf ("curl version: %s\n" ,curl_version ()); +return 0 ; + } +/* * Just report "remote-curl" here (folding all the various aliases * ("git-remote-http", "git-remote-https", and etc.) here since they
Signature · 1 line
2.25.0.341.g760bfbb309-goog
Re: [PATCH v5 07/15] bugreport: add curl version
Quoted 14 lines · On Fri, 24 Jan 2020 at 04:41, <[email protected]> wrote:
bugreport.c +10
-0+static void get_curl_version_info (struct strbuf *curl_info) +{ +struct child_process cp =CHILD_PROCESS_INIT ; + +argv_array_push (&cp.args,"git" ); +argv_array_push (&cp.args,"remote-https" ); +argv_array_push (&cp.args,"--build-info" ); +if (capture_command (&cp, curl_info,0 )) +strbuf_addstr (curl_info,"'git-remote-https --build-info' not supported\n" ); +}static void get_system_info (struct strbuf *sys_info) {
bugreport.c static void get_system_info(struct strbuf *sys_info) · line 48
Quoted 6 lines
bugreport.c +3
-0@@ -31,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)strbuf_addstr (sys_info,"compiler info: " );get_compiler_info (sys_info);strbuf_complete_line (sys_info); + +strbuf_addstr (sys_info,"git-remote-https --build-info:\n" ); +get_curl_version_info (sys_info);
The header here looks a lot like an implementation detail of get_curl_version_info(). Or put differently, these risk getting out of sync. Maybe frame the header a bit more human readable: "curl version". But is this "curl version", or more like "git-remote-https version"? There's some discrepancy here.
bugreport.c static void get_system_info(struct strbuf *sys_info) · line 50
Quoted 2 lines
bugreport.c +1
-0+strbuf_complete_line (sys_info); }
Martin
Re: [PATCH v5 07/15] bugreport: add curl version
Quoted 12 lines · Martin Ågren
Quoted 11 lines · On Fri, 24 Jan 2020 at 04:41, <[email protected]> wrote:
bugreport.c +10
-0+static void get_curl_version_info (struct strbuf *curl_info) +{ +struct child_process cp =CHILD_PROCESS_INIT ; + +argv_array_push (&cp.args,"git" ); +argv_array_push (&cp.args,"remote-https" ); +argv_array_push (&cp.args,"--build-info" ); +if (capture_command (&cp, curl_info,0 )) +strbuf_addstr (curl_info,"'git-remote-https --build-info' not supported\n" ); +}
Quoted 15 lines
bugreport.c static void get_system_info(struct strbuf *sys_info) · line 48
Quoted 9 lines
static void
get_system_info(struct strbuf *sys_info){bugreport.c +3
-0@@ -31,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)strbuf_addstr (sys_info,"compiler info: " );get_compiler_info (sys_info);strbuf_complete_line (sys_info); + +strbuf_addstr (sys_info,"git-remote-https --build-info:\n" ); +get_curl_version_info (sys_info);The header here looks a lot like an implementation detail of
get_curl_version_info(). Or put differently, these risk getting out of sync. Maybe frame the header a bit more human readable: "curl version". But is this "curl version", or more like "git-remote-https version"? There's some discrepancy here.
Hm, I think you're saying "If we switch to future-https-lib instead of cURL for git-remote-https, then this command will be incorrectly named." Sure, I agree. It's true that with this change git-remote-https also tells us some info about itself.
Thanks.
- Emily
PATCH v5 8/15 bugreport: include user interactive shell
In reply to [email protected]From: Emily Shaffer <[email protected]>
It's possible a user may complain about the way that Git interacts with their interactive shell, e.g. autocompletion or shell prompt. In that case, it's useful for us to know which shell they're using interactively.
--- bugreport.c | 5 +++++ 1 file changed, 5 insertions(+)
bugreport.c +5 -0
@@ -22,6 +22,7 @@ static void get_system_info(struct strbuf *sys_info) {struct strbuf version_info =STRBUF_INIT ;struct utsname uname_info; +char *shell =NULL ;/* get git version from native cmd */ strbuf_addstr (sys_info,"git version:\n" );
@@ -44,6 +45,10 @@ static void get_system_info(struct strbuf *sys_info)get_compiler_info (sys_info);strbuf_complete_line (sys_info); + shell =getenv ("SHELL" ); +strbuf_addf (sys_info,"$SHELL (typically, interactive shell): %s\n" , + shell ? shell :"(NULL)" ); +strbuf_addstr (sys_info,"git-remote-https --build-info:\n" );get_curl_version_info (sys_info);strbuf_complete_line (sys_info);
Signature · 1 line
2.25.0.341.g760bfbb309-goog
Re: [PATCH v5 08/15] bugreport: include user interactive shell
Quoted 1 line · On Fri, 24 Jan 2020 at 04:41, <[email protected]> wrote:
+ char *shell = NULL;
(Unnecessary initialization.)
bugreport.c static void get_system_info(struct strbuf *sys_info) · line 50
Quoted 3 lines
bugreport.c +3
-0+ shell =getenv ("SHELL" ); +strbuf_addf (sys_info,"$SHELL (typically, interactive shell): %s\n" , + shell ? shell :"(NULL)" );
Thanks for avoiding a classic pitfall. :-)
"<unused>" instead of "(NULL)"? "NULL" is mostly an implementation detail.
Martin
Re: [PATCH v5 08/15] bugreport: include user interactive shell
Quoted 10 lines · Martin Ågren
Quoted 1 line · On Fri, 24 Jan 2020 at 04:41, <[email protected]> wrote:
+ char *shell = NULL;
(Unnecessary initialization.)
bugreport.c static void get_system_info(struct strbuf *sys_info) · line 50
Quoted 3 lines
bugreport.c +3
-0+ shell =getenv ("SHELL" ); +strbuf_addf (sys_info,"$SHELL (typically, interactive shell): %s\n" , + shell ? shell :"(NULL)" );Thanks for avoiding a classic pitfall. :-)
Thank Junio. I fell right into it because it Just Works in gcc. ;)
Quoted 2 lines
"<unused>" instead of "(NULL)"? "NULL" is mostly an implementation detail.
Sure, why not.
Re: [PATCH v5 08/15] bugreport: include user interactive shell
In reply to Martin ÅgrenQuoted 5 lines · Martin Ågren
Quoted 1 line · On Fri, 24 Jan 2020 at 04:41, <[email protected]> wrote:
+ char *shell = NULL;
(Unnecessary initialization.)
Quoted 8 lines
bugreport.c static void get_system_info(struct strbuf *sys_info) · line 50
Quoted 3 lines
bugreport.c +3
-0+ shell =getenv ("SHELL" ); +strbuf_addf (sys_info,"$SHELL (typically, interactive shell): %s\n" , + shell ? shell :"(NULL)" );Thanks for avoiding a classic pitfall. :-)
"<unused>" instead of "(NULL)"? "NULL" is mostly an implementation detail.
Isn't that <unset>?
Re: [PATCH v5 08/15] bugreport: include user interactive shell
Quoted 12 lines · Junio C Hamano
Quoted 9 lines · Martin Ågren
Quoted 1 line · On Fri, 24 Jan 2020 at 04:41, <[email protected]> wrote:
+ char *shell = NULL;
(Unnecessary initialization.)
Quoted 3 lines
bugreport.c +3
-0+ shell =getenv ("SHELL" ); +strbuf_addf (sys_info,"$SHELL (typically, interactive shell): %s\n" , + shell ? shell :"(NULL)" );
Quoted 6 lines
Quoted 4 lines
Thanks for avoiding a classic pitfall. :-)
"<unused>" instead of "(NULL)"? "NULL" is mostly an implementation detail.
Isn't that <unset>?
Heh, yes, "SHELL: unused" sounds wrong. :-)
Martin
PATCH v5 9/15 bugreport: generate config safelist based on docs
In reply to [email protected]From: Emily Shaffer <[email protected]>
Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in Documentation/config with the "bugreport" macro, which is recognized by AsciiDoc and AsciiDoctor.
Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.
In order to prevent staleness of the safelist, add a mechanism to generate the safelist from annotations in the config documentation, where contributors are already used to documenting their new config keys.
Implement a new no-op "bugreport" macro for use as "bugreport:include[x]" to annotate the config keys that should be included in the automatically generated safelist. Use "exclude" for the others.
With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc seems to want something between the brackets. A bit unfortunate, but not a huge problem -- we'll just provide an "x".
"doc-diff" reports that this macro doesn't render at all. That is, these are both empty after this commit:
cd Documentation ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD ./doc-diff --asciidoc :/"bugreport: add tool" HEAD
Diffing the rendered HTML shows that there is some small amount of whitespace and comments added. That shouldn't be a problem.
We could perhaps let the implementation verify that the "action" is one of "include" and "exclude". For the Asciidoctor implementation that should be straightforward, but for AsciiDoc I don't immediately know how to do it. Anyway, if someone stumbles on the keyboard and writes "bugreport:icndule", they'll "only" miss out on the config key being included in the safelist. If this were a blocklist, the consequences of a misspelled target could be a lot more severe.
Additionally, add annotations to the sendemail config documentation in order to demonstrate a proof of concept.
--- .gitignore | 1 + Documentation/asciidoc.conf | 8 +++ Documentation/asciidoctor-extensions.rb | 7 +++ Documentation/config/sendemail.txt | 68 ++++++++++++------------- Makefile | 7 +++ generate-bugreport-config-safelist.sh | 17 +++++++ 6 files changed, 74 insertions(+), 34 deletions(-) create mode 100755 generate-bugreport-config-safelist.sh
.gitignore +1 -0
@@ -192,6 +192,7 @@ /gitweb/static/gitweb.min.* /config-list.h /command-list.h +/bugreport-config-safelist.h *.tar.gz *.dsc *.deb
Documentation/asciidoc.conf +8 -0
@@ -6,9 +6,13 @@ # # Show Git link as: <command>(<section>); if section is defined, else just show # the command. +# +# The bugreport macro does nothing as far as rendering is +# concerned -- we just grep for it in the sources. [macros] (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]= +(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]= [attributes] asterisk=*
@@ -28,6 +32,8 @@ ifdef::backend-docbook[] {0#<citerefentry>} {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>} {0#</citerefentry>} +[bugreport-inlinemacro] +{0#} endif::backend-docbook[] ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[] git-relative-html-prefix= [linkgit-inlinemacro] <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a> +[bugreport-inlinemacro] +<!-- --> endif::backend-xhtml11[]
Documentation/asciidoctor-extensions.rb +7 -0
@@ -37,6 +37,10 @@ module Git output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>") end output + + class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor + def process(parent, action, attrs) + "" end end end
@@ -45,4 +49,7 @@ end Asciidoctor::Extensions.register do inline_macro Git::Documentation::LinkGitProcessor, :linkgit postprocessor Git::Documentation::DocumentPostProcessor + # The bugreport macro does nothing as far as rendering is + # concerned -- we just grep for it in the sources. + inline_macro Git::Documentation::BugReportProcessor, :bugreport end
Documentation/config/sendemail.txt +34 -34
97 lines not on this page · raw message
Makefile +7 -0
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a GENERATED_H += config-list.h GENERATED_H += command-list.h +GENERATED_H += bugreport-config-safelist.h LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \ $(FIND) . \
@@ -2163,6 +2164,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc $(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \ command-list.txt >$@+ && mv $@+ $@ +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh + +bugreport-config-safelist.h: Documentation/config/*.txt + $(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \ + >$@+ && mv $@+ $@ + SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\ $(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\ $(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
generate-bugreport-config-safelist.sh new file +17 -0
@@ -0,0 +1,17 @@ +#!/bin/sh + +cat <<EOF +/* Automatically generated by bugreport-generate-config-safelist.sh */ + + +static const char *bugreport_config_safelist[] = { +EOF + +# cat all regular files in Documentation/config +find Documentation/config -type f -exec cat {} \; | +# print the command name which matches the bugreport-include macro +sed -n 's/^\(.*\) \+bugreport:include.* ::$/ "\1",/p' | sort + +cat <<EOF +}; +EOF
Signature · 1 line
2.25.0.341.g760bfbb309-goog
Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
Quoted 7 lines · On Fri, 24 Jan 2020 at 04:35, <[email protected]> wrote:
Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in
Documentation/configwith the "bugreport" macro, which is recognized by AsciiDoc and AsciiDoctor.
Quoted 7 lines
Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.
I sort of wonder whether safelist/blocklist is to prefer over whitelist/blacklist, or if it's the other way round. The former are new to me, whereas the latter are the terms I would have used. But that's just me, of course. I was a little surprised, that's all.
Quoted 8 lines
Implement a new no-op "bugreport" macro for use as "bugreport:include[x]" to annotate the config keys that should be included in the automatically generated safelist. Use "exclude" for the others.
With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc seems to want something between the brackets. A bit unfortunate, but not a huge problem -- we'll just provide an "x".
I recognize this reasoning :-) and I'm not terribly opposed to it, but after some nights' sleeping on this, I have to wonder if "annotate:bugreport[include]" wouldn't be better than "bugreport[x]" with that ugly "x". Maybe this isn't the biggest problem, but if we expect this macro to eventually sit right next to ~90% of all our config variables...
Quoted 6 lines
"doc-diff" reports that this macro doesn't render at all. That is, these are both empty after this commit:
cd Documentation ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD ./doc-diff --asciidoc :/"bugreport: add tool" HEAD
That was true in [1], but alas, no more. In that patch, it's sort of obvious from the diff how it adds a "class" which "end"s.
[1] https://lore.kernel.org/git/[email protected]/
Documentation/asciidoctor-extensions.rb module Git · line 46
Quoted 13 lines
Documentation/asciidoctor-extensions.rb +4
-0@@ -37,6 +37,10 @@ module Git output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>") end output + + class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor + def process(parent, action, attrs) + "" end end end
But this one doesn't add an "end", and Asciidoctor trips up badly.
Documentation/asciidoctor-extensions.rb end · line 54
Quoted 3 lines
Documentation/asciidoctor-extensions.rb +3
-0+ # The bugreport macro does nothing as far as rendering is + # concerned -- we just grep for it in the sources. + inline_macro Git::Documentation::BugReportProcessor, :bugreport
(I never much liked this copy-paste comment then, and I still don't, to be honest.)
Martin
Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
Quoted 16 lines · Martin Ågren
Quoted 7 lines · On Fri, 24 Jan 2020 at 04:35, <[email protected]> wrote:
Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in
Documentation/configwith the "bugreport" macro, which is recognized by AsciiDoc and AsciiDoctor.Quoted 7 lines
Some configs are private in nature, and can contain remote URLs, passwords, or other sensitive information. In the event that a user doesn't notice their information while reviewing a bugreport, that user may leak their credentials to other individuals, mailing lists, or bug tracking tools inadvertently. Heuristic blocklisting of configuration keys is imperfect and prone to false negatives; given the nature of the information which can be leaked, a safelist is more reliable.
Quoted 4 lines
I sort of wonder whether safelist/blocklist is to prefer over whitelist/blacklist, or if it's the other way round. The former are new to me, whereas the latter are the terms I would have used. But that's just me, of course. I was a little surprised, that's all.
Eh. I think the following things are true:
- Whitelist/blacklist has a "smell" of discrimination, whether that's the true etymology or not.
- Those with experience in the field can easily understand what whitelist or blacklist means.
- Safelist/blocklist do not "smell" the same way.
- It is easy to tell what "safelist" means: "a list of stuff which is safe." No experience needed.
So, while it's new, I think it's not harmful. I see only a no-op or positive impact from using this term instead of whitelist/blacklist. Computer science seems to have quite a few terms which fall into this long-standing but potentially negative area, so I don't mind looking for alternatives where it's harmless to do so.
Quoted 10 lines
Quoted 9 lines
Implement a new no-op "bugreport" macro for use as "bugreport:include[x]" to annotate the config keys that should be included in the automatically generated safelist. Use "exclude" for the others.
With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc seems to want something between the brackets. A bit unfortunate, but not a huge problem -- we'll just provide an "x".
Quoted 6 lines
I recognize this reasoning :-) and I'm not terribly opposed to it, but after some nights' sleeping on this, I have to wonder if "annotate:bugreport[include]" wouldn't be better than "bugreport[x]" with that ugly "x". Maybe this isn't the biggest problem, but if we expect this macro to eventually sit right next to ~90% of all our config variables...
Hm. I wanted to say, "Ok, but I don't know how to do that, so can you help?" But I think that's all the more reason that I should do it ;)
Ok. I will try and change it to annotate:bugreport[include] like you suggested, and hopefully learn more about asciidoc macros :)
Quoted 13 lines
Quoted 7 lines
"doc-diff" reports that this macro doesn't render at all. That is, these are both empty after this commit:
cd Documentation ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD ./doc-diff --asciidoc :/"bugreport: add tool" HEADThat was true in [1], but alas, no more. In that patch, it's sort of obvious from the diff how it adds a "class" which "end"s.
Quoted 15 lines
Documentation/asciidoctor-extensions.rb module Git · line 46
Quoted 13 lines
Documentation/asciidoctor-extensions.rb +4
-0@@ -37,6 +37,10 @@ module Git output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>") end output + + class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor + def process(parent, action, attrs) + "" end end endBut this one doesn't add an "end", and Asciidoctor trips up badly.
Ok, I'll have a look. I'm sure I copied something badly.
Quoted 7 lines
Documentation/asciidoctor-extensions.rb end · line 54
Quoted 4 lines
Documentation/asciidoctor-extensions.rb +3
-0+ # The bugreport macro does nothing as far as rendering is + # concerned -- we just grep for it in the sources. + inline_macro Git::Documentation::BugReportProcessor, :bugreport(I never much liked this copy-paste comment then, and I still don't, to be honest.)
I'll see if I can find a reasonable alternative (or remove it).
- Emily
Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
Quoted 16 lines · Emily Shaffer
Quoted 5 lines
I sort of wonder whether safelist/blocklist is to prefer over whitelist/blacklist, or if it's the other way round. The former are new to me, whereas the latter are the terms I would have used. But that's just me, of course. I was a little surprised, that's all.
Eh. I think the following things are true:
- Whitelist/blacklist has a "smell" of discrimination, whether that's the true etymology or not.
- Those with experience in the field can easily understand what whitelist or blacklist means.
- Safelist/blocklist do not "smell" the same way.
- It is easy to tell what "safelist" means: "a list of stuff which is safe." No experience needed.
Quoted 5 lines
So, while it's new, I think it's not harmful. I see only a no-op or positive impact from using this term instead of whitelist/blacklist. Computer science seems to have quite a few terms which fall into this long-standing but potentially negative area, so I don't mind looking for alternatives where it's harmless to do so.
Ok, that all makes sense.
Thanks
Martin
Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
In reply to [email protected]Quoted 6 lines · On Fri, 24 Jan 2020 at 04:35, <[email protected]> wrote:
Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in
Documentation/configwith the "bugreport" macro, which is recognized by AsciiDoc and AsciiDoctor.
"which is recognized by" sounds like it's built-in. Maybe "with a new no-op 'bugreport' macro" or something like that.
Documentation/config/sendemail.txt line 59
Quoted 10 lines
Documentation/config/sendemail.txt +2
-2-sendemail.signedoffcc (deprecated):: +sendemail.signedoffcc (deprecated) bugreport:exclude[x] :: Deprecated alias for `sendemail.signedoffbycc`. -sendemail.smtpBatchSize:: +sendemail.smtpBatchSize bugreport:include[x] :: Number of messages to be sent per connection, after that a relogin will happen. If the value is 0 or undefined, send all messages in one connection. See also the `--batch-size` option of linkgit:git-send-email[1].
Do we really need to list includes *and* excludes? I could see the point of adding an exclude here and there to signal that "this might look innocent enough, but trust me, we really need to exclude this" in order to avoid future commits to more or less accidentally over-include. Should we add some internal documentation and/or a remark in the commit message about this? As an example, is "sendemail.signedofcc" sensitive enough that we need to explicitly exclude it? If someone wants to come along and include it, they don't just need to argue for an inclusion, but also for lifting the exclusion. Hmm?
Martin
Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
Quoted 10 lines · Martin Ågren
Quoted 6 lines · On Fri, 24 Jan 2020 at 04:35, <[email protected]> wrote:
Add a new step to the build to generate a safelist of git-config variables which are appropriate to include in the output of git-bugreport. New variables can be added to the safelist by annotating their documentation in
Documentation/configwith the "bugreport" macro, which is recognized by AsciiDoc and AsciiDoctor."which is recognized by" sounds like it's built-in. Maybe "with a new no-op 'bugreport' macro" or something like that.
Sure, I'll wordsmith this a little.
Quoted 5 lines
Quoted 5 lines
Documentation/config/sendemail.txt +1
-1-sendemail.signedoffcc (deprecated):: +sendemail.signedoffcc (deprecated) bugreport:exclude[x] :: Deprecated alias for `sendemail.signedoffbycc`.
Quoted 8 lines
Documentation/config/sendemail.txt line 59
Quoted 6 lines
Documentation/config/sendemail.txt +1
-1-sendemail.smtpBatchSize:: +sendemail.smtpBatchSize bugreport:include[x] :: Number of messages to be sent per connection, after that a relogin will happen. If the value is 0 or undefined, send all messages in one connection. See also the `--batch-size` option of linkgit:git-send-email[1].Do we really need to list includes *and* excludes?
Need? Nah. The exclude markers are not even acknowledged by the current implementation.
Quoted 4 lines
I could see the point of adding an exclude here and there to signal that "this might look innocent enough, but trust me, we really need to exclude this" in order to avoid future commits to more or less accidentally over-include.
This was more of my line of thinking, along with the thought that some folks may use bug reporting internally, where they have A) more control over which extensions/tools are in use, and B) less concern about sharing things like remote URLs, branch names, etc. and may want to go for a blocklist as opposed to a safelist, to make it easier for their support folks to diagnose.
Quoted 5 lines
Should we add some internal documentation and/or a remark in the commit message about this? As an example, is "sendemail.signedofcc" sensitive enough that we need to explicitly exclude it? If someone wants to come along and include it, they don't just need to argue for an inclusion, but also for lifting the exclusion. Hmm?
Yeah, I think this is a good point. I'll try to figure out how to reword the commit message, and take another look at the sample include/exclude change I made to see where I can omit entirely.
- Emily
Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
Quoted 3 lines · Emily Shaffer
Yeah, I think this is a good point. I'll try to figure out how to reword the commit message, and take another look at the sample
include/excludechange I made to see where I can omit entirely.
Eh. I'm going to try and reword the whole commit message. I combined your scissors patch, plus something from Dscho's scissors patch, plus my original commit message, and now it's huge and hard to follow.
- Emily
PATCH v5 10/15 bugreport: add config values from safelist
In reply to [email protected]From: Emily Shaffer <[email protected]>
Teach bugreport to gather the values of config options which are present in 'bugreport-config-safelist.h'.
Many config options are sensitive, and many Git add-ons use config options which git-core does not know about; it is better only to gather config options which we know to be safe, rather than excluding options which we know to be unsafe.
Taking the build-time generated array and putting it into a set saves us time - since git_config_bugreport() is called for every option the user has configured, performing option lookup in constant time is a useful optimization.
--- Makefile | 2 +- bugreport.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 76 insertions(+), 1 deletion(-)
Makefile +1 -1
@@ -2465,7 +2465,7 @@ endif git-%$X: %.o GIT-LDFLAGS $(GITLIBS) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS) -git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS) +git-bugreport$X: bugreport-config-safelist.h bugreport.o GIT-LDFLAGS $(GITLIBS) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \ $(LIBS)
bugreport.c +75 -0
111 lines not on this page · raw message
Signature · 1 line
2.25.0.341.g760bfbb309-goog
Re: [PATCH v5 10/15] bugreport: add config values from safelist
Makefile endif · line 2470
Quoted 10 lines · On Fri, 24 Jan 2020 at 04:40, <[email protected]> wrote:
Makefile +1
-1@@ -2465,7 +2465,7 @@ endif git-%$X: %.o GIT-LDFLAGS $(GITLIBS) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS) -git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS) +git-bugreport$X: bugreport-config-safelist.h bugreport.o GIT-LDFLAGS $(GITLIBS) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \ $(LIBS)
Haven't looked at this patch at all, except I've noticed that something is up with the dependencies. I think bugreport.o needs to depend on bugreport-config-safelist.h. As it is, the latter may or may not be available as bugreport.o is built. (Reproduces fairly well for me with something like make clean && make -j16.)
I'll try to continue tomorrow. :-)
Martin