git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
* [PATCH v4 00/15] add git-bugreport tool
@ 2019-12-13  0:42 Emily Shaffer
  2019-12-13  0:42 ` [PATCH v4 01/15] bugreport: add tool to generate debugging info Emily Shaffer
                   ` (15 more replies)
  0 siblings, 16 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:42 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Derrick Stolee, Johannes Schindelin,
	Junio C Hamano, Martin Ågren

This topic branch depends on the patch mailed in
lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to
display scopes for configs gathered during "bugreport: add config values from
safelist".

Quite a few changes since v3. I'll try to summarize the big picture
stuff, plus highlight new changes and difference in patches inline in
the commit list. I'll also stick an interdiff since v3 at the bottom of
this cover letter.

Some overall changes:
- The entire command now exists in one file, bugreport.c, which is
  compiled as a standalone.
- static-ness of local functions has been cleaned up.
- strbuf lifetime management has been cleaned up.
- Mention of "whitelist" and "blacklist" has been changed to "safelist"
  and "blocklist" while dealing with Git-approved bugreportable configs.
  Thanks Dscho for pointing this out; "safelist" makes sense to me as
  these are configs we have deemed safe for public inspection.
- Inquiry into both Git's $SHELL_PATH and the user's $SHELL is new to
  v4.
- The 'man git bugreport' page which was lost from the script version of
  git-bugreport has been recovered, as have the "tests" (which only test
  whether it runs and doesn't crash).


Emily Shaffer (15):
  bugreport: add tool to generate debugging info
Moved from library + builtin to a standalone git-bugreport binary.

  help: move list_config_help to builtin/help
Resolves a circular dependency where using help.o required all builtins
to exist. This also moves generation of config-list.h into its own
script (it was previously roommates with command-list.h).

The following 4 commits were previously a single commit which used
run-command.h to call different processes; this has been removed except
for the cURL version.

  bugreport: gather git version and build info
Additionally capture the "--build-info" output from 'git version' where it
was not previously captured.

  help: add shell-path to --build-options
Adding $SHELL_PATH to 'git --version --build-options' gets it in the
bugreport for free. (This is the shell Git uses, determined at build
time.)

  bugreport: add uname info
Capture uname from the library instead of invoking a different process.

  bugreport: add glibc version
Capture glibc version info from the library instead of invoking the
'ldd' process.

  bugreport: add curl version
Capture the cURL version from git-http-fetch, and add a -V flag to that
binary for this purpose.


  bugreport: include user interactive shell
A new feature; check what $SHELL the user is using (at runtime)

  bugreport: generate config safelist based on docs
This commit squashes in suggestions both from Johannes Schindelin and
Martin Ågren, hopefully resulting in Asciidoc(tor) not showing the
safelist macros, and a more portable safelist generation script. It also
puts the safelist, sorted, directly into a header instead of into a text
file, saving us a file read at runtime.

  bugreport: add config values from safelist
At Dscho's suggestion, now we put these into a map (based on khash and
oidset.h's example) for constant-time lookup during the config parse.

  bugreport: collect list of populated hooks
Only a minor change - now this works if 'git bugreport' is invoked
outside of a Git repo.

  bugreport: count loose objects
  bugreport: add packed object summary
  bugreport: list contents of $OBJDIR/info
For the above 3, the entire 'string_list' has been refactored out; now
the obj_info buffer is written to directly during the directory
traversal. Also paid more attention to when to free strbuf used for paths.

  bugreport: summarize contents of alternates file
Rather than printing the contents of the alternates file, now the number
of specified alternate stores are listed, and they are tested to check
whether they exist. This way we stop looking at user filepaths
(sensitive) and start learning whether they have a broken alternate (new
and useful info).

 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   8 +
 Documentation/asciidoctor-extensions.rb |   7 +
 Documentation/config/sendemail.txt      |  68 ++--
 Documentation/git-bugreport.txt         |  43 +++
 Documentation/git-http-fetch.txt        |   5 +
 Makefile                                |  31 +-
 bugreport.c                             | 434 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 generate-bugreport-config-safelist.sh   |  22 ++
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  24 ++
 help.c                                  | 131 ++-----
 help.h                                  |   2 +-
 http-fetch.c                            |  13 +-
 t/t0091-bugreport.sh                    |  41 +++
 16 files changed, 774 insertions(+), 163 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index 055a84c4a8..30935621d9 100644
--- a/.gitignore
+++ b/.gitignore
@@ -189,7 +189,9 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..5d5359fcf9 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,13 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The bugreport macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +32,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[bugreport-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[bugreport-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..750bdff9af 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, action, attrs)
+        ""
       end
     end
   end
@@ -45,4 +49,7 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  # The bugreport macro does nothing as far as rendering is
+  # concerned -- we just grep for it in the sources.
+  inline_macro Git::Documentation::BugReportProcessor, :bugreport
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 69f3e4f219..92f5082013 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -1,63 +1,63 @@
-sendemail.identity:: // bugreport-exclude
+sendemail.identity bugreport:exclude[x] ::
 	A configuration identity. When given, causes values in the
 	'sendemail.<identity>' subsection to take precedence over
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption:: // bugreport-include
+sendemail.smtpEncryption bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
-sendemail.smtpssl (deprecated):: // bugreport-exclude
+sendemail.smtpssl (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for 'sendemail.smtpEncryption = ssl'.
 
-sendemail.smtpsslcertpath:: // bugreport-exclude
+sendemail.smtpsslcertpath bugreport:exclude[x] ::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*:: // bugreport-exclude
+sendemail.<identity>.* bugreport:exclude[x] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
 	`sendemail.identity`.
 
-sendemail.aliasesFile:: // bugreport-exclude
-sendemail.aliasFileType:: // bugreport-exclude
-sendemail.annotate:: // bugreport-include
-sendemail.bcc:: // bugreport-include
-sendemail.cc:: // bugreport-include
-sendemail.ccCmd:: // bugreport-include
-sendemail.chainReplyTo:: // bugreport-include
-sendemail.confirm:: // bugreport-include
-sendemail.envelopeSender:: // bugreport-include
-sendemail.from:: // bugreport-include
-sendemail.multiEdit:: // bugreport-include
-sendemail.signedoffbycc:: // bugreport-include
-sendemail.smtpPass:: // bugreport-exclude
-sendemail.suppresscc:: // bugreport-include
-sendemail.suppressFrom:: // bugreport-include
-sendemail.to:: // bugreport-include
-sendemail.tocmd:: // bugreport-include
-sendemail.smtpDomain:: // bugreport-include
-sendemail.smtpServer:: // bugreport-include
-sendemail.smtpServerPort:: // bugreport-include
-sendemail.smtpServerOption:: // bugreport-include
-sendemail.smtpUser:: // bugreport-exclude
-sendemail.thread:: // bugreport-include
-sendemail.transferEncoding:: // bugreport-include
-sendemail.validate:: // bugreport-include
-sendemail.xmailer:: // bugreport-include
+sendemail.aliasesFile bugreport:exclude[x] ::
+sendemail.aliasFileType bugreport:exclude[x] ::
+sendemail.annotate bugreport:include[x] ::
+sendemail.bcc bugreport:include[x] ::
+sendemail.cc bugreport:include[x] ::
+sendemail.ccCmd bugreport:include[x] ::
+sendemail.chainReplyTo bugreport:include[x] ::
+sendemail.confirm bugreport:include[x] ::
+sendemail.envelopeSender bugreport:include[x] ::
+sendemail.from bugreport:include[x] ::
+sendemail.multiEdit bugreport:include[x] ::
+sendemail.signedoffbycc bugreport:include[x] ::
+sendemail.smtpPass bugreport:exclude[x] ::
+sendemail.suppresscc bugreport:include[x] ::
+sendemail.suppressFrom bugreport:include[x] ::
+sendemail.to bugreport:include[x] ::
+sendemail.tocmd bugreport:include[x] ::
+sendemail.smtpDomain bugreport:include[x] ::
+sendemail.smtpServer bugreport:include[x] ::
+sendemail.smtpServerPort bugreport:include[x] ::
+sendemail.smtpServerOption bugreport:include[x] ::
+sendemail.smtpUser bugreport:exclude[x] ::
+sendemail.thread bugreport:include[x] ::
+sendemail.transferEncoding bugreport:include[x] ::
+sendemail.validate bugreport:include[x] ::
+sendemail.xmailer bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.
 
-sendemail.signedoffcc (deprecated):: // bugreport-exclude
+sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize:: // bugreport-include
+sendemail.smtpBatchSize bugreport:include[x] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay:: // bugreport-include
+sendemail.smtpReloginDelay bugreport:include[x] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..75f0c80acf
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,43 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [-o | --output <path>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+The following information is captured automatically:
+
+ - Git version (`git version --build-options`)
+ - Machine information (`uname -a`)
+ - Versions of various dependencies
+ - Git config contents (`git config --show-origin --list`)
+ - The names of all configured git-hooks in `.git/hooks/`
+
+OPTIONS
+-------
+-o [<path>]::
+--output [<path>]::
+	Place the resulting bug report file in <path> instead of the root of the
+	Git repository.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Documentation/git-http-fetch.txt b/Documentation/git-http-fetch.txt
index 666b042679..2894c5e82b 100644
--- a/Documentation/git-http-fetch.txt
+++ b/Documentation/git-http-fetch.txt
@@ -10,6 +10,7 @@ SYNOPSIS
 --------
 [verse]
 'git http-fetch' [-c] [-t] [-a] [-d] [-v] [-w filename] [--recover] [--stdin] <commit> <url>
+'git http-fetch' [-V]
 
 DESCRIPTION
 -----------
@@ -30,6 +31,10 @@ commit-id::
 -v::
 	Report what is downloaded.
 
+-V::
+	Report information about the version of git-http-fetch, including the
+	versions of its dependencies.
+
 -w <filename>::
         Writes the commit-id into the filename under $GIT_DIR/refs/<filename> on
         the local end after the transfer is complete.
diff --git a/Makefile b/Makefile
index 6e361eed64..76dc51e2b1 100644
--- a/Makefile
+++ b/Makefile
@@ -634,10 +634,6 @@ SCRIPT_PYTHON += git-p4.py
 SCRIPT_SH_GEN = $(patsubst %.sh,%,$(SCRIPT_SH))
 SCRIPT_PERL_GEN = $(patsubst %.perl,%,$(SCRIPT_PERL))
 SCRIPT_PYTHON_GEN = $(patsubst %.py,%,$(SCRIPT_PYTHON))
-SCRIPT_DEPENDENCIES = git-bugreport-config-whitelist
-
-$(SCRIPT_DEPENDENCIES): Documentation/config/*.txt
-	sh bugreport-generate-config-whitelist.sh
 
 # Individual rules to allow e.g.
 # "make -C ../.. SCRIPT_PERL=contrib/foo/bar.perl build-perl-script"
@@ -662,13 +658,10 @@ clean-perl-script:
 	$(RM) $(SCRIPT_PERL_GEN)
 clean-python-script:
 	$(RM) $(SCRIPT_PYTHON_GEN)
-clean-script-dependencies:
-	$(RM) $(SCRIPT_DEPENDENCIES)
 
 SCRIPTS = $(SCRIPT_SH_GEN) \
 	  $(SCRIPT_PERL_GEN) \
 	  $(SCRIPT_PYTHON_GEN) \
-	  $(SCRIPT_DEPENDENCIES) \
 	  git-instaweb
 
 ETAGS_TARGET = TAGS
@@ -688,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -821,7 +815,9 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -846,7 +842,6 @@ LIB_OBJS += bisect.o
 LIB_OBJS += blame.o
 LIB_OBJS += blob.o
 LIB_OBJS += branch.o
-LIB_OBJS += bugreport.o
 LIB_OBJS += bulk-checkin.o
 LIB_OBJS += bundle.o
 LIB_OBJS += cache-tree.o
@@ -1049,7 +1044,6 @@ BUILTIN_OBJS += builtin/archive.o
 BUILTIN_OBJS += builtin/bisect--helper.o
 BUILTIN_OBJS += builtin/blame.o
 BUILTIN_OBJS += builtin/branch.o
-BUILTIN_OBJS += builtin/bugreport.o
 BUILTIN_OBJS += builtin/bundle.o
 BUILTIN_OBJS += builtin/cat-file.o
 BUILTIN_OBJS += builtin/check-attr.o
@@ -2135,7 +2129,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2155,6 +2149,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2162,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
@@ -2457,6 +2463,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
@@ -2788,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-GEN_HDRS := command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -2811,7 +2821,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -3114,7 +3124,9 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo
+	$(RM) config-list.h command-list.h bugreport-config-safelist.h
+	$(RM) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
diff --git a/bugreport-generate-config-whitelist.sh b/bugreport-generate-config-whitelist.sh
deleted file mode 100755
index ca6b232024..0000000000
--- a/bugreport-generate-config-whitelist.sh
+++ /dev/null
@@ -1,4 +0,0 @@
-#!/bin/sh
-
-grep -RhPo ".*(?=:: \/\/ bugreport-include)" Documentation/config \
-  >git-bugreport-config-whitelist
diff --git a/bugreport.c b/bugreport.c
index a7bdc72b7f..d2a3fb1f67 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,114 +1,128 @@
 #include "cache.h"
-
-#include "bugreport.h"
-#include "config.h"
-#include "exec-cmd.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
 #include "help.h"
+#include <gnu/libc-version.h>
 #include "run-command.h"
-#include "strbuf.h"
-#include "string-list.h"
-#include "version.h"
-
-#include "dirent.h"
-
-/**
- * A sorted list of config options which we will add to the bugreport. Managed
- * by 'gather_whitelist(...)'.
- */
-struct string_list whitelist = STRING_LIST_INIT_DUP;
-struct strbuf configs_and_values = STRBUF_INIT;
-
-// git version --build-options
-// uname -a
-// curl-config --version
-// ldd --version
-// echo $SHELL
-void get_system_info(struct strbuf *sys_info)
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
+#include "run-command.h"
+
+static void get_http_version_info(struct strbuf *http_info)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;
-	struct strbuf std_out = STRBUF_INIT;
 
-	strbuf_reset(sys_info);
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "http-fetch");
+	argv_array_push(&cp.args, "-V");
+	if (capture_command(&cp, http_info, 0))
+	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
+}
 
-	// get git version from native cmd
-	strbuf_addstr(sys_info, "git version: ");
-	strbuf_addstr(sys_info, git_version_string);
-	strbuf_complete_line(sys_info);
+KHASH_INIT(cfg_set, const char*, int, 0, kh_str_hash_func, kh_str_hash_equal);
 
-	// system call for other version info
-	argv_array_push(&cp.args, "uname");
-	argv_array_push(&cp.args, "-a");
-	capture_command(&cp, &std_out, 0);
+struct cfgset {
+	kh_cfg_set_t set;
+};
 
-	strbuf_addstr(sys_info, "uname -a: ");
-	strbuf_addbuf(sys_info, &std_out);
-	strbuf_complete_line(sys_info);
-
-	argv_array_clear(&cp.args);
-	strbuf_reset(&std_out);
+struct cfgset safelist;
 
+static void cfgset_init(struct cfgset *set, size_t initial_size)
+{
+	memset(&set->set, 0, sizeof(set->set));
+	if (initial_size)
+		kh_resize_cfg_set(&set->set, initial_size);
+}
 
-	argv_array_push(&cp.args, "curl-config");
-	argv_array_push(&cp.args, "--version");
-	capture_command(&cp, &std_out, 0);
+static int cfgset_insert(struct cfgset *set, const char *cfg_key)
+{
+	int added;
+	kh_put_cfg_set(&set->set, cfg_key, &added);
+	printf("ESS: added %s\n", cfg_key);
+	return !added;
+}
 
-	strbuf_addstr(sys_info, "curl-config --version: ");
-	strbuf_addbuf(sys_info, &std_out);
-	strbuf_complete_line(sys_info);
+static int cfgset_contains(struct cfgset *set, const char *cfg_key)
+{
+	khiter_t pos = kh_get_cfg_set(&set->set, cfg_key);
+	return pos != kh_end(&set->set);
+}
 
-	argv_array_clear(&cp.args);
-	strbuf_reset(&std_out);
+static void cfgset_clear(struct cfgset *set)
+{
+	kh_release_cfg_set(&set->set);
+	cfgset_init(set, 0);
+}
 
+static void get_system_info(struct strbuf *sys_info)
+{
+	struct strbuf version_info = STRBUF_INIT;
+	struct utsname uname_info;
 
-	argv_array_push(&cp.args, "ldd");
-	argv_array_push(&cp.args, "--version");
-	capture_command(&cp, &std_out, 0);
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	list_version_info(&version_info, 1);
+	strbuf_addbuf(sys_info, &version_info);
+	strbuf_complete_line(sys_info);
 
-	strbuf_addstr(sys_info, "ldd --version: ");
-	strbuf_addbuf(sys_info, &std_out);
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.nodename,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "glibc version: ");
+	strbuf_addstr(sys_info, gnu_get_libc_version());
 	strbuf_complete_line(sys_info);
 
-	argv_array_clear(&cp.args);
-	strbuf_reset(&std_out);
-}
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    getenv("SHELL"));
 
-void gather_whitelist(struct strbuf *path)
-{
-	struct strbuf tmp = STRBUF_INIT;
-	strbuf_read_file(&tmp, path->buf, 0);
-	string_list_init(&whitelist, 1);
-	string_list_split(&whitelist, tmp.buf, '\n', -1);
-	string_list_sort(&whitelist);
+	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
+	get_http_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
-int git_config_bugreport(const char *var, const char *value, void *cb)
+static void gather_safelist()
 {
-	if (string_list_has_string(&whitelist, var)) {
-		strbuf_addf(&configs_and_values,
-			    "%s : %s\n",
-			    var, value);
-	}
+	int index;
+	int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
+	cfgset_init(&safelist, safelist_len);
+	for (index = 0; index < safelist_len; index++)
+		cfgset_insert(&safelist, bugreport_config_safelist[index]);
 
-	return 0;
 }
 
-void get_whitelisted_config(struct strbuf *config_info)
+static int git_config_bugreport(const char *var, const char *value, void *cb)
 {
-	struct strbuf path = STRBUF_INIT;
+	struct strbuf *config_info = (struct strbuf *)cb;
 
-	strbuf_addstr(&path, git_exec_path());
-	strbuf_addstr(&path, "/git-bugreport-config-whitelist");
+	if (cfgset_contains(&safelist, var))
+		strbuf_addf(config_info,
+			    "%s (%s) : %s\n",
+			    var, config_scope_to_string(current_config_scope()),
+			    value);
 
-	gather_whitelist(&path);
-	strbuf_init(&configs_and_values, whitelist.nr);
-
-	git_config(git_config_bugreport, NULL);
+	return 0;
+}
 
-	strbuf_reset(config_info);
-	strbuf_addbuf(config_info, &configs_and_values);
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	gather_safelist();
+	git_config(git_config_bugreport, config_info);
+	cfgset_clear(&safelist);
 }
 
-void get_populated_hooks(struct strbuf *hook_info)
+static void get_populated_hooks(struct strbuf *hook_info)
 {
 	/*
 	 * Doesn't look like there is a list of all possible hooks; so below is
@@ -139,8 +153,15 @@ void get_populated_hooks(struct strbuf *hook_info)
 			    "post-index-changex";
 	struct string_list hooks_list = STRING_LIST_INIT_DUP;
 	struct string_list_item *iter = NULL;
+	int nongit_ok;
+
+	setup_git_directory_gently(&nongit_ok);
 
-	strbuf_reset(hook_info);
+	if (nongit_ok) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
 
 	string_list_split(&hooks_list, hooks, ',', -1);
 
@@ -152,163 +173,262 @@ void get_populated_hooks(struct strbuf *hook_info)
 	}
 }
 
-/**
- * Fill 'contents' with the contents of the dir at 'dirpath'.
- * If 'filter' is nonzero, the contents are filtered on d_type as 'type' - see
- * 'man readdir'. opendir() doesn't take string length as an arg, so don't
- * bother passing it in.
- */
-void list_contents_of_dir(struct string_list *contents, struct strbuf *dirpath,
-			  int filter, unsigned char type)
+static int is_hex(const char *string, size_t count)
 {
-	struct dirent *dir = NULL;
-	DIR *dh = NULL;
-
-	dh = opendir(dirpath->buf);
-	while (dh && (dir = readdir(dh))) {
-		if (!filter || type == dir->d_type) {
-			string_list_append(contents, dir->d_name);
-		}
+	for (; count; string++, count--) {
+		if (!isxdigit(*string))
+			return 0;
 	}
+	return 1;
 }
 
-/**
- * Fills 'contents' with a list of all directories within the provided
- * directory, recursing into each directory.
- */
-void list_contents_of_dir_recursively(struct string_list *contents,
-				      struct strbuf *dirpath)
-{
-	struct string_list current_contents = STRING_LIST_INIT_DUP;
-	struct string_list current_subdirs = STRING_LIST_INIT_DUP;
-	struct string_list_item *it;
-	struct strbuf buf = STRBUF_INIT;
-
-	list_contents_of_dir(&current_contents, dirpath, 0, 0);
-	for_each_string_list_item(it, &current_contents) {
-		strbuf_reset(&buf);
-		strbuf_addbuf(&buf, dirpath);
-		strbuf_complete(&buf, '/');
-		strbuf_addstr(&buf, it->string);
-
-		string_list_append(contents, buf.buf);
-	}
+static void get_loose_object_summary(struct strbuf *obj_info) {
+	struct dirent *d = NULL;
+	DIR *dir, *subdir = NULL;
+	size_t dir_len;
+	struct strbuf dirpath = STRBUF_INIT;
 
-	list_contents_of_dir(&current_subdirs, dirpath, 1, DT_DIR);
-	for_each_string_list_item(it, &current_subdirs) {
-		if (strcmp(it->string, ".") == 0
-		    || strcmp(it->string, "..") == 0)
-			continue;
-		strbuf_reset(&buf);
-		strbuf_addbuf(&buf, dirpath);
-		strbuf_complete(&buf, '/');
-		strbuf_addstr(&buf, it->string);
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
 
-		list_contents_of_dir_recursively(contents, &buf);
+	dir = opendir(dirpath.buf);
+	if (!dir) {
+		strbuf_addf(obj_info, "could not open object directory '%s'\n",
+			    dirpath.buf);
+		strbuf_release(&dirpath);
+		return;
 	}
-}
 
-void get_object_counts(struct strbuf *obj_info)
-{
-	struct child_process cp = CHILD_PROCESS_INIT;
-	struct strbuf std_out = STRBUF_INIT;
-
-	argv_array_push(&cp.args, "count-objects");
-	argv_array_push(&cp.args, "-vH");
-	cp.git_cmd = 1;
-	capture_command(&cp, &std_out, 0);
+	dir_len = dirpath.len;
 
-	strbuf_reset(obj_info);
-	strbuf_addstr(obj_info, "git-count-objects -vH:\n");
-	strbuf_addbuf(obj_info, &std_out);
-}
+	while ((d = readdir(dir))) {
+		int object_count = 0;
+		char subdir_name[3];
 
-void get_loose_object_summary(struct strbuf *obj_info)
-{
-	struct strbuf dirpath = STRBUF_INIT;
-	struct string_list subdirs = STRING_LIST_INIT_DUP;
-	struct string_list_item *subdir;
-
-	strbuf_reset(obj_info);
+		if (d->d_type != DT_DIR)
+			continue;
 
-	strbuf_addstr(&dirpath, get_object_directory());
-	strbuf_complete(&dirpath, '/');
+		if ((strlen(d->d_name) != 2) || (!is_hex(d->d_name, 2)))
+			continue;
 
-	list_contents_of_dir(&subdirs, &dirpath, 1, DT_DIR);
+		/* copy directory name + \0 */
+		memcpy(subdir_name, d->d_name, 3);
 
-	for_each_string_list_item(subdir, &subdirs)
-	{
-		struct strbuf subdir_buf = STRBUF_INIT;
-		struct string_list objects = STRING_LIST_INIT_DUP;
+		strbuf_setlen(&dirpath, dir_len);
+		strbuf_addstr(&dirpath, d->d_name);
 
-		/*
-		 * Only interested in loose objects - so dirs named with the
-		 * first byte of the object ID
-		 */
-		if (strlen(subdir->string) != 2 || !strcmp(subdir->string, ".."))
+		subdir = opendir(dirpath.buf);
+		if (!subdir)
 			continue;
+		while ((d = readdir(subdir)))
+			if (d->d_type == DT_REG)
+				object_count++;
+
+		closedir(subdir);
 
-		strbuf_addbuf(&subdir_buf, &dirpath);
-		strbuf_addstr(&subdir_buf, subdir->string);
-		list_contents_of_dir(&objects, &subdir_buf, 0, 0);
-		strbuf_addf(obj_info, "%s: %d objects\n", subdir->string,
-			    objects.nr);
+		strbuf_addf(obj_info, "%s: %d\n", subdir_name, object_count);
 	}
+
+
+	closedir(dir);
+	strbuf_release(&dirpath);
 }
 
-void get_packed_object_summary(struct strbuf *obj_info)
+static void get_packed_object_summary(struct strbuf *obj_info)
 {
 	struct strbuf dirpath = STRBUF_INIT;
-	struct string_list contents = STRING_LIST_INIT_DUP;
-	struct string_list_item *entry;
-
-	strbuf_reset(obj_info);
+	struct dirent *d;
+	DIR *dir = NULL;
 
 	strbuf_addstr(&dirpath, get_object_directory());
 	strbuf_complete(&dirpath, '/');
 	strbuf_addstr(&dirpath, "pack/");
-	list_contents_of_dir(&contents, &dirpath, 0, 0);
 
-	// list full contents of $GIT_OBJECT_DIRECTORY/pack/
-	for_each_string_list_item(entry, &contents) {
+	dir = opendir(dirpath.buf);
+	if (!dir) {
+		strbuf_addf(obj_info, "could not open packed object directory '%s'\n",
+			    dirpath.buf);
+		strbuf_release(&dirpath);
+		return;
+	}
+
+	while ((d = readdir(dir))) {
 		strbuf_addbuf(obj_info, &dirpath);
-		strbuf_addstr(obj_info, entry->string);
+		strbuf_addstr(obj_info, d->d_name);
 		strbuf_complete_line(obj_info);
 	}
+
+	closedir(dir);
+	strbuf_release(&dirpath);
 }
 
-void get_object_info_summary(struct strbuf *obj_info)
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
 {
-	// strbuf += GITDIR/info/:
-	// recursively list contents of $GIT_OBJECT_DIRECTORY/info
-	struct strbuf dirpath = STRBUF_INIT;
-	struct string_list contents = STRING_LIST_INIT_DUP;
-	struct string_list_item *entry;
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
 
-	strbuf_reset(obj_info);
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
 
 	strbuf_addstr(&dirpath, get_object_directory());
 	strbuf_complete(&dirpath, '/');
 	strbuf_addstr(&dirpath, "info/");
 
-	list_contents_of_dir_recursively(&contents, &dirpath);
+	list_contents_of_dir_recursively(obj_info, &dirpath);
 
-	for_each_string_list_item(entry, &contents) {
-		strbuf_addstr(obj_info, entry->string);
-		strbuf_complete_line(obj_info);
-	}
+	strbuf_release(&dirpath);
 }
 
-void get_alternates_file(struct strbuf *alternates_info)
+static void get_alternates_summary(struct strbuf *alternates_info)
 {
 	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
 
 	strbuf_addstr(&alternates_path, get_object_directory());
 	strbuf_complete(&alternates_path, '/');
 	strbuf_addstr(&alternates_path, "info/alternates");
 
-	strbuf_reset(alternates_info);
-	strbuf_addbuf(alternates_info, &alternates_path);
-	strbuf_complete_line(alternates_info);
-	strbuf_read_file(alternates_info, alternates_path.buf, 0);
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output <file>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to send.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+
+	get_bug_template(&buffer);
+
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
+	get_header(&buffer, "Configured Hooks");
+	get_populated_hooks(&buffer);
+
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer);
+
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer);
+
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	launch_editor(report_path.buf, NULL, NULL);
+	return 0;
 }
diff --git a/bugreport.h b/bugreport.h
deleted file mode 100644
index 74d1f79960..0000000000
--- a/bugreport.h
+++ /dev/null
@@ -1,44 +0,0 @@
-#include "strbuf.h"
-
-/**
- * Adds the Git version, `uname -a`, and `curl-config --version` to sys_info.
- * The previous contents of sys_info will be discarded.
- */
-void get_system_info(struct strbuf *sys_info);
-
-/**
- * Adds the values of the config items listed in
- * 'git-bugreport-config-whitelist' to config_info. The previous contents of
- * config_info will be discarded.
- */
-void get_whitelisted_config(struct strbuf *sys_info);
-
-/**
- * Adds the paths to all configured hooks (but not their contents). The previous
- * contents of hook_info will be discarded.
- */
-void get_populated_hooks(struct strbuf *hook_info);
-
-/**
- * Adds the output of `git count-object -vH`. The previous contents of hook_info
- * will be discarded.
- */
-void get_loose_object_summary(struct strbuf *obj_info);
-
-/**
- * Adds a list of the contents of '.git/objects/pack'. The previous contents of
- * hook_info will be discarded.
- */
-void get_packed_object_summary(struct strbuf *obj_info);
-
-/**
- * Adds a list of all contents (recursively) of '.git/objects/info'. The
- * previous contents of hook_info will be discarded.
- */
-void get_object_info_summary(struct strbuf *obj_info);
-
-/**
- * Adds the contents of '.git/info/alternates'. The previous contents of
- * alternates_info will be discarded.
- */
-void get_alternates_file(struct strbuf *alt_info);
diff --git a/builtin.h b/builtin.h
index c6373d3289..5cf5df69f7 100644
--- a/builtin.h
+++ b/builtin.h
@@ -135,7 +135,6 @@ int cmd_archive(int argc, const char **argv, const char *prefix);
 int cmd_bisect__helper(int argc, const char **argv, const char *prefix);
 int cmd_blame(int argc, const char **argv, const char *prefix);
 int cmd_branch(int argc, const char **argv, const char *prefix);
-int cmd_bugreport(int argc, const char **argv, const char *prefix);
 int cmd_bundle(int argc, const char **argv, const char *prefix);
 int cmd_cat_file(int argc, const char **argv, const char *prefix);
 int cmd_checkout(int argc, const char **argv, const char *prefix);
diff --git a/builtin/bugreport.c b/builtin/bugreport.c
deleted file mode 100644
index 0784bdc42a..0000000000
--- a/builtin/bugreport.c
+++ /dev/null
@@ -1,90 +0,0 @@
-#include "builtin.h"
-#include "bugreport.h"
-#include "stdio.h"
-#include "strbuf.h"
-#include "time.h"
-
-int get_bug_template(struct strbuf *template)
-{
-	const char template_text[] =
-"Thank you for filling out a Git bug report!\n"
-"Please answer the following questions to help us understand your issue.\n"
-"\n"
-"What did you do before the bug happened? (Steps to reproduce your issue)\n"
-"\n"
-"What did you expect to happen? (Expected behavior)\n"
-"\n"
-"What happened instead? (Actual behavior)\n"
-"\n"
-"What's different between what you expected and what actually happened?\n"
-"\n"
-"Anything else you want to add:\n"
-"\n"
-"Please review the rest of the bug report below.\n"
-"You can delete any lines you don't wish to send.\n";
-
-	strbuf_reset(template);
-	strbuf_add(template, template_text, strlen(template_text));
-	return 0;
-}
-
-void add_header(FILE *report, const char *title)
-{
-	struct strbuf buffer = STRBUF_INIT;
-	strbuf_addf(&buffer, "\n\n[%s]\n", title);
-	strbuf_write(&buffer, report);
-}
-
-int cmd_bugreport(int argc, const char **argv, const char *prefix)
-{
-	struct strbuf buffer = STRBUF_INIT;
-	struct strbuf report_path = STRBUF_INIT;
-	FILE *report;
-	time_t now = time(NULL);
-
-	strbuf_addstr(&report_path, "git-bugreport-");
-	strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
-	strbuf_addstr(&report_path, ".txt");
-
-	report = fopen_for_writing(report_path.buf);
-
-	get_bug_template(&buffer);
-	strbuf_write(&buffer, report);
-
-	// add other contents
-	add_header(report, "System Info");
-	get_system_info(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Whitelisted Config");
-	get_whitelisted_config(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Populated Hooks");
-	get_populated_hooks(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Object Counts");
-	get_loose_object_summary(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Packed Object Summary");
-	get_packed_object_summary(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Object Info Data");
-	get_object_info_summary(&buffer);
-	strbuf_write(&buffer, report);
-
-	add_header(report, "Alternates File");
-	get_alternates_file(&buffer);
-	strbuf_write(&buffer, report);
-
-	// Close file
-	// open file in editor
-	launch_editor(report_path, NULL, NULL);
-	fclose(report);
-
-	launch_editor(report_path.buf, NULL, NULL);
-	return 0;
-}
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/config.c b/config.c
index e7052b3977..baab4a916e 100644
--- a/config.c
+++ b/config.c
@@ -3312,6 +3312,23 @@ enum config_scope current_config_scope(void)
 		return current_parsing_scope;
 }
 
+const char *config_scope_to_string(enum config_scope scope)
+{
+	switch (scope) {
+	case CONFIG_SCOPE_SYSTEM:
+		return "system";
+	case CONFIG_SCOPE_GLOBAL:
+		return "global";
+	case CONFIG_SCOPE_REPO:
+		return "repo";
+	case CONFIG_SCOPE_CMDLINE:
+		return "cmdline";
+	case CONFIG_SCOPE_UNKNOWN:
+	default:
+		return "unknown";
+	}
+}
+
 int lookup_config(const char **mapping, int nr_mapping, const char *var)
 {
 	int i;
diff --git a/config.h b/config.h
index 91fd4c5e96..c8bf296dcc 100644
--- a/config.h
+++ b/config.h
@@ -303,6 +303,7 @@ enum config_scope {
 };
 
 enum config_scope current_config_scope(void);
+const char *config_scope_to_string(enum config_scope);
 const char *current_config_origin_type(void);
 const char *current_config_name(void);
 
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..06b8e0c3c4
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,22 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the bugreport-include macro
+sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |
+sort |
+while read line
+do
+	echo "	\"$line\","
+done
+
+cat <<EOF
+};
+EOF
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..eca6a00c30
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/git.c b/git.c
index 2d6a64f019..ce6ab0ece2 100644
--- a/git.c
+++ b/git.c
@@ -473,7 +473,6 @@ static struct cmd_struct commands[] = {
 	{ "bisect--helper", cmd_bisect__helper, RUN_SETUP },
 	{ "blame", cmd_blame, RUN_SETUP },
 	{ "branch", cmd_branch, RUN_SETUP | DELAY_PAGER_CONFIG },
-	{ "bugreport", cmd_bugreport, RUN_SETUP },
 	{ "bundle", cmd_bundle, RUN_SETUP_GENTLY | NO_PARSEOPT },
 	{ "cat-file", cmd_cat_file, RUN_SETUP },
 	{ "check-attr", cmd_check_attr, RUN_SETUP },
diff --git a/help.c b/help.c
index cf67624a94..6d1bd17971 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
@@ -707,8 +622,34 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void list_version_info(struct strbuf *buf, int build_options)
+{
+	strbuf_reset(buf);
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addf(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -722,25 +663,9 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	list_version_info(&buf, build_options);
+	printf("%s", buf.buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 7a455beeb7..54f6b5f793 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
@@ -38,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void list_version_info(struct strbuf *buf, int build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
diff --git a/http-fetch.c b/http-fetch.c
index a32ac118d9..31844812a1 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -3,9 +3,18 @@
 #include "exec-cmd.h"
 #include "http.h"
 #include "walker.h"
+#include "version.h"
 
 static const char http_fetch_usage[] = "git http-fetch "
-"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
+"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
+
+void NORETURN version_info()
+{
+	printf("git-http-fetch version: %s\n", git_version_string);
+	printf("built from commit: %s\n", git_built_from_commit_string);
+	printf("curl version: %s\n", curl_version());
+	exit(0);
+}
 
 int cmd_main(int argc, const char **argv)
 {
@@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
 		} else if (argv[arg][1] == 'a') {
 		} else if (argv[arg][1] == 'v') {
 			get_verbosely = 1;
+		} else if (argv[arg][1] == 'V') {
+			version_info();
 		} else if (argv[arg][1] == 'w') {
 			write_ref = &argv[arg + 1];
 			arg++;
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..6eb2ee4f66
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,41 @@
+#!/bin/bash
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if [$(grep $HEADER_PATTERN $line)]; then
+			read -r nextline
+			if [-z $nextline]; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	check_all_headers_populated <git-bugreport-* &&
+	rm git-bugreport-*
+'
+
+test_expect_success '--output puts the report in the provided dir' '
+	mkdir foo/ &&
+	git bugreport -o foo/ &&
+	test -f foo/git-bugreport-* &&
+	rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	grep usage output &&
+	test ! -f git-bugreport-*
+'
+
+test_done

-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v4 01/15] bugreport: add tool to generate debugging info
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
@ 2019-12-13  0:42 ` Emily Shaffer
  2019-12-13  0:42 ` [PATCH v4 02/15] help: move list_config_help to builtin/help Emily Shaffer
                   ` (14 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:42 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction
steps, expected behavior, and actual behavior. Later, Git can learn how
to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic
information we would otherwise need to ask the user for, we can reduce
the number of question-and-answer round trips between the reporter and
the Git contributor.

Users may also wish to send a report like this to their local "Git
expert" if they have put their repository into a state they are confused
by.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 43 ++++++++++++++++++++
 Makefile                        |  5 +++
 bugreport.c                     | 69 +++++++++++++++++++++++++++++++++
 t/t0091-bugreport.sh            | 41 ++++++++++++++++++++
 5 files changed, 159 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index 89b3b79c1a..055a84c4a8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..75f0c80acf
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,43 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [-o | --output <path>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+The following information is captured automatically:
+
+ - Git version (`git version --build-options`)
+ - Machine information (`uname -a`)
+ - Versions of various dependencies
+ - Git config contents (`git config --show-origin --list`)
+ - The names of all configured git-hooks in `.git/hooks/`
+
+OPTIONS
+-------
+-o [<path>]::
+--output [<path>]::
+	Place the resulting bug report file in <path> instead of the root of the
+	Git repository.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index b7d7374dac..9dff91436e 100644
--- a/Makefile
+++ b/Makefile
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2448,6 +2449,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
diff --git a/bugreport.c b/bugreport.c
new file mode 100644
index 0000000000..5495b31674
--- /dev/null
+++ b/bugreport.c
@@ -0,0 +1,69 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output <file>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to send.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+
+	get_bug_template(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	launch_editor(report_path.buf, NULL, NULL);
+	return 0;
+}
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..6eb2ee4f66
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,41 @@
+#!/bin/bash
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if [$(grep $HEADER_PATTERN $line)]; then
+			read -r nextline
+			if [-z $nextline]; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	check_all_headers_populated <git-bugreport-* &&
+	rm git-bugreport-*
+'
+
+test_expect_success '--output puts the report in the provided dir' '
+	mkdir foo/ &&
+	git bugreport -o foo/ &&
+	test -f foo/git-bugreport-* &&
+	rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	grep usage output &&
+	test ! -f git-bugreport-*
+'
+
+test_done
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v4 02/15] help: move list_config_help to builtin/help
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
  2019-12-13  0:42 ` [PATCH v4 01/15] bugreport: add tool to generate debugging info Emily Shaffer
@ 2019-12-13  0:42 ` Emily Shaffer
  2019-12-13 20:51   ` Junio C Hamano
  2019-12-13  0:43 ` [PATCH v4 03/15] bugreport: gather git version and build info Emily Shaffer
                   ` (13 subsequent siblings)
  15 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:42 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o,
builtin/clean.o, and builtin/config.o. This meant that help.o was
unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help()
into builtin/help.c (where it makes sense to assume other builtin libraries
are present).

When command-list.h is included but a member is not used, we start to
hear a compiler warning. Since the config list is generated in a fairly
different way than the command list, and since commands and config
options are semantically different, move the config list into its own
header and move the generator into its own script and build rule.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore             |  1 +
 Makefile               | 16 ++++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 24 ++++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 123 insertions(+), 109 deletions(-)
 create mode 100755 generate-configlist.sh

diff --git a/.gitignore b/.gitignore
index 055a84c4a8..5dde2cc4c8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index 9dff91436e..c49f55a521 100644
--- a/Makefile
+++ b/Makefile
@@ -815,6 +815,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2127,7 +2128,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2147,6 +2148,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2784,7 +2791,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-GEN_HDRS := command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -2807,7 +2814,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -3110,7 +3117,8 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
+	$(RM) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..eca6a00c30
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/help.c b/help.c
index cf67624a94..a21487db77 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
diff --git a/help.h b/help.h
index 7a455beeb7..9071894e8c 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v4 03/15] bugreport: gather git version and build info
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
  2019-12-13  0:42 ` [PATCH v4 01/15] bugreport: add tool to generate debugging info Emily Shaffer
  2019-12-13  0:42 ` [PATCH v4 02/15] help: move list_config_help to builtin/help Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:06   ` Junio C Hamano
  2019-12-17 18:45   ` Johannes Schindelin
  2019-12-13  0:43 ` [PATCH v4 04/15] help: add shell-path to --build-options Emily Shaffer
                   ` (12 subsequent siblings)
  15 siblings, 2 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Knowing which version of Git a user has and how it was built allows us
to more precisely pin down the circumstances when a certain issue
occurs, so teach bugreport how to tell us the same output as 'git
version --build-options'.

It's not ideal to directly call 'git version --build-options' because
that output goes to stdout. Instead, wrap the version string in a helper
within help.[ch] library, and call that helper from within the bugreport
library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 23 ++++++++++++++++++++++-
 help.c      | 45 +++++++++++++++++++++++++++------------------
 help.h      |  1 +
 3 files changed, 50 insertions(+), 19 deletions(-)

diff --git a/bugreport.c b/bugreport.c
index 5495b31674..59d8b5a3af 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,8 +1,20 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	struct strbuf version_info = STRBUF_INIT;
+
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	list_version_info(&version_info, 1);
+	strbuf_addbuf(sys_info, &version_info);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
@@ -32,6 +44,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -60,6 +77,10 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	// add other contents
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
diff --git a/help.c b/help.c
index a21487db77..a43693fca5 100644
--- a/help.c
+++ b/help.c
@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void list_version_info(struct strbuf *buf, int build_options)
+{
+	strbuf_reset(buf);
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addf(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -637,25 +662,9 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	list_version_info(&buf, build_options);
+	printf("%s", buf.buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 9071894e8c..54f6b5f793 100644
--- a/help.h
+++ b/help.h
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void list_version_info(struct strbuf *buf, int build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v4 04/15] help: add shell-path to --build-options
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (2 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 03/15] bugreport: gather git version and build info Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13  0:43 ` [PATCH v4 05/15] bugreport: add uname info Emily Shaffer
                   ` (11 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It may be useful to know which shell Git was built to try to point to,
in the event that shell-based Git commands are failing. $SHELL_PATH is
set during the build and used to launch the manpage viewer, as well as
by git-compat-util.h, and it's used during tests. 'git version
--build-options' is encouraged for use in bug reports, so it makes sense
to include this information there.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 help.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/help.c b/help.c
index a43693fca5..6d1bd17971 100644
--- a/help.c
+++ b/help.c
@@ -642,6 +642,7 @@ void list_version_info(struct strbuf *buf, int build_options)
 			strbuf_addf(buf, "no commit associated with this build\n");
 		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
 		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 }
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v4 05/15] bugreport: add uname info
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (3 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 04/15] help: add shell-path to --build-options Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:12   ` Junio C Hamano
  2020-01-10  2:05   ` Aaron Schrab
  2019-12-13  0:43 ` [PATCH v4 06/15] bugreport: add glibc version Emily Shaffer
                   ` (10 subsequent siblings)
  15 siblings, 2 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

The contents of uname() can give us some insight into what sort of
system the user is running on, and help us replicate their setup if need
be. The domainname field is not guaranteed to be available, so don't
collect it.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 59d8b5a3af..9c69e3fa34 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -8,12 +8,25 @@
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
+	struct utsname uname_info;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	list_version_info(&version_info, 1);
 	strbuf_addbuf(sys_info, &version_info);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.nodename,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v4 06/15] bugreport: add glibc version
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (4 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 05/15] bugreport: add uname info Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:18   ` Junio C Hamano
  2019-12-13  0:43 ` [PATCH v4 07/15] bugreport: add curl version Emily Shaffer
                   ` (9 subsequent siblings)
  15 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

To help pinpoint the source of a regression, it is useful to know the
version of libc which the user's Git client was built with. Let's
include it alongside the other versioning information, which is used to
identify how the client was built.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 9c69e3fa34..af715dc157 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include <gnu/libc-version.h>
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -27,6 +28,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "glibc version: ");
+	strbuf_addstr(sys_info, gnu_get_libc_version());
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v4 07/15] bugreport: add curl version
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (5 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 06/15] bugreport: add glibc version Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:27   ` Junio C Hamano
  2019-12-17 18:47   ` Johannes Schindelin
  2019-12-13  0:43 ` [PATCH v4 08/15] bugreport: include user interactive shell Emily Shaffer
                   ` (8 subsequent siblings)
  15 siblings, 2 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It's possible for git-http* to be built separately from git; in that
case we want to know what version of cURL is used by git-http*, not
necessarily which version was present at git-bugreport's build time.
So instead, ask git-http-fetch for the version information it knows
about.

git-http-fetch was chosen as git-http-backend was described as a
server-side implementation, and as an accidental fetch in case of
problems was considered less harmful than an accidental push.

Since it could have been built at a different time, also report the
version and built-from commit of git-http-fetch alongside the cURL info.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-http-fetch.txt |  5 +++++
 bugreport.c                      | 16 ++++++++++++++++
 http-fetch.c                     | 13 ++++++++++++-
 3 files changed, 33 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-http-fetch.txt b/Documentation/git-http-fetch.txt
index 666b042679..2894c5e82b 100644
--- a/Documentation/git-http-fetch.txt
+++ b/Documentation/git-http-fetch.txt
@@ -10,6 +10,7 @@ SYNOPSIS
 --------
 [verse]
 'git http-fetch' [-c] [-t] [-a] [-d] [-v] [-w filename] [--recover] [--stdin] <commit> <url>
+'git http-fetch' [-V]
 
 DESCRIPTION
 -----------
@@ -30,6 +31,10 @@ commit-id::
 -v::
 	Report what is downloaded.
 
+-V::
+	Report information about the version of git-http-fetch, including the
+	versions of its dependencies.
+
 -w <filename>::
         Writes the commit-id into the filename under $GIT_DIR/refs/<filename> on
         the local end after the transfer is complete.
diff --git a/bugreport.c b/bugreport.c
index af715dc157..f5598513d9 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include <gnu/libc-version.h>
+#include "run-command.h"
+
+static void get_http_version_info(struct strbuf *http_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "http-fetch");
+	argv_array_push(&cp.args, "-V");
+	if (capture_command(&cp, http_info, 0))
+	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -32,6 +44,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "glibc version: ");
 	strbuf_addstr(sys_info, gnu_get_libc_version());
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
+	get_http_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/http-fetch.c b/http-fetch.c
index a32ac118d9..31844812a1 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -3,9 +3,18 @@
 #include "exec-cmd.h"
 #include "http.h"
 #include "walker.h"
+#include "version.h"
 
 static const char http_fetch_usage[] = "git http-fetch "
-"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
+"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
+
+void NORETURN version_info()
+{
+	printf("git-http-fetch version: %s\n", git_version_string);
+	printf("built from commit: %s\n", git_built_from_commit_string);
+	printf("curl version: %s\n", curl_version());
+	exit(0);
+}
 
 int cmd_main(int argc, const char **argv)
 {
@@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
 		} else if (argv[arg][1] == 'a') {
 		} else if (argv[arg][1] == 'v') {
 			get_verbosely = 1;
+		} else if (argv[arg][1] == 'V') {
+			version_info();
 		} else if (argv[arg][1] == 'w') {
 			write_ref = &argv[arg + 1];
 			arg++;
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v4 08/15] bugreport: include user interactive shell
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (6 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 07/15] bugreport: add curl version Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:38   ` Junio C Hamano
  2019-12-13  0:43 ` [PATCH v4 09/15] bugreport: generate config safelist based on docs Emily Shaffer
                   ` (7 subsequent siblings)
  15 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It's possible a user may complain about the way that Git interacts with
their interactive shell, e.g. autocompletion or shell prompt. In that
case, it's useful for us to know which shell they're using
interactively.

$SHELL isn't set by every shell, but getenv() returns NULL in the
event the variable isn't found, so we'll see a line like "$SHELL:
(null)" to tell us that variable wasn't set.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index f5598513d9..759cc0b0f8 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -45,6 +45,9 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, gnu_get_libc_version());
 	strbuf_complete_line(sys_info);
 
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    getenv("SHELL"));
+
 	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
 	get_http_version_info(sys_info);
 	strbuf_complete_line(sys_info);
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v4 09/15] bugreport: generate config safelist based on docs
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (7 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 08/15] bugreport: include user interactive shell Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 22:57   ` Junio C Hamano
                     ` (2 more replies)
  2019-12-13  0:43 ` [PATCH v4 10/15] bugreport: add config values from safelist Emily Shaffer
                   ` (6 subsequent siblings)
  15 siblings, 3 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Martin Ågren, Johannes Schindelin

Add a new step to the build to generate a safelist of git-config
variables which are appropriate to include in the output of
git-bugreport. New variables can be added to the safelist by annotating
their documentation in Documentation/config with the
"bugreport" macro, which is recognized by AsciiDoc and
AsciiDoctor.

Some configs are private in nature, and can contain remote URLs,
passwords, or other sensitive information. In the event that a user
doesn't notice their information while reviewing a bugreport, that user
may leak their credentials to other individuals, mailing lists, or bug
tracking tools inadvertently. Heuristic blocklisting of configuration
keys is imperfect and prone to false negatives; given the nature of the
information which can be leaked, a safelist is more reliable.

In order to prevent staleness of the safelist, add a mechanism to
generate the safelist from annotations in the config documentation,
where contributors are already used to documenting their new config
keys.

Implement a new no-op "bugreport" macro for use as
"bugreport:include[x]" to annotate the config keys that should be
included in the automatically generated safelist. Use "exclude" for the
others.

With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc
seems to want something between the brackets. A bit unfortunate, but
not a huge problem -- we'll just provide an "x".

"doc-diff" reports that this macro doesn't render at all. That is,
these are both empty after this commit:

  cd Documentation
  ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
  ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD

Diffing the rendered HTML shows that there is some small amount of
whitespace and comments added. That shouldn't be a problem.

We could perhaps let the implementation verify that the "action" is one
of "include" and "exclude". For the Asciidoctor implementation that
should be straightforward, but for AsciiDoc I don't immediately know how
to do it. Anyway, if someone stumbles on the keyboard and writes
"bugreport:icndule", they'll "only" miss out on the config key being
included in the safelist. If this were a blocklist, the consequences of
a misspelled target could be a lot more severe.

Additionally, add annotations to the sendemail config documentation in
order to demonstrate a proof of concept.

Helped-by: Martin Ågren <martin.agren@gmail.com>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  8 +++
 Documentation/asciidoctor-extensions.rb |  7 +++
 Documentation/config/sendemail.txt      | 68 ++++++++++++-------------
 Makefile                                | 14 +++--
 generate-bugreport-config-safelist.sh   | 22 ++++++++
 6 files changed, 83 insertions(+), 37 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh

diff --git a/.gitignore b/.gitignore
index 5dde2cc4c8..30935621d9 100644
--- a/.gitignore
+++ b/.gitignore
@@ -191,6 +191,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..5d5359fcf9 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,13 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The bugreport macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +32,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[bugreport-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[bugreport-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..750bdff9af 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, action, attrs)
+        ""
       end
     end
   end
@@ -45,4 +49,7 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  # The bugreport macro does nothing as far as rendering is
+  # concerned -- we just grep for it in the sources.
+  inline_macro Git::Documentation::BugReportProcessor, :bugreport
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 0006faf800..92f5082013 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -1,63 +1,63 @@
-sendemail.identity::
+sendemail.identity bugreport:exclude[x] ::
 	A configuration identity. When given, causes values in the
 	'sendemail.<identity>' subsection to take precedence over
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption::
+sendemail.smtpEncryption bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
-sendemail.smtpssl (deprecated)::
+sendemail.smtpssl (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for 'sendemail.smtpEncryption = ssl'.
 
-sendemail.smtpsslcertpath::
+sendemail.smtpsslcertpath bugreport:exclude[x] ::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*::
+sendemail.<identity>.* bugreport:exclude[x] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
 	`sendemail.identity`.
 
-sendemail.aliasesFile::
-sendemail.aliasFileType::
-sendemail.annotate::
-sendemail.bcc::
-sendemail.cc::
-sendemail.ccCmd::
-sendemail.chainReplyTo::
-sendemail.confirm::
-sendemail.envelopeSender::
-sendemail.from::
-sendemail.multiEdit::
-sendemail.signedoffbycc::
-sendemail.smtpPass::
-sendemail.suppresscc::
-sendemail.suppressFrom::
-sendemail.to::
-sendemail.tocmd::
-sendemail.smtpDomain::
-sendemail.smtpServer::
-sendemail.smtpServerPort::
-sendemail.smtpServerOption::
-sendemail.smtpUser::
-sendemail.thread::
-sendemail.transferEncoding::
-sendemail.validate::
-sendemail.xmailer::
+sendemail.aliasesFile bugreport:exclude[x] ::
+sendemail.aliasFileType bugreport:exclude[x] ::
+sendemail.annotate bugreport:include[x] ::
+sendemail.bcc bugreport:include[x] ::
+sendemail.cc bugreport:include[x] ::
+sendemail.ccCmd bugreport:include[x] ::
+sendemail.chainReplyTo bugreport:include[x] ::
+sendemail.confirm bugreport:include[x] ::
+sendemail.envelopeSender bugreport:include[x] ::
+sendemail.from bugreport:include[x] ::
+sendemail.multiEdit bugreport:include[x] ::
+sendemail.signedoffbycc bugreport:include[x] ::
+sendemail.smtpPass bugreport:exclude[x] ::
+sendemail.suppresscc bugreport:include[x] ::
+sendemail.suppressFrom bugreport:include[x] ::
+sendemail.to bugreport:include[x] ::
+sendemail.tocmd bugreport:include[x] ::
+sendemail.smtpDomain bugreport:include[x] ::
+sendemail.smtpServer bugreport:include[x] ::
+sendemail.smtpServerPort bugreport:include[x] ::
+sendemail.smtpServerOption bugreport:include[x] ::
+sendemail.smtpUser bugreport:exclude[x] ::
+sendemail.thread bugreport:include[x] ::
+sendemail.transferEncoding bugreport:include[x] ::
+sendemail.validate bugreport:include[x] ::
+sendemail.xmailer bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.
 
-sendemail.signedoffcc (deprecated)::
+sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize bugreport:include[x] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay::
+sendemail.smtpReloginDelay bugreport:include[x] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Makefile b/Makefile
index c49f55a521..76dc51e2b1 100644
--- a/Makefile
+++ b/Makefile
@@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
 install-python-script: $(SCRIPT_PYTHON_GEN)
 	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
 
-.PHONY: clean-perl-script clean-sh-script clean-python-script
+.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
 clean-sh-script:
 	$(RM) $(SCRIPT_SH_GEN)
 clean-perl-script:
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
@@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-GEN_HDRS := config-list.h command-list.h unicode-width.h
+GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
 EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
@@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
 	$(RM) $(HCC)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo
+	$(RM) config-list.h command-list.h bugreport-config-safelist.h
 	$(RM) $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..06b8e0c3c4
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,22 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the bugreport-include macro
+sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |
+sort |
+while read line
+do
+	echo "	\"$line\","
+done
+
+cat <<EOF
+};
+EOF
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v4 10/15] bugreport: add config values from safelist
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (8 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 09/15] bugreport: generate config safelist based on docs Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:45   ` Junio C Hamano
  2019-12-29 20:17   ` Johannes Schindelin
  2019-12-13  0:43 ` [PATCH v4 11/15] bugreport: collect list of populated hooks Emily Shaffer
                   ` (5 subsequent siblings)
  15 siblings, 2 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach bugreport to gather the values of config options which are present
in 'bugreport-config-safelist.h'.

Many config options are sensitive, and many Git add-ons use config
options which git-core does not know about; it is better only to gather
config options which we know to be safe, rather than excluding options
which we know to be unsafe.

Taking the build-time generated array and putting it into a set saves us
time - since git_config_bugreport() is called for every option the user
has configured, performing option lookup in constant time is a useful
optimization.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 71 insertions(+), 1 deletion(-)

diff --git a/bugreport.c b/bugreport.c
index 759cc0b0f8..1fca28f0b9 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -6,6 +6,9 @@
 #include "help.h"
 #include <gnu/libc-version.h>
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_http_version_info(struct strbuf *http_info)
 {
@@ -18,6 +21,41 @@ static void get_http_version_info(struct strbuf *http_info)
 	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
 }
 
+KHASH_INIT(cfg_set, const char*, int, 0, kh_str_hash_func, kh_str_hash_equal);
+
+struct cfgset {
+	kh_cfg_set_t set;
+};
+
+struct cfgset safelist;
+
+static void cfgset_init(struct cfgset *set, size_t initial_size)
+{
+	memset(&set->set, 0, sizeof(set->set));
+	if (initial_size)
+		kh_resize_cfg_set(&set->set, initial_size);
+}
+
+static int cfgset_insert(struct cfgset *set, const char *cfg_key)
+{
+	int added;
+	kh_put_cfg_set(&set->set, cfg_key, &added);
+	printf("ESS: added %s\n", cfg_key);
+	return !added;
+}
+
+static int cfgset_contains(struct cfgset *set, const char *cfg_key)
+{
+	khiter_t pos = kh_get_cfg_set(&set->set, cfg_key);
+	return pos != kh_end(&set->set);
+}
+
+static void cfgset_clear(struct cfgset *set)
+{
+	kh_release_cfg_set(&set->set);
+	cfgset_init(set, 0);
+}
+
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
@@ -53,6 +91,36 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void gather_safelist()
+{
+	int index;
+	int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
+	cfgset_init(&safelist, safelist_len);
+	for (index = 0; index < safelist_len; index++)
+		cfgset_insert(&safelist, bugreport_config_safelist[index]);
+
+}
+
+static int git_config_bugreport(const char *var, const char *value, void *cb)
+{
+	struct strbuf *config_info = (struct strbuf *)cb;
+
+	if (cfgset_contains(&safelist, var))
+		strbuf_addf(config_info,
+			    "%s (%s) : %s\n",
+			    var, config_scope_to_string(current_config_scope()),
+			    value);
+
+	return 0;
+}
+
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	gather_safelist();
+	git_config(git_config_bugreport, config_info);
+	cfgset_clear(&safelist);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -114,10 +182,12 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
-	// add other contents
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v4 11/15] bugreport: collect list of populated hooks
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (9 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 10/15] bugreport: add config values from safelist Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:47   ` Junio C Hamano
  2019-12-13  0:43 ` [PATCH v4 12/15] bugreport: count loose objects Emily Shaffer
                   ` (4 subsequent siblings)
  15 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Occasionally a failure a user is seeing may be related to a specific
hook which is being run, perhaps without the user realizing. While the
contents of hooks can be sensitive - containing user data or process
information specific to the user's organization - simply knowing that a
hook is being run at a certain stage can help us to understand whether
something is going wrong.

Without a definitive list of hook names within the code, we compile our
own list from the documentation. This is likely prone to bitrot. To
reduce the amount of code humans need to read, we turn the list into a
string_list and iterate over it (as we are calling the same find_hook
operation on each string). However, since bugreport should primarily be
called by the user, the performance loss from massaging the string
seems acceptable.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 1fca28f0b9..f89cb8d754 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "bugreport-config-safelist.h"
 #include "khash.h"
+#include "run-command.h"
 
 static void get_http_version_info(struct strbuf *http_info)
 {
@@ -121,6 +122,57 @@ static void get_safelisted_config(struct strbuf *config_info)
 	cfgset_clear(&safelist);
 }
 
+static void get_populated_hooks(struct strbuf *hook_info)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hook`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+	int nongit_ok;
+
+	setup_git_directory_gently(&nongit_ok);
+
+	if (nongit_ok) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -188,6 +240,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Configured Hooks");
+	get_populated_hooks(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v4 12/15] bugreport: count loose objects
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (10 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 11/15] bugreport: collect list of populated hooks Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:51   ` Junio C Hamano
  2019-12-13  0:43 ` [PATCH v4 13/15] bugreport: add packed object summary Emily Shaffer
                   ` (3 subsequent siblings)
  15 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Johannes Schindelin

The number of unpacked objects in a user's repository may help us
understand the root of the problem they're seeing, especially if a
command is running unusually slowly.

Rather than directly invoking 'git-count-objects', which may sometimes
fail unexpectedly on Git for Windows, manually count the contents of
.git/objects. Additionally, since we may wish to inspect other
directories' contents for bugreport in the future, put the directory
listing into a helper function.

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index f89cb8d754..3abb83d77f 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -173,6 +173,67 @@ static void get_populated_hooks(struct strbuf *hook_info)
 	}
 }
 
+static int is_hex(const char *string, size_t count)
+{
+	for (; count; string++, count--) {
+		if (!isxdigit(*string))
+			return 0;
+	}
+	return 1;
+}
+
+static void get_loose_object_summary(struct strbuf *obj_info) {
+	struct dirent *d = NULL;
+	DIR *dir, *subdir = NULL;
+	size_t dir_len;
+	struct strbuf dirpath = STRBUF_INIT;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+
+	dir = opendir(dirpath.buf);
+	if (!dir) {
+		strbuf_addf(obj_info, "could not open object directory '%s'\n",
+			    dirpath.buf);
+		strbuf_release(&dirpath);
+		return;
+	}
+
+	dir_len = dirpath.len;
+
+	while ((d = readdir(dir))) {
+		int object_count = 0;
+		char subdir_name[3];
+
+		if (d->d_type != DT_DIR)
+			continue;
+
+		if ((strlen(d->d_name) != 2) || (!is_hex(d->d_name, 2)))
+			continue;
+
+		/* copy directory name + \0 */
+		memcpy(subdir_name, d->d_name, 3);
+
+		strbuf_setlen(&dirpath, dir_len);
+		strbuf_addstr(&dirpath, d->d_name);
+
+		subdir = opendir(dirpath.buf);
+		if (!subdir)
+			continue;
+		while ((d = readdir(subdir)))
+			if (d->d_type == DT_REG)
+				object_count++;
+
+		closedir(subdir);
+
+		strbuf_addf(obj_info, "%s: %d\n", subdir_name, object_count);
+	}
+
+
+	closedir(dir);
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -243,6 +304,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Configured Hooks");
 	get_populated_hooks(&buffer);
 
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v4 13/15] bugreport: add packed object summary
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (11 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 12/15] bugreport: count loose objects Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13 21:56   ` Junio C Hamano
  2019-12-13  0:43 ` [PATCH v4 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
                   ` (2 subsequent siblings)
  15 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Alongside the list of loose objects, it's useful to see the list of
object packs as well. It can help us to examine what Git did and did not
pack.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 3abb83d77f..992d8f9de7 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -234,6 +234,34 @@ static void get_loose_object_summary(struct strbuf *obj_info) {
 	strbuf_release(&dirpath);
 }
 
+static void get_packed_object_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+	struct dirent *d;
+	DIR *dir = NULL;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "pack/");
+
+	dir = opendir(dirpath.buf);
+	if (!dir) {
+		strbuf_addf(obj_info, "could not open packed object directory '%s'\n",
+			    dirpath.buf);
+		strbuf_release(&dirpath);
+		return;
+	}
+
+	while ((d = readdir(dir))) {
+		strbuf_addbuf(obj_info, &dirpath);
+		strbuf_addstr(obj_info, d->d_name);
+		strbuf_complete_line(obj_info);
+	}
+
+	closedir(dir);
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -307,6 +335,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v4 14/15] bugreport: list contents of $OBJDIR/info
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (12 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 13/15] bugreport: add packed object summary Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2019-12-13  0:43 ` [PATCH v4 15/15] bugreport: summarize contents of alternates file Emily Shaffer
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Miscellaneous information used about the object store can end up in
.git/objects/info; this can help us understand what may be going on with
the object store when the user is reporting a bug. Otherwise, it could
be difficult to track down what is going wrong with an object which
isn't kept locally to .git/objects/ or .git/objects/pack. Having some
understanding of where the user's objects may be kept can save us some
hops during the bug reporting process.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 992d8f9de7..5d300f1c5f 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -262,6 +262,51 @@ static void get_packed_object_summary(struct strbuf *obj_info)
 	strbuf_release(&dirpath);
 }
 
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
+{
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
+
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(obj_info, &dirpath);
+
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -338,6 +383,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Packed Object Summary");
 	get_packed_object_summary(&buffer);
 
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v4 15/15] bugreport: summarize contents of alternates file
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (13 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
@ 2019-12-13  0:43 ` Emily Shaffer
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-13  0:43 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

In some cases, it could be that the user is having a problem with an
object which isn't present in their normal object directory. We can get
a hint that that might be the case by examining the list of alternates
where their object may be stored instead. Since paths to alternates may
be sensitive, we'll instead count how many alternates have been
specified and note how many of them exist or are broken.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 5d300f1c5f..d2a3fb1f67 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -307,6 +307,42 @@ static void get_object_info_summary(struct strbuf *obj_info)
 	strbuf_release(&dirpath);
 }
 
+static void get_alternates_summary(struct strbuf *alternates_info)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -386,6 +422,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.24.1.735.g03f4e72817-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 02/15] help: move list_config_help to builtin/help
  2019-12-13  0:42 ` [PATCH v4 02/15] help: move list_config_help to builtin/help Emily Shaffer
@ 2019-12-13 20:51   ` Junio C Hamano
  2019-12-16 21:36     ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2019-12-13 20:51 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o,
> builtin/clean.o, and builtin/config.o. This meant that help.o was
> unusable outside of the context of the main Git executable.
>
> To make help.o usable by other commands again, move list_config_help()
> into builtin/help.c (where it makes sense to assume other builtin libraries
> are present).
>
> When command-list.h is included but a member is not used, we start to
> hear a compiler warning. Since the config list is generated in a fairly
> different way than the command list, and since commands and config
> options are semantically different, move the config list into its own
> header and move the generator into its own script and build rule.
>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---

OK.  Looks like a clean-up that is very worthy even without the
remainder of the topic.

Thanks.

>  .gitignore             |  1 +
>  Makefile               | 16 ++++++--
>  builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
>  generate-cmdlist.sh    | 19 ----------
>  generate-configlist.sh | 24 ++++++++++++
>  help.c                 | 85 -----------------------------------------
>  help.h                 |  1 -
>  7 files changed, 123 insertions(+), 109 deletions(-)
>  create mode 100755 generate-configlist.sh
>
> diff --git a/.gitignore b/.gitignore
> index 055a84c4a8..5dde2cc4c8 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -189,6 +189,7 @@
>  /gitweb/gitweb.cgi
>  /gitweb/static/gitweb.js
>  /gitweb/static/gitweb.min.*
> +/config-list.h
>  /command-list.h
>  *.tar.gz
>  *.dsc
> diff --git a/Makefile b/Makefile
> index 9dff91436e..c49f55a521 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -815,6 +815,7 @@ LIB_FILE = libgit.a
>  XDIFF_LIB = xdiff/lib.a
>  VCSSVN_LIB = vcs-svn/lib.a
>  
> +GENERATED_H += config-list.h
>  GENERATED_H += command-list.h
>  
>  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
> @@ -2127,7 +2128,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
>  
>  help.sp help.s help.o: command-list.h
>  
> -builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
> +builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
>  builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
>  	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
>  	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
> @@ -2147,6 +2148,12 @@ $(BUILT_INS): git$X
>  	ln -s $< $@ 2>/dev/null || \
>  	cp $< $@
>  
> +config-list.h: generate-configlist.sh
> +
> +config-list.h:
> +	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
> +		>$@+ && mv $@+ $@
> +
>  command-list.h: generate-cmdlist.sh command-list.txt
>  
>  command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
> @@ -2784,7 +2791,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
>  .PHONY: sparse $(SP_OBJ)
>  sparse: $(SP_OBJ)
>  
> -GEN_HDRS := command-list.h unicode-width.h
> +GEN_HDRS := config-list.h command-list.h unicode-width.h
>  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
>  ifndef GCRYPT_SHA256
>  	EXCEPT_HDRS += sha256/gcrypt.h
> @@ -2807,7 +2814,7 @@ hdr-check: $(HCO)
>  style:
>  	git clang-format --style file --diff --extensions c,h
>  
> -check: command-list.h
> +check: config-list.h command-list.h
>  	@if sparse; \
>  	then \
>  		echo >&2 "Use 'make sparse' instead"; \
> @@ -3110,7 +3117,8 @@ clean: profile-clean coverage-clean cocciclean
>  	$(RM) $(HCC)
>  	$(RM) -r bin-wrappers $(dep_dirs)
>  	$(RM) -r po/build/
> -	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
> +	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> +	$(RM) $(ETAGS_TARGET) tags cscope*
>  	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
>  	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
>  	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
> diff --git a/builtin/help.c b/builtin/help.c
> index e5590d7787..1c5f2b9255 100644
> --- a/builtin/help.c
> +++ b/builtin/help.c
> @@ -8,6 +8,7 @@
>  #include "parse-options.h"
>  #include "run-command.h"
>  #include "column.h"
> +#include "config-list.h"
>  #include "help.h"
>  #include "alias.h"
>  
> @@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
>  	NULL
>  };
>  
> +struct slot_expansion {
> +	const char *prefix;
> +	const char *placeholder;
> +	void (*fn)(struct string_list *list, const char *prefix);
> +	int found;
> +};
> +
> +static void list_config_help(int for_human)
> +{
> +	struct slot_expansion slot_expansions[] = {
> +		{ "advice", "*", list_config_advices },
> +		{ "color.branch", "<slot>", list_config_color_branch_slots },
> +		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
> +		{ "color.diff", "<slot>", list_config_color_diff_slots },
> +		{ "color.grep", "<slot>", list_config_color_grep_slots },
> +		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
> +		{ "color.remote", "<slot>", list_config_color_sideband_slots },
> +		{ "color.status", "<slot>", list_config_color_status_slots },
> +		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
> +		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
> +		{ NULL, NULL, NULL }
> +	};
> +	const char **p;
> +	struct slot_expansion *e;
> +	struct string_list keys = STRING_LIST_INIT_DUP;
> +	int i;
> +
> +	for (p = config_name_list; *p; p++) {
> +		const char *var = *p;
> +		struct strbuf sb = STRBUF_INIT;
> +
> +		for (e = slot_expansions; e->prefix; e++) {
> +
> +			strbuf_reset(&sb);
> +			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
> +			if (!strcasecmp(var, sb.buf)) {
> +				e->fn(&keys, e->prefix);
> +				e->found++;
> +				break;
> +			}
> +		}
> +		strbuf_release(&sb);
> +		if (!e->prefix)
> +			string_list_append(&keys, var);
> +	}
> +
> +	for (e = slot_expansions; e->prefix; e++)
> +		if (!e->found)
> +			BUG("slot_expansion %s.%s is not used",
> +			    e->prefix, e->placeholder);
> +
> +	string_list_sort(&keys);
> +	for (i = 0; i < keys.nr; i++) {
> +		const char *var = keys.items[i].string;
> +		const char *wildcard, *tag, *cut;
> +
> +		if (for_human) {
> +			puts(var);
> +			continue;
> +		}
> +
> +		wildcard = strchr(var, '*');
> +		tag = strchr(var, '<');
> +
> +		if (!wildcard && !tag) {
> +			puts(var);
> +			continue;
> +		}
> +
> +		if (wildcard && !tag)
> +			cut = wildcard;
> +		else if (!wildcard && tag)
> +			cut = tag;
> +		else
> +			cut = wildcard < tag ? wildcard : tag;
> +
> +		/*
> +		 * We may produce duplicates, but that's up to
> +		 * git-completion.bash to handle
> +		 */
> +		printf("%.*s\n", (int)(cut - var), var);
> +	}
> +	string_list_clear(&keys, 0);
> +}
> +
>  static enum help_format parse_help_format(const char *format)
>  {
>  	if (!strcmp(format, "man"))
> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> index 71158f7d8b..45fecf8bdf 100755
> --- a/generate-cmdlist.sh
> +++ b/generate-cmdlist.sh
> @@ -76,23 +76,6 @@ print_command_list () {
>  	echo "};"
>  }
>  
> -print_config_list () {
> -	cat <<EOF
> -static const char *config_name_list[] = {
> -EOF
> -	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> -	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> -	sort |
> -	while read line
> -	do
> -		echo "	\"$line\","
> -	done
> -	cat <<EOF
> -	NULL,
> -};
> -EOF
> -}
> -
>  exclude_programs=
>  while test "--exclude-program" = "$1"
>  do
> @@ -113,5 +96,3 @@ echo
>  define_category_names "$1"
>  echo
>  print_command_list "$1"
> -echo
> -print_config_list
> diff --git a/generate-configlist.sh b/generate-configlist.sh
> new file mode 100755
> index 0000000000..eca6a00c30
> --- /dev/null
> +++ b/generate-configlist.sh
> @@ -0,0 +1,24 @@
> +#!/bin/sh
> +
> +echo "/* Automatically generated by generate-configlist.sh */"
> +echo
> +
> +print_config_list () {
> +	cat <<EOF
> +static const char *config_name_list[] = {
> +EOF
> +	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> +	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> +	sort |
> +	while read line
> +	do
> +		echo "	\"$line\","
> +	done
> +	cat <<EOF
> +	NULL,
> +};
> +EOF
> +}
> +
> +echo
> +print_config_list
> diff --git a/help.c b/help.c
> index cf67624a94..a21487db77 100644
> --- a/help.c
> +++ b/help.c
> @@ -407,91 +407,6 @@ void list_common_guides_help(void)
>  	putchar('\n');
>  }
>  
> -struct slot_expansion {
> -	const char *prefix;
> -	const char *placeholder;
> -	void (*fn)(struct string_list *list, const char *prefix);
> -	int found;
> -};
> -
> -void list_config_help(int for_human)
> -{
> -	struct slot_expansion slot_expansions[] = {
> -		{ "advice", "*", list_config_advices },
> -		{ "color.branch", "<slot>", list_config_color_branch_slots },
> -		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
> -		{ "color.diff", "<slot>", list_config_color_diff_slots },
> -		{ "color.grep", "<slot>", list_config_color_grep_slots },
> -		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
> -		{ "color.remote", "<slot>", list_config_color_sideband_slots },
> -		{ "color.status", "<slot>", list_config_color_status_slots },
> -		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
> -		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
> -		{ NULL, NULL, NULL }
> -	};
> -	const char **p;
> -	struct slot_expansion *e;
> -	struct string_list keys = STRING_LIST_INIT_DUP;
> -	int i;
> -
> -	for (p = config_name_list; *p; p++) {
> -		const char *var = *p;
> -		struct strbuf sb = STRBUF_INIT;
> -
> -		for (e = slot_expansions; e->prefix; e++) {
> -
> -			strbuf_reset(&sb);
> -			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
> -			if (!strcasecmp(var, sb.buf)) {
> -				e->fn(&keys, e->prefix);
> -				e->found++;
> -				break;
> -			}
> -		}
> -		strbuf_release(&sb);
> -		if (!e->prefix)
> -			string_list_append(&keys, var);
> -	}
> -
> -	for (e = slot_expansions; e->prefix; e++)
> -		if (!e->found)
> -			BUG("slot_expansion %s.%s is not used",
> -			    e->prefix, e->placeholder);
> -
> -	string_list_sort(&keys);
> -	for (i = 0; i < keys.nr; i++) {
> -		const char *var = keys.items[i].string;
> -		const char *wildcard, *tag, *cut;
> -
> -		if (for_human) {
> -			puts(var);
> -			continue;
> -		}
> -
> -		wildcard = strchr(var, '*');
> -		tag = strchr(var, '<');
> -
> -		if (!wildcard && !tag) {
> -			puts(var);
> -			continue;
> -		}
> -
> -		if (wildcard && !tag)
> -			cut = wildcard;
> -		else if (!wildcard && tag)
> -			cut = tag;
> -		else
> -			cut = wildcard < tag ? wildcard : tag;
> -
> -		/*
> -		 * We may produce duplicates, but that's up to
> -		 * git-completion.bash to handle
> -		 */
> -		printf("%.*s\n", (int)(cut - var), var);
> -	}
> -	string_list_clear(&keys, 0);
> -}
> -
>  static int get_alias(const char *var, const char *value, void *data)
>  {
>  	struct string_list *list = data;
> diff --git a/help.h b/help.h
> index 7a455beeb7..9071894e8c 100644
> --- a/help.h
> +++ b/help.h
> @@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
>  void list_common_cmds_help(void);
>  void list_all_cmds_help(void);
>  void list_common_guides_help(void);
> -void list_config_help(int for_human);
>  
>  void list_all_main_cmds(struct string_list *list);
>  void list_all_other_cmds(struct string_list *list);

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 03/15] bugreport: gather git version and build info
  2019-12-13  0:43 ` [PATCH v4 03/15] bugreport: gather git version and build info Emily Shaffer
@ 2019-12-13 21:06   ` Junio C Hamano
  2019-12-20  1:46     ` Emily Shaffer
  2019-12-17 18:45   ` Johannes Schindelin
  1 sibling, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:06 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> Knowing which version of Git a user has and how it was built allows us
> to more precisely pin down the circumstances when a certain issue
> occurs, so teach bugreport how to tell us the same output as 'git
> version --build-options'.
>
> It's not ideal to directly call 'git version --build-options' because
> that output goes to stdout. Instead, wrap the version string in a helper
> within help.[ch] library, and call that helper from within the bugreport
> library.

Move to strbuf() from stdio makes sense.  

> +	// add other contents

Style.

> diff --git a/help.h b/help.h
> index 9071894e8c..54f6b5f793 100644
> --- a/help.h
> +++ b/help.h
> @@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
>  void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
>  int is_in_cmdlist(struct cmdnames *cmds, const char *name);


Many new helpers are called get_frotz(), but only one among these is
called list_version_info().  I do not think the naming of the
get_*() ones, that are private to the bugreport tool, matters that
much, but unlike "list_commands()" whose purpose is to list the
commands ;-), the new function does not list versions---it just
gives information about a single version which is the one that is
being run, so perhaps it is a misnomer.

>  void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
> +void list_version_info(struct strbuf *buf, int build_options);

It is not clear to the readers build_options is a boolean that
tells the function to include (or not to include) build options.
Perhaps rename it to "int show_build_options" or something?


^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 05/15] bugreport: add uname info
  2019-12-13  0:43 ` [PATCH v4 05/15] bugreport: add uname info Emily Shaffer
@ 2019-12-13 21:12   ` Junio C Hamano
  2020-01-10  2:05   ` Aaron Schrab
  1 sibling, 0 replies; 177+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:12 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> The contents of uname() can give us some insight into what sort of
> system the user is running on, and help us replicate their setup if need
> be. The domainname field is not guaranteed to be available, so don't
> collect it.

It was surprising to me that we do use "struct utsname" somewhere in
the system ;-) ... so at least we know this is just as portable as
the remainder of Git, which is good.

> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  bugreport.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
>
> diff --git a/bugreport.c b/bugreport.c
> index 59d8b5a3af..9c69e3fa34 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -8,12 +8,25 @@
>  static void get_system_info(struct strbuf *sys_info)
>  {
>  	struct strbuf version_info = STRBUF_INIT;
> +	struct utsname uname_info;
>  
>  	/* get git version from native cmd */
>  	strbuf_addstr(sys_info, "git version:\n");
>  	list_version_info(&version_info, 1);
>  	strbuf_addbuf(sys_info, &version_info);
>  	strbuf_complete_line(sys_info);
> +
> +	/* system call for other version info */
> +	strbuf_addstr(sys_info, "uname -a: ");
> +	if (uname(&uname_info))
> +		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
> +	else
> +		strbuf_addf(sys_info, "%s %s %s %s %s\n",
> +			    uname_info.sysname,
> +			    uname_info.nodename,
> +			    uname_info.release,
> +			    uname_info.version,
> +			    uname_info.machine);
>  }
>  
>  static const char * const bugreport_usage[] = {

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 06/15] bugreport: add glibc version
  2019-12-13  0:43 ` [PATCH v4 06/15] bugreport: add glibc version Emily Shaffer
@ 2019-12-13 21:18   ` Junio C Hamano
  2019-12-16 22:39     ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:18 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> To help pinpoint the source of a regression, it is useful to know the
> version of libc which the user's Git client was built with. Let's
> include it alongside the other versioning information, which is used to
> identify how the client was built.
>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  bugreport.c | 5 +++++
>  1 file changed, 5 insertions(+)

This is the first iffy one in the series.  It is unreasonable to
assume that we can dictate that Git can be built only on glibc
systems, no?

Making this conditional, perhaps make "bugreport.c" depend on
"extern void get_compiler_info(struct strbuf *sys_info)" and require
the function to be defined in compat/; the glibc variant that uses
gnu_get_libc_version() would become just one of them.

> diff --git a/bugreport.c b/bugreport.c
> index 9c69e3fa34..af715dc157 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -4,6 +4,7 @@
>  #include "strbuf.h"
>  #include "time.h"
>  #include "help.h"
> +#include <gnu/libc-version.h>
>  
>  static void get_system_info(struct strbuf *sys_info)
>  {
> @@ -27,6 +28,10 @@ static void get_system_info(struct strbuf *sys_info)
>  			    uname_info.release,
>  			    uname_info.version,
>  			    uname_info.machine);
> +
> +	strbuf_addstr(sys_info, "glibc version: ");
> +	strbuf_addstr(sys_info, gnu_get_libc_version());
> +	strbuf_complete_line(sys_info);
>  }
>  
>  static const char * const bugreport_usage[] = {

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 07/15] bugreport: add curl version
  2019-12-13  0:43 ` [PATCH v4 07/15] bugreport: add curl version Emily Shaffer
@ 2019-12-13 21:27   ` Junio C Hamano
  2019-12-16 22:49     ` Emily Shaffer
  2019-12-17 18:47   ` Johannes Schindelin
  1 sibling, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:27 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> It's possible for git-http* to be built separately from git; in that
> case we want to know what version of cURL is used by git-http*, not
> necessarily which version was present at git-bugreport's build time.
> So instead, ask git-http-fetch for the version information it knows
> about.
>
> git-http-fetch was chosen as git-http-backend was described as a
> server-side implementation, and as an accidental fetch in case of
> problems was considered less harmful than an accidental push.
>
> Since it could have been built at a different time, also report the
> version and built-from commit of git-http-fetch alongside the cURL info.

One possible issue I have is that I was hoping that eventually we
can discard "git http-fetch" altogether sometime in the future.
Does anybody still use the dumb HTTP transport seriously?  

And the first move in that direction would be to allow the system be
built without http-fetch, even if git-remote-curl (and its aliases)
would still be built to access smart-http transports.

So, I am not sure.  This is just the matter of adding an out-of-line
hidden option used only for environment inspection, so if it can be
done to git-remote-curl, that would probably be much more future
proof.

> diff --git a/bugreport.c b/bugreport.c
> index af715dc157..f5598513d9 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -5,6 +5,18 @@
>  #include "time.h"
>  #include "help.h"
>  #include <gnu/libc-version.h>
> +#include "run-command.h"
> +
> +static void get_http_version_info(struct strbuf *http_info)
> +{
> +	struct child_process cp = CHILD_PROCESS_INIT;
> +
> +	argv_array_push(&cp.args, "git");
> +	argv_array_push(&cp.args, "http-fetch");
> +	argv_array_push(&cp.args, "-V");
> +	if (capture_command(&cp, http_info, 0))
> +	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");

OK.  We probably can also take the compile-time "NO_CURL" into account,
so that we can tell a misconfigured installation that wanted to have
CURL but failed to install a usable http-fetch and an installation
that deliberately omitted anything cURL?

>  static void get_system_info(struct strbuf *sys_info)
>  {
> @@ -32,6 +44,10 @@ static void get_system_info(struct strbuf *sys_info)
>  	strbuf_addstr(sys_info, "glibc version: ");
>  	strbuf_addstr(sys_info, gnu_get_libc_version());
>  	strbuf_complete_line(sys_info);
> +
> +	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
> +	get_http_version_info(sys_info);
> +	strbuf_complete_line(sys_info);
>  }
>  
>  static const char * const bugreport_usage[] = {
> diff --git a/http-fetch.c b/http-fetch.c
> index a32ac118d9..31844812a1 100644
> --- a/http-fetch.c
> +++ b/http-fetch.c
> @@ -3,9 +3,18 @@
>  #include "exec-cmd.h"
>  #include "http.h"
>  #include "walker.h"
> +#include "version.h"
>  
>  static const char http_fetch_usage[] = "git http-fetch "
> -"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
> +"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
> +
> +void NORETURN version_info()

void NORETURN version_info(void)


> +{
> +	printf("git-http-fetch version: %s\n", git_version_string);
> +	printf("built from commit: %s\n", git_built_from_commit_string);
> +	printf("curl version: %s\n", curl_version());
> +	exit(0);
> +}
>  
>  int cmd_main(int argc, const char **argv)
>  {
> @@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
>  		} else if (argv[arg][1] == 'a') {
>  		} else if (argv[arg][1] == 'v') {
>  			get_verbosely = 1;
> +		} else if (argv[arg][1] == 'V') {
> +			version_info();
>  		} else if (argv[arg][1] == 'w') {
>  			write_ref = &argv[arg + 1];
>  			arg++;

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 08/15] bugreport: include user interactive shell
  2019-12-13  0:43 ` [PATCH v4 08/15] bugreport: include user interactive shell Emily Shaffer
@ 2019-12-13 21:38   ` Junio C Hamano
  0 siblings, 0 replies; 177+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:38 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> It's possible a user may complain about the way that Git interacts with
> their interactive shell, e.g. autocompletion or shell prompt. In that
> case, it's useful for us to know which shell they're using
> interactively.
>
> $SHELL isn't set by every shell, but getenv() returns NULL in the
> event the variable isn't found, so we'll see a line like "$SHELL:
> (null)" to tell us that variable wasn't set.

Do not depend on vsnprintf() from glibc that accepts NULL and shows
"(null)".  You can segfault on systems without glibc that way.

> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  bugreport.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/bugreport.c b/bugreport.c
> index f5598513d9..759cc0b0f8 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -45,6 +45,9 @@ static void get_system_info(struct strbuf *sys_info)
>  	strbuf_addstr(sys_info, gnu_get_libc_version());
>  	strbuf_complete_line(sys_info);
>  
> +	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
> +		    getenv("SHELL"));
> +
>  	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
>  	get_http_version_info(sys_info);
>  	strbuf_complete_line(sys_info);

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 10/15] bugreport: add config values from safelist
  2019-12-13  0:43 ` [PATCH v4 10/15] bugreport: add config values from safelist Emily Shaffer
@ 2019-12-13 21:45   ` Junio C Hamano
  2019-12-16 23:40     ` Emily Shaffer
  2019-12-29 20:17   ` Johannes Schindelin
  1 sibling, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:45 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> Teach bugreport to gather the values of config options which are present
> in 'bugreport-config-safelist.h'.
>
> Many config options are sensitive, and many Git add-ons use config
> options which git-core does not know about; it is better only to gather
> config options which we know to be safe, rather than excluding options
> which we know to be unsafe.
>
> Taking the build-time generated array and putting it into a set saves us
> time - since git_config_bugreport() is called for every option the user
> has configured, performing option lookup in constant time is a useful
> optimization.

Interesting.  I actually was expecting the look-up to go the other
way around.  We know the safe keys, so iterate over them and grab
their values, if defined.  No need for hashes or anything, but just
a simple linear list of safe stuff.

But that is too simple-minded.  If we wanted to safelist foo.*.bar,
where '*' can be anything, walking on the list of safe variables
would not work.  We must have a hash table that knows "foo.*.bar" is
allowed, and while walking all the configuration keys, when we see
foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if
it is whitelisted, or something like that.

But then I am not sure if this implementation does something like
this for three-level names.  If not, I do not see much point in use
of the hash there either.

Puzzled.

>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  bugreport.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 71 insertions(+), 1 deletion(-)
>
> diff --git a/bugreport.c b/bugreport.c
> index 759cc0b0f8..1fca28f0b9 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -6,6 +6,9 @@
>  #include "help.h"
>  #include <gnu/libc-version.h>
>  #include "run-command.h"
> +#include "config.h"
> +#include "bugreport-config-safelist.h"
> +#include "khash.h"
>  
>  static void get_http_version_info(struct strbuf *http_info)
>  {
> @@ -18,6 +21,41 @@ static void get_http_version_info(struct strbuf *http_info)
>  	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
>  }
>  
> +KHASH_INIT(cfg_set, const char*, int, 0, kh_str_hash_func, kh_str_hash_equal);
> +
> +struct cfgset {
> +	kh_cfg_set_t set;
> +};
> +
> +struct cfgset safelist;
> +
> +static void cfgset_init(struct cfgset *set, size_t initial_size)
> +{
> +	memset(&set->set, 0, sizeof(set->set));
> +	if (initial_size)
> +		kh_resize_cfg_set(&set->set, initial_size);
> +}
> +
> +static int cfgset_insert(struct cfgset *set, const char *cfg_key)
> +{
> +	int added;
> +	kh_put_cfg_set(&set->set, cfg_key, &added);
> +	printf("ESS: added %s\n", cfg_key);
> +	return !added;
> +}
> +
> +static int cfgset_contains(struct cfgset *set, const char *cfg_key)
> +{
> +	khiter_t pos = kh_get_cfg_set(&set->set, cfg_key);
> +	return pos != kh_end(&set->set);
> +}
> +
> +static void cfgset_clear(struct cfgset *set)
> +{
> +	kh_release_cfg_set(&set->set);
> +	cfgset_init(set, 0);
> +}
> +
>  static void get_system_info(struct strbuf *sys_info)
>  {
>  	struct strbuf version_info = STRBUF_INIT;
> @@ -53,6 +91,36 @@ static void get_system_info(struct strbuf *sys_info)
>  	strbuf_complete_line(sys_info);
>  }
>  
> +static void gather_safelist()
> +{
> +	int index;
> +	int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
> +	cfgset_init(&safelist, safelist_len);
> +	for (index = 0; index < safelist_len; index++)
> +		cfgset_insert(&safelist, bugreport_config_safelist[index]);
> +
> +}
> +
> +static int git_config_bugreport(const char *var, const char *value, void *cb)
> +{
> +	struct strbuf *config_info = (struct strbuf *)cb;
> +
> +	if (cfgset_contains(&safelist, var))
> +		strbuf_addf(config_info,
> +			    "%s (%s) : %s\n",
> +			    var, config_scope_to_string(current_config_scope()),
> +			    value);
> +
> +	return 0;
> +}
> +
> +static void get_safelisted_config(struct strbuf *config_info)
> +{
> +	gather_safelist();
> +	git_config(git_config_bugreport, config_info);
> +	cfgset_clear(&safelist);
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output <file>]"),
>  	NULL
> @@ -114,10 +182,12 @@ int cmd_main(int argc, const char **argv)
>  
>  	get_bug_template(&buffer);
>  
> -	// add other contents
>  	get_header(&buffer, "System Info");
>  	get_system_info(&buffer);
>  
> +	get_header(&buffer, "Safelisted Config Info");
> +	get_safelisted_config(&buffer);
> +
>  	report = fopen_for_writing(report_path.buf);
>  	strbuf_write(&buffer, report);
>  	fclose(report);

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 11/15] bugreport: collect list of populated hooks
  2019-12-13  0:43 ` [PATCH v4 11/15] bugreport: collect list of populated hooks Emily Shaffer
@ 2019-12-13 21:47   ` Junio C Hamano
  2019-12-16 23:51     ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:47 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +	/*
> +	 * Doesn't look like there is a list of all possible hooks; so below is
> +	 * a transcription of `git help hook`.
> +	 */

We probably would want to employ technique similar to what you used
for the list of safe configuration we saw in an earlier patch in the
series.  What's not even documented is not worth reporting ;-)

> +	const char *hooks = "applypatch-msg,"
> +			    "pre-applypatch,"
> +			    "post-applypatch,"
> +			    "pre-commit,"
> +			    "pre-merge-commit,"
> +			    "prepare-commit-msg,"
> +			    "commit-msg,"
> +			    "post-commit,"
> +			    "pre-rebase,"
> +			    "post-checkout,"
> +			    "post-merge,"
> +			    "pre-push,"
> +			    "pre-receive,"
> +			    "update,"
> +			    "post-receive,"
> +			    "post-update,"
> +			    "push-to-checkout,"
> +			    "pre-auto-gc,"
> +			    "post-rewrite,"
> +			    "sendemail-validate,"
> +			    "fsmonitor-watchman,"
> +			    "p4-pre-submit,"
> +			    "post-index-changex";
> +	struct string_list hooks_list = STRING_LIST_INIT_DUP;
> +	struct string_list_item *iter = NULL;
> +	int nongit_ok;
> +
> +	setup_git_directory_gently(&nongit_ok);
> +
> +	if (nongit_ok) {
> +		strbuf_addstr(hook_info,
> +			"not run from a git repository - no hooks to show\n");
> +		return;
> +	}
> +
> +	string_list_split(&hooks_list, hooks, ',', -1);
> +
> +	for_each_string_list_item(iter, &hooks_list) {
> +		if (find_hook(iter->string)) {
> +			strbuf_addstr(hook_info, iter->string);
> +			strbuf_complete_line(hook_info);
> +		}
> +	}
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output <file>]"),
>  	NULL
> @@ -188,6 +240,9 @@ int cmd_main(int argc, const char **argv)
>  	get_header(&buffer, "Safelisted Config Info");
>  	get_safelisted_config(&buffer);
>  
> +	get_header(&buffer, "Configured Hooks");
> +	get_populated_hooks(&buffer);
> +
>  	report = fopen_for_writing(report_path.buf);
>  	strbuf_write(&buffer, report);
>  	fclose(report);

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 12/15] bugreport: count loose objects
  2019-12-13  0:43 ` [PATCH v4 12/15] bugreport: count loose objects Emily Shaffer
@ 2019-12-13 21:51   ` Junio C Hamano
  2019-12-16 23:54     ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:51 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git, Johannes Schindelin

Emily Shaffer <emilyshaffer@google.com> writes:

> The number of unpacked objects in a user's repository may help us
> understand the root of the problem they're seeing, especially if a
> command is running unusually slowly.
>
> Rather than directly invoking 'git-count-objects', which may sometimes
> fail unexpectedly on Git for Windows, manually count the contents of
> .git/objects.

Is for_each_loose_object() or for_each_loose_file_in_objdir() not
sufficient?  We really do *not* want a redundant implementation of
something we already use elsewhere in the system, especially for the
purpose of this program, because you would end up reporting what you
computed in a way that may be quite different from what the rest of
the system that is actually used by the end users computes.


^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 13/15] bugreport: add packed object summary
  2019-12-13  0:43 ` [PATCH v4 13/15] bugreport: add packed object summary Emily Shaffer
@ 2019-12-13 21:56   ` Junio C Hamano
  2019-12-16 23:56     ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2019-12-13 21:56 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> Alongside the list of loose objects, it's useful to see the list of
> object packs as well. It can help us to examine what Git did and did not
> pack.

Not exactly the same comment, but is in the same spirit, as the
previous step applies to this one.

Would it be too much work to libify the bulk of cmd_count_objects()
that computes numbers, making the cmd_count_objects() into a thin
wrapper that calls the libified "counter/collector" function and
prints the resulting numbers received from it?

That way, the get_packed_object_summary() can be another consumer       
of the same "counter/collector" function, no?

> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  bugreport.c | 31 +++++++++++++++++++++++++++++++
>  1 file changed, 31 insertions(+)
>
> diff --git a/bugreport.c b/bugreport.c
> index 3abb83d77f..992d8f9de7 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -234,6 +234,34 @@ static void get_loose_object_summary(struct strbuf *obj_info) {
>  	strbuf_release(&dirpath);
>  }
>  
> +static void get_packed_object_summary(struct strbuf *obj_info)
> +{
> +	struct strbuf dirpath = STRBUF_INIT;
> +	struct dirent *d;
> +	DIR *dir = NULL;
> +
> +	strbuf_addstr(&dirpath, get_object_directory());
> +	strbuf_complete(&dirpath, '/');
> +	strbuf_addstr(&dirpath, "pack/");
> +
> +	dir = opendir(dirpath.buf);
> +	if (!dir) {
> +		strbuf_addf(obj_info, "could not open packed object directory '%s'\n",
> +			    dirpath.buf);
> +		strbuf_release(&dirpath);
> +		return;
> +	}
> +
> +	while ((d = readdir(dir))) {
> +		strbuf_addbuf(obj_info, &dirpath);
> +		strbuf_addstr(obj_info, d->d_name);
> +		strbuf_complete_line(obj_info);
> +	}
> +
> +	closedir(dir);
> +	strbuf_release(&dirpath);
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output <file>]"),
>  	NULL
> @@ -307,6 +335,9 @@ int cmd_main(int argc, const char **argv)
>  	get_header(&buffer, "Loose Object Counts");
>  	get_loose_object_summary(&buffer);
>  
> +	get_header(&buffer, "Packed Object Summary");
> +	get_packed_object_summary(&buffer);
> +
>  	report = fopen_for_writing(report_path.buf);
>  	strbuf_write(&buffer, report);
>  	fclose(report);

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
  2019-12-13  0:43 ` [PATCH v4 09/15] bugreport: generate config safelist based on docs Emily Shaffer
@ 2019-12-13 22:57   ` Junio C Hamano
  2019-12-16 23:01     ` Emily Shaffer
  2019-12-15 20:17   ` Johannes Schindelin
  2019-12-17 18:38   ` Johannes Schindelin
  2 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2019-12-13 22:57 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git, Martin Ågren, Johannes Schindelin

Emily Shaffer <emilyshaffer@google.com> writes:

> diff --git a/Makefile b/Makefile
> index c49f55a521..76dc51e2b1 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
>  install-python-script: $(SCRIPT_PYTHON_GEN)
>  	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
>  
> -.PHONY: clean-perl-script clean-sh-script clean-python-script
> +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
>  clean-sh-script:
>  	$(RM) $(SCRIPT_SH_GEN)
>  clean-perl-script:
> @@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
>  
>  GENERATED_H += config-list.h
>  GENERATED_H += command-list.h
> +GENERATED_H += bugreport-config-safelist.h

OK.

>  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
>  	$(FIND) . \
> @@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
>  		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
>  		command-list.txt >$@+ && mv $@+ $@
>  
> +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
> +
> +bugreport-config-safelist.h: Documentation/config/*.txt
> +	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
> +		>$@+ && mv $@+ $@

OK.

>  SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
>  	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
>  	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\

But bugreport.o needs this *.h file generated before a compiler
attempts to produce it out of bugreport.c; that dependency is
missing from this update to the Makefile.

> @@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
>  .PHONY: sparse $(SP_OBJ)
>  sparse: $(SP_OBJ)
>  
> -GEN_HDRS := config-list.h command-list.h unicode-width.h
> +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
>  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
>  ifndef GCRYPT_SHA256
>  	EXCEPT_HDRS += sha256/gcrypt.h
> @@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
>  	$(RM) $(HCC)
>  	$(RM) -r bin-wrappers $(dep_dirs)
>  	$(RM) -r po/build/
> -	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> +	$(RM) *.pyc *.pyo */*.pyc */*.pyo
> +	$(RM) config-list.h command-list.h bugreport-config-safelist.h

It is kind of sad that GEN_HDRS defines the list of build artifact
that we should be able to clean, and then we manually list them to
be removed here independently.  Can we fix it up too?

We probably clean up the build/update procedure around unicode-width.h
before we can do so, probably.  It may be generatable using contrib/
script, but as far as our normal build is concerned, it is a tracked
source and not part of the build byproducts, so we probably would
want to remove it from GEN_HDRS.  When that happens, we can $(RM)
all of the $(GEN_HDRS) in the "clean" target.


^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
  2019-12-13  0:43 ` [PATCH v4 09/15] bugreport: generate config safelist based on docs Emily Shaffer
  2019-12-13 22:57   ` Junio C Hamano
@ 2019-12-15 20:17   ` Johannes Schindelin
  2019-12-16 22:52     ` Emily Shaffer
  2019-12-17 18:38   ` Johannes Schindelin
  2 siblings, 1 reply; 177+ messages in thread
From: Johannes Schindelin @ 2019-12-15 20:17 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git, Martin Ågren

Hi Emily,

On Thu, 12 Dec 2019, Emily Shaffer wrote:

> diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
> new file mode 100755
> index 0000000000..06b8e0c3c4
> --- /dev/null
> +++ b/generate-bugreport-config-safelist.sh
> @@ -0,0 +1,22 @@
> +#!/bin/sh
> +
> +cat <<EOF
> +/* Automatically generated by bugreport-generate-config-safelist.sh */
> +
> +
> +static const char *bugreport_config_safelist[] = {
> +EOF
> +
> +# cat all regular files in Documentation/config
> +find Documentation/config -type f -exec cat {} \; |
> +# print the command name which matches the bugreport-include macro
> +sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |

If you use `/  "\1",/` as replacement, you can totally avoid that ugly
`while` loop (that is unfortunately quite slow in MSYS2/Cygwin). You can
still pipe the result into `sort` just the same.

Ciao,
Dscho

> +sort |
> +while read line
> +do
> +	echo "	\"$line\","
> +done
> +
> +cat <<EOF
> +};
> +EOF
> --
> 2.24.1.735.g03f4e72817-goog
>
>

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 02/15] help: move list_config_help to builtin/help
  2019-12-13 20:51   ` Junio C Hamano
@ 2019-12-16 21:36     ` Emily Shaffer
  2019-12-16 22:19       ` Junio C Hamano
  0 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2019-12-16 21:36 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Dec 13, 2019 at 12:51:58PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o,
> > builtin/clean.o, and builtin/config.o. This meant that help.o was
> > unusable outside of the context of the main Git executable.
> >
> > To make help.o usable by other commands again, move list_config_help()
> > into builtin/help.c (where it makes sense to assume other builtin libraries
> > are present).
> >
> > When command-list.h is included but a member is not used, we start to
> > hear a compiler warning. Since the config list is generated in a fairly
> > different way than the command list, and since commands and config
> > options are semantically different, move the config list into its own
> > header and move the generator into its own script and build rule.
> >
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> 
> OK.  Looks like a clean-up that is very worthy even without the
> remainder of the topic.
> 
> Thanks.

Ok. Are you going to take it on its own? If so, I'll be relying on two
branches in pu/ - what's the best way to let you know I need them both
if/when I send another version of this chain?

> 
> >  .gitignore             |  1 +
> >  Makefile               | 16 ++++++--
> >  builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
> >  generate-cmdlist.sh    | 19 ----------
> >  generate-configlist.sh | 24 ++++++++++++
> >  help.c                 | 85 -----------------------------------------
> >  help.h                 |  1 -
> >  7 files changed, 123 insertions(+), 109 deletions(-)
> >  create mode 100755 generate-configlist.sh
> >
> > diff --git a/.gitignore b/.gitignore
> > index 055a84c4a8..5dde2cc4c8 100644
> > --- a/.gitignore
> > +++ b/.gitignore
> > @@ -189,6 +189,7 @@
> >  /gitweb/gitweb.cgi
> >  /gitweb/static/gitweb.js
> >  /gitweb/static/gitweb.min.*
> > +/config-list.h
> >  /command-list.h
> >  *.tar.gz
> >  *.dsc
> > diff --git a/Makefile b/Makefile
> > index 9dff91436e..c49f55a521 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -815,6 +815,7 @@ LIB_FILE = libgit.a
> >  XDIFF_LIB = xdiff/lib.a
> >  VCSSVN_LIB = vcs-svn/lib.a
> >  
> > +GENERATED_H += config-list.h
> >  GENERATED_H += command-list.h
> >  
> >  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
> > @@ -2127,7 +2128,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
> >  
> >  help.sp help.s help.o: command-list.h
> >  
> > -builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
> > +builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
> >  builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
> >  	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
> >  	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
> > @@ -2147,6 +2148,12 @@ $(BUILT_INS): git$X
> >  	ln -s $< $@ 2>/dev/null || \
> >  	cp $< $@
> >  
> > +config-list.h: generate-configlist.sh
> > +
> > +config-list.h:
> > +	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
> > +		>$@+ && mv $@+ $@
> > +
> >  command-list.h: generate-cmdlist.sh command-list.txt
> >  
> >  command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
> > @@ -2784,7 +2791,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
> >  .PHONY: sparse $(SP_OBJ)
> >  sparse: $(SP_OBJ)
> >  
> > -GEN_HDRS := command-list.h unicode-width.h
> > +GEN_HDRS := config-list.h command-list.h unicode-width.h
> >  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
> >  ifndef GCRYPT_SHA256
> >  	EXCEPT_HDRS += sha256/gcrypt.h
> > @@ -2807,7 +2814,7 @@ hdr-check: $(HCO)
> >  style:
> >  	git clang-format --style file --diff --extensions c,h
> >  
> > -check: command-list.h
> > +check: config-list.h command-list.h
> >  	@if sparse; \
> >  	then \
> >  		echo >&2 "Use 'make sparse' instead"; \
> > @@ -3110,7 +3117,8 @@ clean: profile-clean coverage-clean cocciclean
> >  	$(RM) $(HCC)
> >  	$(RM) -r bin-wrappers $(dep_dirs)
> >  	$(RM) -r po/build/
> > -	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
> > +	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> > +	$(RM) $(ETAGS_TARGET) tags cscope*
> >  	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
> >  	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
> >  	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
> > diff --git a/builtin/help.c b/builtin/help.c
> > index e5590d7787..1c5f2b9255 100644
> > --- a/builtin/help.c
> > +++ b/builtin/help.c
> > @@ -8,6 +8,7 @@
> >  #include "parse-options.h"
> >  #include "run-command.h"
> >  #include "column.h"
> > +#include "config-list.h"
> >  #include "help.h"
> >  #include "alias.h"
> >  
> > @@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
> >  	NULL
> >  };
> >  
> > +struct slot_expansion {
> > +	const char *prefix;
> > +	const char *placeholder;
> > +	void (*fn)(struct string_list *list, const char *prefix);
> > +	int found;
> > +};
> > +
> > +static void list_config_help(int for_human)
> > +{
> > +	struct slot_expansion slot_expansions[] = {
> > +		{ "advice", "*", list_config_advices },
> > +		{ "color.branch", "<slot>", list_config_color_branch_slots },
> > +		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
> > +		{ "color.diff", "<slot>", list_config_color_diff_slots },
> > +		{ "color.grep", "<slot>", list_config_color_grep_slots },
> > +		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
> > +		{ "color.remote", "<slot>", list_config_color_sideband_slots },
> > +		{ "color.status", "<slot>", list_config_color_status_slots },
> > +		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
> > +		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
> > +		{ NULL, NULL, NULL }
> > +	};
> > +	const char **p;
> > +	struct slot_expansion *e;
> > +	struct string_list keys = STRING_LIST_INIT_DUP;
> > +	int i;
> > +
> > +	for (p = config_name_list; *p; p++) {
> > +		const char *var = *p;
> > +		struct strbuf sb = STRBUF_INIT;
> > +
> > +		for (e = slot_expansions; e->prefix; e++) {
> > +
> > +			strbuf_reset(&sb);
> > +			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
> > +			if (!strcasecmp(var, sb.buf)) {
> > +				e->fn(&keys, e->prefix);
> > +				e->found++;
> > +				break;
> > +			}
> > +		}
> > +		strbuf_release(&sb);
> > +		if (!e->prefix)
> > +			string_list_append(&keys, var);
> > +	}
> > +
> > +	for (e = slot_expansions; e->prefix; e++)
> > +		if (!e->found)
> > +			BUG("slot_expansion %s.%s is not used",
> > +			    e->prefix, e->placeholder);
> > +
> > +	string_list_sort(&keys);
> > +	for (i = 0; i < keys.nr; i++) {
> > +		const char *var = keys.items[i].string;
> > +		const char *wildcard, *tag, *cut;
> > +
> > +		if (for_human) {
> > +			puts(var);
> > +			continue;
> > +		}
> > +
> > +		wildcard = strchr(var, '*');
> > +		tag = strchr(var, '<');
> > +
> > +		if (!wildcard && !tag) {
> > +			puts(var);
> > +			continue;
> > +		}
> > +
> > +		if (wildcard && !tag)
> > +			cut = wildcard;
> > +		else if (!wildcard && tag)
> > +			cut = tag;
> > +		else
> > +			cut = wildcard < tag ? wildcard : tag;
> > +
> > +		/*
> > +		 * We may produce duplicates, but that's up to
> > +		 * git-completion.bash to handle
> > +		 */
> > +		printf("%.*s\n", (int)(cut - var), var);
> > +	}
> > +	string_list_clear(&keys, 0);
> > +}
> > +
> >  static enum help_format parse_help_format(const char *format)
> >  {
> >  	if (!strcmp(format, "man"))
> > diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> > index 71158f7d8b..45fecf8bdf 100755
> > --- a/generate-cmdlist.sh
> > +++ b/generate-cmdlist.sh
> > @@ -76,23 +76,6 @@ print_command_list () {
> >  	echo "};"
> >  }
> >  
> > -print_config_list () {
> > -	cat <<EOF
> > -static const char *config_name_list[] = {
> > -EOF
> > -	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> > -	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> > -	sort |
> > -	while read line
> > -	do
> > -		echo "	\"$line\","
> > -	done
> > -	cat <<EOF
> > -	NULL,
> > -};
> > -EOF
> > -}
> > -
> >  exclude_programs=
> >  while test "--exclude-program" = "$1"
> >  do
> > @@ -113,5 +96,3 @@ echo
> >  define_category_names "$1"
> >  echo
> >  print_command_list "$1"
> > -echo
> > -print_config_list
> > diff --git a/generate-configlist.sh b/generate-configlist.sh
> > new file mode 100755
> > index 0000000000..eca6a00c30
> > --- /dev/null
> > +++ b/generate-configlist.sh
> > @@ -0,0 +1,24 @@
> > +#!/bin/sh
> > +
> > +echo "/* Automatically generated by generate-configlist.sh */"
> > +echo
> > +
> > +print_config_list () {
> > +	cat <<EOF
> > +static const char *config_name_list[] = {
> > +EOF
> > +	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> > +	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> > +	sort |
> > +	while read line
> > +	do
> > +		echo "	\"$line\","
> > +	done
> > +	cat <<EOF
> > +	NULL,
> > +};
> > +EOF
> > +}
> > +
> > +echo
> > +print_config_list
> > diff --git a/help.c b/help.c
> > index cf67624a94..a21487db77 100644
> > --- a/help.c
> > +++ b/help.c
> > @@ -407,91 +407,6 @@ void list_common_guides_help(void)
> >  	putchar('\n');
> >  }
> >  
> > -struct slot_expansion {
> > -	const char *prefix;
> > -	const char *placeholder;
> > -	void (*fn)(struct string_list *list, const char *prefix);
> > -	int found;
> > -};
> > -
> > -void list_config_help(int for_human)
> > -{
> > -	struct slot_expansion slot_expansions[] = {
> > -		{ "advice", "*", list_config_advices },
> > -		{ "color.branch", "<slot>", list_config_color_branch_slots },
> > -		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
> > -		{ "color.diff", "<slot>", list_config_color_diff_slots },
> > -		{ "color.grep", "<slot>", list_config_color_grep_slots },
> > -		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
> > -		{ "color.remote", "<slot>", list_config_color_sideband_slots },
> > -		{ "color.status", "<slot>", list_config_color_status_slots },
> > -		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
> > -		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
> > -		{ NULL, NULL, NULL }
> > -	};
> > -	const char **p;
> > -	struct slot_expansion *e;
> > -	struct string_list keys = STRING_LIST_INIT_DUP;
> > -	int i;
> > -
> > -	for (p = config_name_list; *p; p++) {
> > -		const char *var = *p;
> > -		struct strbuf sb = STRBUF_INIT;
> > -
> > -		for (e = slot_expansions; e->prefix; e++) {
> > -
> > -			strbuf_reset(&sb);
> > -			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
> > -			if (!strcasecmp(var, sb.buf)) {
> > -				e->fn(&keys, e->prefix);
> > -				e->found++;
> > -				break;
> > -			}
> > -		}
> > -		strbuf_release(&sb);
> > -		if (!e->prefix)
> > -			string_list_append(&keys, var);
> > -	}
> > -
> > -	for (e = slot_expansions; e->prefix; e++)
> > -		if (!e->found)
> > -			BUG("slot_expansion %s.%s is not used",
> > -			    e->prefix, e->placeholder);
> > -
> > -	string_list_sort(&keys);
> > -	for (i = 0; i < keys.nr; i++) {
> > -		const char *var = keys.items[i].string;
> > -		const char *wildcard, *tag, *cut;
> > -
> > -		if (for_human) {
> > -			puts(var);
> > -			continue;
> > -		}
> > -
> > -		wildcard = strchr(var, '*');
> > -		tag = strchr(var, '<');
> > -
> > -		if (!wildcard && !tag) {
> > -			puts(var);
> > -			continue;
> > -		}
> > -
> > -		if (wildcard && !tag)
> > -			cut = wildcard;
> > -		else if (!wildcard && tag)
> > -			cut = tag;
> > -		else
> > -			cut = wildcard < tag ? wildcard : tag;
> > -
> > -		/*
> > -		 * We may produce duplicates, but that's up to
> > -		 * git-completion.bash to handle
> > -		 */
> > -		printf("%.*s\n", (int)(cut - var), var);
> > -	}
> > -	string_list_clear(&keys, 0);
> > -}
> > -
> >  static int get_alias(const char *var, const char *value, void *data)
> >  {
> >  	struct string_list *list = data;
> > diff --git a/help.h b/help.h
> > index 7a455beeb7..9071894e8c 100644
> > --- a/help.h
> > +++ b/help.h
> > @@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
> >  void list_common_cmds_help(void);
> >  void list_all_cmds_help(void);
> >  void list_common_guides_help(void);
> > -void list_config_help(int for_human);
> >  
> >  void list_all_main_cmds(struct string_list *list);
> >  void list_all_other_cmds(struct string_list *list);

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 02/15] help: move list_config_help to builtin/help
  2019-12-16 21:36     ` Emily Shaffer
@ 2019-12-16 22:19       ` Junio C Hamano
  2019-12-16 22:34         ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2019-12-16 22:19 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

>> OK.  Looks like a clean-up that is very worthy even without the
>> remainder of the topic.
>> 
>> Thanks.
>
> Ok. Are you going to take it on its own?

Not really.  Unless you abandon the rest of the topic, in which case
it may be worth resurrecting this one on its own, that is.  But that
is not what you plan to do anyway, so I expect to see it as part of
an updated series ;-)

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 02/15] help: move list_config_help to builtin/help
  2019-12-16 22:19       ` Junio C Hamano
@ 2019-12-16 22:34         ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-16 22:34 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Mon, Dec 16, 2019 at 02:19:48PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> >> OK.  Looks like a clean-up that is very worthy even without the
> >> remainder of the topic.
> >> 
> >> Thanks.
> >
> > Ok. Are you going to take it on its own?
> 
> Not really.  Unless you abandon the rest of the topic, in which case
> it may be worth resurrecting this one on its own, that is.  But that
> is not what you plan to do anyway, so I expect to see it as part of
> an updated series ;-)

Ok! That's fine and makes my life easier. Just making sure. :)

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 06/15] bugreport: add glibc version
  2019-12-13 21:18   ` Junio C Hamano
@ 2019-12-16 22:39     ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-16 22:39 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Dec 13, 2019 at 01:18:44PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > To help pinpoint the source of a regression, it is useful to know the
> > version of libc which the user's Git client was built with. Let's
> > include it alongside the other versioning information, which is used to
> > identify how the client was built.
> >
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> >  bugreport.c | 5 +++++
> >  1 file changed, 5 insertions(+)
> 
> This is the first iffy one in the series.  It is unreasonable to
> assume that we can dictate that Git can be built only on glibc
> systems, no?
> 
> Making this conditional, perhaps make "bugreport.c" depend on
> "extern void get_compiler_info(struct strbuf *sys_info)" and require
> the function to be defined in compat/; the glibc variant that uses
> gnu_get_libc_version() would become just one of them.

Interesting. This sounds like a good approach - thanks, I'll look into
it.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 07/15] bugreport: add curl version
  2019-12-13 21:27   ` Junio C Hamano
@ 2019-12-16 22:49     ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-16 22:49 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Dec 13, 2019 at 01:27:31PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > It's possible for git-http* to be built separately from git; in that
> > case we want to know what version of cURL is used by git-http*, not
> > necessarily which version was present at git-bugreport's build time.
> > So instead, ask git-http-fetch for the version information it knows
> > about.
> >
> > git-http-fetch was chosen as git-http-backend was described as a
> > server-side implementation, and as an accidental fetch in case of
> > problems was considered less harmful than an accidental push.
> >
> > Since it could have been built at a different time, also report the
> > version and built-from commit of git-http-fetch alongside the cURL info.
> 
> One possible issue I have is that I was hoping that eventually we
> can discard "git http-fetch" altogether sometime in the future.
> Does anybody still use the dumb HTTP transport seriously?  

Oh, interesting. I was about to say, "I still use it to fetch, when I
don't really care" - but that isn't even true, as I fetch via https (and
have so much muscle memory to type https that I don't even notice).`

> 
> And the first move in that direction would be to allow the system be
> built without http-fetch, even if git-remote-curl (and its aliases)
> would still be built to access smart-http transports.
> 
> So, I am not sure.  This is just the matter of adding an out-of-line
> hidden option used only for environment inspection, so if it can be
> done to git-remote-curl, that would probably be much more future
> proof.

Ok. I'll move it.

> 
> > diff --git a/bugreport.c b/bugreport.c
> > index af715dc157..f5598513d9 100644
> > --- a/bugreport.c
> > +++ b/bugreport.c
> > @@ -5,6 +5,18 @@
> >  #include "time.h"
> >  #include "help.h"
> >  #include <gnu/libc-version.h>
> > +#include "run-command.h"
> > +
> > +static void get_http_version_info(struct strbuf *http_info)
> > +{
> > +	struct child_process cp = CHILD_PROCESS_INIT;
> > +
> > +	argv_array_push(&cp.args, "git");
> > +	argv_array_push(&cp.args, "http-fetch");
> > +	argv_array_push(&cp.args, "-V");
> > +	if (capture_command(&cp, http_info, 0))
> > +	    strbuf_addstr(http_info, "'git-http-fetch -V' not supported\n");
> 
> OK.  We probably can also take the compile-time "NO_CURL" into account,
> so that we can tell a misconfigured installation that wanted to have
> CURL but failed to install a usable http-fetch and an installation
> that deliberately omitted anything cURL?

Oh, interesting idea! I'll add that.

> 
> >  static void get_system_info(struct strbuf *sys_info)
> >  {
> > @@ -32,6 +44,10 @@ static void get_system_info(struct strbuf *sys_info)
> >  	strbuf_addstr(sys_info, "glibc version: ");
> >  	strbuf_addstr(sys_info, gnu_get_libc_version());
> >  	strbuf_complete_line(sys_info);
> > +
> > +	strbuf_addstr(sys_info, "git-http-fetch -V:\n");
> > +	get_http_version_info(sys_info);
> > +	strbuf_complete_line(sys_info);
> >  }
> >  
> >  static const char * const bugreport_usage[] = {
> > diff --git a/http-fetch.c b/http-fetch.c
> > index a32ac118d9..31844812a1 100644
> > --- a/http-fetch.c
> > +++ b/http-fetch.c
> > @@ -3,9 +3,18 @@
> >  #include "exec-cmd.h"
> >  #include "http.h"
> >  #include "walker.h"
> > +#include "version.h"
> >  
> >  static const char http_fetch_usage[] = "git http-fetch "
> > -"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
> > +"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
> > +
> > +void NORETURN version_info()
> 
> void NORETURN version_info(void)
> 
> 
> > +{
> > +	printf("git-http-fetch version: %s\n", git_version_string);
> > +	printf("built from commit: %s\n", git_built_from_commit_string);
> > +	printf("curl version: %s\n", curl_version());
> > +	exit(0);
> > +}
> >  
> >  int cmd_main(int argc, const char **argv)
> >  {
> > @@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
> >  		} else if (argv[arg][1] == 'a') {
> >  		} else if (argv[arg][1] == 'v') {
> >  			get_verbosely = 1;
> > +		} else if (argv[arg][1] == 'V') {
> > +			version_info();
> >  		} else if (argv[arg][1] == 'w') {
> >  			write_ref = &argv[arg + 1];
> >  			arg++;

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
  2019-12-15 20:17   ` Johannes Schindelin
@ 2019-12-16 22:52     ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-16 22:52 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git, Martin Ågren

On Sun, Dec 15, 2019 at 09:17:57PM +0100, Johannes Schindelin wrote:
> Hi Emily,
> 
> On Thu, 12 Dec 2019, Emily Shaffer wrote:
> 
> > diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
> > new file mode 100755
> > index 0000000000..06b8e0c3c4
> > --- /dev/null
> > +++ b/generate-bugreport-config-safelist.sh
> > @@ -0,0 +1,22 @@
> > +#!/bin/sh
> > +
> > +cat <<EOF
> > +/* Automatically generated by bugreport-generate-config-safelist.sh */
> > +
> > +
> > +static const char *bugreport_config_safelist[] = {
> > +EOF
> > +
> > +# cat all regular files in Documentation/config
> > +find Documentation/config -type f -exec cat {} \; |
> > +# print the command name which matches the bugreport-include macro
> > +sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |
> 
> If you use `/  "\1",/` as replacement, you can totally avoid that ugly
> `while` loop (that is unfortunately quite slow in MSYS2/Cygwin). You can
> still pipe the result into `sort` just the same.

Oooof. How embarrassing - I will do that.

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
  2019-12-13 22:57   ` Junio C Hamano
@ 2019-12-16 23:01     ` Emily Shaffer
  2019-12-17  0:41       ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2019-12-16 23:01 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Martin Ågren, Johannes Schindelin

On Fri, Dec 13, 2019 at 02:57:17PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > diff --git a/Makefile b/Makefile
> > index c49f55a521..76dc51e2b1 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
> >  install-python-script: $(SCRIPT_PYTHON_GEN)
> >  	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
> >  
> > -.PHONY: clean-perl-script clean-sh-script clean-python-script
> > +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
> >  clean-sh-script:
> >  	$(RM) $(SCRIPT_SH_GEN)
> >  clean-perl-script:
> > @@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
> >  
> >  GENERATED_H += config-list.h
> >  GENERATED_H += command-list.h
> > +GENERATED_H += bugreport-config-safelist.h
> 
> OK.
> 
> >  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
> >  	$(FIND) . \
> > @@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
> >  		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
> >  		command-list.txt >$@+ && mv $@+ $@
> >  
> > +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
> > +
> > +bugreport-config-safelist.h: Documentation/config/*.txt
> > +	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
> > +		>$@+ && mv $@+ $@
> 
> OK.
> 
> >  SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
> >  	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
> >  	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
> 
> But bugreport.o needs this *.h file generated before a compiler
> attempts to produce it out of bugreport.c; that dependency is
> missing from this update to the Makefile.

Hm. Somewhere I misformatted my commits, then. My goal was to add the
new header here, but not add it as a dependency to anybody; then, in the
next commit, add it as a dependency to git-bugreport and start to use it
(commit 1: make the thing; commit 2: use the thing). But now when I look
at the next commit, I don't see a Makefile change. So I missed something
while I was shuffling around fixups.  Thanks for noticing.

> 
> > @@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
> >  .PHONY: sparse $(SP_OBJ)
> >  sparse: $(SP_OBJ)
> >  
> > -GEN_HDRS := config-list.h command-list.h unicode-width.h
> > +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
> >  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
> >  ifndef GCRYPT_SHA256
> >  	EXCEPT_HDRS += sha256/gcrypt.h
> > @@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
> >  	$(RM) $(HCC)
> >  	$(RM) -r bin-wrappers $(dep_dirs)
> >  	$(RM) -r po/build/
> > -	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> > +	$(RM) *.pyc *.pyo */*.pyc */*.pyo
> > +	$(RM) config-list.h command-list.h bugreport-config-safelist.h
> 
> It is kind of sad that GEN_HDRS defines the list of build artifact
> that we should be able to clean, and then we manually list them to
> be removed here independently.  Can we fix it up too?

Yeah, I can do that. I thought so too when I was updating this.

I *think* what happened is that when I split config-list away (in the
earleir commit in this chain) I didn't notice that command-list.h was
already in GEN_HDRS, and instead just grepped Makefile for
"command-list.h" and added config-list.h next to it. So I'll try to fix
it much earlier, in that commit, and then simply add to the appropriate
variables in this commit.

> 
> We probably clean up the build/update procedure around unicode-width.h
> before we can do so, probably.  It may be generatable using contrib/
> script, but as far as our normal build is concerned, it is a tracked
> source and not part of the build byproducts, so we probably would
> want to remove it from GEN_HDRS.  When that happens, we can $(RM)
> all of the $(GEN_HDRS) in the "clean" target.

Noted. Thanks.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 10/15] bugreport: add config values from safelist
  2019-12-13 21:45   ` Junio C Hamano
@ 2019-12-16 23:40     ` Emily Shaffer
  2019-12-17 17:43       ` Junio C Hamano
  0 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2019-12-16 23:40 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Dec 13, 2019 at 01:45:47PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Teach bugreport to gather the values of config options which are present
> > in 'bugreport-config-safelist.h'.
> >
> > Many config options are sensitive, and many Git add-ons use config
> > options which git-core does not know about; it is better only to gather
> > config options which we know to be safe, rather than excluding options
> > which we know to be unsafe.
> >
> > Taking the build-time generated array and putting it into a set saves us
> > time - since git_config_bugreport() is called for every option the user
> > has configured, performing option lookup in constant time is a useful
> > optimization.
> 
> Interesting.  I actually was expecting the look-up to go the other
> way around.  We know the safe keys, so iterate over them and grab
> their values, if defined.  No need for hashes or anything, but just
> a simple linear list of safe stuff.

Hm. Without looking at the code, I said to myself,
"repo_config_get_value() will open all the available config files to
find the resolved value, so I don't want to do n*4 file open/closes, I
only want to do 4 total."

Now I look at the code and see that the configs are already being read
into a hashset before now. So you're right that it doesn't make sense
for me to do it this way....

> 
> But that is too simple-minded.  If we wanted to safelist foo.*.bar,
> where '*' can be anything, walking on the list of safe variables
> would not work.  We must have a hash table that knows "foo.*.bar" is
> allowed, and while walking all the configuration keys, when we see
> foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if
> it is whitelisted, or something like that.

...unless we want to use wildcards like you suggest.

But I'm not sure it's a good idea. I envision someone writing another
Git add-on, which offers someone to specify "user.password" and
automagically do some Bitbucket interaction, or something. (An extreme
and misguided example, but I hope the point remains.) Then if I say,
"all the user.* configs I can see in 'git help config' are safe, so I
will just safelist user.*," I'm not accounting for this other tool's
known configs which are a superset of what Git knows and expects.

The point of this safelist-generation exercise is to avoid accidentally
printing some sensitive user config, and the point of using a safelist
instead of a blocklist is to avoid printing a third-party config we
didn't know about (or messing up our pattern matching). So I suggest we
avoid pattern matching entirely.

> 
> But then I am not sure if this implementation does something like
> this for three-level names.  If not, I do not see much point in use
> of the hash there either.
> 
> Puzzled.

I'd prefer to solve it by iterating over the compile-time-generated
safelist using get_config_value_multi(), which I clock at O(n) for n =
safelist length: performing n calls to an O(1) hashset lookup. That
saves us the O(n*log(n)) hashset population, and the implementation of
Yet Another Set in the source.

Thanks.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 11/15] bugreport: collect list of populated hooks
  2019-12-13 21:47   ` Junio C Hamano
@ 2019-12-16 23:51     ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-16 23:51 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Dec 13, 2019 at 01:47:27PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > +	/*
> > +	 * Doesn't look like there is a list of all possible hooks; so below is
> > +	 * a transcription of `git help hook`.
> > +	 */
> 
> We probably would want to employ technique similar to what you used
> for the list of safe configuration we saw in an earlier patch in the
> series.  What's not even documented is not worth reporting ;-)

I somewhat slightly would rather not, since I hope to include a list of
the available hooks in my work with git-hook:
https://lore.kernel.org/20191211205114.GD107889@google.com

It's true that the way this patch is written now is prone to bitrot.
It's also true that I could win the lottery tomorrow and never finish my
hopes of git-hook implementation. And it's finally true that I don't
want to write a fourth build-time-generated header which scrapes docs,
especially if I also plan to delete it two months from now.

I wonder whether a better approach might be to drop hook support
entirely here, and add it later on when this particular bit of tech debt
(no canonical list of hook names) is solved, one way or another.

Another approach may be to just list all contents of $(core.hookdir)/
which doesn't end in ".sample", which is what the bash first attempt of
bugreport did. This does mean that those who have something like:

  .git/hooks/
    pre-commit
    pre-commit.d/

  pre-commit:
    sh pre-commit.d/a
    sh pre-commit.d/b

will have their multihook config exposed, although the contents of those
hooks won't be.

I'm interested to know everyone else's opinion, because mine is biased
by my hope that I can solve all the world's problems with git-hook ;)

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 12/15] bugreport: count loose objects
  2019-12-13 21:51   ` Junio C Hamano
@ 2019-12-16 23:54     ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-16 23:54 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Johannes Schindelin

On Fri, Dec 13, 2019 at 01:51:27PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > The number of unpacked objects in a user's repository may help us
> > understand the root of the problem they're seeing, especially if a
> > command is running unusually slowly.
> >
> > Rather than directly invoking 'git-count-objects', which may sometimes
> > fail unexpectedly on Git for Windows, manually count the contents of
> > .git/objects.
> 
> Is for_each_loose_object() or for_each_loose_file_in_objdir() not
> sufficient?  We really do *not* want a redundant implementation of
> something we already use elsewhere in the system, especially for the
> purpose of this program, because you would end up reporting what you
> computed in a way that may be quite different from what the rest of
> the system that is actually used by the end users computes.
> 
Oh, thanks for the pointer. That looks sufficient - I will investigate
the differences between the two and stop filesystem browsing by hand in
this case.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 13/15] bugreport: add packed object summary
  2019-12-13 21:56   ` Junio C Hamano
@ 2019-12-16 23:56     ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-16 23:56 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Dec 13, 2019 at 01:56:07PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Alongside the list of loose objects, it's useful to see the list of
> > object packs as well. It can help us to examine what Git did and did not
> > pack.
> 
> Not exactly the same comment, but is in the same spirit, as the
> previous step applies to this one.
> 
> Would it be too much work to libify the bulk of cmd_count_objects()
> that computes numbers, making the cmd_count_objects() into a thin
> wrapper that calls the libified "counter/collector" function and
> prints the resulting numbers received from it?
> 
> That way, the get_packed_object_summary() can be another consumer       
> of the same "counter/collector" function, no?

That sounds like a good approach. Will do.

> 
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> >  bugreport.c | 31 +++++++++++++++++++++++++++++++
> >  1 file changed, 31 insertions(+)
> >
> > diff --git a/bugreport.c b/bugreport.c
> > index 3abb83d77f..992d8f9de7 100644
> > --- a/bugreport.c
> > +++ b/bugreport.c
> > @@ -234,6 +234,34 @@ static void get_loose_object_summary(struct strbuf *obj_info) {
> >  	strbuf_release(&dirpath);
> >  }
> >  
> > +static void get_packed_object_summary(struct strbuf *obj_info)
> > +{
> > +	struct strbuf dirpath = STRBUF_INIT;
> > +	struct dirent *d;
> > +	DIR *dir = NULL;
> > +
> > +	strbuf_addstr(&dirpath, get_object_directory());
> > +	strbuf_complete(&dirpath, '/');
> > +	strbuf_addstr(&dirpath, "pack/");
> > +
> > +	dir = opendir(dirpath.buf);
> > +	if (!dir) {
> > +		strbuf_addf(obj_info, "could not open packed object directory '%s'\n",
> > +			    dirpath.buf);
> > +		strbuf_release(&dirpath);
> > +		return;
> > +	}
> > +
> > +	while ((d = readdir(dir))) {
> > +		strbuf_addbuf(obj_info, &dirpath);
> > +		strbuf_addstr(obj_info, d->d_name);
> > +		strbuf_complete_line(obj_info);
> > +	}
> > +
> > +	closedir(dir);
> > +	strbuf_release(&dirpath);
> > +}
> > +
> >  static const char * const bugreport_usage[] = {
> >  	N_("git bugreport [-o|--output <file>]"),
> >  	NULL
> > @@ -307,6 +335,9 @@ int cmd_main(int argc, const char **argv)
> >  	get_header(&buffer, "Loose Object Counts");
> >  	get_loose_object_summary(&buffer);
> >  
> > +	get_header(&buffer, "Packed Object Summary");
> > +	get_packed_object_summary(&buffer);
> > +
> >  	report = fopen_for_writing(report_path.buf);
> >  	strbuf_write(&buffer, report);
> >  	fclose(report);

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
  2019-12-16 23:01     ` Emily Shaffer
@ 2019-12-17  0:41       ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-17  0:41 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Martin Ågren, Johannes Schindelin

On Mon, Dec 16, 2019 at 03:01:24PM -0800, Emily Shaffer wrote:
> On Fri, Dec 13, 2019 at 02:57:17PM -0800, Junio C Hamano wrote:
> > Emily Shaffer <emilyshaffer@google.com> writes:
> > 
> > > diff --git a/Makefile b/Makefile
> > > index c49f55a521..76dc51e2b1 100644
> > > --- a/Makefile
> > > +++ b/Makefile
> > > @@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
> > >  install-python-script: $(SCRIPT_PYTHON_GEN)
> > >  	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
> > >  
> > > -.PHONY: clean-perl-script clean-sh-script clean-python-script
> > > +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
> > >  clean-sh-script:
> > >  	$(RM) $(SCRIPT_SH_GEN)
> > >  clean-perl-script:
> > > @@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
> > >  
> > >  GENERATED_H += config-list.h
> > >  GENERATED_H += command-list.h
> > > +GENERATED_H += bugreport-config-safelist.h
> > 
> > OK.
> > 
> > >  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
> > >  	$(FIND) . \
> > > @@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
> > >  		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
> > >  		command-list.txt >$@+ && mv $@+ $@
> > >  
> > > +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
> > > +
> > > +bugreport-config-safelist.h: Documentation/config/*.txt
> > > +	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
> > > +		>$@+ && mv $@+ $@
> > 
> > OK.
> > 
> > >  SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
> > >  	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
> > >  	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
> > 
> > But bugreport.o needs this *.h file generated before a compiler
> > attempts to produce it out of bugreport.c; that dependency is
> > missing from this update to the Makefile.
> 
> Hm. Somewhere I misformatted my commits, then. My goal was to add the
> new header here, but not add it as a dependency to anybody; then, in the
> next commit, add it as a dependency to git-bugreport and start to use it
> (commit 1: make the thing; commit 2: use the thing). But now when I look
> at the next commit, I don't see a Makefile change. So I missed something
> while I was shuffling around fixups.  Thanks for noticing.
> 
> > 
> > > @@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
> > >  .PHONY: sparse $(SP_OBJ)
> > >  sparse: $(SP_OBJ)
> > >  
> > > -GEN_HDRS := config-list.h command-list.h unicode-width.h
> > > +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
> > >  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
> > >  ifndef GCRYPT_SHA256
> > >  	EXCEPT_HDRS += sha256/gcrypt.h
> > > @@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
> > >  	$(RM) $(HCC)
> > >  	$(RM) -r bin-wrappers $(dep_dirs)
> > >  	$(RM) -r po/build/
> > > -	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> > > +	$(RM) *.pyc *.pyo */*.pyc */*.pyo
> > > +	$(RM) config-list.h command-list.h bugreport-config-safelist.h
> > 
> > It is kind of sad that GEN_HDRS defines the list of build artifact
> > that we should be able to clean, and then we manually list them to
> > be removed here independently.  Can we fix it up too?
> 
> Yeah, I can do that. I thought so too when I was updating this.
> 
> I *think* what happened is that when I split config-list away (in the
> earleir commit in this chain) I didn't notice that command-list.h was
> already in GEN_HDRS, and instead just grepped Makefile for
> "command-list.h" and added config-list.h next to it. So I'll try to fix
> it much earlier, in that commit, and then simply add to the appropriate
> variables in this commit.
> 
> > 
> > We probably clean up the build/update procedure around unicode-width.h
> > before we can do so, probably.  It may be generatable using contrib/
> > script, but as far as our normal build is concerned, it is a tracked
> > source and not part of the build byproducts, so we probably would
> > want to remove it from GEN_HDRS.  When that happens, we can $(RM)
> > all of the $(GEN_HDRS) in the "clean" target.
> 
> Noted. Thanks.

Ah, after I sent this mail I saw your patch. For those playing along at
home, https://lore.kernel.org/xmqq1rt7hkp6.fsf@gitster-ct.c.googlers.com

Will review.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 10/15] bugreport: add config values from safelist
  2019-12-16 23:40     ` Emily Shaffer
@ 2019-12-17 17:43       ` Junio C Hamano
  2020-01-24  3:29         ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2019-12-17 17:43 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

>> But that is too simple-minded.  If we wanted to safelist foo.*.bar,
>> where '*' can be anything, walking on the list of safe variables
>> would not work.  We must have a hash table that knows "foo.*.bar" is
>> allowed, and while walking all the configuration keys, when we see
>> foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if
>> it is whitelisted, or something like that.
>
> ...unless we want to use wildcards like you suggest.
>
> But I'm not sure it's a good idea. I envision someone writing another
> Git add-on, which offers someone to specify "user.password" ...

Wildcarding the leaf level of two (or for that matter three) level
names like "user.*" in your example is of course a nonsense way to
use the safelist.  But think about three-level names where the
second level is designed to be used for user-supplied token to give
things of similar nature a name the user can use to distinguish
things.  If remote.origin.url is worth reporting, wouldn't
remote.upstream.url also be?  Shouldn't remote.*.url be the way to
say "the URL field for each and every remote is a safe thing to
report" in the safelist?

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 09/15] bugreport: generate config safelist based on docs
  2019-12-13  0:43 ` [PATCH v4 09/15] bugreport: generate config safelist based on docs Emily Shaffer
  2019-12-13 22:57   ` Junio C Hamano
  2019-12-15 20:17   ` Johannes Schindelin
@ 2019-12-17 18:38   ` Johannes Schindelin
  2 siblings, 0 replies; 177+ messages in thread
From: Johannes Schindelin @ 2019-12-17 18:38 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git, Martin Ågren

Hi Emily,

On Thu, 12 Dec 2019, Emily Shaffer wrote:

> diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
> index d906a00803..750bdff9af 100644
> --- a/Documentation/asciidoctor-extensions.rb
> +++ b/Documentation/asciidoctor-extensions.rb
> @@ -37,6 +37,10 @@ module Git
>            output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
>          end
>          output
> +
> +    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
> +      def process(parent, action, attrs)
> +        ""

The Azure Pipeline fails on this hunk, saying:

/usr/bin/asciidoctor: /home/vsts/work/1/s/Documentation/asciidoctor-extensions.rb:41: class definition in method body (SyntaxError)
/home/vsts/work/1/s/Documentation/asciidoctor-extensions.rb:55: syntax error, unexpected end-of-input, expecting keyword_end

See
https://dev.azure.com/gitgitgadget/git/_build/results?buildId=23830&view=logs&jobId=253e5128-1058-5bd4-fdf1-9b556d3207f8&j=253e5128-1058-5bd4-fdf1-9b556d3207f8&t=95c65aec-3627-54dc-fc17-47625f123bb3
for details.

Ciao,
Dscho

>        end
>      end
>    end
> @@ -45,4 +49,7 @@ end
>  Asciidoctor::Extensions.register do
>    inline_macro Git::Documentation::LinkGitProcessor, :linkgit
>    postprocessor Git::Documentation::DocumentPostProcessor
> +  # The bugreport macro does nothing as far as rendering is
> +  # concerned -- we just grep for it in the sources.
> +  inline_macro Git::Documentation::BugReportProcessor, :bugreport
>  end
> diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
> index 0006faf800..92f5082013 100644
> --- a/Documentation/config/sendemail.txt
> +++ b/Documentation/config/sendemail.txt
> @@ -1,63 +1,63 @@
> -sendemail.identity::
> +sendemail.identity bugreport:exclude[x] ::
>  	A configuration identity. When given, causes values in the
>  	'sendemail.<identity>' subsection to take precedence over
>  	values in the 'sendemail' section. The default identity is
>  	the value of `sendemail.identity`.
>
> -sendemail.smtpEncryption::
> +sendemail.smtpEncryption bugreport:include[x] ::
>  	See linkgit:git-send-email[1] for description.  Note that this
>  	setting is not subject to the 'identity' mechanism.
>
> -sendemail.smtpssl (deprecated)::
> +sendemail.smtpssl (deprecated) bugreport:exclude[x] ::
>  	Deprecated alias for 'sendemail.smtpEncryption = ssl'.
>
> -sendemail.smtpsslcertpath::
> +sendemail.smtpsslcertpath bugreport:exclude[x] ::
>  	Path to ca-certificates (either a directory or a single file).
>  	Set it to an empty string to disable certificate verification.
>
> -sendemail.<identity>.*::
> +sendemail.<identity>.* bugreport:exclude[x] ::
>  	Identity-specific versions of the 'sendemail.*' parameters
>  	found below, taking precedence over those when this
>  	identity is selected, through either the command-line or
>  	`sendemail.identity`.
>
> -sendemail.aliasesFile::
> -sendemail.aliasFileType::
> -sendemail.annotate::
> -sendemail.bcc::
> -sendemail.cc::
> -sendemail.ccCmd::
> -sendemail.chainReplyTo::
> -sendemail.confirm::
> -sendemail.envelopeSender::
> -sendemail.from::
> -sendemail.multiEdit::
> -sendemail.signedoffbycc::
> -sendemail.smtpPass::
> -sendemail.suppresscc::
> -sendemail.suppressFrom::
> -sendemail.to::
> -sendemail.tocmd::
> -sendemail.smtpDomain::
> -sendemail.smtpServer::
> -sendemail.smtpServerPort::
> -sendemail.smtpServerOption::
> -sendemail.smtpUser::
> -sendemail.thread::
> -sendemail.transferEncoding::
> -sendemail.validate::
> -sendemail.xmailer::
> +sendemail.aliasesFile bugreport:exclude[x] ::
> +sendemail.aliasFileType bugreport:exclude[x] ::
> +sendemail.annotate bugreport:include[x] ::
> +sendemail.bcc bugreport:include[x] ::
> +sendemail.cc bugreport:include[x] ::
> +sendemail.ccCmd bugreport:include[x] ::
> +sendemail.chainReplyTo bugreport:include[x] ::
> +sendemail.confirm bugreport:include[x] ::
> +sendemail.envelopeSender bugreport:include[x] ::
> +sendemail.from bugreport:include[x] ::
> +sendemail.multiEdit bugreport:include[x] ::
> +sendemail.signedoffbycc bugreport:include[x] ::
> +sendemail.smtpPass bugreport:exclude[x] ::
> +sendemail.suppresscc bugreport:include[x] ::
> +sendemail.suppressFrom bugreport:include[x] ::
> +sendemail.to bugreport:include[x] ::
> +sendemail.tocmd bugreport:include[x] ::
> +sendemail.smtpDomain bugreport:include[x] ::
> +sendemail.smtpServer bugreport:include[x] ::
> +sendemail.smtpServerPort bugreport:include[x] ::
> +sendemail.smtpServerOption bugreport:include[x] ::
> +sendemail.smtpUser bugreport:exclude[x] ::
> +sendemail.thread bugreport:include[x] ::
> +sendemail.transferEncoding bugreport:include[x] ::
> +sendemail.validate bugreport:include[x] ::
> +sendemail.xmailer bugreport:include[x] ::
>  	See linkgit:git-send-email[1] for description.
>
> -sendemail.signedoffcc (deprecated)::
> +sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
>  	Deprecated alias for `sendemail.signedoffbycc`.
>
> -sendemail.smtpBatchSize::
> +sendemail.smtpBatchSize bugreport:include[x] ::
>  	Number of messages to be sent per connection, after that a relogin
>  	will happen.  If the value is 0 or undefined, send all messages in
>  	one connection.
>  	See also the `--batch-size` option of linkgit:git-send-email[1].
>
> -sendemail.smtpReloginDelay::
> +sendemail.smtpReloginDelay bugreport:include[x] ::
>  	Seconds wait before reconnecting to smtp server.
>  	See also the `--relogin-delay` option of linkgit:git-send-email[1].
> diff --git a/Makefile b/Makefile
> index c49f55a521..76dc51e2b1 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -651,7 +651,7 @@ install-perl-script: $(SCRIPT_PERL_GEN)
>  install-python-script: $(SCRIPT_PYTHON_GEN)
>  	$(INSTALL) $^ '$(DESTDIR_SQ)$(gitexec_instdir_SQ)'
>
> -.PHONY: clean-perl-script clean-sh-script clean-python-script
> +.PHONY: clean-perl-script clean-sh-script clean-python-script clean-script-dependencies
>  clean-sh-script:
>  	$(RM) $(SCRIPT_SH_GEN)
>  clean-perl-script:
> @@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
>
>  GENERATED_H += config-list.h
>  GENERATED_H += command-list.h
> +GENERATED_H += bugreport-config-safelist.h
>
>  LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
>  	$(FIND) . \
> @@ -2161,6 +2162,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
>  		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
>  		command-list.txt >$@+ && mv $@+ $@
>
> +bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
> +
> +bugreport-config-safelist.h: Documentation/config/*.txt
> +	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
> +		>$@+ && mv $@+ $@
> +
>  SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
>  	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
>  	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
> @@ -2791,7 +2798,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
>  .PHONY: sparse $(SP_OBJ)
>  sparse: $(SP_OBJ)
>
> -GEN_HDRS := config-list.h command-list.h unicode-width.h
> +GEN_HDRS := config-list.h command-list.h unicode-width.h bugreport-config-safelist.h
>  EXCEPT_HDRS := $(GEN_HDRS) compat/% xdiff/%
>  ifndef GCRYPT_SHA256
>  	EXCEPT_HDRS += sha256/gcrypt.h
> @@ -3117,7 +3124,8 @@ clean: profile-clean coverage-clean cocciclean
>  	$(RM) $(HCC)
>  	$(RM) -r bin-wrappers $(dep_dirs)
>  	$(RM) -r po/build/
> -	$(RM) *.pyc *.pyo */*.pyc */*.pyo config-list.h command-list.h
> +	$(RM) *.pyc *.pyo */*.pyc */*.pyo
> +	$(RM) config-list.h command-list.h bugreport-config-safelist.h
>  	$(RM) $(ETAGS_TARGET) tags cscope*
>  	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
>  	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
> diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
> new file mode 100755
> index 0000000000..06b8e0c3c4
> --- /dev/null
> +++ b/generate-bugreport-config-safelist.sh
> @@ -0,0 +1,22 @@
> +#!/bin/sh
> +
> +cat <<EOF
> +/* Automatically generated by bugreport-generate-config-safelist.sh */
> +
> +
> +static const char *bugreport_config_safelist[] = {
> +EOF
> +
> +# cat all regular files in Documentation/config
> +find Documentation/config -type f -exec cat {} \; |
> +# print the command name which matches the bugreport-include macro
> +sed -n 's/^\(.*\) \+bugreport:include.* ::$/\1/p' |
> +sort |
> +while read line
> +do
> +	echo "	\"$line\","
> +done
> +
> +cat <<EOF
> +};
> +EOF
> --
> 2.24.1.735.g03f4e72817-goog
>
>
>

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 03/15] bugreport: gather git version and build info
  2019-12-13  0:43 ` [PATCH v4 03/15] bugreport: gather git version and build info Emily Shaffer
  2019-12-13 21:06   ` Junio C Hamano
@ 2019-12-17 18:45   ` Johannes Schindelin
  2019-12-17 20:34     ` Junio C Hamano
  1 sibling, 1 reply; 177+ messages in thread
From: Johannes Schindelin @ 2019-12-17 18:45 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Thu, 12 Dec 2019, Emily Shaffer wrote:

> diff --git a/help.c b/help.c
> index a21487db77..a43693fca5 100644
> --- a/help.c
> +++ b/help.c
> @@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
>  	exit(1);
>  }
>
> +void list_version_info(struct strbuf *buf, int build_options)
> +{
> +	strbuf_reset(buf);
> +	/*
> +	 * The format of this string should be kept stable for compatibility
> +	 * with external projects that rely on the output of "git version".
> +	 *
> +	 * Always show the version, even if other options are given.
> +	 */
> +	strbuf_addf(buf, "git version %s\n", git_version_string);
> +
> +	if (build_options) {
> +		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
> +		if (git_built_from_commit_string[0])
> +			strbuf_addf(buf, "built from commit: %s\n",
> +			       git_built_from_commit_string);
> +		else
> +			strbuf_addf(buf, "no commit associated with this build\n");

The "StaticAnalysis" job of the Azure Pipeline is not happy with this,
claiming that this should be an `strbuf_addstr()` call instead. For
details, see:

https://dev.azure.com/gitgitgadget/8fc4a374-71dc-4558-a5ea-dd1c081ea621/_apis/build/builds/23830/logs/68

Ciao,
Dscho

> +		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
> +		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
> +		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
> +	}
> +}
> +
>  int cmd_version(int argc, const char **argv, const char *prefix)
>  {
> +	struct strbuf buf = STRBUF_INIT;
>  	int build_options = 0;
>  	const char * const usage[] = {
>  		N_("git version [<options>]"),
> @@ -637,25 +662,9 @@ int cmd_version(int argc, const char **argv, const char *prefix)
>
>  	argc = parse_options(argc, argv, prefix, options, usage, 0);
>
> -	/*
> -	 * The format of this string should be kept stable for compatibility
> -	 * with external projects that rely on the output of "git version".
> -	 *
> -	 * Always show the version, even if other options are given.
> -	 */
> -	printf("git version %s\n", git_version_string);
> +	list_version_info(&buf, build_options);
> +	printf("%s", buf.buf);
>
> -	if (build_options) {
> -		printf("cpu: %s\n", GIT_HOST_CPU);
> -		if (git_built_from_commit_string[0])
> -			printf("built from commit: %s\n",
> -			       git_built_from_commit_string);
> -		else
> -			printf("no commit associated with this build\n");
> -		printf("sizeof-long: %d\n", (int)sizeof(long));
> -		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
> -		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
> -	}
>  	return 0;
>  }
>
> diff --git a/help.h b/help.h
> index 9071894e8c..54f6b5f793 100644
> --- a/help.h
> +++ b/help.h
> @@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
>  void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
>  int is_in_cmdlist(struct cmdnames *cmds, const char *name);
>  void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
> +void list_version_info(struct strbuf *buf, int build_options);
>
>  /*
>   * call this to die(), when it is suspected that the user mistyped a
> --
> 2.24.1.735.g03f4e72817-goog
>
>
>

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 07/15] bugreport: add curl version
  2019-12-13  0:43 ` [PATCH v4 07/15] bugreport: add curl version Emily Shaffer
  2019-12-13 21:27   ` Junio C Hamano
@ 2019-12-17 18:47   ` Johannes Schindelin
  1 sibling, 0 replies; 177+ messages in thread
From: Johannes Schindelin @ 2019-12-17 18:47 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Thu, 12 Dec 2019, Emily Shaffer wrote:

> diff --git a/http-fetch.c b/http-fetch.c
> index a32ac118d9..31844812a1 100644
> --- a/http-fetch.c
> +++ b/http-fetch.c
> @@ -3,9 +3,18 @@
>  #include "exec-cmd.h"
>  #include "http.h"
>  #include "walker.h"
> +#include "version.h"
>
>  static const char http_fetch_usage[] = "git http-fetch "
> -"[-c] [-t] [-a] [-v] [--recover] [-w ref] [--stdin] commit-id url";
> +"[-c] [-t] [-a] [-v] [-V] [--recover] [-w ref] [--stdin] commit-id url";
> +
> +void NORETURN version_info()

Pretty much all the builds in
https://dev.azure.com/gitgitgadget/git/_build/results?buildId=23830&view=logs&jobId=253e5128-1058-5bd4-fdf1-9b556d3207f8&j=fd490c07-0b22-5182-fac9-6d67fe1e939b&t=ce91d5d6-0c55-50f5-8ab9-6695c03ab102
fail because this function definition needs `(void)` instead of `()`.

Ciao,
Dscho

> +{
> +	printf("git-http-fetch version: %s\n", git_version_string);
> +	printf("built from commit: %s\n", git_built_from_commit_string);
> +	printf("curl version: %s\n", curl_version());
> +	exit(0);
> +}
>
>  int cmd_main(int argc, const char **argv)
>  {
> @@ -26,6 +35,8 @@ int cmd_main(int argc, const char **argv)
>  		} else if (argv[arg][1] == 'a') {
>  		} else if (argv[arg][1] == 'v') {
>  			get_verbosely = 1;
> +		} else if (argv[arg][1] == 'V') {
> +			version_info();
>  		} else if (argv[arg][1] == 'w') {
>  			write_ref = &argv[arg + 1];
>  			arg++;
> --
> 2.24.1.735.g03f4e72817-goog
>
>
>

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 03/15] bugreport: gather git version and build info
  2019-12-17 18:45   ` Johannes Schindelin
@ 2019-12-17 20:34     ` Junio C Hamano
  2019-12-20  1:25       ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2019-12-17 20:34 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Emily Shaffer, git

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> +	if (build_options) {
>> +		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
>> +		if (git_built_from_commit_string[0])
>> +			strbuf_addf(buf, "built from commit: %s\n",
>> +			       git_built_from_commit_string);
>> +		else
>> +			strbuf_addf(buf, "no commit associated with this build\n");
>
> The "StaticAnalysis" job of the Azure Pipeline is not happy with this,
> claiming that this should be an `strbuf_addstr()` call instead.

You mean the "else" clause, right?  That feels similar to say

	printf("Hello world\n");

should better be written as

	fputs("Hello world\n", stdout);

which I do not agree with at all.  IOW, I view the distinction more
like "once it is written one way or the other way, it is not worth
spending bits and braincycles to see if it is worth changing it"
kind of minor stylistic preference.



^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 03/15] bugreport: gather git version and build info
  2019-12-17 20:34     ` Junio C Hamano
@ 2019-12-20  1:25       ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-20  1:25 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Johannes Schindelin, git

On Tue, Dec 17, 2019 at 12:34:53PM -0800, Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> +	if (build_options) {
> >> +		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
> >> +		if (git_built_from_commit_string[0])
> >> +			strbuf_addf(buf, "built from commit: %s\n",
> >> +			       git_built_from_commit_string);
> >> +		else
> >> +			strbuf_addf(buf, "no commit associated with this build\n");
> >
> > The "StaticAnalysis" job of the Azure Pipeline is not happy with this,
> > claiming that this should be an `strbuf_addstr()` call instead.
> 
> You mean the "else" clause, right?  That feels similar to say
> 
> 	printf("Hello world\n");
> 
> should better be written as
> 
> 	fputs("Hello world\n", stdout);
> 
> which I do not agree with at all.  IOW, I view the distinction more
> like "once it is written one way or the other way, it is not worth
> spending bits and braincycles to see if it is worth changing it"
> kind of minor stylistic preference.

I think I side with Junio here, although it's true that when
strbuf_addstr() exists it doesn't make that much sense to use
strbuf_addf(). Since there's some other comments, though, I'll change
this too to make your CI shut up. :)

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 03/15] bugreport: gather git version and build info
  2019-12-13 21:06   ` Junio C Hamano
@ 2019-12-20  1:46     ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2019-12-20  1:46 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Dec 13, 2019 at 01:06:29PM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Knowing which version of Git a user has and how it was built allows us
> > to more precisely pin down the circumstances when a certain issue
> > occurs, so teach bugreport how to tell us the same output as 'git
> > version --build-options'.
> >
> > It's not ideal to directly call 'git version --build-options' because
> > that output goes to stdout. Instead, wrap the version string in a helper
> > within help.[ch] library, and call that helper from within the bugreport
> > library.
> 
> Move to strbuf() from stdio makes sense.  
> 
> > +	// add other contents
> 
> Style.

Sure, dropped this entirely. I think with the helpers the code is
self-documenting there.

> 
> > diff --git a/help.h b/help.h
> > index 9071894e8c..54f6b5f793 100644
> > --- a/help.h
> > +++ b/help.h
> > @@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
> >  void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
> >  int is_in_cmdlist(struct cmdnames *cmds, const char *name);
> 
> 
> Many new helpers are called get_frotz(), but only one among these is
> called list_version_info().  I do not think the naming of the
> get_*() ones, that are private to the bugreport tool, matters that
> much, but unlike "list_commands()" whose purpose is to list the
> commands ;-), the new function does not list versions---it just
> gives information about a single version which is the one that is
> being run, so perhaps it is a misnomer.

Hm, sure. I renamed it to get_version_info(); I had named it list_*
because all the other helpers in help.h are named list_*, and it does
print more than one piece of info. But I do see your point (all the info
is about the same version) so I've renamed it.

> 
> >  void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
> > +void list_version_info(struct strbuf *buf, int build_options);
> 
> It is not clear to the readers build_options is a boolean that
> tells the function to include (or not to include) build options.
> Perhaps rename it to "int show_build_options" or something?

Agree, done.

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 10/15] bugreport: add config values from safelist
  2019-12-13  0:43 ` [PATCH v4 10/15] bugreport: add config values from safelist Emily Shaffer
  2019-12-13 21:45   ` Junio C Hamano
@ 2019-12-29 20:17   ` Johannes Schindelin
  1 sibling, 0 replies; 177+ messages in thread
From: Johannes Schindelin @ 2019-12-29 20:17 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Thu, 12 Dec 2019, Emily Shaffer wrote:

> diff --git a/bugreport.c b/bugreport.c
> index 759cc0b0f8..1fca28f0b9 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -6,6 +6,9 @@
>  #include "help.h"
>  #include <gnu/libc-version.h>
>  #include "run-command.h"
> +#include "config.h"
> +#include "bugreport-config-safelist.h"
> +#include "khash.h"

Seems that this patch makes things fail in the CI build
(https://dev.azure.com/gitgitgadget/git/_build/results?buildId=23830&view=results):

-- snipsnap --
bugreport.c:10:10: fatal error: 'bugreport-config-safelist.h' file not found
#include "bugreport-config-safelist.h"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
Makefile:2395: recipe for target 'bugreport.o' failed
make: *** [bugreport.o] Error 1
make: *** Waiting for unfinished jobs....


^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 05/15] bugreport: add uname info
  2019-12-13  0:43 ` [PATCH v4 05/15] bugreport: add uname info Emily Shaffer
  2019-12-13 21:12   ` Junio C Hamano
@ 2020-01-10  2:05   ` Aaron Schrab
  1 sibling, 0 replies; 177+ messages in thread
From: Aaron Schrab @ 2020-01-10  2:05 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

At 16:43 -0800 12 Dec 2019, Emily Shaffer <emilyshaffer@google.com> wrote:
>The contents of uname() can give us some insight into what sort of
>system the user is running on, and help us replicate their setup if need
>be. The domainname field is not guaranteed to be available, so don't
>collect it.

The manpage on Linux says that it's a GNU extension; on Mac OS it isn't 
mentioned. So it would be more accurate to say that it's known not to be 
available on many systems rather than just not being guaranteed.

Besides that I think in some cases it may be considered sensitive 
information, so another reason to not include it. Perhaps even worthy of 
being mentioned as the primary reason.

>+			    uname_info.nodename,

I think in some cases this could also be considered as sensitive 
information, and is unlikely to help in diagnosing problems.  So I'd 
move to exclude this as well.

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v4 10/15] bugreport: add config values from safelist
  2019-12-17 17:43       ` Junio C Hamano
@ 2020-01-24  3:29         ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-01-24  3:29 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Dec 17, 2019 at 09:43:23AM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> >> But that is too simple-minded.  If we wanted to safelist foo.*.bar,
> >> where '*' can be anything, walking on the list of safe variables
> >> would not work.  We must have a hash table that knows "foo.*.bar" is
> >> allowed, and while walking all the configuration keys, when we see
> >> foo.a.bar, we consult "foo.*.bar" as well as "foo.a.bar" to see if
> >> it is whitelisted, or something like that.
> >
> > ...unless we want to use wildcards like you suggest.
> >
> > But I'm not sure it's a good idea. I envision someone writing another
> > Git add-on, which offers someone to specify "user.password" ...
> 
> Wildcarding the leaf level of two (or for that matter three) level
> names like "user.*" in your example is of course a nonsense way to
> use the safelist.  But think about three-level names where the
> second level is designed to be used for user-supplied token to give
> things of similar nature a name the user can use to distinguish
> things.  If remote.origin.url is worth reporting, wouldn't
> remote.upstream.url also be?  Shouldn't remote.*.url be the way to
> say "the URL field for each and every remote is a safe thing to
> report" in the safelist?

Bah, somehow this mail slipped through the cracks for me.

When you put it that way, I see how it can be useful. Although I admit
to fatigue on this patchset and now I'm hesitant about feature creep. :)

I'm going to send the reroll for v5 as I have it, not having seen this
mail, and think more about how I'd approach wildcarding in the middle
like this. I want to take a look at the other configs which use a middle
level as a user token and try to reason about whether we ought to be
collecting them. I'll hope to report back sooner (or later).

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v5 00/15] add git-bugreport tool
  2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
                   ` (14 preceding siblings ...)
  2019-12-13  0:43 ` [PATCH v4 15/15] bugreport: summarize contents of alternates file Emily Shaffer
@ 2020-01-24  3:34 ` emilyshaffer
  2020-01-24  3:34   ` [PATCH v5 01/15] bugreport: add tool to generate debugging info emilyshaffer
                     ` (18 more replies)
  15 siblings, 19 replies; 177+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Derrick Stolee, Johannes Schindelin,
	Junio C Hamano, Martin Ågren, Aaron Schrab

From: Emily Shaffer <emilyshaffer@google.com>

This topic branch depends on the patch mailed in
lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to
display scopes for configs gathered during "bugreport: add config values from
safelist".

I'll summarize v4-v5. Since v4 has languished for some time, I don't
think an interdiff is too helpful, so I won't include one. Bonus, the
code is much simplified from some suggestions from Junio on how to
inspect objects, so I hope it's easy to review anyways.

Throughout, did some style changes away from C++ bad habits. Otherwise,
as listed:

Emily Shaffer (15):
  bugreport: add tool to generate debugging info
    (no change)
  help: move list_config_help to builtin/help
    Changed to agree with f3719846134

  bugreport: gather git version and build info
    Style only

  help: add shell-path to --build-options
    (no change)

  bugreport: add uname info
    Removed nodename for privacy reasons

  bugreport: add compiler info
    Moved glibc tattling into compat/.

    I appreciate a close look at this one - I think I understood the
    right way to go about a compat/ util but it's the first one I've
    done.

  bugreport: add curl version
    Moved curl tattling from git-http-fetch into git-remote-curl

  bugreport: include user interactive shell
    Stop depending on a compiler quirk to save us from segfault

  bugreport: generate config safelist based on docs
    Changed to agree with f3719846134
    Cleaned up script a little

  bugreport: add config values from safelist
    Made git-bugreport dependent on generated safelist header

  bugreport: collect list of populated hooks
    No change.
    Per https://lore.kernel.org/git/20191216235131.GL135450@google.com,
    should we even keep this patch?

  bugreport: count loose objects
    Use helpers from object-store.h instead of manually walking the
    filesystem.

  bugreport: add packed object summary
    Use helpers from object-store.h instead of manually walking the
    filesystem.

  bugreport: list contents of $OBJDIR/info
    (no change)

  bugreport: summarize contents of alternates file
    Rephrase commit message to explain why I can't use the helpers in
    object-store.h.


Thanks.
 - Emily


 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   8 +
 Documentation/asciidoctor-extensions.rb |   7 +
 Documentation/config/sendemail.txt      |  68 ++--
 Documentation/git-bugreport.txt         |  43 +++
 Makefile                                |  25 +-
 bugreport.c                             | 427 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 compat/compiler.h                       |  24 ++
 generate-bugreport-config-safelist.sh   |  22 ++
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  24 ++
 help.c                                  | 133 ++------
 help.h                                  |   2 +-
 remote-curl.c                           |   8 +
 t/t0091-bugreport.sh                    |  41 +++
 16 files changed, 780 insertions(+), 160 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100644 compat/compiler.h
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh

-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v5 01/15] bugreport: add tool to generate debugging info
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:18     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 02/15] help: move list_config_help to builtin/help emilyshaffer
                     ` (17 subsequent siblings)
  18 siblings, 1 reply; 177+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

Teach Git how to prompt the user for a good bug report: reproduction
steps, expected behavior, and actual behavior. Later, Git can learn how
to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic
information we would otherwise need to ask the user for, we can reduce
the number of question-and-answer round trips between the reporter and
the Git contributor.

Users may also wish to send a report like this to their local "Git
expert" if they have put their repository into a state they are confused
by.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 43 ++++++++++++++++++++
 Makefile                        |  5 +++
 bugreport.c                     | 69 +++++++++++++++++++++++++++++++++
 t/t0091-bugreport.sh            | 41 ++++++++++++++++++++
 5 files changed, 159 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index aebe7c0908..ca301bc890 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..75f0c80acf
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,43 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [-o | --output <path>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+The following information is captured automatically:
+
+ - Git version (`git version --build-options`)
+ - Machine information (`uname -a`)
+ - Versions of various dependencies
+ - Git config contents (`git config --show-origin --list`)
+ - The names of all configured git-hooks in `.git/hooks/`
+
+OPTIONS
+-------
+-o [<path>]::
+--output [<path>]::
+	Place the resulting bug report file in <path> instead of the root of the
+	Git repository.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index 09f98b777c..f271619371 100644
--- a/Makefile
+++ b/Makefile
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2450,6 +2451,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
diff --git a/bugreport.c b/bugreport.c
new file mode 100644
index 0000000000..5495b31674
--- /dev/null
+++ b/bugreport.c
@@ -0,0 +1,69 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output <file>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to send.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+
+	get_bug_template(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	launch_editor(report_path.buf, NULL, NULL);
+	return 0;
+}
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..6eb2ee4f66
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,41 @@
+#!/bin/bash
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if [$(grep $HEADER_PATTERN $line)]; then
+			read -r nextline
+			if [-z $nextline]; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	check_all_headers_populated <git-bugreport-* &&
+	rm git-bugreport-*
+'
+
+test_expect_success '--output puts the report in the provided dir' '
+	mkdir foo/ &&
+	git bugreport -o foo/ &&
+	test -f foo/git-bugreport-* &&
+	rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	grep usage output &&
+	test ! -f git-bugreport-*
+'
+
+test_done
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v5 02/15] help: move list_config_help to builtin/help
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
  2020-01-24  3:34   ` [PATCH v5 01/15] bugreport: add tool to generate debugging info emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:19     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 03/15] bugreport: gather git version and build info emilyshaffer
                     ` (16 subsequent siblings)
  18 siblings, 1 reply; 177+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o,
builtin/clean.o, and builtin/config.o. This meant that help.o was
unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help()
into builtin/help.c (where it makes sense to assume other builtin libraries
are present).

When command-list.h is included but a member is not used, we start to
hear a compiler warning. Since the config list is generated in a fairly
different way than the command list, and since commands and config
options are semantically different, move the config list into its own
header and move the generator into its own script and build rule.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore             |  1 +
 Makefile               | 13 +++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 24 ++++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 121 insertions(+), 108 deletions(-)
 create mode 100755 generate-configlist.sh

diff --git a/.gitignore b/.gitignore
index ca301bc890..d89bf9e11e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -190,6 +190,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index f271619371..a01a050aa3 100644
--- a/Makefile
+++ b/Makefile
@@ -815,6 +815,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2129,7 +2130,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2149,6 +2150,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2786,7 +2793,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2808,7 +2815,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..eca6a00c30
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/help.c b/help.c
index cf67624a94..a21487db77 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
diff --git a/help.h b/help.h
index 7a455beeb7..9071894e8c 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v5 03/15] bugreport: gather git version and build info
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
  2020-01-24  3:34   ` [PATCH v5 01/15] bugreport: add tool to generate debugging info emilyshaffer
  2020-01-24  3:34   ` [PATCH v5 02/15] help: move list_config_help to builtin/help emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:19     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 04/15] help: add shell-path to --build-options emilyshaffer
                     ` (15 subsequent siblings)
  18 siblings, 1 reply; 177+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

Knowing which version of Git a user has and how it was built allows us
to more precisely pin down the circumstances when a certain issue
occurs, so teach bugreport how to tell us the same output as 'git
version --build-options'.

It's not ideal to directly call 'git version --build-options' because
that output goes to stdout. Instead, wrap the version string in a helper
within help.[ch] library, and call that helper from within the bugreport
library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 22 +++++++++++++++++++++-
 help.c      | 47 +++++++++++++++++++++++++++++------------------
 help.h      |  1 +
 3 files changed, 51 insertions(+), 19 deletions(-)

diff --git a/bugreport.c b/bugreport.c
index 5495b31674..b1a5a279ac 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,8 +1,20 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	struct strbuf version_info = STRBUF_INIT;
+
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(&version_info, 1);
+	strbuf_addbuf(sys_info, &version_info);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
@@ -32,6 +44,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -60,6 +77,9 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
diff --git a/help.c b/help.c
index a21487db77..156a6cdea3 100644
--- a/help.c
+++ b/help.c
@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void get_version_info(struct strbuf *buf, int show_build_options)
+{
+	strbuf_reset(buf);
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (show_build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addstr(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -637,25 +662,11 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	get_version_info(&buf, build_options);
+	printf("%s", buf.buf);
+
+	strbuf_release(&buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 9071894e8c..500521b908 100644
--- a/help.h
+++ b/help.h
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void get_version_info(struct strbuf *buf, int show_build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v5 04/15] help: add shell-path to --build-options
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (2 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 03/15] bugreport: gather git version and build info emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:21     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 05/15] bugreport: add uname info emilyshaffer
                     ` (14 subsequent siblings)
  18 siblings, 1 reply; 177+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

It may be useful to know which shell Git was built to try to point to,
in the event that shell-based Git commands are failing. $SHELL_PATH is
set during the build and used to launch the manpage viewer, as well as
by git-compat-util.h, and it's used during tests. 'git version
--build-options' is encouraged for use in bug reports, so it makes sense
to include this information there.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 help.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/help.c b/help.c
index 156a6cdea3..d3f941824f 100644
--- a/help.c
+++ b/help.c
@@ -642,6 +642,7 @@ void get_version_info(struct strbuf *buf, int show_build_options)
 			strbuf_addstr(buf, "no commit associated with this build\n");
 		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
 		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 }
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v5 05/15] bugreport: add uname info
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (3 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 04/15] help: add shell-path to --build-options emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-24  3:34   ` [PATCH v5 06/15] bugreport: add compiler info emilyshaffer
                     ` (13 subsequent siblings)
  18 siblings, 0 replies; 177+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

The contents of uname() can give us some insight into what sort of
system the user is running on, and help us replicate their setup if need
be. The domainname field is not guaranteed to be available, so don't
collect it.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index b1a5a279ac..720c91e1bd 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -8,12 +8,24 @@
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
+	struct utsname uname_info;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	get_version_info(&version_info, 1);
 	strbuf_addbuf(sys_info, &version_info);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v5 06/15] bugreport: add compiler info
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (4 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 05/15] bugreport: add uname info emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:21     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 07/15] bugreport: add curl version emilyshaffer
                     ` (12 subsequent siblings)
  18 siblings, 1 reply; 177+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

To help pinpoint the source of a regression, it is useful to know some
info about the compiler which the user's Git client was built with. By
adding a generic get_compiler_info() in 'compat/' we can choose which
relevant information to share per compiler; to get started, let's
demonstrate the version of glibc if the user built with 'gcc'.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c       |  5 +++++
 compat/compiler.h | 24 ++++++++++++++++++++++++
 2 files changed, 29 insertions(+)
 create mode 100644 compat/compiler.h

diff --git a/bugreport.c b/bugreport.c
index 720c91e1bd..818ccb385c 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include "compat/compiler.h"
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -26,6 +27,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/compat/compiler.h b/compat/compiler.h
new file mode 100644
index 0000000000..bda5098e1b
--- /dev/null
+++ b/compat/compiler.h
@@ -0,0 +1,24 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
+}
+
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addstr(info, "get_compiler_info() not implemented");
+}
+
+#endif
+
+#endif /* COMPILER_H */
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v5 07/15] bugreport: add curl version
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (5 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 06/15] bugreport: add compiler info emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:27     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 08/15] bugreport: include user interactive shell emilyshaffer
                     ` (11 subsequent siblings)
  18 siblings, 1 reply; 177+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

It's possible for git-remote-curl to be built separately from git; in that
case we want to know what version of cURL is used by git-remote-curl, not
necessarily which version was present at git-bugreport's build time.
So instead, ask git-remote-curl for the version information it knows
about.

For longevity purposes, invoke the alias "git-remote-https" instead of
"git-remote-http".

Since it could have been built at a different time, also report the
version and built-from commit of git-remote-curl alongside the cURL info.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c   | 16 ++++++++++++++++
 remote-curl.c |  8 ++++++++
 2 files changed, 24 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 818ccb385c..73f6d39517 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include "compat/compiler.h"
+#include "run-command.h"
+
+static void get_curl_version_info(struct strbuf *curl_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, curl_info, 0))
+	    strbuf_addstr(curl_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -31,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "compiler info: ");
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+	get_curl_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/remote-curl.c b/remote-curl.c
index 350d92a074..c590fbfae3 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -17,6 +17,7 @@
 #include "protocol.h"
 #include "quote.h"
 #include "transport.h"
+#include "version.h"
 
 static struct remote *remote;
 /* always ends with a trailing slash */
@@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {
+		printf("git-http-fetch version: %s\n", git_version_string);
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v5 08/15] bugreport: include user interactive shell
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (6 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 07/15] bugreport: add curl version emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:28     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 09/15] bugreport: generate config safelist based on docs emilyshaffer
                     ` (10 subsequent siblings)
  18 siblings, 1 reply; 177+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

It's possible a user may complain about the way that Git interacts with
their interactive shell, e.g. autocompletion or shell prompt. In that
case, it's useful for us to know which shell they're using
interactively.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 73f6d39517..07b84b9c94 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -22,6 +22,7 @@ static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
 	struct utsname uname_info;
+	char *shell = NULL;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
@@ -44,6 +45,10 @@ static void get_system_info(struct strbuf *sys_info)
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
 
+	shell = getenv("SHELL");
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    shell ? shell : "(NULL)");
+
 	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
 	get_curl_version_info(sys_info);
 	strbuf_complete_line(sys_info);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v5 09/15] bugreport: generate config safelist based on docs
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (7 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 08/15] bugreport: include user interactive shell emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:34     ` Martin Ågren
  2020-01-31 21:20     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 10/15] bugreport: add config values from safelist emilyshaffer
                     ` (9 subsequent siblings)
  18 siblings, 2 replies; 177+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Martin Ågren, Johannes Schindelin

From: Emily Shaffer <emilyshaffer@google.com>

Add a new step to the build to generate a safelist of git-config
variables which are appropriate to include in the output of
git-bugreport. New variables can be added to the safelist by annotating
their documentation in Documentation/config with the
"bugreport" macro, which is recognized by AsciiDoc and
AsciiDoctor.

Some configs are private in nature, and can contain remote URLs,
passwords, or other sensitive information. In the event that a user
doesn't notice their information while reviewing a bugreport, that user
may leak their credentials to other individuals, mailing lists, or bug
tracking tools inadvertently. Heuristic blocklisting of configuration
keys is imperfect and prone to false negatives; given the nature of the
information which can be leaked, a safelist is more reliable.

In order to prevent staleness of the safelist, add a mechanism to
generate the safelist from annotations in the config documentation,
where contributors are already used to documenting their new config
keys.

Implement a new no-op "bugreport" macro for use as
"bugreport:include[x]" to annotate the config keys that should be
included in the automatically generated safelist. Use "exclude" for the
others.

With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc
seems to want something between the brackets. A bit unfortunate, but
not a huge problem -- we'll just provide an "x".

"doc-diff" reports that this macro doesn't render at all. That is,
these are both empty after this commit:

  cd Documentation
  ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
  ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD

Diffing the rendered HTML shows that there is some small amount of
whitespace and comments added. That shouldn't be a problem.

We could perhaps let the implementation verify that the "action" is one
of "include" and "exclude". For the Asciidoctor implementation that
should be straightforward, but for AsciiDoc I don't immediately know how
to do it. Anyway, if someone stumbles on the keyboard and writes
"bugreport:icndule", they'll "only" miss out on the config key being
included in the safelist. If this were a blocklist, the consequences of
a misspelled target could be a lot more severe.

Additionally, add annotations to the sendemail config documentation in
order to demonstrate a proof of concept.

Helped-by: Martin Ågren <martin.agren@gmail.com>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  8 +++
 Documentation/asciidoctor-extensions.rb |  7 +++
 Documentation/config/sendemail.txt      | 68 ++++++++++++-------------
 Makefile                                |  7 +++
 generate-bugreport-config-safelist.sh   | 17 +++++++
 6 files changed, 74 insertions(+), 34 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh

diff --git a/.gitignore b/.gitignore
index d89bf9e11e..bd2f49b996 100644
--- a/.gitignore
+++ b/.gitignore
@@ -192,6 +192,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..5d5359fcf9 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,13 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The bugreport macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>bugreport):(?P<action>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +32,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[bugreport-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +100,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[bugreport-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..750bdff9af 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, action, attrs)
+        ""
       end
     end
   end
@@ -45,4 +49,7 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  # The bugreport macro does nothing as far as rendering is
+  # concerned -- we just grep for it in the sources.
+  inline_macro Git::Documentation::BugReportProcessor, :bugreport
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 0006faf800..92f5082013 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -1,63 +1,63 @@
-sendemail.identity::
+sendemail.identity bugreport:exclude[x] ::
 	A configuration identity. When given, causes values in the
 	'sendemail.<identity>' subsection to take precedence over
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption::
+sendemail.smtpEncryption bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
-sendemail.smtpssl (deprecated)::
+sendemail.smtpssl (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for 'sendemail.smtpEncryption = ssl'.
 
-sendemail.smtpsslcertpath::
+sendemail.smtpsslcertpath bugreport:exclude[x] ::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*::
+sendemail.<identity>.* bugreport:exclude[x] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
 	`sendemail.identity`.
 
-sendemail.aliasesFile::
-sendemail.aliasFileType::
-sendemail.annotate::
-sendemail.bcc::
-sendemail.cc::
-sendemail.ccCmd::
-sendemail.chainReplyTo::
-sendemail.confirm::
-sendemail.envelopeSender::
-sendemail.from::
-sendemail.multiEdit::
-sendemail.signedoffbycc::
-sendemail.smtpPass::
-sendemail.suppresscc::
-sendemail.suppressFrom::
-sendemail.to::
-sendemail.tocmd::
-sendemail.smtpDomain::
-sendemail.smtpServer::
-sendemail.smtpServerPort::
-sendemail.smtpServerOption::
-sendemail.smtpUser::
-sendemail.thread::
-sendemail.transferEncoding::
-sendemail.validate::
-sendemail.xmailer::
+sendemail.aliasesFile bugreport:exclude[x] ::
+sendemail.aliasFileType bugreport:exclude[x] ::
+sendemail.annotate bugreport:include[x] ::
+sendemail.bcc bugreport:include[x] ::
+sendemail.cc bugreport:include[x] ::
+sendemail.ccCmd bugreport:include[x] ::
+sendemail.chainReplyTo bugreport:include[x] ::
+sendemail.confirm bugreport:include[x] ::
+sendemail.envelopeSender bugreport:include[x] ::
+sendemail.from bugreport:include[x] ::
+sendemail.multiEdit bugreport:include[x] ::
+sendemail.signedoffbycc bugreport:include[x] ::
+sendemail.smtpPass bugreport:exclude[x] ::
+sendemail.suppresscc bugreport:include[x] ::
+sendemail.suppressFrom bugreport:include[x] ::
+sendemail.to bugreport:include[x] ::
+sendemail.tocmd bugreport:include[x] ::
+sendemail.smtpDomain bugreport:include[x] ::
+sendemail.smtpServer bugreport:include[x] ::
+sendemail.smtpServerPort bugreport:include[x] ::
+sendemail.smtpServerOption bugreport:include[x] ::
+sendemail.smtpUser bugreport:exclude[x] ::
+sendemail.thread bugreport:include[x] ::
+sendemail.transferEncoding bugreport:include[x] ::
+sendemail.validate bugreport:include[x] ::
+sendemail.xmailer bugreport:include[x] ::
 	See linkgit:git-send-email[1] for description.
 
-sendemail.signedoffcc (deprecated)::
+sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize bugreport:include[x] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay::
+sendemail.smtpReloginDelay bugreport:include[x] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Makefile b/Makefile
index a01a050aa3..2bc9f112ea 100644
--- a/Makefile
+++ b/Makefile
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2163,6 +2164,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..44612d5538
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the bugreport-include macro
+sed -n 's/^\(.*\) \+bugreport:include.* ::$/  "\1",/p' | sort
+
+cat <<EOF
+};
+EOF
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v5 10/15] bugreport: add config values from safelist
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (8 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 09/15] bugreport: generate config safelist based on docs emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-30 22:36     ` Martin Ågren
  2020-01-31 21:25     ` Martin Ågren
  2020-01-24  3:34   ` [PATCH v5 11/15] bugreport: collect list of populated hooks emilyshaffer
                     ` (8 subsequent siblings)
  18 siblings, 2 replies; 177+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

Teach bugreport to gather the values of config options which are present
in 'bugreport-config-safelist.h'.

Many config options are sensitive, and many Git add-ons use config
options which git-core does not know about; it is better only to gather
config options which we know to be safe, rather than excluding options
which we know to be unsafe.

Taking the build-time generated array and putting it into a set saves us
time - since git_config_bugreport() is called for every option the user
has configured, performing option lookup in constant time is a useful
optimization.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Makefile    |  2 +-
 bugreport.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 76 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 2bc9f112ea..eb17ece120 100644
--- a/Makefile
+++ b/Makefile
@@ -2465,7 +2465,7 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
-git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+git-bugreport$X: bugreport-config-safelist.h bugreport.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(LIBS)
 
diff --git a/bugreport.c b/bugreport.c
index 07b84b9c94..7a9fd36b60 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -6,6 +6,9 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_curl_version_info(struct strbuf *curl_info)
 {
@@ -18,6 +21,40 @@ static void get_curl_version_info(struct strbuf *curl_info)
 	    strbuf_addstr(curl_info, "'git-remote-https --build-info' not supported\n");
 }
 
+KHASH_INIT(cfg_set, const char*, int, 0, kh_str_hash_func, kh_str_hash_equal);
+
+struct cfgset {
+	kh_cfg_set_t set;
+};
+
+struct cfgset safelist;
+
+static void cfgset_init(struct cfgset *set, size_t initial_size)
+{
+	memset(&set->set, 0, sizeof(set->set));
+	if (initial_size)
+		kh_resize_cfg_set(&set->set, initial_size);
+}
+
+static int cfgset_insert(struct cfgset *set, const char *cfg_key)
+{
+	int added;
+	kh_put_cfg_set(&set->set, cfg_key, &added);
+	return !added;
+}
+
+static int cfgset_contains(struct cfgset *set, const char *cfg_key)
+{
+	khiter_t pos = kh_get_cfg_set(&set->set, cfg_key);
+	return pos != kh_end(&set->set);
+}
+
+static void cfgset_clear(struct cfgset *set)
+{
+	kh_release_cfg_set(&set->set);
+	cfgset_init(set, 0);
+}
+
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct strbuf version_info = STRBUF_INIT;
@@ -54,6 +91,36 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void gather_safelist(void)
+{
+	int index;
+	int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
+	cfgset_init(&safelist, safelist_len);
+	for (index = 0; index < safelist_len; index++)
+		cfgset_insert(&safelist, bugreport_config_safelist[index]);
+
+}
+
+static int git_config_bugreport(const char *var, const char *value, void *cb)
+{
+	struct strbuf *config_info = (struct strbuf *)cb;
+
+	if (cfgset_contains(&safelist, var))
+		strbuf_addf(config_info,
+			    "%s (%s) : %s\n",
+			    var, config_scope_to_string(current_config_scope()),
+			    value);
+
+	return 0;
+}
+
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	gather_safelist();
+	git_config(git_config_bugreport, config_info);
+	cfgset_clear(&safelist);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -94,12 +161,17 @@ int cmd_main(int argc, const char **argv)
 	FILE *report;
 	time_t now = time(NULL);
 	char *option_output = NULL;
+	int nongit_ok = 0;
 
 	const struct option bugreport_options[] = {
 		OPT_STRING('o', "output", &option_output, N_("path"),
 			   N_("specify a destination for the bugreport file")),
 		OPT_END()
 	};
+
+	/* Prerequisite for hooks and config checks */
+	setup_git_directory_gently(&nongit_ok);
+
 	argc = parse_options(argc, argv, "", bugreport_options,
 			     bugreport_usage, 0);
 
@@ -118,6 +190,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v5 11/15] bugreport: collect list of populated hooks
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (9 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 10/15] bugreport: add config values from safelist emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-02-04 18:44     ` Junio C Hamano
  2020-01-24  3:34   ` [PATCH v5 12/15] bugreport: count loose objects emilyshaffer
                     ` (7 subsequent siblings)
  18 siblings, 1 reply; 177+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

Occasionally a failure a user is seeing may be related to a specific
hook which is being run, perhaps without the user realizing. While the
contents of hooks can be sensitive - containing user data or process
information specific to the user's organization - simply knowing that a
hook is being run at a certain stage can help us to understand whether
something is going wrong.

Without a definitive list of hook names within the code, we compile our
own list from the documentation. This is likely prone to bitrot. To
reduce the amount of code humans need to read, we turn the list into a
string_list and iterate over it (as we are calling the same find_hook
operation on each string). However, since bugreport should primarily be
called by the user, the performance loss from massaging the string
seems acceptable.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 7a9fd36b60..4c77009f1b 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "bugreport-config-safelist.h"
 #include "khash.h"
+#include "run-command.h"
 
 static void get_curl_version_info(struct strbuf *curl_info)
 {
@@ -121,6 +122,55 @@ static void get_safelisted_config(struct strbuf *config_info)
 	cfgset_clear(&safelist);
 }
 
+static void get_populated_hooks(struct strbuf *hook_info, int nongit)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hook`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+
+
+	if (nongit) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -193,6 +243,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Configured Hooks");
+	get_populated_hooks(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v5 12/15] bugreport: count loose objects
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (10 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 11/15] bugreport: collect list of populated hooks emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-02-04 18:48     ` Junio C Hamano
  2020-01-24  3:34   ` [PATCH v5 13/15] bugreport: add packed object summary emilyshaffer
                     ` (6 subsequent siblings)
  18 siblings, 1 reply; 177+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Johannes Schindelin

From: Emily Shaffer <emilyshaffer@google.com>

The number of unpacked objects in a user's repository may help us
understand the root of the problem they're seeing, especially if a
command is running unusually slowly.

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
The refactor removed much of the code Dscho suggested; and yet it
remains true that he helped me while developing this commit (although
his suggestions didn't survive). Shall I leave the Helped-by line or
remove it?

 - Emily


 bugreport.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 4c77009f1b..bf10857183 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -10,6 +10,7 @@
 #include "bugreport-config-safelist.h"
 #include "khash.h"
 #include "run-command.h"
+#include "object-store.h"
 
 static void get_curl_version_info(struct strbuf *curl_info)
 {
@@ -171,6 +172,48 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 }
 
+static int loose_object_cb(const struct object_id *oid, const char *path,
+			   void *data) {
+	int *loose_object_count = data;
+
+	if (loose_object_count) {
+		(*loose_object_count)++;
+		return 0;
+	}
+
+	return 1;
+}
+
+static void get_loose_object_summary(struct strbuf *obj_info) {
+
+	int local_loose_object_count = 0, total_loose_object_count = 0;
+	int local_count_questionable = 0, total_count_questionable = 0;
+
+	local_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&local_loose_object_count,
+					FOR_EACH_OBJECT_LOCAL_ONLY);
+
+	total_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&total_loose_object_count,
+					0);
+
+	strbuf_addf(obj_info, "%d local loose objects%s\n",
+		    local_loose_object_count,
+		    local_count_questionable ? " (problem during count)" : "");
+
+	strbuf_addf(obj_info, "%d alternate loose objects%s\n",
+		    total_loose_object_count - local_loose_object_count,
+		    (local_count_questionable || total_count_questionable)
+			? " (problem during count)"
+			: "");
+
+	strbuf_addf(obj_info, "%d total loose objects%s\n",
+		    total_loose_object_count,
+		    total_count_questionable ? " (problem during count)" : "");
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -246,6 +289,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Configured Hooks");
 	get_populated_hooks(&buffer, nongit_ok);
 
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v5 13/15] bugreport: add packed object summary
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (11 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 12/15] bugreport: count loose objects emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-02-04 19:00     ` Junio C Hamano
  2020-02-04 19:03     ` Junio C Hamano
  2020-01-24  3:34   ` [PATCH v5 14/15] bugreport: list contents of $OBJDIR/info emilyshaffer
                     ` (5 subsequent siblings)
  18 siblings, 2 replies; 177+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

Alongside the loose object counts, it can be useful to show the number
of packs and packed objects. This way we can check whether the repo has
an appropriate ratio of packed to loose objects to help determine
whether it's behaving correctly.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index bf10857183..45cc1764e0 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -214,6 +214,41 @@ static void get_loose_object_summary(struct strbuf *obj_info) {
 		    total_count_questionable ? " (problem during count)" : "");
 }
 
+struct packed_object_cb_data {
+	struct packed_git *last_pack;
+	int pack_count;
+	int object_count;
+};
+
+static int packed_object_cb(const struct object_id *oid,
+			    struct packed_git *pack,
+			    uint32_t pos,
+			    void *data) {
+	struct packed_object_cb_data *cb_data = data;
+
+	if (!cb_data)
+		return 1;
+
+	if (pack && pack != cb_data->last_pack) {
+		cb_data->last_pack = pack;
+		cb_data->pack_count++;
+	}
+
+	cb_data->object_count++;
+
+	return 0;
+}
+
+static void get_packed_object_summary(struct strbuf *obj_info)
+{
+	struct packed_object_cb_data cb_data = {NULL, 0, 0};
+
+	for_each_packed_object(packed_object_cb, &cb_data, 0);
+
+	strbuf_addf(obj_info, "%d total packs (%d objects)\n",
+		    cb_data.pack_count, cb_data.object_count);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -292,6 +327,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v5 14/15] bugreport: list contents of $OBJDIR/info
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (12 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 13/15] bugreport: add packed object summary emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-24  3:34   ` [PATCH v5 15/15] bugreport: summarize contents of alternates file emilyshaffer
                     ` (4 subsequent siblings)
  18 siblings, 0 replies; 177+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

Miscellaneous information used about the object store can end up in
.git/objects/info; this can help us understand what may be going on with
the object store when the user is reporting a bug. Otherwise, it could
be difficult to track down what is going wrong with an object which
isn't kept locally to .git/objects/ or .git/objects/pack. Having some
understanding of where the user's objects may be kept can save us some
hops during the bug reporting process.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 45cc1764e0..12463ffdc7 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -249,6 +249,51 @@ static void get_packed_object_summary(struct strbuf *obj_info)
 		    cb_data.pack_count, cb_data.object_count);
 }
 
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
+{
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
+
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(obj_info, &dirpath);
+
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -330,6 +375,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Packed Object Summary");
 	get_packed_object_summary(&buffer);
 
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v5 15/15] bugreport: summarize contents of alternates file
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (13 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 14/15] bugreport: list contents of $OBJDIR/info emilyshaffer
@ 2020-01-24  3:34   ` emilyshaffer
  2020-01-24  3:38   ` [PATCH v5 00/15] add git-bugreport tool Emily Shaffer
                     ` (3 subsequent siblings)
  18 siblings, 0 replies; 177+ messages in thread
From: emilyshaffer @ 2020-01-24  3:34 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

From: Emily Shaffer <emilyshaffer@google.com>

In some cases, it could be that the user is having a problem with an
object which isn't present in their normal object directory. We can get
a hint that that might be the case by examining the list of alternates
where their object may be stored instead. Since paths to alternates may
be sensitive, we'll instead count how many alternates have been
specified and note how many of them exist or are broken.

While object-cache.h describes a function "foreach_alt_odb()", this
function does not provide information on broken alternates, which are
skipped over in "link_alt_odb_entry()". Since the goal is to identify
missing alternates, we can gather the contents of
.git/objects/info/alternates manually.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 bugreport.c | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)

diff --git a/bugreport.c b/bugreport.c
index 12463ffdc7..2c540182dd 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -294,6 +294,42 @@ static void get_object_info_summary(struct strbuf *obj_info)
 	strbuf_release(&dirpath);
 }
 
+static void get_alternates_summary(struct strbuf *alternates_info)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output <file>]"),
 	NULL
@@ -378,6 +414,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 	strbuf_write(&buffer, report);
 	fclose(report);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 00/15] add git-bugreport tool
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (14 preceding siblings ...)
  2020-01-24  3:34   ` [PATCH v5 15/15] bugreport: summarize contents of alternates file emilyshaffer
@ 2020-01-24  3:38   ` Emily Shaffer
  2020-01-28 23:04   ` Jonathan Tan
                     ` (2 subsequent siblings)
  18 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-01-24  3:38 UTC (permalink / raw)
  To: git
  Cc: Derrick Stolee, Johannes Schindelin, Junio C Hamano,
	Martin Ågren, Aaron Schrab

A sample bugreport generated from this patchset follows:

Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)

What did you expect to happen? (Expected behavior)

What happened instead? (Actual behavior)

What's different between what you expected and what actually happened?

Anything else you want to add:

Please review the rest of the bug report below.
You can delete any lines you don't wish to send.


[System Info]
git version:
git version 2.25.0.18.g682ab0d3eb
cpu: x86_64
built from commit: 682ab0d3eb8b84f8af4db1a161d24ca53d2f39fc
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
uname -a: Linux 5.2.17-1rodete3-amd64 #1 SMP Debian 5.2.17-1rodete3 (2019-10-21 > 2018) x86_64
compiler info: glibc: 2.28
$SHELL (typically, interactive shell): /bin/bash
git-remote-https --build-info:
git-http-fetch version: 2.25.0.18.g682ab0d3eb
built from commit: 682ab0d3eb8b84f8af4db1a161d24ca53d2f39fc
curl version: libcurl/7.66.0 GnuTLS/3.6.9 zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.20.2 (+libidn2/2.2.0) libssh2/1.8.0 nghttp2/1.39.2 librtmp/2.3


[Safelisted Config Info]
sendemail.from (global) : emilyshaffer@google.com
sendemail.from (repo) : emilyshaffer@google.com


[Configured Hooks]
pre-commit
prepare-commit-msg


[Loose Object Counts]
2549 local loose objects
2641 alternate loose objects
5190 total loose objects


[Packed Object Summary]
60 total packs (591308 objects)


[Object Info Summary]
.git/objects/info/foo
.git/objects/info/bar
.git/objects/info/bar/baz
.git/objects/info/alternates


[Alternates]
2 alternates found (1 working, 1 broken)

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 00/15] add git-bugreport tool
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (15 preceding siblings ...)
  2020-01-24  3:38   ` [PATCH v5 00/15] add git-bugreport tool Emily Shaffer
@ 2020-01-28 23:04   ` Jonathan Tan
  2020-01-28 23:26     ` Emily Shaffer
  2020-01-30 22:15   ` Martin Ågren
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
  18 siblings, 1 reply; 177+ messages in thread
From: Jonathan Tan @ 2020-01-28 23:04 UTC (permalink / raw)
  To: emilyshaffer
  Cc: git, stolee, Johannes.Schindelin, gitster, martin.agren, aaron,
	Jonathan Tan

> From: Emily Shaffer <emilyshaffer@google.com>
> 
> This topic branch depends on the patch mailed in
> lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to
> display scopes for configs gathered during "bugreport: add config values from
> safelist".
> 
> I'll summarize v4-v5. Since v4 has languished for some time, I don't
> think an interdiff is too helpful, so I won't include one. Bonus, the
> code is much simplified from some suggestions from Junio on how to
> inspect objects, so I hope it's easy to review anyways.

To everyone in the developer community interested in this set: what is
the status of this?

If this needs further review, then maybe it would be best if only
patches 1-4 were put up for submission first, with a note in the
bugreport documentation that more information may be added in future Git
versions. For me, patches 1-4 look good and I don't have enough
experience with uname (especially across libcs and OSes) to determine
what should or should not be included - if this is typical of reviewers
in the Git project, it might be better to submit patches 1-4 first, and
then send each additional diagnostic separately, so that people who know
what's going on in one area but not another can just comment on the area
they know about.

Having said that, I see that a few people have already looked at the
entire patchset and made comments, so if they are OK with it, we don't
need to split it up.

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 00/15] add git-bugreport tool
  2020-01-28 23:04   ` Jonathan Tan
@ 2020-01-28 23:26     ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-01-28 23:26 UTC (permalink / raw)
  To: Jonathan Tan
  Cc: git, stolee, Johannes.Schindelin, gitster, martin.agren, aaron

On Tue, Jan 28, 2020 at 03:04:21PM -0800, Jonathan Tan wrote:
> > From: Emily Shaffer <emilyshaffer@google.com>
> > 
> > This topic branch depends on the patch mailed in
> > lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to
> > display scopes for configs gathered during "bugreport: add config values from
> > safelist".
> > 
> > I'll summarize v4-v5. Since v4 has languished for some time, I don't
> > think an interdiff is too helpful, so I won't include one. Bonus, the
> > code is much simplified from some suggestions from Junio on how to
> > inspect objects, so I hope it's easy to review anyways.
> 
> To everyone in the developer community interested in this set: what is
> the status of this?
> 
> If this needs further review, then maybe it would be best if only
> patches 1-4 were put up for submission first, with a note in the
> bugreport documentation that more information may be added in future Git
> versions. For me, patches 1-4 look good and I don't have enough
> experience with uname (especially across libcs and OSes) to determine
> what should or should not be included - if this is typical of reviewers
> in the Git project, it might be better to submit patches 1-4 first, and
> then send each additional diagnostic separately, so that people who know
> what's going on in one area but not another can just comment on the area
> they know about.
> 
> Having said that, I see that a few people have already looked at the
> entire patchset and made comments, so if they are OK with it, we don't
> need to split it up.

I'd be fine with either; from now, my gut says the only ones I'm not
comfortable merging today are maybe 10, as Junio had some concerns about
whether to allow glob expansion, and 11, as it's prone to rot and I'm
doing other work in that area which would cause 11 to be
dropped/refactored anyways once it goes in. Otherwise I think they're
all pretty OK to go in and let the other work continue later in a
different thread.

Of course, as an author my opinion that they're good to go doesn't mean
much ;)

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 00/15] add git-bugreport tool
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (16 preceding siblings ...)
  2020-01-28 23:04   ` Jonathan Tan
@ 2020-01-30 22:15   ` Martin Ågren
  2020-02-04  0:07     ` Emily Shaffer
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
  18 siblings, 1 reply; 177+ messages in thread
From: Martin Ågren @ 2020-01-30 22:15 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: Git Mailing List, Derrick Stolee, Johannes Schindelin,
	Junio C Hamano, Aaron Schrab

On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
> This topic branch depends on the patch mailed in
> lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to
> display scopes for configs gathered during "bugreport: add config values from
> safelist".

Should this use `config_scope_name()` which looks like it's about
to graduate [1]? Disclaimer: I haven't followed that patch set too closely.

[1] https://lore.kernel.org/git/xmqqzhe66dav.fsf@gitster-ct.c.googlers.com/



Martin

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 01/15] bugreport: add tool to generate debugging info
  2020-01-24  3:34   ` [PATCH v5 01/15] bugreport: add tool to generate debugging info emilyshaffer
@ 2020-01-30 22:18     ` Martin Ågren
  2020-02-04 22:00       ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Martin Ågren @ 2020-01-30 22:18 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 05:56, <emilyshaffer@google.com> wrote:
>
> From: Emily Shaffer <emilyshaffer@google.com>
>
> Teach Git how to prompt the user for a good bug report: reproduction
> steps, expected behavior, and actual behavior. Later, Git can learn how
> to collect some diagnostic information from the repository.

("Later" meaning "later in this series", or "any day now"? ;-) )

> +SYNOPSIS
> +--------
> +[verse]
> +'git bugreport' [-o | --output <path>]

Hmm. Should that be "[(-o | --output) <path>]"?

> +DESCRIPTION
> +-----------
> +Captures information about the user's machine, Git client, and repository state,
> +as well as a form requesting information about the behavior the user observed,
> +into a single text file which the user can then share, for example to the Git
> +mailing list, in order to report an observed bug.

Nice description. Got it.

> +The following information is requested from the user:
> +
> + - Reproduction steps
> + - Expected behavior
> + - Actual behavior
> +
> +The following information is captured automatically:
> +
> + - Git version (`git version --build-options`)
> + - Machine information (`uname -a`)
> + - Versions of various dependencies
> + - Git config contents (`git config --show-origin --list`)
> + - The names of all configured git-hooks in `.git/hooks/`

I would have expected these points to appear later, both to make it
clear what this does commit does (and not), and to highlight what
user-visible (documentation-worthy) changes later commits bring along.

> +OPTIONS
> +-------
> +-o [<path>]::
> +--output [<path>]::

Drop the "[" and "]"? If you give -o, you'd better give a path as well?

> +       Place the resulting bug report file in <path> instead of the root of the

`<path>`

> +"Please review the rest of the bug report below.\n"
> +"You can delete any lines you don't wish to send.\n");

"send" sounds like we're *just* about to send this report somewhere, but
it's "only" going to be written to the disk. Maybe "share", instead?

> +       if (option_output) {
> +               strbuf_addstr(&report_path, option_output);
> +               strbuf_complete(&report_path, '/');
> +       }

I thought I'd use `-o` to indicate the filename, but it turns out it's
the *directory* where the file (of some semi-random, generated name)
will end up. Re-reading the docs further up, I can see how this is
consistent. I sort of wonder if this should be `--output*-directory*`
for symmetry with `git format-patch`.

> +       strbuf_addstr(&report_path, "git-bugreport-");
> +       strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
> +       strbuf_addstr(&report_path, ".txt");
> +
> +

(Double blank line?)

> +       get_bug_template(&buffer);
> +
> +       report = fopen_for_writing(report_path.buf);

Report might be NULL here.

If there's already such a file, we overwrite. Should we generate the
filename using not just today's date (two bug reports in a day wouldn't
be unheard of?) but also something like hh:mm:ss?

> +       strbuf_write(&buffer, report);
> +       fclose(report);

Maybe clear the strbuf around here...

> +       launch_editor(report_path.buf, NULL, NULL);
> +       return 0;

... and/or UNLEAK it here, together with report_path.

Maybe "return -launch_editor(...)"?

> +#!/bin/bash

Use /bin/sh instead?

> +# Headers "[System Info]" will be followed by a non-empty line if we put some
> +# information there; we can make sure all our headers were followed by some
> +# information to check if the command was successful.
> +HEADER_PATTERN="^\[.*\]$"
> +check_all_headers_populated() {
> +       while read -r line; do
> +               if [$(grep $HEADER_PATTERN $line)]; then

I think this is a bash-ism.

> +                       read -r nextline
> +                       if [-z $nextline]; then

Likewise.

> +                               return 1;
> +                       fi
> +               fi
> +       done
> +}
> +
> +test_expect_success 'creates a report with content in the right places' '
> +       git bugreport &&
> +       check_all_headers_populated <git-bugreport-* &&
> +       rm git-bugreport-*
> +'
> +
> +test_expect_success '--output puts the report in the provided dir' '
> +       mkdir foo/ &&

If foo isn't there, do we not create it? Apparently not -- in my
testing, we segfault. (We don't check for NULL after opening the file.)

> +       git bugreport -o foo/ &&
> +       test -f foo/git-bugreport-* &&

test_path_is_file

> +       rm -fr foo/
> +'
> +
> +test_expect_success 'incorrect arguments abort with usage' '
> +       test_must_fail git bugreport --false 2>output &&
> +       grep usage output &&
> +       test ! -f git-bugreport-*

test_path_is_missing


Martin

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 02/15] help: move list_config_help to builtin/help
  2020-01-24  3:34   ` [PATCH v5 02/15] help: move list_config_help to builtin/help emilyshaffer
@ 2020-01-30 22:19     ` Martin Ågren
  2020-02-04  0:53       ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Martin Ågren @ 2020-01-30 22:19 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> To make help.o usable by other commands again, move list_config_help()
> into builtin/help.c (where it makes sense to assume other builtin libraries
> are present).

Nit: I think this could be patch 01/15. It shuffles things around --
*then*, starting with patch 02/15, we can add exciting stuff.


Martin

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 03/15] bugreport: gather git version and build info
  2020-01-24  3:34   ` [PATCH v5 03/15] bugreport: gather git version and build info emilyshaffer
@ 2020-01-30 22:19     ` Martin Ågren
  2020-02-04 22:21       ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Martin Ågren @ 2020-01-30 22:19 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> +static void get_system_info(struct strbuf *sys_info)
> +{
> +       struct strbuf version_info = STRBUF_INIT;
> +
> +       /* get git version from native cmd */
> +       strbuf_addstr(sys_info, "git version:\n");
> +       get_version_info(&version_info, 1);
> +       strbuf_addbuf(sys_info, &version_info);
> +       strbuf_complete_line(sys_info);

This leaks version_info.

> +}


Martin

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 04/15] help: add shell-path to --build-options
  2020-01-24  3:34   ` [PATCH v5 04/15] help: add shell-path to --build-options emilyshaffer
@ 2020-01-30 22:21     ` Martin Ågren
  0 siblings, 0 replies; 177+ messages in thread
From: Martin Ågren @ 2020-01-30 22:21 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> It may be useful to know which shell Git was built to try to point to,
> in the event that shell-based Git commands are failing. $SHELL_PATH is
> set during the build and used to launch the manpage viewer, as well as
> by git-compat-util.h, and it's used during tests. 'git version
> --build-options' is encouraged for use in bug reports, so it makes sense
> to include this information there.

Hmm, similar nit as for patch 02/15: This could go in as, e.g., 02/15 and
your 01/15 would be 03/15: This patch would be useful even without `git
bugreport`.


Martin

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 06/15] bugreport: add compiler info
  2020-01-24  3:34   ` [PATCH v5 06/15] bugreport: add compiler info emilyshaffer
@ 2020-01-30 22:21     ` Martin Ågren
  2020-02-04 22:51       ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Martin Ågren @ 2020-01-30 22:21 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> +#else
> +
> +static inline void get_compiler_info(struct strbuf *info)
> +{
> +       strbuf_addstr(info, "get_compiler_info() not implemented");

Maybe "no compiler info available" (or s/ available//, or
s/available/reported/), or something else more human-readable?


> +}
> +
> +#endif
> +
> +#endif /* COMPILER_H */

Martin

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 07/15] bugreport: add curl version
  2020-01-24  3:34   ` [PATCH v5 07/15] bugreport: add curl version emilyshaffer
@ 2020-01-30 22:27     ` Martin Ågren
  2020-02-04 22:54       ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Martin Ågren @ 2020-01-30 22:27 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
> +static void get_curl_version_info(struct strbuf *curl_info)
> +{
> +       struct child_process cp = CHILD_PROCESS_INIT;
> +
> +       argv_array_push(&cp.args, "git");
> +       argv_array_push(&cp.args, "remote-https");
> +       argv_array_push(&cp.args, "--build-info");
> +       if (capture_command(&cp, curl_info, 0))
> +           strbuf_addstr(curl_info, "'git-remote-https --build-info' not supported\n");
> +}
>
>  static void get_system_info(struct strbuf *sys_info)
>  {
> @@ -31,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
>         strbuf_addstr(sys_info, "compiler info: ");
>         get_compiler_info(sys_info);
>         strbuf_complete_line(sys_info);
> +
> +       strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
> +       get_curl_version_info(sys_info);

The header here looks a lot like an implementation detail of
`get_curl_version_info()`. Or put differently, these risk getting out of
sync. Maybe frame the header a bit more human readable: "curl version".
But is this "curl version", or more like "git-remote-https version"?
There's some discrepancy here.

> +       strbuf_complete_line(sys_info);
>  }


Martin

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 08/15] bugreport: include user interactive shell
  2020-01-24  3:34   ` [PATCH v5 08/15] bugreport: include user interactive shell emilyshaffer
@ 2020-01-30 22:28     ` Martin Ågren
  2020-02-04 23:16       ` Emily Shaffer
  2020-02-05 20:06       ` Junio C Hamano
  0 siblings, 2 replies; 177+ messages in thread
From: Martin Ågren @ 2020-01-30 22:28 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
> +       char *shell = NULL;

(Unnecessary initialization.)

> +       shell = getenv("SHELL");
> +       strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
> +                   shell ? shell : "(NULL)");

Thanks for avoiding a classic pitfall. :-)

"<unused>" instead of "(NULL)"? "NULL" is mostly an implementation
detail.


Martin

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
  2020-01-24  3:34   ` [PATCH v5 09/15] bugreport: generate config safelist based on docs emilyshaffer
@ 2020-01-30 22:34     ` Martin Ågren
  2020-02-05  0:44       ` Emily Shaffer
  2020-01-31 21:20     ` Martin Ågren
  1 sibling, 1 reply; 177+ messages in thread
From: Martin Ågren @ 2020-01-30 22:34 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List, Johannes Schindelin

On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
> Add a new step to the build to generate a safelist of git-config
> variables which are appropriate to include in the output of
> git-bugreport. New variables can be added to the safelist by annotating
> their documentation in Documentation/config with the
> "bugreport" macro, which is recognized by AsciiDoc and
> AsciiDoctor.
>
> Some configs are private in nature, and can contain remote URLs,
> passwords, or other sensitive information. In the event that a user
> doesn't notice their information while reviewing a bugreport, that user
> may leak their credentials to other individuals, mailing lists, or bug
> tracking tools inadvertently. Heuristic blocklisting of configuration
> keys is imperfect and prone to false negatives; given the nature of the
> information which can be leaked, a safelist is more reliable.

I sort of wonder whether safelist/blocklist is to prefer over
whitelist/blacklist, or if it's the other way round. The former are new
to me, whereas the latter are the terms I would have used. But that's
just me, of course. I was a little surprised, that's all.

> Implement a new no-op "bugreport" macro for use as
> "bugreport:include[x]" to annotate the config keys that should be
> included in the automatically generated safelist. Use "exclude" for the
> others.
>
> With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc
> seems to want something between the brackets. A bit unfortunate, but
> not a huge problem -- we'll just provide an "x".

I recognize this reasoning :-) and I'm not terribly opposed to it, but
after some nights' sleeping on this, I have to wonder if
"annotate:bugreport[include]" wouldn't be better than "bugreport[x]"
with that ugly "x". Maybe this isn't the biggest problem, but if we
expect this macro to eventually sit right next to ~90% of all our config
variables...

> "doc-diff" reports that this macro doesn't render at all. That is,
> these are both empty after this commit:
>
>   cd Documentation
>   ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
>   ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD

That was true in [1], but alas, no more. In that patch, it's sort of
obvious from the diff how it adds a "class" which "end"s.

[1] https://lore.kernel.org/git/20190817203846.31609-1-martin.agren@gmail.com/

> --- a/Documentation/asciidoctor-extensions.rb
> +++ b/Documentation/asciidoctor-extensions.rb
> @@ -37,6 +37,10 @@ module Git
>            output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
>          end
>          output
> +
> +    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
> +      def process(parent, action, attrs)
> +        ""
>        end
>      end
>    end

But this one doesn't add an "end", and Asciidoctor trips up badly.

> +  # The bugreport macro does nothing as far as rendering is
> +  # concerned -- we just grep for it in the sources.
> +  inline_macro Git::Documentation::BugReportProcessor, :bugreport

(I never much liked this copy-paste comment then, and I still don't, to
be honest.)



Martin

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 10/15] bugreport: add config values from safelist
  2020-01-24  3:34   ` [PATCH v5 10/15] bugreport: add config values from safelist emilyshaffer
@ 2020-01-30 22:36     ` Martin Ågren
  2020-02-05  1:34       ` Emily Shaffer
  2020-01-31 21:25     ` Martin Ågren
  1 sibling, 1 reply; 177+ messages in thread
From: Martin Ågren @ 2020-01-30 22:36 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> --- a/Makefile
> +++ b/Makefile
> @@ -2465,7 +2465,7 @@ endif
>  git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
>         $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
>
> -git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
> +git-bugreport$X: bugreport-config-safelist.h bugreport.o GIT-LDFLAGS $(GITLIBS)
>         $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
>                 $(LIBS)

Haven't looked at this patch at all, except I've noticed that something
is up with the dependencies. I think bugreport.o needs to depend on
bugreport-config-safelist.h. As it is, the latter may or may not be
available as bugreport.o is built. (Reproduces fairly well for me with
something like `make clean && make -j16`.)

I'll try to continue tomorrow. :-)


Martin

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
  2020-01-24  3:34   ` [PATCH v5 09/15] bugreport: generate config safelist based on docs emilyshaffer
  2020-01-30 22:34     ` Martin Ågren
@ 2020-01-31 21:20     ` Martin Ågren
  2020-02-05  0:30       ` Emily Shaffer
  1 sibling, 1 reply; 177+ messages in thread
From: Martin Ågren @ 2020-01-31 21:20 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List, Johannes Schindelin

On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
> Add a new step to the build to generate a safelist of git-config
> variables which are appropriate to include in the output of
> git-bugreport. New variables can be added to the safelist by annotating
> their documentation in Documentation/config with the
> "bugreport" macro, which is recognized by AsciiDoc and
> AsciiDoctor.

"which is recognized by" sounds like it's built-in. Maybe "with a new
no-op 'bugreport' macro" or something like that.

> -sendemail.signedoffcc (deprecated)::
> +sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
>         Deprecated alias for `sendemail.signedoffbycc`.
>
> -sendemail.smtpBatchSize::
> +sendemail.smtpBatchSize bugreport:include[x] ::
>         Number of messages to be sent per connection, after that a relogin
>         will happen.  If the value is 0 or undefined, send all messages in
>         one connection.
>         See also the `--batch-size` option of linkgit:git-send-email[1].

Do we really need to list includes *and* excludes? I could see the point
of adding an exclude here and there to signal that "this might look
innocent enough, but trust me, we really need to exclude this" in order
to avoid future commits to more or less accidentally over-include.
Should we add some internal documentation and/or a remark in the commit
message about this? As an example, is "sendemail.signedofcc" sensitive
enough that we need to explicitly exclude it? If someone wants to come
along and include it, they don't just need to argue for an inclusion,
but also for lifting the exclusion. Hmm?

Martin

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 10/15] bugreport: add config values from safelist
  2020-01-24  3:34   ` [PATCH v5 10/15] bugreport: add config values from safelist emilyshaffer
  2020-01-30 22:36     ` Martin Ågren
@ 2020-01-31 21:25     ` Martin Ågren
  2020-02-05  2:31       ` Emily Shaffer
  1 sibling, 1 reply; 177+ messages in thread
From: Martin Ågren @ 2020-01-31 21:25 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> Taking the build-time generated array and putting it into a set saves us
> time - since git_config_bugreport() is called for every option the user
> has configured, performing option lookup in constant time is a useful
> optimization.

I'm sympathetic to your sending out what you have to obtain comments,
knowing that it's not perfect. It would have saved me some time and
effort if I'd known that this was the case though. An "[RFC]" tag,
perhaps. Or at least tweaking the above part of this commit message to
say that this might be over-engineered, with a reference to [1].

[1] https://lore.kernel.org/git/20200124032905.GA37541@google.com/

> +       int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);

I was going to suggest ARRAY_SIZE, but then I realized there are some
outstanding questions around whether you need this stuff in the first
place. I'd be inclined to guess that the first version of this would be
"for each safelisted item, obtain it and include", ignoring any "a.*.b"
business. In which case you wouldn't really need this hashset stuff.


Martin

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 00/15] add git-bugreport tool
  2020-01-30 22:15   ` Martin Ågren
@ 2020-02-04  0:07     ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-04  0:07 UTC (permalink / raw)
  To: Martin Ågren
  Cc: Git Mailing List, Derrick Stolee, Johannes Schindelin,
	Junio C Hamano, Aaron Schrab

On Thu, Jan 30, 2020 at 11:15:25PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
> > This topic branch depends on the patch mailed in
> > lore.kernel.org/git/20191211233820.185153-1-emilyshaffer@google.com in order to
> > display scopes for configs gathered during "bugreport: add config values from
> > safelist".
> 
> Should this use `config_scope_name()` which looks like it's about
> to graduate [1]? Disclaimer: I haven't followed that patch set too closely.

Yeah, you're right that I can use it. config.h:config_scope_name() is a
new feature on that patch since last time I looked. Thanks for the tip!

 - Emily

> [1] https://lore.kernel.org/git/xmqqzhe66dav.fsf@gitster-ct.c.googlers.com/

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 02/15] help: move list_config_help to builtin/help
  2020-01-30 22:19     ` Martin Ågren
@ 2020-02-04  0:53       ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-04  0:53 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List

On Thu, Jan 30, 2020 at 11:19:26PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > To make help.o usable by other commands again, move list_config_help()
> > into builtin/help.c (where it makes sense to assume other builtin libraries
> > are present).
> 
> Nit: I think this could be patch 01/15. It shuffles things around --
> *then*, starting with patch 02/15, we can add exciting stuff.

Sounds reasonable, here and 04/15. Will do.

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 11/15] bugreport: collect list of populated hooks
  2020-01-24  3:34   ` [PATCH v5 11/15] bugreport: collect list of populated hooks emilyshaffer
@ 2020-02-04 18:44     ` Junio C Hamano
  2020-02-05  2:48       ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2020-02-04 18:44 UTC (permalink / raw)
  To: emilyshaffer; +Cc: git

emilyshaffer@google.com writes:

> +	/*
> +	 * Doesn't look like there is a list of all possible hooks; so below is
> +	 * a transcription of `git help hook`.

That's "git help hooks", if I tried my reproduction correctly.  A
straight-forward (in the sense of "what we want in the outcome is
quite clear" and not in the sense of "anybody can design and
implement it with a single 30-line patch") follow-up we can make
after this series lands is to rethink how Documentation/githooks.txt
is maintained and the list we have here is synchronized with it.

The design could me just the matter of running "grep" of some sort,
with appropriate markups that are no-op to AsciiDoctor/AsciiDoc
added to the documentation source, to produce this list.

> +	 */
> +	const char *hooks = "applypatch-msg,"
> +			    "pre-applypatch,"
> +			    "post-applypatch,"
> +			    "pre-commit,"
> +			    "pre-merge-commit,"
> +			    "prepare-commit-msg,"
> +			    "commit-msg,"
> +			    "post-commit,"
> +			    "pre-rebase,"
> +			    "post-checkout,"
> +			    "post-merge,"
> +			    "pre-push,"
> +			    "pre-receive,"
> +			    "update,"
> +			    "post-receive,"
> +			    "post-update,"
> +			    "push-to-checkout,"
> +			    "pre-auto-gc,"
> +			    "post-rewrite,"
> +			    "sendemail-validate,"
> +			    "fsmonitor-watchman,"
> +			    "p4-pre-submit,"
> +			    "post-index-changex";
> +	struct string_list hooks_list = STRING_LIST_INIT_DUP;
> +	struct string_list_item *iter = NULL;
> +
> +
> +	if (nongit) {
> +		strbuf_addstr(hook_info,
> +			"not run from a git repository - no hooks to show\n");
> +		return;
> +	}
> +
> +	string_list_split(&hooks_list, hooks, ',', -1);
> +
> +	for_each_string_list_item(iter, &hooks_list) {
> +		if (find_hook(iter->string)) {
> +			strbuf_addstr(hook_info, iter->string);
> +			strbuf_complete_line(hook_info);
> +		}
> +	}
> +}
> +
>  static const char * const bugreport_usage[] = {
>  	N_("git bugreport [-o|--output <file>]"),
>  	NULL
> @@ -193,6 +243,9 @@ int cmd_main(int argc, const char **argv)
>  	get_header(&buffer, "Safelisted Config Info");
>  	get_safelisted_config(&buffer);
>  
> +	get_header(&buffer, "Configured Hooks");

Phrase nit.  There may be many people who just enabled hooks without
configuring, so "Enabled Hooks" may be more appropriate.  We do not
have to inspect what is in the hook to determine if it is enabled,
but we do need to if we want to tell if a hook is "configured".

> +	get_populated_hooks(&buffer, nongit_ok);
> +
>  	report = fopen_for_writing(report_path.buf);
>  	strbuf_write(&buffer, report);
>  	fclose(report);

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 12/15] bugreport: count loose objects
  2020-01-24  3:34   ` [PATCH v5 12/15] bugreport: count loose objects emilyshaffer
@ 2020-02-04 18:48     ` Junio C Hamano
  2020-02-05  2:50       ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2020-02-04 18:48 UTC (permalink / raw)
  To: emilyshaffer; +Cc: git, Johannes Schindelin

emilyshaffer@google.com writes:

> From: Emily Shaffer <emilyshaffer@google.com>
>
> The number of unpacked objects in a user's repository may help us
> understand the root of the problem they're seeing, especially if a
> command is running unusually slowly.
>
> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
> The refactor removed much of the code Dscho suggested; and yet it
> remains true that he helped me while developing this commit (although
> his suggestions didn't survive). Shall I leave the Helped-by line or
> remove it?

You two collectively thought about viable alternatives and decided
to reject what was not wanted in the final result, and not having
that rejected code was good for the project, right?  If so, I would
say it still is the help that deserves recognition.  After all,
making the result better by removing things is harder than by adding
things ;-)

>  bugreport.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 46 insertions(+)
> ...

The patch text looked sensible.

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 13/15] bugreport: add packed object summary
  2020-01-24  3:34   ` [PATCH v5 13/15] bugreport: add packed object summary emilyshaffer
@ 2020-02-04 19:00     ` Junio C Hamano
  2020-02-05  3:15       ` Emily Shaffer
  2020-02-04 19:03     ` Junio C Hamano
  1 sibling, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2020-02-04 19:00 UTC (permalink / raw)
  To: emilyshaffer; +Cc: git

emilyshaffer@google.com writes:

> From: Emily Shaffer <emilyshaffer@google.com>
>
> Alongside the loose object counts, it can be useful to show the number
> of packs and packed objects. This way we can check whether the repo has
> an appropriate ratio of packed to loose objects to help determine
> whether it's behaving correctly.

This lets the enumeration machinery to enumerate at the individual
object level, relies on the enumeration machinery to show your
callback all objects from the same pack before an object from a
different pack, to count both objects and packs.

But given that packfiles record how many objects there are in the
pack, and that the packfile.c layer must know what are the pack
files we have available, I wonder if we have an API already to allow
us to enumerate each packfile, and while counting the number of
times our callback was called, add the objects contained in each of
them.  If there isn't, it does not sound like a bad API function to
add (i.e. given a repository r, iterate over r->objects->packed_git
and let the API caller see each of them; the API caller can grab
num_objects and add them up).


^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 13/15] bugreport: add packed object summary
  2020-01-24  3:34   ` [PATCH v5 13/15] bugreport: add packed object summary emilyshaffer
  2020-02-04 19:00     ` Junio C Hamano
@ 2020-02-04 19:03     ` Junio C Hamano
  2020-02-05  3:09       ` Emily Shaffer
  1 sibling, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2020-02-04 19:03 UTC (permalink / raw)
  To: emilyshaffer; +Cc: git, Derrick Stolee

emilyshaffer@google.com writes:

> From: Emily Shaffer <emilyshaffer@google.com>
>
> Alongside the loose object counts, it can be useful to show the number
> of packs and packed objects. This way we can check whether the repo has
> an appropriate ratio of packed to loose objects to help determine
> whether it's behaving correctly.

This step makes me wonder if we want to see the midx as well.
Didn't we have a bug that manifests only when midx is (or is not) in
use?

Similarly, I think use (or non-use) of the commit graph may also be
a useful information for diagnosing.




^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 01/15] bugreport: add tool to generate debugging info
  2020-01-30 22:18     ` Martin Ågren
@ 2020-02-04 22:00       ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-04 22:00 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List

On Thu, Jan 30, 2020 at 11:18:55PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 05:56, <emilyshaffer@google.com> wrote:
> >
> > From: Emily Shaffer <emilyshaffer@google.com>
> >
> > Teach Git how to prompt the user for a good bug report: reproduction
> > steps, expected behavior, and actual behavior. Later, Git can learn how
> > to collect some diagnostic information from the repository.
> 
> ("Later" meaning "later in this series", or "any day now"? ;-) )
> 
> > +SYNOPSIS
> > +--------
> > +[verse]
> > +'git bugreport' [-o | --output <path>]
> 
> Hmm. Should that be "[(-o | --output) <path>]"?

Done.

> > +DESCRIPTION
> > +-----------
> > +Captures information about the user's machine, Git client, and repository state,
> > +as well as a form requesting information about the behavior the user observed,
> > +into a single text file which the user can then share, for example to the Git
> > +mailing list, in order to report an observed bug.
> 
> Nice description. Got it.
> 
> > +The following information is requested from the user:
> > +
> > + - Reproduction steps
> > + - Expected behavior
> > + - Actual behavior
> > +
> > +The following information is captured automatically:
> > +
> > + - Git version (`git version --build-options`)
> > + - Machine information (`uname -a`)
> > + - Versions of various dependencies
> > + - Git config contents (`git config --show-origin --list`)
> > + - The names of all configured git-hooks in `.git/hooks/`
> 
> I would have expected these points to appear later, both to make it
> clear what this does commit does (and not), and to highlight what
> user-visible (documentation-worthy) changes later commits bring along.

Sure, agreed.

> 
> > +OPTIONS
> > +-------
> > +-o [<path>]::
> > +--output [<path>]::
> 
> Drop the "[" and "]"? If you give -o, you'd better give a path as well?

Done.

> > +       Place the resulting bug report file in <path> instead of the root of the
> 
> `<path>`

Done.

> > +"Please review the rest of the bug report below.\n"
> > +"You can delete any lines you don't wish to send.\n");
> 
> "send" sounds like we're *just* about to send this report somewhere, but
> it's "only" going to be written to the disk. Maybe "share", instead?

Nice turn of phrase, done.

> > +       if (option_output) {
> > +               strbuf_addstr(&report_path, option_output);
> > +               strbuf_complete(&report_path, '/');
> > +       }
> 
> I thought I'd use `-o` to indicate the filename, but it turns out it's
> the *directory* where the file (of some semi-random, generated name)
> will end up. Re-reading the docs further up, I can see how this is
> consistent. I sort of wonder if this should be `--output*-directory*`
> for symmetry with `git format-patch`.

Sure.

> > +       strbuf_addstr(&report_path, "git-bugreport-");
> > +       strbuf_addftime(&report_path, "%F", gmtime(&now), 0, 0);
> > +       strbuf_addstr(&report_path, ".txt");
> > +
> > +
> 
> (Double blank line?)

Done.
 
> > +       get_bug_template(&buffer);
> > +
> > +       report = fopen_for_writing(report_path.buf);
> 
> Report might be NULL here.

Nice.

> If there's already such a file, we overwrite. Should we generate the
> filename using not just today's date (two bug reports in a day wouldn't
> be unheard of?) but also something like hh:mm:ss?

Sure. For the sake of brevity I'll probably neglect seconds; I hope
someone is spending more than 1 minute filling in the provided form.

I'm a little worried about including : in a filename, so I went for
'git-bugreport-YYYY-MM-DD-HHMM' (24-hour).

As I started to write a test to ensure duplicate filenames were handled
well, Jonathan Tan pointed out that it would be easy to add an arg like
--suffix to allow specifying a custom strftime string. That would allow
users to easily create a file named
`git-bugreport-fetch-failing.txt` or `git-bugreport-March-19.txt` or
whatever they want; it also makes testing easy. So I'll add this for the
next rollup.

> 
> > +       strbuf_write(&buffer, report);
> > +       fclose(report);
> 
> Maybe clear the strbuf around here...
> 
> > +       launch_editor(report_path.buf, NULL, NULL);
> > +       return 0;
> 
> ... and/or UNLEAK it here, together with report_path.
> 
> Maybe "return -launch_editor(...)"?

Hm, sure. I see that builtin/tag.c does mark strbufs this way, so I
don't see a problem using UNLEAK and tail-calling launch_editor().


As a final bonus, I also added a line to report to stderr the name of
the file that was created. I noticed it's sort of unclear what the
command actually did otherwise.

> > +#!/bin/bash
> 
> Use /bin/sh instead?

Yeah, doing so immediately pointed out the bashisms you mentioned, plus
some more. :facepalm:

> 
> > +# Headers "[System Info]" will be followed by a non-empty line if we put some
> > +# information there; we can make sure all our headers were followed by some
> > +# information to check if the command was successful.
> > +HEADER_PATTERN="^\[.*\]$"
> > +check_all_headers_populated() {
> > +       while read -r line; do
> > +               if [$(grep $HEADER_PATTERN $line)]; then
> 
> I think this is a bash-ism.
> 
> > +                       read -r nextline
> > +                       if [-z $nextline]; then
> 
> Likewise.
> 
> > +                               return 1;
> > +                       fi
> > +               fi
> > +       done
> > +}
> > +
> > +test_expect_success 'creates a report with content in the right places' '
> > +       git bugreport &&
> > +       check_all_headers_populated <git-bugreport-* &&
> > +       rm git-bugreport-*
> > +'
> > +
> > +test_expect_success '--output puts the report in the provided dir' '
> > +       mkdir foo/ &&
> 
> If foo isn't there, do we not create it? Apparently not -- in my
> testing, we segfault. (We don't check for NULL after opening the file.)

Yeah, at the moment I just added a die() if we can't open the provided
path. I think other utilties can create the path (e.g. git-format-patch)
but where it makes sense to do so, I'd prefer to keep bugreport very
simple.

I'll die instead of overwriting, too; you're right that spending quite a
while on a bug report and then accidentally writing over it with a blank
one would be a very bad user experience.

> 
> > +       git bugreport -o foo/ &&
> > +       test -f foo/git-bugreport-* &&
> 
> test_path_is_file

Sure.

> > +       rm -fr foo/
> > +'
> > +
> > +test_expect_success 'incorrect arguments abort with usage' '
> > +       test_must_fail git bugreport --false 2>output &&
> > +       grep usage output &&
> > +       test ! -f git-bugreport-*
> 
> test_path_is_missing

OK.

Thanks very much, Martin, for the thorough review. This is incredibly
helpful.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 03/15] bugreport: gather git version and build info
  2020-01-30 22:19     ` Martin Ågren
@ 2020-02-04 22:21       ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-04 22:21 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List

On Thu, Jan 30, 2020 at 11:19:55PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > +static void get_system_info(struct strbuf *sys_info)
> > +{
> > +       struct strbuf version_info = STRBUF_INIT;
> > +
> > +       /* get git version from native cmd */
> > +       strbuf_addstr(sys_info, "git version:\n");
> > +       get_version_info(&version_info, 1);
> > +       strbuf_addbuf(sys_info, &version_info);
> > +       strbuf_complete_line(sys_info);
> 
> This leaks version_info.

So it does.

I think the easiest thing is to drop the 'strbuf_reset()' at the top of
help.c:get_version_info(), and just pass in sys_info from bugreport.c.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 06/15] bugreport: add compiler info
  2020-01-30 22:21     ` Martin Ågren
@ 2020-02-04 22:51       ` Emily Shaffer
  2020-02-05 19:47         ` Martin Ågren
  0 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-04 22:51 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List

On Thu, Jan 30, 2020 at 11:21:39PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > +#else
> > +
> > +static inline void get_compiler_info(struct strbuf *info)
> > +{
> > +       strbuf_addstr(info, "get_compiler_info() not implemented");
> 
> Maybe "no compiler info available" (or s/ available//, or
> s/available/reported/), or something else more human-readable?

Hm. I envisioned the target audience for this as "Git developers" (and
that's why I didn't mark the unimplemented string for translation); Git
developers know exactly what "get_compiler_info() not implemented"
means. But I suppose it's just as easy to grep for one string as the
other.

I am hesitant to say "no info available" or "no info reported" when I'm
certain there is some environment variable set at build time which
contains the info we'd want to show here; it really is just a question
of some Git developer having put in the time to implement this function.

So, I think I will leave it. Thanks for your suggestions.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 07/15] bugreport: add curl version
  2020-01-30 22:27     ` Martin Ågren
@ 2020-02-04 22:54       ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-04 22:54 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List

On Thu, Jan 30, 2020 at 11:27:45PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
> > +static void get_curl_version_info(struct strbuf *curl_info)
> > +{
> > +       struct child_process cp = CHILD_PROCESS_INIT;
> > +
> > +       argv_array_push(&cp.args, "git");
> > +       argv_array_push(&cp.args, "remote-https");
> > +       argv_array_push(&cp.args, "--build-info");
> > +       if (capture_command(&cp, curl_info, 0))
> > +           strbuf_addstr(curl_info, "'git-remote-https --build-info' not supported\n");
> > +}
> >
> >  static void get_system_info(struct strbuf *sys_info)
> >  {
> > @@ -31,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
> >         strbuf_addstr(sys_info, "compiler info: ");
> >         get_compiler_info(sys_info);
> >         strbuf_complete_line(sys_info);
> > +
> > +       strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
> > +       get_curl_version_info(sys_info);
> 
> The header here looks a lot like an implementation detail of
> `get_curl_version_info()`. Or put differently, these risk getting out of
> sync. Maybe frame the header a bit more human readable: "curl version".
> But is this "curl version", or more like "git-remote-https version"?
> There's some discrepancy here.

Hm, I think you're saying "If we switch to future-https-lib instead of
cURL for git-remote-https, then this command will be incorrectly named."
Sure, I agree. It's true that with this change git-remote-https also
tells us some info about itself.

Thanks.
 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 08/15] bugreport: include user interactive shell
  2020-01-30 22:28     ` Martin Ågren
@ 2020-02-04 23:16       ` Emily Shaffer
  2020-02-05 20:06       ` Junio C Hamano
  1 sibling, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-04 23:16 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List

On Thu, Jan 30, 2020 at 11:28:40PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
> > +       char *shell = NULL;
> 
> (Unnecessary initialization.)
> 
> > +       shell = getenv("SHELL");
> > +       strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
> > +                   shell ? shell : "(NULL)");
> 
> Thanks for avoiding a classic pitfall. :-)

Thank Junio. I fell right into it because it Just Works in gcc. ;)

> "<unused>" instead of "(NULL)"? "NULL" is mostly an implementation
> detail.

Sure, why not.

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
  2020-01-31 21:20     ` Martin Ågren
@ 2020-02-05  0:30       ` Emily Shaffer
  2020-02-05  0:52         ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-05  0:30 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List, Johannes Schindelin

On Fri, Jan 31, 2020 at 10:20:05PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
> > Add a new step to the build to generate a safelist of git-config
> > variables which are appropriate to include in the output of
> > git-bugreport. New variables can be added to the safelist by annotating
> > their documentation in Documentation/config with the
> > "bugreport" macro, which is recognized by AsciiDoc and
> > AsciiDoctor.
> 
> "which is recognized by" sounds like it's built-in. Maybe "with a new
> no-op 'bugreport' macro" or something like that.

Sure, I'll wordsmith this a little.

> 
> > -sendemail.signedoffcc (deprecated)::
> > +sendemail.signedoffcc (deprecated) bugreport:exclude[x] ::
> >         Deprecated alias for `sendemail.signedoffbycc`.
> >
> > -sendemail.smtpBatchSize::
> > +sendemail.smtpBatchSize bugreport:include[x] ::
> >         Number of messages to be sent per connection, after that a relogin
> >         will happen.  If the value is 0 or undefined, send all messages in
> >         one connection.
> >         See also the `--batch-size` option of linkgit:git-send-email[1].
> 
> Do we really need to list includes *and* excludes?

Need? Nah. The exclude markers are not even acknowledged by the current
implementation.

> I could see the point of adding an exclude here and there to signal
> that "this might look innocent enough, but trust me, we really need to
> exclude this" in order to avoid future commits to more or less
> accidentally over-include.

This was more of my line of thinking, along with the thought that some
folks may use bug reporting internally, where they have A) more control
over which extensions/tools are in use, and B) less concern about
sharing things like remote URLs, branch names, etc. and may want to go
for a blocklist as opposed to a safelist, to make it easier for their
support folks to diagnose.

> Should we add some internal documentation and/or a remark in the commit
> message about this? As an example, is "sendemail.signedofcc" sensitive
> enough that we need to explicitly exclude it? If someone wants to come
> along and include it, they don't just need to argue for an inclusion,
> but also for lifting the exclusion. Hmm?

Yeah, I think this is a good point. I'll try to figure out how to reword
the commit message, and take another look at the sample include/exclude
change I made to see where I can omit entirely.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
  2020-01-30 22:34     ` Martin Ågren
@ 2020-02-05  0:44       ` Emily Shaffer
  2020-02-05 19:53         ` Martin Ågren
  0 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-05  0:44 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List, Johannes Schindelin

On Thu, Jan 30, 2020 at 11:34:24PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:35, <emilyshaffer@google.com> wrote:
> > Add a new step to the build to generate a safelist of git-config
> > variables which are appropriate to include in the output of
> > git-bugreport. New variables can be added to the safelist by annotating
> > their documentation in Documentation/config with the
> > "bugreport" macro, which is recognized by AsciiDoc and
> > AsciiDoctor.
> >
> > Some configs are private in nature, and can contain remote URLs,
> > passwords, or other sensitive information. In the event that a user
> > doesn't notice their information while reviewing a bugreport, that user
> > may leak their credentials to other individuals, mailing lists, or bug
> > tracking tools inadvertently. Heuristic blocklisting of configuration
> > keys is imperfect and prone to false negatives; given the nature of the
> > information which can be leaked, a safelist is more reliable.
> 
> I sort of wonder whether safelist/blocklist is to prefer over
> whitelist/blacklist, or if it's the other way round. The former are new
> to me, whereas the latter are the terms I would have used. But that's
> just me, of course. I was a little surprised, that's all.

Eh. I think the following things are true:

 - Whitelist/blacklist has a "smell" of discrimination, whether that's
   the true etymology or not.
 - Those with experience in the field can easily understand what
   whitelist or blacklist means.
 - Safelist/blocklist do not "smell" the same way.
 - It is easy to tell what "safelist" means: "a list of stuff which is
   safe." No experience needed.

So, while it's new, I think it's not harmful. I see only a no-op or
positive impact from using this term instead of whitelist/blacklist.
Computer science seems to have quite a few terms which fall into this
long-standing but potentially negative area, so I don't mind looking for
alternatives where it's harmless to do so.

> 
> > Implement a new no-op "bugreport" macro for use as
> > "bugreport:include[x]" to annotate the config keys that should be
> > included in the automatically generated safelist. Use "exclude" for the
> > others.
> >
> > With Asciidoctor, it's ok to say "bugreport:include[]", but AsciiDoc
> > seems to want something between the brackets. A bit unfortunate, but
> > not a huge problem -- we'll just provide an "x".
> 
> I recognize this reasoning :-) and I'm not terribly opposed to it, but
> after some nights' sleeping on this, I have to wonder if
> "annotate:bugreport[include]" wouldn't be better than "bugreport[x]"
> with that ugly "x". Maybe this isn't the biggest problem, but if we
> expect this macro to eventually sit right next to ~90% of all our config
> variables...

Hm. I wanted to say, "Ok, but I don't know how to do that, so can you
help?" But I think that's all the more reason that I should do it ;)

Ok. I will try and change it to annotate:bugreport[include] like you
suggested, and hopefully learn more about asciidoc macros :)

> 
> > "doc-diff" reports that this macro doesn't render at all. That is,
> > these are both empty after this commit:
> >
> >   cd Documentation
> >   ./doc-diff --asciidoctor :/"bugreport: add tool" HEAD
> >   ./doc-diff --asciidoc    :/"bugreport: add tool" HEAD
> 
> That was true in [1], but alas, no more. In that patch, it's sort of
> obvious from the diff how it adds a "class" which "end"s.
> 
> [1] https://lore.kernel.org/git/20190817203846.31609-1-martin.agren@gmail.com/
> 
> > --- a/Documentation/asciidoctor-extensions.rb
> > +++ b/Documentation/asciidoctor-extensions.rb
> > @@ -37,6 +37,10 @@ module Git
> >            output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
> >          end
> >          output
> > +
> > +    class BugReportProcessor < Asciidoctor::Extensions::InlineMacroProcessor
> > +      def process(parent, action, attrs)
> > +        ""
> >        end
> >      end
> >    end
> 
> But this one doesn't add an "end", and Asciidoctor trips up badly.

Ok, I'll have a look. I'm sure I copied something badly.

> 
> > +  # The bugreport macro does nothing as far as rendering is
> > +  # concerned -- we just grep for it in the sources.
> > +  inline_macro Git::Documentation::BugReportProcessor, :bugreport
> 
> (I never much liked this copy-paste comment then, and I still don't, to
> be honest.)

I'll see if I can find a reasonable alternative (or remove it).

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
  2020-02-05  0:30       ` Emily Shaffer
@ 2020-02-05  0:52         ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-05  0:52 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List, Johannes Schindelin

On Tue, Feb 04, 2020 at 04:30:43PM -0800, Emily Shaffer wrote:
> Yeah, I think this is a good point. I'll try to figure out how to reword
> the commit message, and take another look at the sample include/exclude
> change I made to see where I can omit entirely.

Eh. I'm going to try and reword the whole commit message. I combined
your scissors patch, plus something from Dscho's scissors patch, plus my
original commit message, and now it's huge and hard to follow.


 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 10/15] bugreport: add config values from safelist
  2020-01-30 22:36     ` Martin Ågren
@ 2020-02-05  1:34       ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-05  1:34 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List

On Thu, Jan 30, 2020 at 11:36:43PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -2465,7 +2465,7 @@ endif
> >  git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
> >         $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
> >
> > -git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
> > +git-bugreport$X: bugreport-config-safelist.h bugreport.o GIT-LDFLAGS $(GITLIBS)
> >         $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
> >                 $(LIBS)
> 
> Haven't looked at this patch at all, except I've noticed that something
> is up with the dependencies. I think bugreport.o needs to depend on
> bugreport-config-safelist.h. As it is, the latter may or may not be
> available as bugreport.o is built. (Reproduces fairly well for me with
> something like `make clean && make -j16`.)

Hum, I thought that the ordering of the dependency list here was
ensured. But I see where help.o relies directly on command-list.h, so
I'll add a dependency for bugreport.o (and drop it from the direct
dependency for git-bugreport).

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 10/15] bugreport: add config values from safelist
  2020-01-31 21:25     ` Martin Ågren
@ 2020-02-05  2:31       ` Emily Shaffer
  2020-02-05 20:12         ` Martin Ågren
  0 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-05  2:31 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Git Mailing List

On Fri, Jan 31, 2020 at 10:25:06PM +0100, Martin Ågren wrote:
> On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > Taking the build-time generated array and putting it into a set saves us
> > time - since git_config_bugreport() is called for every option the user
> > has configured, performing option lookup in constant time is a useful
> > optimization.
> 
> I'm sympathetic to your sending out what you have to obtain comments,
> knowing that it's not perfect. It would have saved me some time and
> effort if I'd known that this was the case though. An "[RFC]" tag,
> perhaps. Or at least tweaking the above part of this commit message to
> say that this might be over-engineered, with a reference to [1].
> 
> [1] https://lore.kernel.org/git/20200124032905.GA37541@google.com/

Yeah, you are right, and thanks for the feedback. I think I had wrapped
up the series before I realized there were those significant outstanding
comments, but I definitely should have said so in this patch.

> 
> > +       int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
> 
> I was going to suggest ARRAY_SIZE, but then I realized there are some
> outstanding questions around whether you need this stuff in the first
> place. I'd be inclined to guess that the first version of this would be
> "for each safelisted item, obtain it and include", ignoring any "a.*.b"
> business. In which case you wouldn't really need this hashset stuff.

Regardless, I think it's worth doing nicely for now.

It seems to me that supporting wildcarding in the config safelist is a
superset of supporting static strings in that safelist - that is, if I
write it simply to support static strings, a later change to support
wildcards would be welcome and non-breaking.

So I'm going to clean this up without making it more featureful, for
now.

Sorry about the confusion!
 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 11/15] bugreport: collect list of populated hooks
  2020-02-04 18:44     ` Junio C Hamano
@ 2020-02-05  2:48       ` Emily Shaffer
  2020-02-05  3:00         ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-05  2:48 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Feb 04, 2020 at 10:44:39AM -0800, Junio C Hamano wrote:
> emilyshaffer@google.com writes:
> 
> > +	/*
> > +	 * Doesn't look like there is a list of all possible hooks; so below is
> > +	 * a transcription of `git help hook`.
> 
> That's "git help hooks", if I tried my reproduction correctly.

Yep. I'll fix it.

> A straight-forward (in the sense of "what we want in the outcome is
> quite clear" and not in the sense of "anybody can design and implement
> it with a single 30-line patch") follow-up we can make after this
> series lands is to rethink how Documentation/githooks.txt is
> maintained and the list we have here is synchronized with it.
> 
> The design could me just the matter of running "grep" of some sort,
> with appropriate markups that are no-op to AsciiDoctor/AsciiDoc
> added to the documentation source, to produce this list.

Well, with Martin's suggestion[1] to use annotate:bugreport[include]
instead of bugreport:include[x], this kind of change becomes a matter of
course to just use "annotate:hook[x]" and write a script nearly
identical to generate-bugreport-config-safelist.sh, and add it to the
Makefile.

Like I said up-thread, I don't like the idea just because it's awfully
temporary (pending work that's right below this on my list for the
coming weeks).

It sounds like you're saying you don't mind any work of this nature
being done as a follow-up - and that's fine with me, too. I'd just
rather do the follow-up with enums and a library than with grep and
sh. :)

> >  static const char * const bugreport_usage[] = {
> >  	N_("git bugreport [-o|--output <file>]"),
> >  	NULL
> > @@ -193,6 +243,9 @@ int cmd_main(int argc, const char **argv)
> >  	get_header(&buffer, "Safelisted Config Info");
> >  	get_safelisted_config(&buffer);
> >  
> > +	get_header(&buffer, "Configured Hooks");
> 
> Phrase nit.  There may be many people who just enabled hooks without
> configuring, so "Enabled Hooks" may be more appropriate.  We do not
> have to inspect what is in the hook to determine if it is enabled,
> but we do need to if we want to tell if a hook is "configured".

Ah, that's a good point. OK, I'll change the wording as you suggested.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 12/15] bugreport: count loose objects
  2020-02-04 18:48     ` Junio C Hamano
@ 2020-02-05  2:50       ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-05  2:50 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Johannes Schindelin

On Tue, Feb 04, 2020 at 10:48:30AM -0800, Junio C Hamano wrote:
> emilyshaffer@google.com writes:
> 
> > From: Emily Shaffer <emilyshaffer@google.com>
> >
> > The number of unpacked objects in a user's repository may help us
> > understand the root of the problem they're seeing, especially if a
> > command is running unusually slowly.
> >
> > Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> > Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> > ---
> > The refactor removed much of the code Dscho suggested; and yet it
> > remains true that he helped me while developing this commit (although
> > his suggestions didn't survive). Shall I leave the Helped-by line or
> > remove it?
> 
> You two collectively thought about viable alternatives and decided
> to reject what was not wanted in the final result, and not having
> that rejected code was good for the project, right?  If so, I would
> say it still is the help that deserves recognition.  After all,
> making the result better by removing things is harder than by adding
> things ;-)

Sounds great to me. I'd rather over-assign credit than under-assign (but
I don't think this is a case of over-assigning here).

> >  bugreport.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 46 insertions(+)
> > ...
> 
> The patch text looked sensible.

Thanks.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 11/15] bugreport: collect list of populated hooks
  2020-02-05  2:48       ` Emily Shaffer
@ 2020-02-05  3:00         ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-05  3:00 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Feb 04, 2020 at 06:48:59PM -0800, Emily Shaffer wrote:
> Well, with Martin's suggestion[1] to use annotate:bugreport[include]
> instead of bugreport:include[x], this kind of change becomes a matter of
> course to just use "annotate:hook[x]" and write a script nearly
> identical to generate-bugreport-config-safelist.sh, and add it to the
> Makefile.

Bah.

[1]: https://lore.kernel.org/git/CAN0heSq_i4EitqYH-qrZyXBU+=PUNcSXOOJDHLSnJ1ufV0WtaQ@mail.gmail.com

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 13/15] bugreport: add packed object summary
  2020-02-04 19:03     ` Junio C Hamano
@ 2020-02-05  3:09       ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-05  3:09 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Derrick Stolee

On Tue, Feb 04, 2020 at 11:03:57AM -0800, Junio C Hamano wrote:
> emilyshaffer@google.com writes:
> 
> > From: Emily Shaffer <emilyshaffer@google.com>
> >
> > Alongside the loose object counts, it can be useful to show the number
> > of packs and packed objects. This way we can check whether the repo has
> > an appropriate ratio of packed to loose objects to help determine
> > whether it's behaving correctly.
> 
> This step makes me wonder if we want to see the midx as well.
> Didn't we have a bug that manifests only when midx is (or is not) in
> use?
> 
> Similarly, I think use (or non-use) of the commit graph may also be
> a useful information for diagnosing.
> 

Sure. I'll look into both of these. Would you rather see them as add-ons
later, or do you want me to hold off on a v6 of this set until they're
ready?

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 13/15] bugreport: add packed object summary
  2020-02-04 19:00     ` Junio C Hamano
@ 2020-02-05  3:15       ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-05  3:15 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Feb 04, 2020 at 11:00:41AM -0800, Junio C Hamano wrote:
> emilyshaffer@google.com writes:
> 
> > From: Emily Shaffer <emilyshaffer@google.com>
> >
> > Alongside the loose object counts, it can be useful to show the number
> > of packs and packed objects. This way we can check whether the repo has
> > an appropriate ratio of packed to loose objects to help determine
> > whether it's behaving correctly.
> 
> This lets the enumeration machinery to enumerate at the individual
> object level, relies on the enumeration machinery to show your
> callback all objects from the same pack before an object from a
> different pack, to count both objects and packs.
> 
> But given that packfiles record how many objects there are in the
> pack, and that the packfile.c layer must know what are the pack
> files we have available, I wonder if we have an API already to allow
> us to enumerate each packfile, and while counting the number of
> times our callback was called, add the objects contained in each of
> them.  If there isn't, it does not sound like a bad API function to
> add (i.e. given a repository r, iterate over r->objects->packed_git
> and let the API caller see each of them; the API caller can grab
> num_objects and add them up).

Yeah, I don't see one. I'll go ahead and add it - that sounds
straightforward enough.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 06/15] bugreport: add compiler info
  2020-02-04 22:51       ` Emily Shaffer
@ 2020-02-05 19:47         ` Martin Ågren
  0 siblings, 0 replies; 177+ messages in thread
From: Martin Ågren @ 2020-02-05 19:47 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Tue, 4 Feb 2020 at 23:51, Emily Shaffer <emilyshaffer@google.com> wrote:
>
> On Thu, Jan 30, 2020 at 11:21:39PM +0100, Martin Ågren wrote:
> > On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > > +#else
> > > +
> > > +static inline void get_compiler_info(struct strbuf *info)
> > > +{
> > > +       strbuf_addstr(info, "get_compiler_info() not implemented");
> >
> > Maybe "no compiler info available" (or s/ available//, or
> > s/available/reported/), or something else more human-readable?
>
> Hm. I envisioned the target audience for this as "Git developers" (and
> that's why I didn't mark the unimplemented string for translation); Git
> developers know exactly what "get_compiler_info() not implemented"
> means. But I suppose it's just as easy to grep for one string as the
> other.
>
> I am hesitant to say "no info available" or "no info reported" when I'm
> certain there is some environment variable set at build time which
> contains the info we'd want to show here; it really is just a question
> of some Git developer having put in the time to implement this function.
>
> So, I think I will leave it. Thanks for your suggestions.

Ok, that makes sense.

Martin

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 09/15] bugreport: generate config safelist based on docs
  2020-02-05  0:44       ` Emily Shaffer
@ 2020-02-05 19:53         ` Martin Ågren
  0 siblings, 0 replies; 177+ messages in thread
From: Martin Ågren @ 2020-02-05 19:53 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List, Johannes Schindelin

On Wed, 5 Feb 2020 at 01:45, Emily Shaffer <emilyshaffer@google.com> wrote:
>
> > I sort of wonder whether safelist/blocklist is to prefer over
> > whitelist/blacklist, or if it's the other way round. The former are new
> > to me, whereas the latter are the terms I would have used. But that's
> > just me, of course. I was a little surprised, that's all.
>
> Eh. I think the following things are true:
>
>  - Whitelist/blacklist has a "smell" of discrimination, whether that's
>    the true etymology or not.
>  - Those with experience in the field can easily understand what
>    whitelist or blacklist means.
>  - Safelist/blocklist do not "smell" the same way.
>  - It is easy to tell what "safelist" means: "a list of stuff which is
>    safe." No experience needed.
>
> So, while it's new, I think it's not harmful. I see only a no-op or
> positive impact from using this term instead of whitelist/blacklist.
> Computer science seems to have quite a few terms which fall into this
> long-standing but potentially negative area, so I don't mind looking for
> alternatives where it's harmless to do so.

Ok, that all makes sense.

Thanks
Martin

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 08/15] bugreport: include user interactive shell
  2020-01-30 22:28     ` Martin Ågren
  2020-02-04 23:16       ` Emily Shaffer
@ 2020-02-05 20:06       ` Junio C Hamano
  2020-02-05 20:14         ` Martin Ågren
  1 sibling, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2020-02-05 20:06 UTC (permalink / raw)
  To: Martin Ågren; +Cc: Emily Shaffer, Git Mailing List

Martin Ågren <martin.agren@gmail.com> writes:

> On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
>> +       char *shell = NULL;
>
> (Unnecessary initialization.)
>
>> +       shell = getenv("SHELL");
>> +       strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
>> +                   shell ? shell : "(NULL)");
>
> Thanks for avoiding a classic pitfall. :-)
>
> "<unused>" instead of "(NULL)"? "NULL" is mostly an implementation
> detail.

Isn't that <unset>?

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 10/15] bugreport: add config values from safelist
  2020-02-05  2:31       ` Emily Shaffer
@ 2020-02-05 20:12         ` Martin Ågren
  0 siblings, 0 replies; 177+ messages in thread
From: Martin Ågren @ 2020-02-05 20:12 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Git Mailing List

On Wed, 5 Feb 2020 at 03:31, Emily Shaffer <emilyshaffer@google.com> wrote:
>
> On Fri, Jan 31, 2020 at 10:25:06PM +0100, Martin Ågren wrote:
> > On Fri, 24 Jan 2020 at 04:40, <emilyshaffer@google.com> wrote:
> > > Taking the build-time generated array and putting it into a set saves us
> > > time - since git_config_bugreport() is called for every option the user
> > > has configured, performing option lookup in constant time is a useful
> > > optimization.
> >
> > I'm sympathetic to your sending out what you have to obtain comments,
> > knowing that it's not perfect. It would have saved me some time and
> > effort if I'd known that this was the case though. An "[RFC]" tag,
> > perhaps. Or at least tweaking the above part of this commit message to
> > say that this might be over-engineered, with a reference to [1].
> >
> > [1] https://lore.kernel.org/git/20200124032905.GA37541@google.com/
>
> Yeah, you are right, and thanks for the feedback. I think I had wrapped
> up the series before I realized there were those significant outstanding
> comments, but I definitely should have said so in this patch.
>
> >
> > > +       int safelist_len = sizeof(bugreport_config_safelist) / sizeof(const char *);
> >
> > I was going to suggest ARRAY_SIZE, but then I realized there are some
> > outstanding questions around whether you need this stuff in the first
> > place. I'd be inclined to guess that the first version of this would be
> > "for each safelisted item, obtain it and include", ignoring any "a.*.b"
> > business. In which case you wouldn't really need this hashset stuff.
>
> Regardless, I think it's worth doing nicely for now.
>
> It seems to me that supporting wildcarding in the config safelist is a
> superset of supporting static strings in that safelist - that is, if I
> write it simply to support static strings, a later change to support
> wildcards would be welcome and non-breaking.

Yeah, agreed. Also, whatever magic the code for gathering the config
items knows, the annotations in Documentation/, i.e., the generated list
in bugreport-config-safelist.h will be "synced". That is, until we know
how to handle "a.*.b" in the safelist, we probably won't be marking
"a.*.b" as safe. And even if we have some crazy mixed build, it's a
*safe*list, so we should be fine. (Famous last words?)

On that topic though, and just so we don't mess up from the beginning,
in the documentation, we typically write something like
"branch.<name>.merge", i.e., not "branch.*.merge". So I guess the
"a.*.b" feature would be more like "look for '.<', something, something,
'>.'." Would that be sufficient? Or would we actually want more
fine-grained control, i.e., something like regexes?

If so, we'd need to provide those regexes somehow, e.g., as part of the
asciidoc macro notation. :-/ Do we need to prepare somehow for such a
future? I don't think we do.

> So I'm going to clean this up without making it more featureful, for
> now.
>
> Sorry about the confusion!
>  - Emily

No worries. :)

Martin

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v5 08/15] bugreport: include user interactive shell
  2020-02-05 20:06       ` Junio C Hamano
@ 2020-02-05 20:14         ` Martin Ågren
  0 siblings, 0 replies; 177+ messages in thread
From: Martin Ågren @ 2020-02-05 20:14 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Emily Shaffer, Git Mailing List

On Wed, 5 Feb 2020 at 21:06, Junio C Hamano <gitster@pobox.com> wrote:
>
> Martin Ågren <martin.agren@gmail.com> writes:
>
> > On Fri, 24 Jan 2020 at 04:41, <emilyshaffer@google.com> wrote:
> >> +       char *shell = NULL;
> >
> > (Unnecessary initialization.)
> >
> >> +       shell = getenv("SHELL");
> >> +       strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
> >> +                   shell ? shell : "(NULL)");
> >
> > Thanks for avoiding a classic pitfall. :-)
> >
> > "<unused>" instead of "(NULL)"? "NULL" is mostly an implementation
> > detail.
>
> Isn't that <unset>?

Heh, yes, "SHELL: unused" sounds wrong. :-)

Martin

^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v6 00/15] add git-bugreport tool
  2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
                     ` (17 preceding siblings ...)
  2020-01-30 22:15   ` Martin Ågren
@ 2020-02-06  0:40   ` " Emily Shaffer
  2020-02-06  0:40     ` [PATCH v6 01/15] help: move list_config_help to builtin/help Emily Shaffer
                       ` (15 more replies)
  18 siblings, 16 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:40 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Derrick Stolee, Johannes Schindelin,
	Junio C Hamano, Martin Ågren, Aaron Schrab

This patchset is based on mr/show-config-scope.

Since v5, I addressed comments from Martin and Junio.

 - Reorganized the series so the ecosystem cleanup comes before the
   interesting new features.
 - Spread out manpage updates to indicate each new feature as it's added.
 - Fixed a number of patches which assumed a repo, so bugreport can now
   run anywhere.
 - Wordsmithing in the report template.
 - die() instead of overwriting an existing report.
 - Added --suffix to allow user to change the name of the report to
   git-bugreport-<strftime format>.txt.
 - General NULL-ptr, leak, and bashism cleanup.
 - Changed safelist macro from bugreport:include[x] to
   annotate:bugreport[include].
 - Use git_config_get_value_multi() instead of a config callback to
   gather the safelist configs.
 - Added a convenience for_each_pack() macro to object-store.h and began
   using it in packfile.c and bugreport.c for pack summary

Emily Shaffer (15):
  help: move list_config_help to builtin/help
  help: add shell-path to --build-options
  bugreport: add tool to generate debugging info
  bugreport: gather git version and build info
  bugreport: add uname info
  bugreport: add compiler info
  bugreport: add git-remote-https version
  bugreport: include user interactive shell
  bugreport: generate config safelist based on docs
  bugreport: add config values from safelist
  bugreport: collect list of populated hooks
  bugreport: count loose objects
  bugreport: add packed object summary
  bugreport: list contents of $OBJDIR/info
  bugreport: summarize contents of alternates file

 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   9 +
 Documentation/asciidoctor-extensions.rb |   5 +
 Documentation/config/sendemail.txt      |  56 ++--
 Documentation/git-bugreport.txt         |  55 ++++
 Makefile                                |  27 +-
 bugreport.c                             | 409 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 compat/compiler.h                       |  24 ++
 generate-bugreport-config-safelist.sh   |  17 +
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  24 ++
 help.c                                  | 132 ++------
 help.h                                  |   2 +-
 object-store.h                          |   6 +
 packfile.c                              |   3 +-
 remote-curl.c                           |   8 +
 t/t0091-bugreport.sh                    |  49 +++
 18 files changed, 778 insertions(+), 156 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100644 compat/compiler.h
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh

-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v6 01/15] help: move list_config_help to builtin/help
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
@ 2020-02-06  0:40     ` Emily Shaffer
  2020-02-06  1:35       ` Danh Doan
  2020-02-06  0:40     ` [PATCH v6 02/15] help: add shell-path to --build-options Emily Shaffer
                       ` (14 subsequent siblings)
  15 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:40 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o,
builtin/clean.o, and builtin/config.o. This meant that help.o was
unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help()
into builtin/help.c (where it makes sense to assume other builtin libraries
are present).

When command-list.h is included but a member is not used, we start to
hear a compiler warning. Since the config list is generated in a fairly
different way than the command list, and since commands and config
options are semantically different, move the config list into its own
header and move the generator into its own script and build rule.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore             |  1 +
 Makefile               | 13 +++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 24 ++++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 121 insertions(+), 108 deletions(-)
 create mode 100755 generate-configlist.sh

diff --git a/.gitignore b/.gitignore
index aebe7c0908..ea97de83f3 100644
--- a/.gitignore
+++ b/.gitignore
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index 09f98b777c..5a022367d4 100644
--- a/Makefile
+++ b/Makefile
@@ -814,6 +814,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2128,7 +2129,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2148,6 +2149,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2781,7 +2788,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2803,7 +2810,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..eca6a00c30
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	while read line
+	do
+		echo "	\"$line\","
+	done
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/help.c b/help.c
index cf67624a94..a21487db77 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
diff --git a/help.h b/help.h
index 7a455beeb7..9071894e8c 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v6 02/15] help: add shell-path to --build-options
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
  2020-02-06  0:40     ` [PATCH v6 01/15] help: move list_config_help to builtin/help Emily Shaffer
@ 2020-02-06  0:40     ` Emily Shaffer
  2020-02-06  0:40     ` [PATCH v6 03/15] bugreport: add tool to generate debugging info Emily Shaffer
                       ` (13 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:40 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It may be useful to know which shell Git was built to try to point to,
in the event that shell-based Git commands are failing. $SHELL_PATH is
set during the build and used to launch the manpage viewer, as well as
by git-compat-util.h, and it's used during tests. 'git version
--build-options' is encouraged for use in bug reports, so it makes sense
to include this information there.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 help.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/help.c b/help.c
index a21487db77..190722fb0a 100644
--- a/help.c
+++ b/help.c
@@ -654,6 +654,7 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 			printf("no commit associated with this build\n");
 		printf("sizeof-long: %d\n", (int)sizeof(long));
 		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
+		printf("shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 	return 0;
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v6 03/15] bugreport: add tool to generate debugging info
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
  2020-02-06  0:40     ` [PATCH v6 01/15] help: move list_config_help to builtin/help Emily Shaffer
  2020-02-06  0:40     ` [PATCH v6 02/15] help: add shell-path to --build-options Emily Shaffer
@ 2020-02-06  0:40     ` Emily Shaffer
  2020-02-07 14:18       ` SZEDER Gábor
                         ` (2 more replies)
  2020-02-06  0:40     ` [PATCH v6 04/15] bugreport: gather git version and build info Emily Shaffer
                       ` (12 subsequent siblings)
  15 siblings, 3 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:40 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction
steps, expected behavior, and actual behavior. Later, Git can learn how
to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic
information we would otherwise need to ask the user for, we can reduce
the number of question-and-answer round trips between the reporter and
the Git contributor.

Users may also wish to send a report like this to their local "Git
expert" if they have put their repository into a state they are confused
by.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 41 ++++++++++++++++
 Makefile                        |  5 ++
 bugreport.c                     | 85 +++++++++++++++++++++++++++++++++
 t/t0091-bugreport.sh            | 49 +++++++++++++++++++
 5 files changed, 181 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index ea97de83f3..d89bf9e11e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..52d49ed7aa
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,41 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [(-o | --output-directory) <path>] [(-s | --suffix) <format>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+OPTIONS
+-------
+-o <path>::
+--output-directory <path>::
+	Place the resulting bug report file in `<path>` instead of the root of
+	the Git repository.
+
+-s <format>::
+--suffix <format>::
+	Specify an alternate suffix for the bugreport name, to create a file
+	named 'git-bugreport-<formatted suffix>'. This should take the form of a
+	link:strftime[3] format string; the current local time will be used.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index 5a022367d4..a01a050aa3 100644
--- a/Makefile
+++ b/Makefile
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2457,6 +2458,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
diff --git a/bugreport.c b/bugreport.c
new file mode 100644
index 0000000000..db46fb88be
--- /dev/null
+++ b/bugreport.c
@@ -0,0 +1,85 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to share.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+	char *option_suffix = "%F-%H%M";
+	struct stat statbuf;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output-directory", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_STRING('s', "suffix", &option_suffix, N_("format"),
+			   N_("specify a strftime format suffix for the filename")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+	if (!stat(report_path.buf, &statbuf))
+		die("'%s' already exists", report_path.buf);
+
+	get_bug_template(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+
+	if (report == NULL) {
+		strbuf_release(&report_path);
+		die("couldn't open '%s' for writing", report_path.buf);
+	}
+
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
+
+	UNLEAK(buffer);
+	UNLEAK(report_path);
+	return -launch_editor(report_path.buf, NULL, NULL);
+}
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..451badff0c
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,49 @@
+#!/bin/sh
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+check_all_headers_populated() {
+	while read -r line; do
+		if test "$(grep "$HEADER_PATTERN" "$line")"; then
+			echo "$line"
+			read -r nextline
+			if test -z "$nextline"; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport &&
+	REPORT="$(ls git-bugreport-*)" &&
+	check_all_headers_populated <$REPORT &&
+	rm $REPORT
+'
+
+test_expect_success 'dies if file with same name as report already exists' '
+	touch git-bugreport-duplicate.txt &&
+	test_must_fail git bugreport --suffix duplicate &&
+	rm git-bugreport-duplicate.txt
+'
+
+test_expect_success '--output-directory puts the report in the provided dir' '
+	mkdir foo/ &&
+	git bugreport -o foo/ &&
+	test_path_is_file foo/git-bugreport-* &&
+	rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	grep usage output &&
+	test_path_is_missing git-bugreport-*
+'
+
+test_done
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v6 04/15] bugreport: gather git version and build info
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (2 preceding siblings ...)
  2020-02-06  0:40     ` [PATCH v6 03/15] bugreport: add tool to generate debugging info Emily Shaffer
@ 2020-02-06  0:40     ` Emily Shaffer
  2020-02-06  0:40     ` [PATCH v6 05/15] bugreport: add uname info Emily Shaffer
                       ` (11 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:40 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Knowing which version of Git a user has and how it was built allows us
to more precisely pin down the circumstances when a certain issue
occurs, so teach bugreport how to tell us the same output as 'git
version --build-options'.

It's not ideal to directly call 'git version --build-options' because
that output goes to stdout. Instead, wrap the version string in a helper
within help.[ch] library, and call that helper from within the bugreport
library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  4 +++
 bugreport.c                     | 19 ++++++++++++-
 help.c                          | 48 ++++++++++++++++++++-------------
 help.h                          |  1 +
 4 files changed, 52 insertions(+), 20 deletions(-)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 52d49ed7aa..6ce2ae2eff 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -23,6 +23,10 @@ The following information is requested from the user:
  - Expected behavior
  - Actual behavior
 
+The following information is captured automatically:
+
+ - 'git version --build-options'
+
 OPTIONS
 -------
 -o <path>::
diff --git a/bugreport.c b/bugreport.c
index db46fb88be..dd21be9d68 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,8 +1,17 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(sys_info, 1);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
@@ -32,6 +41,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -67,6 +81,9 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
diff --git a/help.c b/help.c
index 190722fb0a..44cee69c11 100644
--- a/help.c
+++ b/help.c
@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void get_version_info(struct strbuf *buf, int show_build_options)
+{
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (show_build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addstr(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -637,26 +662,11 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	get_version_info(&buf, build_options);
+	printf("%s", buf.buf);
+
+	strbuf_release(&buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		printf("shell-path: %s\n", SHELL_PATH);
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 9071894e8c..500521b908 100644
--- a/help.h
+++ b/help.h
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void get_version_info(struct strbuf *buf, int show_build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v6 05/15] bugreport: add uname info
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (3 preceding siblings ...)
  2020-02-06  0:40     ` [PATCH v6 04/15] bugreport: gather git version and build info Emily Shaffer
@ 2020-02-06  0:40     ` Emily Shaffer
  2020-02-06  0:40     ` [PATCH v6 06/15] bugreport: add compiler info Emily Shaffer
                       ` (10 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:40 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

The contents of uname() can give us some insight into what sort of
system the user is running on, and help us replicate their setup if need
be. The domainname field is not guaranteed to be available, so don't
collect it.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 13 +++++++++++++
 2 files changed, 14 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 6ce2ae2eff..4dd72c60f5 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -26,6 +26,7 @@ The following information is requested from the user:
 The following information is captured automatically:
 
  - 'git version --build-options'
+ - uname sysname, release, version, and machine strings
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index dd21be9d68..4c4b877ea1 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -7,10 +7,23 @@
 
 static void get_system_info(struct strbuf *sys_info)
 {
+	struct utsname uname_info;
+
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	get_version_info(sys_info, 1);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v6 06/15] bugreport: add compiler info
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (4 preceding siblings ...)
  2020-02-06  0:40     ` [PATCH v6 05/15] bugreport: add uname info Emily Shaffer
@ 2020-02-06  0:40     ` Emily Shaffer
  2020-02-06  0:41     ` [PATCH v6 07/15] bugreport: add git-remote-https version Emily Shaffer
                       ` (9 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:40 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

To help pinpoint the source of a regression, it is useful to know some
info about the compiler which the user's Git client was built with. By
adding a generic get_compiler_info() in 'compat/' we can choose which
relevant information to share per compiler; to get started, let's
demonstrate the version of glibc if the user built with 'gcc'.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     |  5 +++++
 compat/compiler.h               | 24 ++++++++++++++++++++++++
 3 files changed, 30 insertions(+)
 create mode 100644 compat/compiler.h

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 4dd72c60f5..8bbc4c960c 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -27,6 +27,7 @@ The following information is captured automatically:
 
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
+ - Compiler-specific info string
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 4c4b877ea1..aaeb94c10c 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include "compat/compiler.h"
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -24,6 +25,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/compat/compiler.h b/compat/compiler.h
new file mode 100644
index 0000000000..bda5098e1b
--- /dev/null
+++ b/compat/compiler.h
@@ -0,0 +1,24 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
+}
+
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addstr(info, "get_compiler_info() not implemented");
+}
+
+#endif
+
+#endif /* COMPILER_H */
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v6 07/15] bugreport: add git-remote-https version
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (5 preceding siblings ...)
  2020-02-06  0:40     ` [PATCH v6 06/15] bugreport: add compiler info Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-06  0:41     ` [PATCH v6 08/15] bugreport: include user interactive shell Emily Shaffer
                       ` (8 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It's possible for git-remote-curl to be built separately from git; in that
case we want to know what version of cURL is used by git-remote-curl, not
necessarily which version was present at git-bugreport's build time.
So instead, ask git-remote-curl for the version information it knows
about.

Today, "git-remote-http" and "git-remote-https" are aliased to
"git-remote-curl"; but in case we rely on a different library than cURL
in the future, let's not explicitly reference cURL from bugreport.

For longevity purposes, invoke the alias "git-remote-https" instead of
"git-remote-http".

Since it could have been built at a different time, also report the
version and built-from commit of git-remote-curl alongside the cURL info.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 16 ++++++++++++++++
 remote-curl.c                   |  8 ++++++++
 3 files changed, 25 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 8bbc4c960c..33df4dec7f 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -28,6 +28,7 @@ The following information is captured automatically:
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
+ - 'git remote-https --build-info'
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index aaeb94c10c..da5859c131 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include "compat/compiler.h"
+#include "run-command.h"
+
+static void get_git_remote_https_version_info(struct strbuf *version_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, version_info, 0))
+	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -29,6 +41,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "compiler info: ");
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+	get_git_remote_https_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/remote-curl.c b/remote-curl.c
index 350d92a074..c590fbfae3 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -17,6 +17,7 @@
 #include "protocol.h"
 #include "quote.h"
 #include "transport.h"
+#include "version.h"
 
 static struct remote *remote;
 /* always ends with a trailing slash */
@@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {
+		printf("git-http-fetch version: %s\n", git_version_string);
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v6 08/15] bugreport: include user interactive shell
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (6 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 07/15] bugreport: add git-remote-https version Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-06  0:41     ` [PATCH v6 09/15] bugreport: generate config safelist based on docs Emily Shaffer
                       ` (7 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It's possible a user may complain about the way that Git interacts with
their interactive shell, e.g. autocompletion or shell prompt. In that
case, it's useful for us to know which shell they're using
interactively.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt | 1 +
 bugreport.c                     | 5 +++++
 2 files changed, 6 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 33df4dec7f..23265b0d74 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -29,6 +29,7 @@ The following information is captured automatically:
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
  - 'git remote-https --build-info'
+ - $SHELL
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index da5859c131..9b51250155 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -21,6 +21,7 @@ static void get_git_remote_https_version_info(struct strbuf *version_info)
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct utsname uname_info;
+	char *shell = NULL;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
@@ -42,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
 
+	shell = getenv("SHELL");
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    shell ? shell : "<unset>");
+
 	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
 	get_git_remote_https_version_info(sys_info);
 	strbuf_complete_line(sys_info);
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v6 09/15] bugreport: generate config safelist based on docs
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (7 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 08/15] bugreport: include user interactive shell Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-07 15:30       ` SZEDER Gábor
  2020-02-06  0:41     ` [PATCH v6 10/15] bugreport: add config values from safelist Emily Shaffer
                       ` (6 subsequent siblings)
  15 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Martin Ågren, Johannes Schindelin

Add a new step to the build to generate a safelist of git-config
variables which are appropriate to include in the output of
git-bugreport. New variables can be added to the safelist by annotating
their documentation in Documentation/config with the "annotate" macro,
which is a no-op in AsciiDoc and AsciiDoctor.

Some configs are private in nature, and can contain remote URLs,
passwords, or other sensitive information. In the event that a user
doesn't notice their information while reviewing a bugreport, that user
may leak their credentials to other individuals, mailing lists, or bug
tracking tools inadvertently. Heuristic blocklisting of configuration
keys is imperfect and prone to false negatives; given the nature of the
information which can be leaked, a safelist is more reliable.

However, it's possible that in some situations, an organization may be
less concerned with privacy of things like remote URLs and branch names,
and more concerned with ease of diagnosis for their support staff. In
those cases, it may make more sense for that organization to modify the
code to use a blocklist. To that end, we should try to mark configs which
are definitely safe, and configs which are definitely unsafe, and leave
blank configs which are somewhere in between. To mark a config as safe,
add "annotate:bugreport[include]" to the corresponding line in the
config documentation; to mark it as unsafe, add
"annotate:bugreport[exclude]" instead.

Generating bugreport-config-safelist.h at build time by grepping the
documentation for this new macro helps us prevent staleness. The macro
itself is a no-op and should not alter the appearance of the
documentation in either AsciiDoc or AsciiDoctor, confirmable by running:

  cd Documentation
  ./doc-diff --asciidoctor HEAD^ HEAD
  ./doc-diff --asciidoc HEAD^ HEAD

Diffing the rendered HTML shows that only inline comments were added,
which shouldn't be a problem.

Additionally, add annotations to the sendemail config documentation in
order to demonstrate a proof of concept.

Helped-by: Martin Ågren <martin.agren@gmail.com>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  9 ++++
 Documentation/asciidoctor-extensions.rb |  5 +++
 Documentation/config/sendemail.txt      | 56 ++++++++++++-------------
 Makefile                                |  7 ++++
 generate-bugreport-config-safelist.sh   | 17 ++++++++
 6 files changed, 67 insertions(+), 28 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh

diff --git a/.gitignore b/.gitignore
index d89bf9e11e..bd2f49b996 100644
--- a/.gitignore
+++ b/.gitignore
@@ -192,6 +192,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..663e06481f 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,14 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The annotate macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources to populate
+# things like the bugreport safelist.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>annotate):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +33,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[annotate-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +101,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[annotate-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..382bd8f6f4 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -37,6 +37,10 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+
+    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, target, attrs)
+        ""
       end
     end
   end
@@ -45,4 +49,5 @@ end
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  inline_macro Git::Documentation::AnnotateProcessor, :annotate
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 0006faf800..fe27473e44 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -4,7 +4,7 @@ sendemail.identity::
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption::
+sendemail.smtpEncryption annotate:bugreport[include] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
@@ -15,7 +15,7 @@ sendemail.smtpsslcertpath::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*::
+sendemail.<identity>.* annotate:bugreport[exclude] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
@@ -23,41 +23,41 @@ sendemail.<identity>.*::
 
 sendemail.aliasesFile::
 sendemail.aliasFileType::
-sendemail.annotate::
-sendemail.bcc::
-sendemail.cc::
-sendemail.ccCmd::
-sendemail.chainReplyTo::
-sendemail.confirm::
-sendemail.envelopeSender::
-sendemail.from::
-sendemail.multiEdit::
-sendemail.signedoffbycc::
-sendemail.smtpPass::
-sendemail.suppresscc::
-sendemail.suppressFrom::
-sendemail.to::
-sendemail.tocmd::
-sendemail.smtpDomain::
-sendemail.smtpServer::
-sendemail.smtpServerPort::
-sendemail.smtpServerOption::
-sendemail.smtpUser::
-sendemail.thread::
-sendemail.transferEncoding::
-sendemail.validate::
-sendemail.xmailer::
+sendemail.annotate annotate:bugreport[include] ::
+sendemail.bcc annotate:bugreport[include] ::
+sendemail.cc annotate:bugreport[include] ::
+sendemail.ccCmd annotate:bugreport[include] ::
+sendemail.chainReplyTo annotate:bugreport[include] ::
+sendemail.confirm annotate:bugreport[include] ::
+sendemail.envelopeSender annotate:bugreport[include] ::
+sendemail.from annotate:bugreport[include] ::
+sendemail.multiEdit annotate:bugreport[include] ::
+sendemail.signedoffbycc annotate:bugreport[include] ::
+sendemail.smtpPass annotate:bugreport[exclude] ::
+sendemail.suppresscc annotate:bugreport[include] ::
+sendemail.suppressFrom annotate:bugreport[include] ::
+sendemail.to annotate:bugreport[include] ::
+sendemail.tocmd annotate:bugreport[include] ::
+sendemail.smtpDomain annotate:bugreport[include] ::
+sendemail.smtpServer annotate:bugreport[include] ::
+sendemail.smtpServerPort annotate:bugreport[include] ::
+sendemail.smtpServerOption annotate:bugreport[include] ::
+sendemail.smtpUser annotate:bugreport[exclude] ::
+sendemail.thread annotate:bugreport[include] ::
+sendemail.transferEncoding annotate:bugreport[include] ::
+sendemail.validate annotate:bugreport[include] ::
+sendemail.xmailer annotate:bugreport[include] ::
 	See linkgit:git-send-email[1] for description.
 
 sendemail.signedoffcc (deprecated)::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize annotate:bugreport[include] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay::
+sendemail.smtpReloginDelay annotate:bugreport[include] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Makefile b/Makefile
index a01a050aa3..2bc9f112ea 100644
--- a/Makefile
+++ b/Makefile
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2163,6 +2164,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..17d92a91c5
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,17 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the annotate-bugreport macro
+sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
+
+cat <<EOF
+};
+EOF
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v6 10/15] bugreport: add config values from safelist
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (8 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 09/15] bugreport: generate config safelist based on docs Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-07 14:47       ` SZEDER Gábor
  2020-02-06  0:41     ` [PATCH v6 11/15] bugreport: collect list of populated hooks Emily Shaffer
                       ` (5 subsequent siblings)
  15 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach bugreport to gather the values of config options which are present
in 'bugreport-config-safelist.h', and show their origin scope.

Many config options are sensitive, and many Git add-ons use config
options which git-core does not know about; it is better only to gather
config options which we know to be safe, rather than excluding options
which we know to be unsafe.

Rather than including the path to someone's config, which can reveal
filesystem layout and project names, just name the scope (e.g. system,
global, local) of the config source.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 Makefile                        |  2 ++
 bugreport.c                     | 34 +++++++++++++++++++++++++++++++++
 3 files changed, 37 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 23265b0d74..4e9171d1bd 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -30,6 +30,7 @@ The following information is captured automatically:
  - Compiler-specific info string
  - 'git remote-https --build-info'
  - $SHELL
+ - Selected config values
 
 OPTIONS
 -------
diff --git a/Makefile b/Makefile
index 2bc9f112ea..7b139230ae 100644
--- a/Makefile
+++ b/Makefile
@@ -2131,6 +2131,8 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
+bugreport.sp bugreport.s bugreport.o: bugreport-config-safelist.h
+
 builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
diff --git a/bugreport.c b/bugreport.c
index 9b51250155..16216bff8e 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -6,6 +6,9 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -52,6 +55,29 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	size_t idx;
+	struct string_list_item *it = NULL;
+	struct key_value_info *kv_info = NULL;
+
+	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
+		const struct string_list *list =
+			git_config_get_value_multi(bugreport_config_safelist[idx]);
+
+		if (!list)
+			continue;
+
+		strbuf_addf(config_info, "%s:\n", bugreport_config_safelist[idx]);
+		for_each_string_list_item(it, list) {
+			kv_info = it->util;
+			strbuf_addf(config_info, "  %s (%s)\n", it->string,
+				    kv_info ? config_scope_name(kv_info->scope)
+					    : "source unknown");
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -94,6 +120,7 @@ int cmd_main(int argc, const char **argv)
 	char *option_output = NULL;
 	char *option_suffix = "%F-%H%M";
 	struct stat statbuf;
+	int nongit_ok = 0;
 
 	const struct option bugreport_options[] = {
 		OPT_STRING('o', "output-directory", &option_output, N_("path"),
@@ -102,6 +129,10 @@ int cmd_main(int argc, const char **argv)
 			   N_("specify a strftime format suffix for the filename")),
 		OPT_END()
 	};
+
+	/* Prerequisite for hooks and config checks */
+	setup_git_directory_gently(&nongit_ok);
+
 	argc = parse_options(argc, argv, "", bugreport_options,
 			     bugreport_usage, 0);
 
@@ -123,6 +154,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v6 11/15] bugreport: collect list of populated hooks
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (9 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 10/15] bugreport: add config values from safelist Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-06  0:41     ` [PATCH v6 12/15] bugreport: count loose objects Emily Shaffer
                       ` (4 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Occasionally a failure a user is seeing may be related to a specific
hook which is being run, perhaps without the user realizing. While the
contents of hooks can be sensitive - containing user data or process
information specific to the user's organization - simply knowing that a
hook is being run at a certain stage can help us to understand whether
something is going wrong.

Without a definitive list of hook names within the code, we compile our
own list from the documentation. This is likely prone to bitrot. To
reduce the amount of code humans need to read, we turn the list into a
string_list and iterate over it (as we are calling the same find_hook
operation on each string). However, since bugreport should primarily be
called by the user, the performance loss from massaging the string
seems acceptable.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 53 +++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 4e9171d1bd..779c9c7fd9 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -31,6 +31,7 @@ The following information is captured automatically:
  - 'git remote-https --build-info'
  - $SHELL
  - Selected config values
+ - A list of enabled hooks
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 16216bff8e..3b0a2b12cc 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "bugreport-config-safelist.h"
 #include "khash.h"
+#include "run-command.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -78,6 +79,55 @@ static void get_safelisted_config(struct strbuf *config_info)
 	}
 }
 
+static void get_populated_hooks(struct strbuf *hook_info, int nongit)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hooks`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+
+
+	if (nongit) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -157,6 +207,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Enabled Hooks");
+	get_populated_hooks(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v6 12/15] bugreport: count loose objects
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (10 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 11/15] bugreport: collect list of populated hooks Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-06  0:41     ` [PATCH v6 13/15] bugreport: add packed object summary Emily Shaffer
                       ` (3 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Johannes Schindelin

The number of unpacked objects in a user's repository may help us
understand the root of the problem they're seeing, especially if a
command is running unusually slowly.

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 52 +++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 779c9c7fd9..7e1ed44cb4 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -32,6 +32,7 @@ The following information is captured automatically:
  - $SHELL
  - Selected config values
  - A list of enabled hooks
+ - The number of loose objects in the repository
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 3b0a2b12cc..9552e2922e 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -10,6 +10,7 @@
 #include "bugreport-config-safelist.h"
 #include "khash.h"
 #include "run-command.h"
+#include "object-store.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -128,6 +129,54 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 }
 
+static int loose_object_cb(const struct object_id *oid, const char *path,
+			   void *data) {
+	int *loose_object_count = data;
+
+	if (loose_object_count) {
+		(*loose_object_count)++;
+		return 0;
+	}
+
+	return 1;
+}
+
+static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
+
+	int local_loose_object_count = 0, total_loose_object_count = 0;
+	int local_count_questionable = 0, total_count_questionable = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	local_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&local_loose_object_count,
+					FOR_EACH_OBJECT_LOCAL_ONLY);
+
+	total_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&total_loose_object_count,
+					0);
+
+	strbuf_addf(obj_info, "%d local loose objects%s\n",
+		    local_loose_object_count,
+		    local_count_questionable ? " (problem during count)" : "");
+
+	strbuf_addf(obj_info, "%d alternate loose objects%s\n",
+		    total_loose_object_count - local_loose_object_count,
+		    (local_count_questionable || total_count_questionable)
+			? " (problem during count)"
+			: "");
+
+	strbuf_addf(obj_info, "%d total loose objects%s\n",
+		    total_loose_object_count,
+		    total_count_questionable ? " (problem during count)" : "");
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -210,6 +259,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Enabled Hooks");
 	get_populated_hooks(&buffer, nongit_ok);
 
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v6 13/15] bugreport: add packed object summary
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (11 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 12/15] bugreport: count loose objects Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-06  0:41     ` [PATCH v6 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
                       ` (2 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Alongside the loose object counts, it can be useful to show the number
of packs and packed objects. This way we can check whether the repo has
an appropriate ratio of packed to loose objects to help determine
whether it's behaving correctly.

Add a utility to easily traverse all packfiles in a given repository.
Use it in packfile.c and remove a redundant call to
prepare_packed_git(), which is already called in get_all_packs().

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
Perhaps the macro should be in another patch? I didn't spend a lot of
time looking for anybody who could conceivably be iterating packs; if I
were to do so, I'd break this change out separately.

I considered following the pattern in the rest of the file, where the
foreach takes a callback and data pointer, but found it would be
cumbersome to use with for_each_packed_object() - I'd have to add a new
callback just to handle the pack, and bundle that function's callback
data along with the callback pointer. It seemed like overkill.

 - Emily

 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 30 ++++++++++++++++++++++++++++++
 object-store.h                  |  6 ++++++
 packfile.c                      |  3 +--
 4 files changed, 38 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 7e1ed44cb4..309b34fc42 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -33,6 +33,7 @@ The following information is captured automatically:
  - Selected config values
  - A list of enabled hooks
  - The number of loose objects in the repository
+ - The number of packs and packed objects in the repository
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 9552e2922e..d554488f3e 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -177,6 +177,33 @@ static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
 		    total_count_questionable ? " (problem during count)" : "");
 }
 
+static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
+{
+	struct packed_git *pack = NULL;
+	int pack_count = 0;
+	int object_count = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	for_each_pack(the_repository, pack) {
+		pack_count++;
+		/*
+		 * To accurately count how many objects are packed, look inside
+		 * the packfile's index.
+		 */
+		open_pack_index(pack);
+		object_count += pack->num_objects;
+	}
+
+	strbuf_addf(obj_info, "%d total packs (%d objects)\n", pack_count,
+		    object_count);
+
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -262,6 +289,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
diff --git a/object-store.h b/object-store.h
index 55ee639350..518c73469d 100644
--- a/object-store.h
+++ b/object-store.h
@@ -6,6 +6,7 @@
 #include "list.h"
 #include "sha1-array.h"
 #include "strbuf.h"
+#include "packfile.h"
 
 struct object_directory {
 	struct object_directory *next;
@@ -403,4 +404,9 @@ int for_each_object_in_pack(struct packed_git *p,
 int for_each_packed_object(each_packed_object_fn, void *,
 			   enum for_each_object_flags flags);
 
+#define for_each_pack(repo, pack) 		\
+		for (pack = get_all_packs(repo);\
+		     pack;			\
+		     pack = pack->next)
+
 #endif /* OBJECT_STORE_H */
diff --git a/packfile.c b/packfile.c
index 7e7c04e4d8..6873827fb5 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2061,8 +2061,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data,
 	int r = 0;
 	int pack_errors = 0;
 
-	prepare_packed_git(the_repository);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for_each_pack(the_repository, p) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v6 14/15] bugreport: list contents of $OBJDIR/info
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (12 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 13/15] bugreport: add packed object summary Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-06  0:41     ` [PATCH v6 15/15] bugreport: summarize contents of alternates file Emily Shaffer
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Miscellaneous information used about the object store can end up in
.git/objects/info; this can help us understand what may be going on with
the object store when the user is reporting a bug. Otherwise, it could
be difficult to track down what is going wrong with an object which
isn't kept locally to .git/objects/ or .git/objects/pack. Having some
understanding of where the user's objects may be kept can save us some
hops during the bug reporting process.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 54 +++++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 309b34fc42..a21d081616 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -34,6 +34,7 @@ The following information is captured automatically:
  - A list of enabled hooks
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
+ - A list of the contents of .git/objects/info (or equivalent)
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index d554488f3e..3a3d33f680 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -204,6 +204,57 @@ static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
 
 }
 
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
+{
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
+
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info, int nongit)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - object info unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(obj_info, &dirpath);
+
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -292,6 +343,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Packed Object Summary");
 	get_packed_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v6 15/15] bugreport: summarize contents of alternates file
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (13 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
@ 2020-02-06  0:41     ` Emily Shaffer
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-06  0:41 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

In some cases, it could be that the user is having a problem with an
object which isn't present in their normal object directory. We can get
a hint that that might be the case by examining the list of alternates
where their object may be stored instead. Since paths to alternates may
be sensitive, we'll instead count how many alternates have been
specified and note how many of them exist or are broken.

While object-cache.h describes a function "foreach_alt_odb()", this
function does not provide information on broken alternates, which are
skipped over in "link_alt_odb_entry()". Since the goal is to identify
missing alternates, we can gather the contents of
.git/objects/info/alternates manually.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 45 +++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index a21d081616..e870900c80 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -35,6 +35,7 @@ The following information is captured automatically:
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
  - A list of the contents of .git/objects/info (or equivalent)
+ - The number of valid and invalid alternates
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 3a3d33f680..9d4d5c8e6f 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -255,6 +255,48 @@ static void get_object_info_summary(struct strbuf *obj_info, int nongit)
 	strbuf_release(&dirpath);
 }
 
+static void get_alternates_summary(struct strbuf *alternates_info, int nongit)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
+
+	if (nongit) {
+		strbuf_addstr(alternates_info,
+			"not run from a git repository - alternates unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -346,6 +388,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.341.g760bfbb309-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 01/15] help: move list_config_help to builtin/help
  2020-02-06  0:40     ` [PATCH v6 01/15] help: move list_config_help to builtin/help Emily Shaffer
@ 2020-02-06  1:35       ` Danh Doan
  2020-02-13 22:58         ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Danh Doan @ 2020-02-06  1:35 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

On 2020-02-05 16:40:54-0800, Emily Shaffer <emilyshaffer@google.com> wrote:
> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> index 71158f7d8b..45fecf8bdf 100755
> --- a/generate-cmdlist.sh
> +++ b/generate-cmdlist.sh
> @@ -76,23 +76,6 @@ print_command_list () {
>  	echo "};"
>  }
>  
> -print_config_list () {
> -	cat <<EOF
> -static const char *config_name_list[] = {
> -EOF
> -	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> -	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> -	sort |
> -	while read line
> -	do
> -		echo "	\"$line\","
> -	done
> -	cat <<EOF
> -	NULL,
> -};
> -EOF
> -}
> -
>  exclude_programs=
>  while test "--exclude-program" = "$1"
>  do
> @@ -113,5 +96,3 @@ echo
>  define_category_names "$1"
>  echo
>  print_command_list "$1"
> -echo
> -print_config_list
> diff --git a/generate-configlist.sh b/generate-configlist.sh
> new file mode 100755
> index 0000000000..eca6a00c30
> --- /dev/null
> +++ b/generate-configlist.sh
> @@ -0,0 +1,24 @@
> +#!/bin/sh
> +
> +echo "/* Automatically generated by generate-configlist.sh */"
> +echo
> +
> +print_config_list () {
> +	cat <<EOF
> +static const char *config_name_list[] = {
> +EOF
> +	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> +	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> +	sort |
> +	while read line
> +	do
> +		echo "	\"$line\","
> +	done

This while-read-echo was moved from generate-cmdlist.sh,
which has some logic to work with read-variable.

We're moving it out, I think apply this diff on top of it will make the code easier to read.

diff --git a/generate-configlist.sh b/generate-configlist.sh
index eca6a00c30..163dbf30bb 100755
--- a/generate-configlist.sh
+++ b/generate-configlist.sh
@@ -10,10 +10,7 @@ EOF
 	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
 	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
 	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
+	sed 's/^/	"/; s/$/",/'
 	cat <<EOF
 	NULL,
 };



-- 
Danh

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info
  2020-02-06  0:40     ` [PATCH v6 03/15] bugreport: add tool to generate debugging info Emily Shaffer
@ 2020-02-07 14:18       ` SZEDER Gábor
  2020-02-07 18:51         ` Junio C Hamano
  2020-02-07 14:54       ` SZEDER Gábor
  2020-02-12 18:06       ` Junio C Hamano
  2 siblings, 1 reply; 177+ messages in thread
From: SZEDER Gábor @ 2020-02-07 14:18 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

On Wed, Feb 05, 2020 at 04:40:56PM -0800, Emily Shaffer wrote:
> diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
> new file mode 100755
> index 0000000000..451badff0c
> --- /dev/null
> +++ b/t/t0091-bugreport.sh
> @@ -0,0 +1,49 @@
> +#!/bin/sh
> +
> +test_description='git bugreport'
> +
> +. ./test-lib.sh
> +
> +# Headers "[System Info]" will be followed by a non-empty line if we put some
> +# information there; we can make sure all our headers were followed by some
> +# information to check if the command was successful.
> +HEADER_PATTERN="^\[.*\]$"
> +check_all_headers_populated() {
> +	while read -r line; do
> +		if test "$(grep "$HEADER_PATTERN" "$line")"; then
> +			echo "$line"
> +			read -r nextline
> +			if test -z "$nextline"; then
> +				return 1;
> +			fi
> +		fi
> +	done
> +}
> +
> +test_expect_success 'creates a report with content in the right places' '
> +	git bugreport &&
> +	REPORT="$(ls git-bugreport-*)" &&

What if the globbing were to match more than one file?

> +	check_all_headers_populated <$REPORT &&
> +	rm $REPORT

Please register a cleanup commands like this with 'test_when_finished'.

> +'
> +
> +test_expect_success 'dies if file with same name as report already exists' '
> +	touch git-bugreport-duplicate.txt &&
> +	test_must_fail git bugreport --suffix duplicate &&
> +	rm git-bugreport-duplicate.txt
> +'
> +
> +test_expect_success '--output-directory puts the report in the provided dir' '
> +	mkdir foo/ &&

Is it really necessary to create the directory in advance?  Or to put
it in another way: shouldn't 'git bugreport' create any missing
leading directories of the path given for its '-o' option?  FWIW, 'git
format-patch -o dir ...' does create all necessary directories.

> +	git bugreport -o foo/ &&
> +	test_path_is_file foo/git-bugreport-* &&

What if the globbing were to match more than one file? :)

> +	rm -fr foo/
> +'
> +
> +test_expect_success 'incorrect arguments abort with usage' '
> +	test_must_fail git bugreport --false 2>output &&
> +	grep usage output &&

This breaks the GETTEXT_POISON CI job, because "usage" is translated,
so the test should not look for it with plain 'grep'.  Please use
'test_i18ngrep' instead.

> +	test_path_is_missing git-bugreport-*
> +'
> +
> +test_done
> -- 
> 2.25.0.341.g760bfbb309-goog
> 

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 10/15] bugreport: add config values from safelist
  2020-02-06  0:41     ` [PATCH v6 10/15] bugreport: add config values from safelist Emily Shaffer
@ 2020-02-07 14:47       ` SZEDER Gábor
  2020-02-07 15:08         ` SZEDER Gábor
  0 siblings, 1 reply; 177+ messages in thread
From: SZEDER Gábor @ 2020-02-07 14:47 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

On Wed, Feb 05, 2020 at 04:41:03PM -0800, Emily Shaffer wrote:
> +static void get_safelisted_config(struct strbuf *config_info)
> +{
> +	size_t idx;
> +	struct string_list_item *it = NULL;
> +	struct key_value_info *kv_info = NULL;
> +
> +	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {

GCC 9 complains about this loop condition:

      CC bugreport.o
  
  bugreport.c: In function 'get_safelisted_config':
  
  bugreport.c:66:20: error: comparison of unsigned expression < 0 is always false [-Werror=type-limits]
  
     66 |  for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
  
        |                    ^

I don't understand this error, that autogenerated
'bugreport_config_safelist' array clearly has a a non-zero size.
What am I missing?


> +		const struct string_list *list =
> +			git_config_get_value_multi(bugreport_config_safelist[idx]);
> +
> +		if (!list)
> +			continue;
> +
> +		strbuf_addf(config_info, "%s:\n", bugreport_config_safelist[idx]);
> +		for_each_string_list_item(it, list) {
> +			kv_info = it->util;
> +			strbuf_addf(config_info, "  %s (%s)\n", it->string,
> +				    kv_info ? config_scope_name(kv_info->scope)
> +					    : "source unknown");
> +		}
> +	}
> +}
> +

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info
  2020-02-06  0:40     ` [PATCH v6 03/15] bugreport: add tool to generate debugging info Emily Shaffer
  2020-02-07 14:18       ` SZEDER Gábor
@ 2020-02-07 14:54       ` SZEDER Gábor
  2020-02-12 18:06       ` Junio C Hamano
  2 siblings, 0 replies; 177+ messages in thread
From: SZEDER Gábor @ 2020-02-07 14:54 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

On Wed, Feb 05, 2020 at 04:40:56PM -0800, Emily Shaffer wrote:
> Teach Git how to prompt the user for a good bug report: reproduction
> steps, expected behavior, and actual behavior. Later, Git can learn how
> to collect some diagnostic information from the repository.
> 
> If users can send us a well-written bug report which contains diagnostic
> information we would otherwise need to ask the user for, we can reduce
> the number of question-and-answer round trips between the reporter and
> the Git contributor.
> 
> Users may also wish to send a report like this to their local "Git
> expert" if they have put their repository into a state they are confused
> by.
> 
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  .gitignore                      |  1 +
>  Documentation/git-bugreport.txt | 41 ++++++++++++++++
>  Makefile                        |  5 ++
>  bugreport.c                     | 85 +++++++++++++++++++++++++++++++++
>  t/t0091-bugreport.sh            | 49 +++++++++++++++++++
>  5 files changed, 181 insertions(+)
>  create mode 100644 Documentation/git-bugreport.txt
>  create mode 100644 bugreport.c
>  create mode 100755 t/t0091-bugreport.sh

Please add a corresponding entry to 'command-list.txt' as well.
According to the comment at the beginning of that file, all, even
external commands should be included there.


^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 10/15] bugreport: add config values from safelist
  2020-02-07 14:47       ` SZEDER Gábor
@ 2020-02-07 15:08         ` SZEDER Gábor
  2020-02-07 16:24           ` Eric Sunshine
  0 siblings, 1 reply; 177+ messages in thread
From: SZEDER Gábor @ 2020-02-07 15:08 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

On Fri, Feb 07, 2020 at 03:47:25PM +0100, SZEDER Gábor wrote:
> On Wed, Feb 05, 2020 at 04:41:03PM -0800, Emily Shaffer wrote:
> > +static void get_safelisted_config(struct strbuf *config_info)
> > +{
> > +	size_t idx;
> > +	struct string_list_item *it = NULL;
> > +	struct key_value_info *kv_info = NULL;
> > +
> > +	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
> 
> GCC 9 complains about this loop condition:
> 
>       CC bugreport.o
>   
>   bugreport.c: In function 'get_safelisted_config':
>   
>   bugreport.c:66:20: error: comparison of unsigned expression < 0 is always false [-Werror=type-limits]
>   
>      66 |  for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
>   
>         |                    ^
> 
> I don't understand this error, that autogenerated
> 'bugreport_config_safelist' array clearly has a a non-zero size.
> What am I missing?

macOS 'sed', that's what I was missing :)

So that array is in fact empty on macOS, because the entries of that
array are generated with:

  # cat all regular files in Documentation/config
  find Documentation/config -type f -exec cat {} \; |
  # print the command name which matches the annotate-bugreport macro
  sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort

and the 'sed' included in macOS apparently interprets that '\+'
differently than GNU 'sed', and as a result won't match anything.

FWIW, that '\+' doesn't seem to be necessary, though, and after
removing it the resulting generated array looked good to me (and to
the compiler) both on Linux and macOS.


^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 09/15] bugreport: generate config safelist based on docs
  2020-02-06  0:41     ` [PATCH v6 09/15] bugreport: generate config safelist based on docs Emily Shaffer
@ 2020-02-07 15:30       ` SZEDER Gábor
  2020-02-13 23:14         ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: SZEDER Gábor @ 2020-02-07 15:30 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git, Martin Ågren, Johannes Schindelin

On Wed, Feb 05, 2020 at 04:41:02PM -0800, Emily Shaffer wrote:
> Add a new step to the build to generate a safelist of git-config
> variables which are appropriate to include in the output of
> git-bugreport. New variables can be added to the safelist by annotating
> their documentation in Documentation/config with the "annotate" macro,
> which is a no-op in AsciiDoc and AsciiDoctor.
> 
> Some configs are private in nature, and can contain remote URLs,
> passwords, or other sensitive information. In the event that a user
> doesn't notice their information while reviewing a bugreport, that user
> may leak their credentials to other individuals, mailing lists, or bug
> tracking tools inadvertently. Heuristic blocklisting of configuration
> keys is imperfect and prone to false negatives; given the nature of the
> information which can be leaked, a safelist is more reliable.
> 
> However, it's possible that in some situations, an organization may be
> less concerned with privacy of things like remote URLs and branch names,
> and more concerned with ease of diagnosis for their support staff. In
> those cases, it may make more sense for that organization to modify the
> code to use a blocklist. To that end, we should try to mark configs which
> are definitely safe, and configs which are definitely unsafe, and leave
> blank configs which are somewhere in between. To mark a config as safe,
> add "annotate:bugreport[include]" to the corresponding line in the
> config documentation; to mark it as unsafe, add
> "annotate:bugreport[exclude]" instead.
> 
> Generating bugreport-config-safelist.h at build time by grepping the
> documentation for this new macro helps us prevent staleness. The macro
> itself is a no-op and should not alter the appearance of the
> documentation in either AsciiDoc or AsciiDoctor, confirmable by running:
> 
>   cd Documentation
>   ./doc-diff --asciidoctor HEAD^ HEAD
>   ./doc-diff --asciidoc HEAD^ HEAD
> 
> Diffing the rendered HTML shows that only inline comments were added,
> which shouldn't be a problem.
> 
> Additionally, add annotations to the sendemail config documentation in
> order to demonstrate a proof of concept.
> 
> Helped-by: Martin Ågren <martin.agren@gmail.com>
> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  .gitignore                              |  1 +
>  Documentation/asciidoc.conf             |  9 ++++
>  Documentation/asciidoctor-extensions.rb |  5 +++
>  Documentation/config/sendemail.txt      | 56 ++++++++++++-------------
>  Makefile                                |  7 ++++
>  generate-bugreport-config-safelist.sh   | 17 ++++++++
>  6 files changed, 67 insertions(+), 28 deletions(-)
>  create mode 100755 generate-bugreport-config-safelist.sh
> 
> diff --git a/.gitignore b/.gitignore
> index d89bf9e11e..bd2f49b996 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -192,6 +192,7 @@
>  /gitweb/static/gitweb.min.*
>  /config-list.h
>  /command-list.h
> +/bugreport-config-safelist.h
>  *.tar.gz
>  *.dsc
>  *.deb
> diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
> index 8fc4b67081..663e06481f 100644
> --- a/Documentation/asciidoc.conf
> +++ b/Documentation/asciidoc.conf
> @@ -6,9 +6,14 @@
>  #
>  # Show Git link as: <command>(<section>); if section is defined, else just show
>  # the command.
> +#
> +# The annotate macro does nothing as far as rendering is
> +# concerned -- we just grep for it in the sources to populate
> +# things like the bugreport safelist.
>  
>  [macros]
>  (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
> +(?su)[\\]?(?P<name>annotate):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
>  
>  [attributes]
>  asterisk=&#42;
> @@ -28,6 +33,8 @@ ifdef::backend-docbook[]
>  {0#<citerefentry>}
>  {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
>  {0#</citerefentry>}
> +[annotate-inlinemacro]
> +{0#}
>  endif::backend-docbook[]
>  
>  ifdef::backend-docbook[]
> @@ -94,4 +101,6 @@ ifdef::backend-xhtml11[]
>  git-relative-html-prefix=
>  [linkgit-inlinemacro]
>  <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
> +[annotate-inlinemacro]
> +<!-- -->
>  endif::backend-xhtml11[]
> diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
> index d906a00803..382bd8f6f4 100644
> --- a/Documentation/asciidoctor-extensions.rb
> +++ b/Documentation/asciidoctor-extensions.rb
> @@ -37,6 +37,10 @@ module Git
>            output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
>          end
>          output
> +
> +    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
> +      def process(parent, target, attrs)
> +        ""

This change breaks building the documentation with Asciidoctor v1.5.8:

  /home/travis/.rvm/rubies/ruby-2.5.3/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': /home/travis/build/git/git/Documentation/asciidoctor-extensions.rb:41: class definition in method body (SyntaxError)
  
  ...xtensions::InlineMacroProcessor
  
  ...                               ^
  
  /home/travis/build/git/git/Documentation/asciidoctor-extensions.rb:53: syntax error, unexpected end-of-input, expecting keyword_end


  https://travis-ci.org/git/git/jobs/647093871#L1127

>        end
>      end
>    end
> @@ -45,4 +49,5 @@ end
>  Asciidoctor::Extensions.register do
>    inline_macro Git::Documentation::LinkGitProcessor, :linkgit
>    postprocessor Git::Documentation::DocumentPostProcessor
> +  inline_macro Git::Documentation::AnnotateProcessor, :annotate
>  end

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 10/15] bugreport: add config values from safelist
  2020-02-07 15:08         ` SZEDER Gábor
@ 2020-02-07 16:24           ` Eric Sunshine
  2020-02-07 16:51             ` Andreas Schwab
  0 siblings, 1 reply; 177+ messages in thread
From: Eric Sunshine @ 2020-02-07 16:24 UTC (permalink / raw)
  To: SZEDER Gábor; +Cc: Emily Shaffer, Git List

On Fri, Feb 7, 2020 at 10:09 AM SZEDER Gábor <szeder.dev@gmail.com> wrote:
> macOS 'sed', that's what I was missing :)
>
>   sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
>
> and the 'sed' included in macOS apparently interprets that '\+'
> differently than GNU 'sed', and as a result won't match anything.

More generally, this would be a problem with any 'sed' of BSD lineage.

> FWIW, that '\+' doesn't seem to be necessary, though, and after
> removing it the resulting generated array looked good to me [...]

A reasonable replacement for "<SP>\+" would be "<SP><SP>*" (where <SP>
represents 'space').

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 10/15] bugreport: add config values from safelist
  2020-02-07 16:24           ` Eric Sunshine
@ 2020-02-07 16:51             ` Andreas Schwab
  2020-02-13 22:02               ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Andreas Schwab @ 2020-02-07 16:51 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: SZEDER Gábor, Emily Shaffer, Git List

On Feb 07 2020, Eric Sunshine wrote:

> On Fri, Feb 7, 2020 at 10:09 AM SZEDER Gábor <szeder.dev@gmail.com> wrote:
>> macOS 'sed', that's what I was missing :)
>>
>>   sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
>>
>> and the 'sed' included in macOS apparently interprets that '\+'
>> differently than GNU 'sed', and as a result won't match anything.
>
> More generally, this would be a problem with any 'sed' of BSD lineage.
>
>> FWIW, that '\+' doesn't seem to be necessary, though, and after
>> removing it the resulting generated array looked good to me [...]
>
> A reasonable replacement for "<SP>\+" would be "<SP><SP>*" (where <SP>
> represents 'space').

Another problem with that regexp is that it contains two adjacent
repetitions matching the same character.  When there are two or more
spaces before "annotate:" all but the last of them can be matched by
either '\(.*\)' or ' \+'.  To fix that '\(.*\)' needs to be modified to
not match a trailing space.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info
  2020-02-07 14:18       ` SZEDER Gábor
@ 2020-02-07 18:51         ` Junio C Hamano
  2020-02-11 22:40           ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2020-02-07 18:51 UTC (permalink / raw)
  To: SZEDER Gábor; +Cc: Emily Shaffer, git

SZEDER Gábor <szeder.dev@gmail.com> writes:

> On Wed, Feb 05, 2020 at 04:40:56PM -0800, Emily Shaffer wrote:
>> diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
>> new file mode 100755
>> index 0000000000..451badff0c
>> --- /dev/null
>> +++ b/t/t0091-bugreport.sh
>> @@ -0,0 +1,49 @@
>> +#!/bin/sh
>> +
>> +test_description='git bugreport'
>> +
>> +. ./test-lib.sh
>> +
>> +# Headers "[System Info]" will be followed by a non-empty line if we put some
>> +# information there; we can make sure all our headers were followed by some
>> +# information to check if the command was successful.
>> +HEADER_PATTERN="^\[.*\]$"
>> +check_all_headers_populated() {
>> +	while read -r line; do
>> +		if test "$(grep "$HEADER_PATTERN" "$line")"; then
>> +			echo "$line"
>> +			read -r nextline
>> +			if test -z "$nextline"; then
>> +				return 1;
>> +			fi
>> +		fi
>> +	done
>> +}
>> +
>> +test_expect_success 'creates a report with content in the right places' '
>> +	git bugreport &&
>> +	REPORT="$(ls git-bugreport-*)" &&
>
> What if the globbing were to match more than one file?

An often-useful pattern is to make the command report the output
filename, i.e.

	REPORT=$(git butreport) &&

if the design insists that "git bugreport" should allocate a
filename in order to make it easy to guarantee uniqueness.

Of course, we can make the invoker supply filename, e.g.

	REPORT=$(generate-output-filename) &&
	git bugreport -o "$REPORT" ;# or git bugreport >"$REPORT"


^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info
  2020-02-07 18:51         ` Junio C Hamano
@ 2020-02-11 22:40           ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-11 22:40 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: SZEDER Gábor, git

On Fri, Feb 07, 2020 at 10:51:29AM -0800, Junio C Hamano wrote:
> SZEDER Gábor <szeder.dev@gmail.com> writes:
> 
> > On Wed, Feb 05, 2020 at 04:40:56PM -0800, Emily Shaffer wrote:
> >> diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
> >> new file mode 100755
> >> index 0000000000..451badff0c
> >> --- /dev/null
> >> +++ b/t/t0091-bugreport.sh
> >> @@ -0,0 +1,49 @@
> >> +#!/bin/sh
> >> +
> >> +test_description='git bugreport'
> >> +
> >> +. ./test-lib.sh
> >> +
> >> +# Headers "[System Info]" will be followed by a non-empty line if we put some
> >> +# information there; we can make sure all our headers were followed by some
> >> +# information to check if the command was successful.
> >> +HEADER_PATTERN="^\[.*\]$"
> >> +check_all_headers_populated() {
> >> +	while read -r line; do
> >> +		if test "$(grep "$HEADER_PATTERN" "$line")"; then
> >> +			echo "$line"
> >> +			read -r nextline
> >> +			if test -z "$nextline"; then
> >> +				return 1;
> >> +			fi
> >> +		fi
> >> +	done
> >> +}
> >> +
> >> +test_expect_success 'creates a report with content in the right places' '
> >> +	git bugreport &&
> >> +	REPORT="$(ls git-bugreport-*)" &&
> >
> > What if the globbing were to match more than one file?
> 
> An often-useful pattern is to make the command report the output
> filename, i.e.
> 
> 	REPORT=$(git butreport) &&
> 
> if the design insists that "git bugreport" should allocate a
> filename in order to make it easy to guarantee uniqueness.
> 
> Of course, we can make the invoker supply filename, e.g.
> 
> 	REPORT=$(generate-output-filename) &&
> 	git bugreport -o "$REPORT" ;# or git bugreport >"$REPORT"

Yeah, this is an option and I've changed the offending test - which
captures a variable based on globbing - to use --suffix and force a
filename.

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info
  2020-02-06  0:40     ` [PATCH v6 03/15] bugreport: add tool to generate debugging info Emily Shaffer
  2020-02-07 14:18       ` SZEDER Gábor
  2020-02-07 14:54       ` SZEDER Gábor
@ 2020-02-12 18:06       ` Junio C Hamano
  2020-02-12 22:36         ` Emily Shaffer
  2 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2020-02-12 18:06 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

As this topic seems to break under GIT_TEST_GETTEXT_POISON=yes, I'll
apply the following band-aid on top before merging it to 'pu'.  

Most of them are style fixes, but quoting $REPORT, not just makes
the redirection honor the coding guidelines, will ensure that an
error is caught if git-bugreport-* glob matches no paths or more
than on paths.




 t/t0091-bugreport.sh | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
index 451badff0c..f2186941ce 100755
--- a/t/t0091-bugreport.sh
+++ b/t/t0091-bugreport.sh
@@ -8,9 +8,12 @@ test_description='git bugreport'
 # information there; we can make sure all our headers were followed by some
 # information to check if the command was successful.
 HEADER_PATTERN="^\[.*\]$"
-check_all_headers_populated() {
-	while read -r line; do
-		if test "$(grep "$HEADER_PATTERN" "$line")"; then
+
+check_all_headers_populated () {
+	while read -r line
+	do
+		if test "$(grep "$HEADER_PATTERN" "$line")"
+		then
 			echo "$line"
 			read -r nextline
 			if test -z "$nextline"; then
@@ -23,12 +26,12 @@ check_all_headers_populated() {
 test_expect_success 'creates a report with content in the right places' '
 	git bugreport &&
 	REPORT="$(ls git-bugreport-*)" &&
-	check_all_headers_populated <$REPORT &&
-	rm $REPORT
+	check_all_headers_populated <"$REPORT" &&
+	rm "$REPORT"
 '
 
 test_expect_success 'dies if file with same name as report already exists' '
-	touch git-bugreport-duplicate.txt &&
+	>>git-bugreport-duplicate.txt &&
 	test_must_fail git bugreport --suffix duplicate &&
 	rm git-bugreport-duplicate.txt
 '
@@ -42,7 +45,7 @@ test_expect_success '--output-directory puts the report in the provided dir' '
 
 test_expect_success 'incorrect arguments abort with usage' '
 	test_must_fail git bugreport --false 2>output &&
-	grep usage output &&
+	test_i18ngrep usage output &&
 	test_path_is_missing git-bugreport-*
 '
 
-- 
2.25.0-455-gcf3c3a5ab4


^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 03/15] bugreport: add tool to generate debugging info
  2020-02-12 18:06       ` Junio C Hamano
@ 2020-02-12 22:36         ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-12 22:36 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Wed, Feb 12, 2020 at 10:06:44AM -0800, Junio C Hamano wrote:
> As this topic seems to break under GIT_TEST_GETTEXT_POISON=yes, I'll
> apply the following band-aid on top before merging it to 'pu'.  
> 
> Most of them are style fixes, but quoting $REPORT, not just makes
> the redirection honor the coding guidelines, will ensure that an
> error is caught if git-bugreport-* glob matches no paths or more
> than on paths.

Thanks. Most of these I have locally, but not all. I'll apply this as a
fixup on patch 3/n (the one that introduces the tests) and add a
Helped-by line.

I'm hoping to get a reroll out today; it was the first thing on my list
this week, and then a high-priority internal issue shows up (so it
goes). Sorry for the inconvenience with the CI failures.

 - Emily

>  t/t0091-bugreport.sh | 17 ++++++++++-------
>  1 file changed, 10 insertions(+), 7 deletions(-)
> 
> diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
> index 451badff0c..f2186941ce 100755
> --- a/t/t0091-bugreport.sh
> +++ b/t/t0091-bugreport.sh
> @@ -8,9 +8,12 @@ test_description='git bugreport'
>  # information there; we can make sure all our headers were followed by some
>  # information to check if the command was successful.
>  HEADER_PATTERN="^\[.*\]$"
> -check_all_headers_populated() {
> -	while read -r line; do
> -		if test "$(grep "$HEADER_PATTERN" "$line")"; then
> +
> +check_all_headers_populated () {
> +	while read -r line
> +	do
> +		if test "$(grep "$HEADER_PATTERN" "$line")"
> +		then
>  			echo "$line"
>  			read -r nextline
>  			if test -z "$nextline"; then
> @@ -23,12 +26,12 @@ check_all_headers_populated() {
>  test_expect_success 'creates a report with content in the right places' '
>  	git bugreport &&
>  	REPORT="$(ls git-bugreport-*)" &&
> -	check_all_headers_populated <$REPORT &&
> -	rm $REPORT
> +	check_all_headers_populated <"$REPORT" &&
> +	rm "$REPORT"
>  '
>  
>  test_expect_success 'dies if file with same name as report already exists' '
> -	touch git-bugreport-duplicate.txt &&
> +	>>git-bugreport-duplicate.txt &&
>  	test_must_fail git bugreport --suffix duplicate &&
>  	rm git-bugreport-duplicate.txt
>  '
> @@ -42,7 +45,7 @@ test_expect_success '--output-directory puts the report in the provided dir' '
>  
>  test_expect_success 'incorrect arguments abort with usage' '
>  	test_must_fail git bugreport --false 2>output &&
> -	grep usage output &&
> +	test_i18ngrep usage output &&
>  	test_path_is_missing git-bugreport-*
>  '
>  
> -- 
> 2.25.0-455-gcf3c3a5ab4
> 

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 10/15] bugreport: add config values from safelist
  2020-02-07 16:51             ` Andreas Schwab
@ 2020-02-13 22:02               ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-13 22:02 UTC (permalink / raw)
  To: Andreas Schwab; +Cc: Eric Sunshine, SZEDER Gábor, Git List

On Fri, Feb 07, 2020 at 05:51:05PM +0100, Andreas Schwab wrote:
> On Feb 07 2020, Eric Sunshine wrote:
> 
> > On Fri, Feb 7, 2020 at 10:09 AM SZEDER Gábor <szeder.dev@gmail.com> wrote:
> >> macOS 'sed', that's what I was missing :)
> >>
> >>   sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
> >>
> >> and the 'sed' included in macOS apparently interprets that '\+'
> >> differently than GNU 'sed', and as a result won't match anything.
> >
> > More generally, this would be a problem with any 'sed' of BSD lineage.
> >
> >> FWIW, that '\+' doesn't seem to be necessary, though, and after
> >> removing it the resulting generated array looked good to me [...]
> >
> > A reasonable replacement for "<SP>\+" would be "<SP><SP>*" (where <SP>
> > represents 'space').
> 
> Another problem with that regexp is that it contains two adjacent
> repetitions matching the same character.  When there are two or more
> spaces before "annotate:" all but the last of them can be matched by
> either '\(.*\)' or ' \+'.  To fix that '\(.*\)' needs to be modified to
> not match a trailing space.

Hum. I had assumed since the capture group was not greedy, it would not
capture any trailing spaces that could be captured by ' \+'. I don't see
a problem in making it non-explicit, though.

I'll hack on this some more. I do find myself pretty annoyed at the
matcher difference between 'sed' and 'also sed', though, and don't see
in the manpage a way to guarantee which matcher 'sed' should use (a la
'grep -[EFGP]'). :)

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 01/15] help: move list_config_help to builtin/help
  2020-02-06  1:35       ` Danh Doan
@ 2020-02-13 22:58         ` Emily Shaffer
  2020-02-13 23:07           ` Eric Sunshine
  0 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-13 22:58 UTC (permalink / raw)
  To: Danh Doan; +Cc: git

On Thu, Feb 06, 2020 at 08:35:33AM +0700, Danh Doan wrote:
> On 2020-02-05 16:40:54-0800, Emily Shaffer <emilyshaffer@google.com> wrote:
> > diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> > index 71158f7d8b..45fecf8bdf 100755
> > --- a/generate-cmdlist.sh
> > +++ b/generate-cmdlist.sh
> > @@ -76,23 +76,6 @@ print_command_list () {
> >  	echo "};"
> >  }
> >  
> > -print_config_list () {
> > -	cat <<EOF
> > -static const char *config_name_list[] = {
> > -EOF
> > -	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> > -	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> > -	sort |
> > -	while read line
> > -	do
> > -		echo "	\"$line\","
> > -	done
> > -	cat <<EOF
> > -	NULL,
> > -};
> > -EOF
> > -}
> > -
> >  exclude_programs=
> >  while test "--exclude-program" = "$1"
> >  do
> > @@ -113,5 +96,3 @@ echo
> >  define_category_names "$1"
> >  echo
> >  print_command_list "$1"
> > -echo
> > -print_config_list
> > diff --git a/generate-configlist.sh b/generate-configlist.sh
> > new file mode 100755
> > index 0000000000..eca6a00c30
> > --- /dev/null
> > +++ b/generate-configlist.sh
> > @@ -0,0 +1,24 @@
> > +#!/bin/sh
> > +
> > +echo "/* Automatically generated by generate-configlist.sh */"
> > +echo
> > +
> > +print_config_list () {
> > +	cat <<EOF
> > +static const char *config_name_list[] = {
> > +EOF
> > +	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
> > +	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
> > +	sort |
> > +	while read line
> > +	do
> > +		echo "	\"$line\","
> > +	done
> 
> This while-read-echo was moved from generate-cmdlist.sh,
> which has some logic to work with read-variable.
> 
> We're moving it out, I think apply this diff on top of it will make the code easier to read.
> 
> diff --git a/generate-configlist.sh b/generate-configlist.sh
> index eca6a00c30..163dbf30bb 100755
> --- a/generate-configlist.sh
> +++ b/generate-configlist.sh
> @@ -10,10 +10,7 @@ EOF
>  	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
>  	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
>  	sort |
> -	while read line
> -	do
> -		echo "	\"$line\","
> -	done
> +	sed 's/^/	"/; s/$/",/'
>  	cat <<EOF
>  	NULL,
>  };

Thanks for the suggestion. You're right that I didn't look into the
contents of this script much, because I was just moving it; I modified
your sed expression slightly so it performs just one operation:

  sed 's/^.*$/    "\0",/'

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 01/15] help: move list_config_help to builtin/help
  2020-02-13 22:58         ` Emily Shaffer
@ 2020-02-13 23:07           ` Eric Sunshine
  2020-02-13 23:24             ` Junio C Hamano
  0 siblings, 1 reply; 177+ messages in thread
From: Eric Sunshine @ 2020-02-13 23:07 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Danh Doan, Git List

On Thu, Feb 13, 2020 at 5:58 PM Emily Shaffer <emilyshaffer@google.com> wrote:
> On Thu, Feb 06, 2020 at 08:35:33AM +0700, Danh Doan wrote:
> > -     while read line
> > -     do
> > -             echo "  \"$line\","
> > -     done
> > +     sed 's/^/       "/; s/$/",/'
>
> Thanks for the suggestion. You're right that I didn't look into the
> contents of this script much, because I was just moving it; I modified
> your sed expression slightly so it performs just one operation:
>
>   sed 's/^.*$/    "\0",/'

This isn't portable and won't work with 'sed' from BSD lineage,
including Mac OS. More portable would be:

    sed 's/^\(.*\)$/    "\1",/'

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 09/15] bugreport: generate config safelist based on docs
  2020-02-07 15:30       ` SZEDER Gábor
@ 2020-02-13 23:14         ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-13 23:14 UTC (permalink / raw)
  To: SZEDER Gábor; +Cc: git, Martin Ågren, Johannes Schindelin

On Fri, Feb 07, 2020 at 04:30:42PM +0100, SZEDER Gábor wrote:
> On Wed, Feb 05, 2020 at 04:41:02PM -0800, Emily Shaffer wrote:
> > diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
> > index d906a00803..382bd8f6f4 100644
> > --- a/Documentation/asciidoctor-extensions.rb
> > +++ b/Documentation/asciidoctor-extensions.rb
> > @@ -37,6 +37,10 @@ module Git
> >            output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
> >          end
> >          output
> > +
> > +    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
> > +      def process(parent, target, attrs)
> > +        ""
> 
> This change breaks building the documentation with Asciidoctor v1.5.8:
> 
>   /home/travis/.rvm/rubies/ruby-2.5.3/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require': /home/travis/build/git/git/Documentation/asciidoctor-extensions.rb:41: class definition in method body (SyntaxError)
>   
>   ...xtensions::InlineMacroProcessor
>   
>   ...                               ^
>   
>   /home/travis/build/git/git/Documentation/asciidoctor-extensions.rb:53: syntax error, unexpected end-of-input, expecting keyword_end
> 
> 
>   https://travis-ci.org/git/git/jobs/647093871#L1127

Done. It passes locally for me now if I run:

  USE_ASCIIDOCTOR=1 make doc

Thanks for the heads up.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 01/15] help: move list_config_help to builtin/help
  2020-02-13 23:07           ` Eric Sunshine
@ 2020-02-13 23:24             ` Junio C Hamano
  2020-02-13 23:29               ` Eric Sunshine
  0 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2020-02-13 23:24 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: Emily Shaffer, Danh Doan, Git List

Eric Sunshine <sunshine@sunshineco.com> writes:

> On Thu, Feb 13, 2020 at 5:58 PM Emily Shaffer <emilyshaffer@google.com> wrote:
>> On Thu, Feb 06, 2020 at 08:35:33AM +0700, Danh Doan wrote:
>> > -     while read line
>> > -     do
>> > -             echo "  \"$line\","
>> > -     done
>> > +     sed 's/^/       "/; s/$/",/'
>>
>> Thanks for the suggestion. You're right that I didn't look into the
>> contents of this script much, because I was just moving it; I modified
>> your sed expression slightly so it performs just one operation:
>>
>>   sed 's/^.*$/    "\0",/'
>
> This isn't portable and won't work with 'sed' from BSD lineage,
> including Mac OS. More portable would be:
>
>     sed 's/^\(.*\)$/    "\1",/'

Or just replace Emily's \0 with &; that should be in POSIX.

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 01/15] help: move list_config_help to builtin/help
  2020-02-13 23:24             ` Junio C Hamano
@ 2020-02-13 23:29               ` Eric Sunshine
  2020-02-14  1:20                 ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Eric Sunshine @ 2020-02-13 23:29 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Emily Shaffer, Danh Doan, Git List

On Thu, Feb 13, 2020 at 6:24 PM Junio C Hamano <gitster@pobox.com> wrote:
> Eric Sunshine <sunshine@sunshineco.com> writes:
> > On Thu, Feb 13, 2020 at 5:58 PM Emily Shaffer <emilyshaffer@google.com> wrote:
> >> Thanks for the suggestion. You're right that I didn't look into the
> >> contents of this script much, because I was just moving it; I modified
> >> your sed expression slightly so it performs just one operation:
> >>
> >>   sed 's/^.*$/    "\0",/'
> >
> > This isn't portable and won't work with 'sed' from BSD lineage,
> > including Mac OS. More portable would be:
> >
> >     sed 's/^\(.*\)$/    "\1",/'
>
> Or just replace Emily's \0 with &; that should be in POSIX.

Yes, that's nicer.

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v6 01/15] help: move list_config_help to builtin/help
  2020-02-13 23:29               ` Eric Sunshine
@ 2020-02-14  1:20                 ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:20 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: Junio C Hamano, Danh Doan, Git List

On Thu, Feb 13, 2020 at 06:29:39PM -0500, Eric Sunshine wrote:
> On Thu, Feb 13, 2020 at 6:24 PM Junio C Hamano <gitster@pobox.com> wrote:
> > Eric Sunshine <sunshine@sunshineco.com> writes:
> > > On Thu, Feb 13, 2020 at 5:58 PM Emily Shaffer <emilyshaffer@google.com> wrote:
> > >> Thanks for the suggestion. You're right that I didn't look into the
> > >> contents of this script much, because I was just moving it; I modified
> > >> your sed expression slightly so it performs just one operation:
> > >>
> > >>   sed 's/^.*$/    "\0",/'
> > >
> > > This isn't portable and won't work with 'sed' from BSD lineage,
> > > including Mac OS. More portable would be:
> > >
> > >     sed 's/^\(.*\)$/    "\1",/'
> >
> > Or just replace Emily's \0 with &; that should be in POSIX.
> 
> Yes, that's nicer.

Sure, thanks.

^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v7 00/15] add git-bugreport tool
  2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
                       ` (14 preceding siblings ...)
  2020-02-06  0:41     ` [PATCH v6 15/15] bugreport: summarize contents of alternates file Emily Shaffer
@ 2020-02-14  1:53     ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 01/15] help: move list_config_help to builtin/help Emily Shaffer
                         ` (15 more replies)
  15 siblings, 16 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git
  Cc: Emily Shaffer, Derrick Stolee, Johannes Schindelin,
	Junio C Hamano, Martin Ågren, Aaron Schrab, Danh Doan,
	Eric Sunshine, SZEDER Gábor, Andreas Schwab

This patchset is based on mr/show-config-scope, which looks close to
ready for 'next'.

Since v6:

 - Applied Danh's suggestion[1] (with modifications recommended by Eric and
   Junio downthread)
 - Fixed a missed end-of-macro in the Asciidoctor config which was
   pointed out most recently by SZEDER[2]
 - Fixed generate-bugreport-safelist-config.sh to be more friendly to
   BSD-derived 'sed' and to capture trailing ' ' when extra whitespace is
   present(patch 9/15). This now uses POSIX character classes and {}
   notation instead of + and has been tested on OSX; I'd love to hear if
   it's still not working correctly.
 - Applied Junio's patch[3] to make GIT_TEST_GETTEXT_POISON happy and to
   make the tests more idiomatic.
 - Stop globbing into variables, start making the output directory if it
   doesn't exist before[4]
 - Add an entry in 'command-list.txt'[5]. I sure wish someone had
   explained how to do this somewhere in a tutorial about writing new
   commands ;)
 - Added a test to ensure that git-bugreport runs gracefully in a
   non-repo directory.
 - Added nongit_ok flags to relevant pieces of info to ensure the above
   test passes.

[1] https://lore.kernel.org/git/20200206013533.GA3993@danh.dev
[2] https://lore.kernel.org/git/20200207153042.GI2868@szeder.dev
[3] https://lore.kernel.org/git/xmqq36bfvfwr.fsf@gitster-ct.c.googlers.com
[4] https://lore.kernel.org/git/20200207141802.GE2868@szeder.dev
[5] https://lore.kernel.org/git/20200207145409.GG2868@szeder.dev


Emily Shaffer (15):
  help: move list_config_help to builtin/help
  help: add shell-path to --build-options
  bugreport: add tool to generate debugging info
  bugreport: gather git version and build info
  bugreport: add uname info
  bugreport: add compiler info
  bugreport: add git-remote-https version
  bugreport: include user interactive shell
  bugreport: generate config safelist based on docs
  bugreport: add config values from safelist
  bugreport: collect list of populated hooks
  bugreport: count loose objects
  bugreport: add packed object summary
  bugreport: list contents of $OBJDIR/info
  bugreport: summarize contents of alternates file

 .gitignore                              |   3 +
 Documentation/asciidoc.conf             |   9 +
 Documentation/asciidoctor-extensions.rb |   7 +
 Documentation/config/sendemail.txt      |  56 ++--
 Documentation/git-bugreport.txt         |  55 ++++
 Makefile                                |  27 +-
 bugreport.c                             | 418 ++++++++++++++++++++++++
 builtin/help.c                          |  86 +++++
 command-list.txt                        |   1 +
 compat/compiler.h                       |  24 ++
 generate-bugreport-config-safelist.sh   |  18 +
 generate-cmdlist.sh                     |  19 --
 generate-configlist.sh                  |  21 ++
 help.c                                  | 132 ++------
 help.h                                  |   2 +-
 object-store.h                          |   6 +
 packfile.c                              |   3 +-
 remote-curl.c                           |   8 +
 t/t0091-bugreport.sh                    |  55 ++++
 19 files changed, 794 insertions(+), 156 deletions(-)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100644 compat/compiler.h
 create mode 100755 generate-bugreport-config-safelist.sh
 create mode 100755 generate-configlist.sh
 create mode 100755 t/t0091-bugreport.sh

v6 to v7 range-diff:

diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index 382bd8f6f4..03c80af0e5 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -37,6 +37,8 @@ module Git
           output = output.sub(/<\/refmeta>/, new_tags + "</refmeta>")
         end
         output
+      end
+    end
 
     class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
       def process(parent, target, attrs)
diff --git a/bugreport.c b/bugreport.c
index 9d4d5c8e6f..3bc8cb3579 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -368,6 +368,15 @@ int cmd_main(int argc, const char **argv)
 	if (!stat(report_path.buf, &statbuf))
 		die("'%s' already exists", report_path.buf);
 
+	switch (safe_create_leading_directories(report_path.buf)) {
+	case SCLD_OK:
+	case SCLD_EXISTS:
+		break;
+	default:
+		die(_("could not create leading directories for '%s'"),
+		    report_path.buf);
+	}
+
 	get_bug_template(&buffer);
 
 	get_header(&buffer, "System Info");
diff --git a/command-list.txt b/command-list.txt
index 2087894655..185e5e3f05 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -54,6 +54,7 @@ git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
 git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
+git-bugreport                           ancillaryinterrogators
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
 git-check-attr                          purehelpers
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
index 17d92a91c5..2794a8ae77 100755
--- a/generate-bugreport-config-safelist.sh
+++ b/generate-bugreport-config-safelist.sh
@@ -10,7 +10,8 @@ EOF
 # cat all regular files in Documentation/config
 find Documentation/config -type f -exec cat {} \; |
 # print the command name which matches the annotate-bugreport macro
-sed -n 's/^\(.*\) \+annotate:bugreport\[include\].* ::$/  "\1",/p' | sort
+sed -n 's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/  "\1",/p' \
+	| sort
 
 cat <<EOF
 };
diff --git a/generate-configlist.sh b/generate-configlist.sh
index eca6a00c30..8692fe5cf4 100755
--- a/generate-configlist.sh
+++ b/generate-configlist.sh
@@ -10,10 +10,7 @@ EOF
 	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
 	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
 	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
+	sed 's/^.*$/	"&",/'
 	cat <<EOF
 	NULL,
 };
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
index 451badff0c..6585a2d144 100755
--- a/t/t0091-bugreport.sh
+++ b/t/t0091-bugreport.sh
@@ -8,9 +8,12 @@ test_description='git bugreport'
 # information there; we can make sure all our headers were followed by some
 # information to check if the command was successful.
 HEADER_PATTERN="^\[.*\]$"
-check_all_headers_populated() {
-	while read -r line; do
-		if test "$(grep "$HEADER_PATTERN" "$line")"; then
+
+check_all_headers_populated () {
+	while read -r line
+	do
+		if test "$(grep "$HEADER_PATTERN" "$line")"
+		then
 			echo "$line"
 			read -r nextline
 			if test -z "$nextline"; then
@@ -21,29 +24,32 @@ check_all_headers_populated() {
 }
 
 test_expect_success 'creates a report with content in the right places' '
-	git bugreport &&
-	REPORT="$(ls git-bugreport-*)" &&
-	check_all_headers_populated <$REPORT &&
-	rm $REPORT
+	git bugreport -s check-headers &&
+	check_all_headers_populated <git-bugreport-check-headers.txt &&
+	test_when_finished rm git-bugreport-check-headers.txt
 '
 
 test_expect_success 'dies if file with same name as report already exists' '
-	touch git-bugreport-duplicate.txt &&
+	>>git-bugreport-duplicate.txt &&
 	test_must_fail git bugreport --suffix duplicate &&
-	rm git-bugreport-duplicate.txt
+	test_when_finished rm git-bugreport-duplicate.txt
 '
 
 test_expect_success '--output-directory puts the report in the provided dir' '
-	mkdir foo/ &&
 	git bugreport -o foo/ &&
 	test_path_is_file foo/git-bugreport-* &&
-	rm -fr foo/
+	test_when_finished rm -fr foo/
 '
 
 test_expect_success 'incorrect arguments abort with usage' '
 	test_must_fail git bugreport --false 2>output &&
-	grep usage output &&
+	test_i18ngrep usage output &&
 	test_path_is_missing git-bugreport-*
 '
 
+test_expect_success 'runs outside of a git dir' '
+	nongit git bugreport &&
+	test_when_finished rm non-repo/git-bugreport-*
+'
+
 test_done

-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v7 01/15] help: move list_config_help to builtin/help
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 02/15] help: add shell-path to --build-options Emily Shaffer
                         ` (14 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Starting in 3ac68a93fd2, help.o began to depend on builtin/branch.o,
builtin/clean.o, and builtin/config.o. This meant that help.o was
unusable outside of the context of the main Git executable.

To make help.o usable by other commands again, move list_config_help()
into builtin/help.c (where it makes sense to assume other builtin libraries
are present).

When command-list.h is included but a member is not used, we start to
hear a compiler warning. Since the config list is generated in a fairly
different way than the command list, and since commands and config
options are semantically different, move the config list into its own
header and move the generator into its own script and build rule.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore             |  1 +
 Makefile               | 13 +++++--
 builtin/help.c         | 86 ++++++++++++++++++++++++++++++++++++++++++
 generate-cmdlist.sh    | 19 ----------
 generate-configlist.sh | 21 +++++++++++
 help.c                 | 85 -----------------------------------------
 help.h                 |  1 -
 7 files changed, 118 insertions(+), 108 deletions(-)
 create mode 100755 generate-configlist.sh

diff --git a/.gitignore b/.gitignore
index aebe7c0908..ea97de83f3 100644
--- a/.gitignore
+++ b/.gitignore
@@ -189,6 +189,7 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
+/config-list.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index 09f98b777c..5a022367d4 100644
--- a/Makefile
+++ b/Makefile
@@ -814,6 +814,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
+GENERATED_H += config-list.h
 GENERATED_H += command-list.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
@@ -2128,7 +2129,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -2148,6 +2149,12 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
+config-list.h: generate-configlist.sh
+
+config-list.h:
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
+		>$@+ && mv $@+ $@
+
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Documentation/config/*.txt
@@ -2781,7 +2788,7 @@ $(SP_OBJ): %.sp: %.c GIT-CFLAGS FORCE
 .PHONY: sparse $(SP_OBJ)
 sparse: $(SP_OBJ)
 
-EXCEPT_HDRS := command-list.h unicode-width.h compat/% xdiff/%
+EXCEPT_HDRS := command-list.h config-list.h unicode-width.h compat/% xdiff/%
 ifndef GCRYPT_SHA256
 	EXCEPT_HDRS += sha256/gcrypt.h
 endif
@@ -2803,7 +2810,7 @@ hdr-check: $(HCO)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: command-list.h
+check: config-list.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
diff --git a/builtin/help.c b/builtin/help.c
index e5590d7787..1c5f2b9255 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -8,6 +8,7 @@
 #include "parse-options.h"
 #include "run-command.h"
 #include "column.h"
+#include "config-list.h"
 #include "help.h"
 #include "alias.h"
 
@@ -62,6 +63,91 @@ static const char * const builtin_help_usage[] = {
 	NULL
 };
 
+struct slot_expansion {
+	const char *prefix;
+	const char *placeholder;
+	void (*fn)(struct string_list *list, const char *prefix);
+	int found;
+};
+
+static void list_config_help(int for_human)
+{
+	struct slot_expansion slot_expansions[] = {
+		{ "advice", "*", list_config_advices },
+		{ "color.branch", "<slot>", list_config_color_branch_slots },
+		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
+		{ "color.diff", "<slot>", list_config_color_diff_slots },
+		{ "color.grep", "<slot>", list_config_color_grep_slots },
+		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
+		{ "color.remote", "<slot>", list_config_color_sideband_slots },
+		{ "color.status", "<slot>", list_config_color_status_slots },
+		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
+		{ NULL, NULL, NULL }
+	};
+	const char **p;
+	struct slot_expansion *e;
+	struct string_list keys = STRING_LIST_INIT_DUP;
+	int i;
+
+	for (p = config_name_list; *p; p++) {
+		const char *var = *p;
+		struct strbuf sb = STRBUF_INIT;
+
+		for (e = slot_expansions; e->prefix; e++) {
+
+			strbuf_reset(&sb);
+			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
+			if (!strcasecmp(var, sb.buf)) {
+				e->fn(&keys, e->prefix);
+				e->found++;
+				break;
+			}
+		}
+		strbuf_release(&sb);
+		if (!e->prefix)
+			string_list_append(&keys, var);
+	}
+
+	for (e = slot_expansions; e->prefix; e++)
+		if (!e->found)
+			BUG("slot_expansion %s.%s is not used",
+			    e->prefix, e->placeholder);
+
+	string_list_sort(&keys);
+	for (i = 0; i < keys.nr; i++) {
+		const char *var = keys.items[i].string;
+		const char *wildcard, *tag, *cut;
+
+		if (for_human) {
+			puts(var);
+			continue;
+		}
+
+		wildcard = strchr(var, '*');
+		tag = strchr(var, '<');
+
+		if (!wildcard && !tag) {
+			puts(var);
+			continue;
+		}
+
+		if (wildcard && !tag)
+			cut = wildcard;
+		else if (!wildcard && tag)
+			cut = tag;
+		else
+			cut = wildcard < tag ? wildcard : tag;
+
+		/*
+		 * We may produce duplicates, but that's up to
+		 * git-completion.bash to handle
+		 */
+		printf("%.*s\n", (int)(cut - var), var);
+	}
+	string_list_clear(&keys, 0);
+}
+
 static enum help_format parse_help_format(const char *format)
 {
 	if (!strcmp(format, "man"))
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 71158f7d8b..45fecf8bdf 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -76,23 +76,6 @@ print_command_list () {
 	echo "};"
 }
 
-print_config_list () {
-	cat <<EOF
-static const char *config_name_list[] = {
-EOF
-	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
-	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
-	sort |
-	while read line
-	do
-		echo "	\"$line\","
-	done
-	cat <<EOF
-	NULL,
-};
-EOF
-}
-
 exclude_programs=
 while test "--exclude-program" = "$1"
 do
@@ -113,5 +96,3 @@ echo
 define_category_names "$1"
 echo
 print_command_list "$1"
-echo
-print_config_list
diff --git a/generate-configlist.sh b/generate-configlist.sh
new file mode 100755
index 0000000000..8692fe5cf4
--- /dev/null
+++ b/generate-configlist.sh
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+echo "/* Automatically generated by generate-configlist.sh */"
+echo
+
+print_config_list () {
+	cat <<EOF
+static const char *config_name_list[] = {
+EOF
+	grep -h '^[a-zA-Z].*\..*::$' Documentation/*config.txt Documentation/config/*.txt |
+	sed '/deprecated/d; s/::$//; s/,  */\n/g' |
+	sort |
+	sed 's/^.*$/	"&",/'
+	cat <<EOF
+	NULL,
+};
+EOF
+}
+
+echo
+print_config_list
diff --git a/help.c b/help.c
index cf67624a94..a21487db77 100644
--- a/help.c
+++ b/help.c
@@ -407,91 +407,6 @@ void list_common_guides_help(void)
 	putchar('\n');
 }
 
-struct slot_expansion {
-	const char *prefix;
-	const char *placeholder;
-	void (*fn)(struct string_list *list, const char *prefix);
-	int found;
-};
-
-void list_config_help(int for_human)
-{
-	struct slot_expansion slot_expansions[] = {
-		{ "advice", "*", list_config_advices },
-		{ "color.branch", "<slot>", list_config_color_branch_slots },
-		{ "color.decorate", "<slot>", list_config_color_decorate_slots },
-		{ "color.diff", "<slot>", list_config_color_diff_slots },
-		{ "color.grep", "<slot>", list_config_color_grep_slots },
-		{ "color.interactive", "<slot>", list_config_color_interactive_slots },
-		{ "color.remote", "<slot>", list_config_color_sideband_slots },
-		{ "color.status", "<slot>", list_config_color_status_slots },
-		{ "fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ "receive.fsck", "<msg-id>", list_config_fsck_msg_ids },
-		{ NULL, NULL, NULL }
-	};
-	const char **p;
-	struct slot_expansion *e;
-	struct string_list keys = STRING_LIST_INIT_DUP;
-	int i;
-
-	for (p = config_name_list; *p; p++) {
-		const char *var = *p;
-		struct strbuf sb = STRBUF_INIT;
-
-		for (e = slot_expansions; e->prefix; e++) {
-
-			strbuf_reset(&sb);
-			strbuf_addf(&sb, "%s.%s", e->prefix, e->placeholder);
-			if (!strcasecmp(var, sb.buf)) {
-				e->fn(&keys, e->prefix);
-				e->found++;
-				break;
-			}
-		}
-		strbuf_release(&sb);
-		if (!e->prefix)
-			string_list_append(&keys, var);
-	}
-
-	for (e = slot_expansions; e->prefix; e++)
-		if (!e->found)
-			BUG("slot_expansion %s.%s is not used",
-			    e->prefix, e->placeholder);
-
-	string_list_sort(&keys);
-	for (i = 0; i < keys.nr; i++) {
-		const char *var = keys.items[i].string;
-		const char *wildcard, *tag, *cut;
-
-		if (for_human) {
-			puts(var);
-			continue;
-		}
-
-		wildcard = strchr(var, '*');
-		tag = strchr(var, '<');
-
-		if (!wildcard && !tag) {
-			puts(var);
-			continue;
-		}
-
-		if (wildcard && !tag)
-			cut = wildcard;
-		else if (!wildcard && tag)
-			cut = tag;
-		else
-			cut = wildcard < tag ? wildcard : tag;
-
-		/*
-		 * We may produce duplicates, but that's up to
-		 * git-completion.bash to handle
-		 */
-		printf("%.*s\n", (int)(cut - var), var);
-	}
-	string_list_clear(&keys, 0);
-}
-
 static int get_alias(const char *var, const char *value, void *data)
 {
 	struct string_list *list = data;
diff --git a/help.h b/help.h
index 7a455beeb7..9071894e8c 100644
--- a/help.h
+++ b/help.h
@@ -22,7 +22,6 @@ static inline void mput_char(char c, unsigned int num)
 void list_common_cmds_help(void);
 void list_all_cmds_help(void);
 void list_common_guides_help(void);
-void list_config_help(int for_human);
 
 void list_all_main_cmds(struct string_list *list);
 void list_all_other_cmds(struct string_list *list);
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v7 02/15] help: add shell-path to --build-options
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 01/15] help: move list_config_help to builtin/help Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 03/15] bugreport: add tool to generate debugging info Emily Shaffer
                         ` (13 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It may be useful to know which shell Git was built to try to point to,
in the event that shell-based Git commands are failing. $SHELL_PATH is
set during the build and used to launch the manpage viewer, as well as
by git-compat-util.h, and it's used during tests. 'git version
--build-options' is encouraged for use in bug reports, so it makes sense
to include this information there.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 help.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/help.c b/help.c
index a21487db77..190722fb0a 100644
--- a/help.c
+++ b/help.c
@@ -654,6 +654,7 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 			printf("no commit associated with this build\n");
 		printf("sizeof-long: %d\n", (int)sizeof(long));
 		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
+		printf("shell-path: %s\n", SHELL_PATH);
 		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
 	}
 	return 0;
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 01/15] help: move list_config_help to builtin/help Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 02/15] help: add shell-path to --build-options Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14 17:25         ` Junio C Hamano
  2020-02-19 14:18         ` Johannes Schindelin
  2020-02-14  1:53       ` [PATCH v7 04/15] bugreport: gather git version and build info Emily Shaffer
                         ` (12 subsequent siblings)
  15 siblings, 2 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach Git how to prompt the user for a good bug report: reproduction
steps, expected behavior, and actual behavior. Later, Git can learn how
to collect some diagnostic information from the repository.

If users can send us a well-written bug report which contains diagnostic
information we would otherwise need to ask the user for, we can reduce
the number of question-and-answer round trips between the reporter and
the Git contributor.

Users may also wish to send a report like this to their local "Git
expert" if they have put their repository into a state they are confused
by.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                      |  1 +
 Documentation/git-bugreport.txt | 41 ++++++++++++++
 Makefile                        |  5 ++
 bugreport.c                     | 94 +++++++++++++++++++++++++++++++++
 command-list.txt                |  1 +
 t/t0091-bugreport.sh            | 55 +++++++++++++++++++
 6 files changed, 197 insertions(+)
 create mode 100644 Documentation/git-bugreport.txt
 create mode 100644 bugreport.c
 create mode 100755 t/t0091-bugreport.sh

diff --git a/.gitignore b/.gitignore
index ea97de83f3..d89bf9e11e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -25,6 +25,7 @@
 /git-bisect--helper
 /git-blame
 /git-branch
+/git-bugreport
 /git-bundle
 /git-cat-file
 /git-check-attr
diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
new file mode 100644
index 0000000000..52d49ed7aa
--- /dev/null
+++ b/Documentation/git-bugreport.txt
@@ -0,0 +1,41 @@
+git-bugreport(1)
+================
+
+NAME
+----
+git-bugreport - Collect information for user to file a bug report
+
+SYNOPSIS
+--------
+[verse]
+'git bugreport' [(-o | --output-directory) <path>] [(-s | --suffix) <format>]
+
+DESCRIPTION
+-----------
+Captures information about the user's machine, Git client, and repository state,
+as well as a form requesting information about the behavior the user observed,
+into a single text file which the user can then share, for example to the Git
+mailing list, in order to report an observed bug.
+
+The following information is requested from the user:
+
+ - Reproduction steps
+ - Expected behavior
+ - Actual behavior
+
+OPTIONS
+-------
+-o <path>::
+--output-directory <path>::
+	Place the resulting bug report file in `<path>` instead of the root of
+	the Git repository.
+
+-s <format>::
+--suffix <format>::
+	Specify an alternate suffix for the bugreport name, to create a file
+	named 'git-bugreport-<formatted suffix>'. This should take the form of a
+	link:strftime[3] format string; the current local time will be used.
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Makefile b/Makefile
index 5a022367d4..a01a050aa3 100644
--- a/Makefile
+++ b/Makefile
@@ -681,6 +681,7 @@ EXTRA_PROGRAMS =
 # ... and all the rest that could be moved out of bindir to gitexecdir
 PROGRAMS += $(EXTRA_PROGRAMS)
 
+PROGRAM_OBJS += bugreport.o
 PROGRAM_OBJS += credential-store.o
 PROGRAM_OBJS += daemon.o
 PROGRAM_OBJS += fast-import.o
@@ -2457,6 +2458,10 @@ endif
 git-%$X: %.o GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
+git-bugreport$X: bugreport.o GIT-LDFLAGS $(GITLIBS)
+	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
+		$(LIBS)
+
 git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
 		$(IMAP_SEND_LDFLAGS) $(LIBS)
diff --git a/bugreport.c b/bugreport.c
new file mode 100644
index 0000000000..a9398e6a2a
--- /dev/null
+++ b/bugreport.c
@@ -0,0 +1,94 @@
+#include "builtin.h"
+#include "parse-options.h"
+#include "stdio.h"
+#include "strbuf.h"
+#include "time.h"
+
+static const char * const bugreport_usage[] = {
+	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
+	NULL
+};
+
+static int get_bug_template(struct strbuf *template)
+{
+	const char template_text[] = N_(
+"Thank you for filling out a Git bug report!\n"
+"Please answer the following questions to help us understand your issue.\n"
+"\n"
+"What did you do before the bug happened? (Steps to reproduce your issue)\n"
+"\n"
+"What did you expect to happen? (Expected behavior)\n"
+"\n"
+"What happened instead? (Actual behavior)\n"
+"\n"
+"What's different between what you expected and what actually happened?\n"
+"\n"
+"Anything else you want to add:\n"
+"\n"
+"Please review the rest of the bug report below.\n"
+"You can delete any lines you don't wish to share.\n");
+
+	strbuf_addstr(template, template_text);
+	return 0;
+}
+
+int cmd_main(int argc, const char **argv)
+{
+	struct strbuf buffer = STRBUF_INIT;
+	struct strbuf report_path = STRBUF_INIT;
+	FILE *report;
+	time_t now = time(NULL);
+	char *option_output = NULL;
+	char *option_suffix = "%F-%H%M";
+	struct stat statbuf;
+
+	const struct option bugreport_options[] = {
+		OPT_STRING('o', "output-directory", &option_output, N_("path"),
+			   N_("specify a destination for the bugreport file")),
+		OPT_STRING('s', "suffix", &option_suffix, N_("format"),
+			   N_("specify a strftime format suffix for the filename")),
+		OPT_END()
+	};
+	argc = parse_options(argc, argv, "", bugreport_options,
+			     bugreport_usage, 0);
+
+	if (option_output) {
+		strbuf_addstr(&report_path, option_output);
+		strbuf_complete(&report_path, '/');
+	}
+
+
+	strbuf_addstr(&report_path, "git-bugreport-");
+	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
+	strbuf_addstr(&report_path, ".txt");
+
+	if (!stat(report_path.buf, &statbuf))
+		die("'%s' already exists", report_path.buf);
+
+	switch (safe_create_leading_directories(report_path.buf)) {
+	case SCLD_OK:
+	case SCLD_EXISTS:
+		break;
+	default:
+		die(_("could not create leading directories for '%s'"),
+		    report_path.buf);
+	}
+
+	get_bug_template(&buffer);
+
+	report = fopen_for_writing(report_path.buf);
+
+	if (report == NULL) {
+		strbuf_release(&report_path);
+		die("couldn't open '%s' for writing", report_path.buf);
+	}
+
+	strbuf_write(&buffer, report);
+	fclose(report);
+
+	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
+
+	UNLEAK(buffer);
+	UNLEAK(report_path);
+	return -launch_editor(report_path.buf, NULL, NULL);
+}
diff --git a/command-list.txt b/command-list.txt
index 2087894655..185e5e3f05 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -54,6 +54,7 @@ git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
 git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
+git-bugreport                           ancillaryinterrogators
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
 git-check-attr                          purehelpers
diff --git a/t/t0091-bugreport.sh b/t/t0091-bugreport.sh
new file mode 100755
index 0000000000..6585a2d144
--- /dev/null
+++ b/t/t0091-bugreport.sh
@@ -0,0 +1,55 @@
+#!/bin/sh
+
+test_description='git bugreport'
+
+. ./test-lib.sh
+
+# Headers "[System Info]" will be followed by a non-empty line if we put some
+# information there; we can make sure all our headers were followed by some
+# information to check if the command was successful.
+HEADER_PATTERN="^\[.*\]$"
+
+check_all_headers_populated () {
+	while read -r line
+	do
+		if test "$(grep "$HEADER_PATTERN" "$line")"
+		then
+			echo "$line"
+			read -r nextline
+			if test -z "$nextline"; then
+				return 1;
+			fi
+		fi
+	done
+}
+
+test_expect_success 'creates a report with content in the right places' '
+	git bugreport -s check-headers &&
+	check_all_headers_populated <git-bugreport-check-headers.txt &&
+	test_when_finished rm git-bugreport-check-headers.txt
+'
+
+test_expect_success 'dies if file with same name as report already exists' '
+	>>git-bugreport-duplicate.txt &&
+	test_must_fail git bugreport --suffix duplicate &&
+	test_when_finished rm git-bugreport-duplicate.txt
+'
+
+test_expect_success '--output-directory puts the report in the provided dir' '
+	git bugreport -o foo/ &&
+	test_path_is_file foo/git-bugreport-* &&
+	test_when_finished rm -fr foo/
+'
+
+test_expect_success 'incorrect arguments abort with usage' '
+	test_must_fail git bugreport --false 2>output &&
+	test_i18ngrep usage output &&
+	test_path_is_missing git-bugreport-*
+'
+
+test_expect_success 'runs outside of a git dir' '
+	nongit git bugreport &&
+	test_when_finished rm non-repo/git-bugreport-*
+'
+
+test_done
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v7 04/15] bugreport: gather git version and build info
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (2 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 03/15] bugreport: add tool to generate debugging info Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 05/15] bugreport: add uname info Emily Shaffer
                         ` (11 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Knowing which version of Git a user has and how it was built allows us
to more precisely pin down the circumstances when a certain issue
occurs, so teach bugreport how to tell us the same output as 'git
version --build-options'.

It's not ideal to directly call 'git version --build-options' because
that output goes to stdout. Instead, wrap the version string in a helper
within help.[ch] library, and call that helper from within the bugreport
library.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  4 +++
 bugreport.c                     | 19 ++++++++++++-
 help.c                          | 48 ++++++++++++++++++++-------------
 help.h                          |  1 +
 4 files changed, 52 insertions(+), 20 deletions(-)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 52d49ed7aa..6ce2ae2eff 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -23,6 +23,10 @@ The following information is requested from the user:
  - Expected behavior
  - Actual behavior
 
+The following information is captured automatically:
+
+ - 'git version --build-options'
+
 OPTIONS
 -------
 -o <path>::
diff --git a/bugreport.c b/bugreport.c
index a9398e6a2a..b7504fe322 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -1,8 +1,17 @@
-#include "builtin.h"
+#include "cache.h"
 #include "parse-options.h"
 #include "stdio.h"
 #include "strbuf.h"
 #include "time.h"
+#include "help.h"
+
+static void get_system_info(struct strbuf *sys_info)
+{
+	/* get git version from native cmd */
+	strbuf_addstr(sys_info, "git version:\n");
+	get_version_info(sys_info, 1);
+	strbuf_complete_line(sys_info);
+}
 
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
@@ -32,6 +41,11 @@ static int get_bug_template(struct strbuf *template)
 	return 0;
 }
 
+static void get_header(struct strbuf *buf, const char *title)
+{
+	strbuf_addf(buf, "\n\n[%s]\n", title);
+}
+
 int cmd_main(int argc, const char **argv)
 {
 	struct strbuf buffer = STRBUF_INIT;
@@ -76,6 +90,9 @@ int cmd_main(int argc, const char **argv)
 
 	get_bug_template(&buffer);
 
+	get_header(&buffer, "System Info");
+	get_system_info(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
diff --git a/help.c b/help.c
index 190722fb0a..44cee69c11 100644
--- a/help.c
+++ b/help.c
@@ -622,8 +622,33 @@ const char *help_unknown_cmd(const char *cmd)
 	exit(1);
 }
 
+void get_version_info(struct strbuf *buf, int show_build_options)
+{
+	/*
+	 * The format of this string should be kept stable for compatibility
+	 * with external projects that rely on the output of "git version".
+	 *
+	 * Always show the version, even if other options are given.
+	 */
+	strbuf_addf(buf, "git version %s\n", git_version_string);
+
+	if (show_build_options) {
+		strbuf_addf(buf, "cpu: %s\n", GIT_HOST_CPU);
+		if (git_built_from_commit_string[0])
+			strbuf_addf(buf, "built from commit: %s\n",
+			       git_built_from_commit_string);
+		else
+			strbuf_addstr(buf, "no commit associated with this build\n");
+		strbuf_addf(buf, "sizeof-long: %d\n", (int)sizeof(long));
+		strbuf_addf(buf, "sizeof-size_t: %d\n", (int)sizeof(size_t));
+		strbuf_addf(buf, "shell-path: %s\n", SHELL_PATH);
+		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
+	}
+}
+
 int cmd_version(int argc, const char **argv, const char *prefix)
 {
+	struct strbuf buf = STRBUF_INIT;
 	int build_options = 0;
 	const char * const usage[] = {
 		N_("git version [<options>]"),
@@ -637,26 +662,11 @@ int cmd_version(int argc, const char **argv, const char *prefix)
 
 	argc = parse_options(argc, argv, prefix, options, usage, 0);
 
-	/*
-	 * The format of this string should be kept stable for compatibility
-	 * with external projects that rely on the output of "git version".
-	 *
-	 * Always show the version, even if other options are given.
-	 */
-	printf("git version %s\n", git_version_string);
+	get_version_info(&buf, build_options);
+	printf("%s", buf.buf);
+
+	strbuf_release(&buf);
 
-	if (build_options) {
-		printf("cpu: %s\n", GIT_HOST_CPU);
-		if (git_built_from_commit_string[0])
-			printf("built from commit: %s\n",
-			       git_built_from_commit_string);
-		else
-			printf("no commit associated with this build\n");
-		printf("sizeof-long: %d\n", (int)sizeof(long));
-		printf("sizeof-size_t: %d\n", (int)sizeof(size_t));
-		printf("shell-path: %s\n", SHELL_PATH);
-		/* NEEDSWORK: also save and output GIT-BUILD_OPTIONS? */
-	}
 	return 0;
 }
 
diff --git a/help.h b/help.h
index 9071894e8c..500521b908 100644
--- a/help.h
+++ b/help.h
@@ -37,6 +37,7 @@ void add_cmdname(struct cmdnames *cmds, const char *name, int len);
 void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes);
 int is_in_cmdlist(struct cmdnames *cmds, const char *name);
 void list_commands(unsigned int colopts, struct cmdnames *main_cmds, struct cmdnames *other_cmds);
+void get_version_info(struct strbuf *buf, int show_build_options);
 
 /*
  * call this to die(), when it is suspected that the user mistyped a
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v7 05/15] bugreport: add uname info
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (3 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 04/15] bugreport: gather git version and build info Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 06/15] bugreport: add compiler info Emily Shaffer
                         ` (10 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

The contents of uname() can give us some insight into what sort of
system the user is running on, and help us replicate their setup if need
be. The domainname field is not guaranteed to be available, so don't
collect it.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 13 +++++++++++++
 2 files changed, 14 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 6ce2ae2eff..4dd72c60f5 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -26,6 +26,7 @@ The following information is requested from the user:
 The following information is captured automatically:
 
  - 'git version --build-options'
+ - uname sysname, release, version, and machine strings
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index b7504fe322..b76a1dfb2a 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -7,10 +7,23 @@
 
 static void get_system_info(struct strbuf *sys_info)
 {
+	struct utsname uname_info;
+
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
 	get_version_info(sys_info, 1);
 	strbuf_complete_line(sys_info);
+
+	/* system call for other version info */
+	strbuf_addstr(sys_info, "uname -a: ");
+	if (uname(&uname_info))
+		strbuf_addf(sys_info, "uname() failed with code %d\n", errno);
+	else
+		strbuf_addf(sys_info, "%s %s %s %s\n",
+			    uname_info.sysname,
+			    uname_info.release,
+			    uname_info.version,
+			    uname_info.machine);
 }
 
 static const char * const bugreport_usage[] = {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v7 06/15] bugreport: add compiler info
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (4 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 05/15] bugreport: add uname info Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-19 14:23         ` Johannes Schindelin
  2020-02-14  1:53       ` [PATCH v7 07/15] bugreport: add git-remote-https version Emily Shaffer
                         ` (9 subsequent siblings)
  15 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

To help pinpoint the source of a regression, it is useful to know some
info about the compiler which the user's Git client was built with. By
adding a generic get_compiler_info() in 'compat/' we can choose which
relevant information to share per compiler; to get started, let's
demonstrate the version of glibc if the user built with 'gcc'.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     |  5 +++++
 compat/compiler.h               | 24 ++++++++++++++++++++++++
 3 files changed, 30 insertions(+)
 create mode 100644 compat/compiler.h

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 4dd72c60f5..8bbc4c960c 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -27,6 +27,7 @@ The following information is captured automatically:
 
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
+ - Compiler-specific info string
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index b76a1dfb2a..4f9101caeb 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -4,6 +4,7 @@
 #include "strbuf.h"
 #include "time.h"
 #include "help.h"
+#include "compat/compiler.h"
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -24,6 +25,10 @@ static void get_system_info(struct strbuf *sys_info)
 			    uname_info.release,
 			    uname_info.version,
 			    uname_info.machine);
+
+	strbuf_addstr(sys_info, "compiler info: ");
+	get_compiler_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/compat/compiler.h b/compat/compiler.h
new file mode 100644
index 0000000000..bda5098e1b
--- /dev/null
+++ b/compat/compiler.h
@@ -0,0 +1,24 @@
+#ifndef COMPILER_H
+#define COMPILER_H
+
+#include "git-compat-util.h"
+#include "strbuf.h"
+
+#ifdef __GLIBC__
+#include <gnu/libc-version.h>
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
+}
+
+#else
+
+static inline void get_compiler_info(struct strbuf *info)
+{
+	strbuf_addstr(info, "get_compiler_info() not implemented");
+}
+
+#endif
+
+#endif /* COMPILER_H */
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v7 07/15] bugreport: add git-remote-https version
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (5 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 06/15] bugreport: add compiler info Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-19 14:28         ` Johannes Schindelin
  2020-02-14  1:53       ` [PATCH v7 08/15] bugreport: include user interactive shell Emily Shaffer
                         ` (8 subsequent siblings)
  15 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It's possible for git-remote-curl to be built separately from git; in that
case we want to know what version of cURL is used by git-remote-curl, not
necessarily which version was present at git-bugreport's build time.
So instead, ask git-remote-curl for the version information it knows
about.

Today, "git-remote-http" and "git-remote-https" are aliased to
"git-remote-curl"; but in case we rely on a different library than cURL
in the future, let's not explicitly reference cURL from bugreport.

For longevity purposes, invoke the alias "git-remote-https" instead of
"git-remote-http".

Since it could have been built at a different time, also report the
version and built-from commit of git-remote-curl alongside the cURL info.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 16 ++++++++++++++++
 remote-curl.c                   |  8 ++++++++
 3 files changed, 25 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 8bbc4c960c..33df4dec7f 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -28,6 +28,7 @@ The following information is captured automatically:
  - 'git version --build-options'
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
+ - 'git remote-https --build-info'
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 4f9101caeb..bfdff33368 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -5,6 +5,18 @@
 #include "time.h"
 #include "help.h"
 #include "compat/compiler.h"
+#include "run-command.h"
+
+static void get_git_remote_https_version_info(struct strbuf *version_info)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_push(&cp.args, "git");
+	argv_array_push(&cp.args, "remote-https");
+	argv_array_push(&cp.args, "--build-info");
+	if (capture_command(&cp, version_info, 0))
+	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
+}
 
 static void get_system_info(struct strbuf *sys_info)
 {
@@ -29,6 +41,10 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_addstr(sys_info, "compiler info: ");
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
+
+	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
+	get_git_remote_https_version_info(sys_info);
+	strbuf_complete_line(sys_info);
 }
 
 static const char * const bugreport_usage[] = {
diff --git a/remote-curl.c b/remote-curl.c
index 350d92a074..c590fbfae3 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -17,6 +17,7 @@
 #include "protocol.h"
 #include "quote.h"
 #include "transport.h"
+#include "version.h"
 
 static struct remote *remote;
 /* always ends with a trailing slash */
@@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
 	string_list_init(&options.deepen_not, 1);
 	string_list_init(&options.push_options, 1);
 
+	if (!strcmp("--build-info", argv[1])) {
+		printf("git-http-fetch version: %s\n", git_version_string);
+		printf("built from commit: %s\n", git_built_from_commit_string);
+		printf("curl version: %s\n", curl_version());
+		return 0;
+	}
+
 	/*
 	 * Just report "remote-curl" here (folding all the various aliases
 	 * ("git-remote-http", "git-remote-https", and etc.) here since they
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v7 08/15] bugreport: include user interactive shell
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (6 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 07/15] bugreport: add git-remote-https version Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 09/15] bugreport: generate config safelist based on docs Emily Shaffer
                         ` (7 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

It's possible a user may complain about the way that Git interacts with
their interactive shell, e.g. autocompletion or shell prompt. In that
case, it's useful for us to know which shell they're using
interactively.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt | 1 +
 bugreport.c                     | 5 +++++
 2 files changed, 6 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 33df4dec7f..23265b0d74 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -29,6 +29,7 @@ The following information is captured automatically:
  - uname sysname, release, version, and machine strings
  - Compiler-specific info string
  - 'git remote-https --build-info'
+ - $SHELL
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index bfdff33368..c1475601bf 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -21,6 +21,7 @@ static void get_git_remote_https_version_info(struct strbuf *version_info)
 static void get_system_info(struct strbuf *sys_info)
 {
 	struct utsname uname_info;
+	char *shell = NULL;
 
 	/* get git version from native cmd */
 	strbuf_addstr(sys_info, "git version:\n");
@@ -42,6 +43,10 @@ static void get_system_info(struct strbuf *sys_info)
 	get_compiler_info(sys_info);
 	strbuf_complete_line(sys_info);
 
+	shell = getenv("SHELL");
+	strbuf_addf(sys_info, "$SHELL (typically, interactive shell): %s\n",
+		    shell ? shell : "<unset>");
+
 	strbuf_addstr(sys_info, "git-remote-https --build-info:\n");
 	get_git_remote_https_version_info(sys_info);
 	strbuf_complete_line(sys_info);
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v7 09/15] bugreport: generate config safelist based on docs
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (7 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 08/15] bugreport: include user interactive shell Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 10/15] bugreport: add config values from safelist Emily Shaffer
                         ` (6 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Martin Ågren, Johannes Schindelin

Add a new step to the build to generate a safelist of git-config
variables which are appropriate to include in the output of
git-bugreport. New variables can be added to the safelist by annotating
their documentation in Documentation/config with the "annotate" macro,
which is a no-op in AsciiDoc and AsciiDoctor.

Some configs are private in nature, and can contain remote URLs,
passwords, or other sensitive information. In the event that a user
doesn't notice their information while reviewing a bugreport, that user
may leak their credentials to other individuals, mailing lists, or bug
tracking tools inadvertently. Heuristic blocklisting of configuration
keys is imperfect and prone to false negatives; given the nature of the
information which can be leaked, a safelist is more reliable.

However, it's possible that in some situations, an organization may be
less concerned with privacy of things like remote URLs and branch names,
and more concerned with ease of diagnosis for their support staff. In
those cases, it may make more sense for that organization to modify the
code to use a blocklist. To that end, we should try to mark configs which
are definitely safe, and configs which are definitely unsafe, and leave
blank configs which are somewhere in between. To mark a config as safe,
add "annotate:bugreport[include]" to the corresponding line in the
config documentation; to mark it as unsafe, add
"annotate:bugreport[exclude]" instead.

Generating bugreport-config-safelist.h at build time by grepping the
documentation for this new macro helps us prevent staleness. The macro
itself is a no-op and should not alter the appearance of the
documentation in either AsciiDoc or AsciiDoctor, confirmable by running:

  cd Documentation
  ./doc-diff --asciidoctor HEAD^ HEAD
  ./doc-diff --asciidoc HEAD^ HEAD

Diffing the rendered HTML shows that only inline comments were added,
which shouldn't be a problem.

Additionally, add annotations to the sendemail config documentation in
order to demonstrate a proof of concept.

Helped-by: Martin Ågren <martin.agren@gmail.com>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 .gitignore                              |  1 +
 Documentation/asciidoc.conf             |  9 ++++
 Documentation/asciidoctor-extensions.rb |  7 ++++
 Documentation/config/sendemail.txt      | 56 ++++++++++++-------------
 Makefile                                |  7 ++++
 generate-bugreport-config-safelist.sh   | 18 ++++++++
 6 files changed, 70 insertions(+), 28 deletions(-)
 create mode 100755 generate-bugreport-config-safelist.sh

diff --git a/.gitignore b/.gitignore
index d89bf9e11e..bd2f49b996 100644
--- a/.gitignore
+++ b/.gitignore
@@ -192,6 +192,7 @@
 /gitweb/static/gitweb.min.*
 /config-list.h
 /command-list.h
+/bugreport-config-safelist.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Documentation/asciidoc.conf b/Documentation/asciidoc.conf
index 8fc4b67081..663e06481f 100644
--- a/Documentation/asciidoc.conf
+++ b/Documentation/asciidoc.conf
@@ -6,9 +6,14 @@
 #
 # Show Git link as: <command>(<section>); if section is defined, else just show
 # the command.
+#
+# The annotate macro does nothing as far as rendering is
+# concerned -- we just grep for it in the sources to populate
+# things like the bugreport safelist.
 
 [macros]
 (?su)[\\]?(?P<name>linkgit):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
+(?su)[\\]?(?P<name>annotate):(?P<target>\S*?)\[(?P<attrlist>.*?)\]=
 
 [attributes]
 asterisk=&#42;
@@ -28,6 +33,8 @@ ifdef::backend-docbook[]
 {0#<citerefentry>}
 {0#<refentrytitle>{target}</refentrytitle><manvolnum>{0}</manvolnum>}
 {0#</citerefentry>}
+[annotate-inlinemacro]
+{0#}
 endif::backend-docbook[]
 
 ifdef::backend-docbook[]
@@ -94,4 +101,6 @@ ifdef::backend-xhtml11[]
 git-relative-html-prefix=
 [linkgit-inlinemacro]
 <a href="{git-relative-html-prefix}{target}.html">{target}{0?({0})}</a>
+[annotate-inlinemacro]
+<!-- -->
 endif::backend-xhtml11[]
diff --git a/Documentation/asciidoctor-extensions.rb b/Documentation/asciidoctor-extensions.rb
index d906a00803..03c80af0e5 100644
--- a/Documentation/asciidoctor-extensions.rb
+++ b/Documentation/asciidoctor-extensions.rb
@@ -39,10 +39,17 @@ module Git
         output
       end
     end
+
+    class AnnotateProcessor < Asciidoctor::Extensions::InlineMacroProcessor
+      def process(parent, target, attrs)
+        ""
+      end
+    end
   end
 end
 
 Asciidoctor::Extensions.register do
   inline_macro Git::Documentation::LinkGitProcessor, :linkgit
   postprocessor Git::Documentation::DocumentPostProcessor
+  inline_macro Git::Documentation::AnnotateProcessor, :annotate
 end
diff --git a/Documentation/config/sendemail.txt b/Documentation/config/sendemail.txt
index 0006faf800..fe27473e44 100644
--- a/Documentation/config/sendemail.txt
+++ b/Documentation/config/sendemail.txt
@@ -4,7 +4,7 @@ sendemail.identity::
 	values in the 'sendemail' section. The default identity is
 	the value of `sendemail.identity`.
 
-sendemail.smtpEncryption::
+sendemail.smtpEncryption annotate:bugreport[include] ::
 	See linkgit:git-send-email[1] for description.  Note that this
 	setting is not subject to the 'identity' mechanism.
 
@@ -15,7 +15,7 @@ sendemail.smtpsslcertpath::
 	Path to ca-certificates (either a directory or a single file).
 	Set it to an empty string to disable certificate verification.
 
-sendemail.<identity>.*::
+sendemail.<identity>.* annotate:bugreport[exclude] ::
 	Identity-specific versions of the 'sendemail.*' parameters
 	found below, taking precedence over those when this
 	identity is selected, through either the command-line or
@@ -23,41 +23,41 @@ sendemail.<identity>.*::
 
 sendemail.aliasesFile::
 sendemail.aliasFileType::
-sendemail.annotate::
-sendemail.bcc::
-sendemail.cc::
-sendemail.ccCmd::
-sendemail.chainReplyTo::
-sendemail.confirm::
-sendemail.envelopeSender::
-sendemail.from::
-sendemail.multiEdit::
-sendemail.signedoffbycc::
-sendemail.smtpPass::
-sendemail.suppresscc::
-sendemail.suppressFrom::
-sendemail.to::
-sendemail.tocmd::
-sendemail.smtpDomain::
-sendemail.smtpServer::
-sendemail.smtpServerPort::
-sendemail.smtpServerOption::
-sendemail.smtpUser::
-sendemail.thread::
-sendemail.transferEncoding::
-sendemail.validate::
-sendemail.xmailer::
+sendemail.annotate annotate:bugreport[include] ::
+sendemail.bcc annotate:bugreport[include] ::
+sendemail.cc annotate:bugreport[include] ::
+sendemail.ccCmd annotate:bugreport[include] ::
+sendemail.chainReplyTo annotate:bugreport[include] ::
+sendemail.confirm annotate:bugreport[include] ::
+sendemail.envelopeSender annotate:bugreport[include] ::
+sendemail.from annotate:bugreport[include] ::
+sendemail.multiEdit annotate:bugreport[include] ::
+sendemail.signedoffbycc annotate:bugreport[include] ::
+sendemail.smtpPass annotate:bugreport[exclude] ::
+sendemail.suppresscc annotate:bugreport[include] ::
+sendemail.suppressFrom annotate:bugreport[include] ::
+sendemail.to annotate:bugreport[include] ::
+sendemail.tocmd annotate:bugreport[include] ::
+sendemail.smtpDomain annotate:bugreport[include] ::
+sendemail.smtpServer annotate:bugreport[include] ::
+sendemail.smtpServerPort annotate:bugreport[include] ::
+sendemail.smtpServerOption annotate:bugreport[include] ::
+sendemail.smtpUser annotate:bugreport[exclude] ::
+sendemail.thread annotate:bugreport[include] ::
+sendemail.transferEncoding annotate:bugreport[include] ::
+sendemail.validate annotate:bugreport[include] ::
+sendemail.xmailer annotate:bugreport[include] ::
 	See linkgit:git-send-email[1] for description.
 
 sendemail.signedoffcc (deprecated)::
 	Deprecated alias for `sendemail.signedoffbycc`.
 
-sendemail.smtpBatchSize::
+sendemail.smtpBatchSize annotate:bugreport[include] ::
 	Number of messages to be sent per connection, after that a relogin
 	will happen.  If the value is 0 or undefined, send all messages in
 	one connection.
 	See also the `--batch-size` option of linkgit:git-send-email[1].
 
-sendemail.smtpReloginDelay::
+sendemail.smtpReloginDelay annotate:bugreport[include] ::
 	Seconds wait before reconnecting to smtp server.
 	See also the `--relogin-delay` option of linkgit:git-send-email[1].
diff --git a/Makefile b/Makefile
index a01a050aa3..2bc9f112ea 100644
--- a/Makefile
+++ b/Makefile
@@ -817,6 +817,7 @@ VCSSVN_LIB = vcs-svn/lib.a
 
 GENERATED_H += config-list.h
 GENERATED_H += command-list.h
+GENERATED_H += bugreport-config-safelist.h
 
 LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
 	$(FIND) . \
@@ -2163,6 +2164,12 @@ command-list.h: $(wildcard Documentation/git*.txt) Documentation/*config.txt Doc
 		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \
 		command-list.txt >$@+ && mv $@+ $@
 
+bugreport-config-safelist.h: generate-bugreport-config-safelist.sh
+
+bugreport-config-safelist.h: Documentation/config/*.txt
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-bugreport-config-safelist.sh \
+		>$@+ && mv $@+ $@
+
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
 	$(gitwebdir_SQ):$(PERL_PATH_SQ):$(SANE_TEXT_GREP):$(PAGER_ENV):\
diff --git a/generate-bugreport-config-safelist.sh b/generate-bugreport-config-safelist.sh
new file mode 100755
index 0000000000..2794a8ae77
--- /dev/null
+++ b/generate-bugreport-config-safelist.sh
@@ -0,0 +1,18 @@
+#!/bin/sh
+
+cat <<EOF
+/* Automatically generated by bugreport-generate-config-safelist.sh */
+
+
+static const char *bugreport_config_safelist[] = {
+EOF
+
+# cat all regular files in Documentation/config
+find Documentation/config -type f -exec cat {} \; |
+# print the command name which matches the annotate-bugreport macro
+sed -n 's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/  "\1",/p' \
+	| sort
+
+cat <<EOF
+};
+EOF
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v7 10/15] bugreport: add config values from safelist
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (8 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 09/15] bugreport: generate config safelist based on docs Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 11/15] bugreport: collect list of populated hooks Emily Shaffer
                         ` (5 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Teach bugreport to gather the values of config options which are present
in 'bugreport-config-safelist.h', and show their origin scope.

Many config options are sensitive, and many Git add-ons use config
options which git-core does not know about; it is better only to gather
config options which we know to be safe, rather than excluding options
which we know to be unsafe.

Rather than including the path to someone's config, which can reveal
filesystem layout and project names, just name the scope (e.g. system,
global, local) of the config source.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 Makefile                        |  2 ++
 bugreport.c                     | 34 +++++++++++++++++++++++++++++++++
 3 files changed, 37 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 23265b0d74..4e9171d1bd 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -30,6 +30,7 @@ The following information is captured automatically:
  - Compiler-specific info string
  - 'git remote-https --build-info'
  - $SHELL
+ - Selected config values
 
 OPTIONS
 -------
diff --git a/Makefile b/Makefile
index 2bc9f112ea..7b139230ae 100644
--- a/Makefile
+++ b/Makefile
@@ -2131,6 +2131,8 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 
 help.sp help.s help.o: command-list.h
 
+bugreport.sp bugreport.s bugreport.o: bugreport-config-safelist.h
+
 builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
diff --git a/bugreport.c b/bugreport.c
index c1475601bf..1a9430a857 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -6,6 +6,9 @@
 #include "help.h"
 #include "compat/compiler.h"
 #include "run-command.h"
+#include "config.h"
+#include "bugreport-config-safelist.h"
+#include "khash.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -52,6 +55,29 @@ static void get_system_info(struct strbuf *sys_info)
 	strbuf_complete_line(sys_info);
 }
 
+static void get_safelisted_config(struct strbuf *config_info)
+{
+	size_t idx;
+	struct string_list_item *it = NULL;
+	struct key_value_info *kv_info = NULL;
+
+	for (idx = 0; idx < ARRAY_SIZE(bugreport_config_safelist); idx++) {
+		const struct string_list *list =
+			git_config_get_value_multi(bugreport_config_safelist[idx]);
+
+		if (!list)
+			continue;
+
+		strbuf_addf(config_info, "%s:\n", bugreport_config_safelist[idx]);
+		for_each_string_list_item(it, list) {
+			kv_info = it->util;
+			strbuf_addf(config_info, "  %s (%s)\n", it->string,
+				    kv_info ? config_scope_name(kv_info->scope)
+					    : "source unknown");
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -94,6 +120,7 @@ int cmd_main(int argc, const char **argv)
 	char *option_output = NULL;
 	char *option_suffix = "%F-%H%M";
 	struct stat statbuf;
+	int nongit_ok = 0;
 
 	const struct option bugreport_options[] = {
 		OPT_STRING('o', "output-directory", &option_output, N_("path"),
@@ -102,6 +129,10 @@ int cmd_main(int argc, const char **argv)
 			   N_("specify a strftime format suffix for the filename")),
 		OPT_END()
 	};
+
+	/* Prerequisite for hooks and config checks */
+	setup_git_directory_gently(&nongit_ok);
+
 	argc = parse_options(argc, argv, "", bugreport_options,
 			     bugreport_usage, 0);
 
@@ -132,6 +163,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "System Info");
 	get_system_info(&buffer);
 
+	get_header(&buffer, "Safelisted Config Info");
+	get_safelisted_config(&buffer);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v7 11/15] bugreport: collect list of populated hooks
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (9 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 10/15] bugreport: add config values from safelist Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 12/15] bugreport: count loose objects Emily Shaffer
                         ` (4 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Occasionally a failure a user is seeing may be related to a specific
hook which is being run, perhaps without the user realizing. While the
contents of hooks can be sensitive - containing user data or process
information specific to the user's organization - simply knowing that a
hook is being run at a certain stage can help us to understand whether
something is going wrong.

Without a definitive list of hook names within the code, we compile our
own list from the documentation. This is likely prone to bitrot. To
reduce the amount of code humans need to read, we turn the list into a
string_list and iterate over it (as we are calling the same find_hook
operation on each string). However, since bugreport should primarily be
called by the user, the performance loss from massaging the string
seems acceptable.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 53 +++++++++++++++++++++++++++++++++
 2 files changed, 54 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 4e9171d1bd..779c9c7fd9 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -31,6 +31,7 @@ The following information is captured automatically:
  - 'git remote-https --build-info'
  - $SHELL
  - Selected config values
+ - A list of enabled hooks
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 1a9430a857..7e6147a935 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -9,6 +9,7 @@
 #include "config.h"
 #include "bugreport-config-safelist.h"
 #include "khash.h"
+#include "run-command.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -78,6 +79,55 @@ static void get_safelisted_config(struct strbuf *config_info)
 	}
 }
 
+static void get_populated_hooks(struct strbuf *hook_info, int nongit)
+{
+	/*
+	 * Doesn't look like there is a list of all possible hooks; so below is
+	 * a transcription of `git help hooks`.
+	 */
+	const char *hooks = "applypatch-msg,"
+			    "pre-applypatch,"
+			    "post-applypatch,"
+			    "pre-commit,"
+			    "pre-merge-commit,"
+			    "prepare-commit-msg,"
+			    "commit-msg,"
+			    "post-commit,"
+			    "pre-rebase,"
+			    "post-checkout,"
+			    "post-merge,"
+			    "pre-push,"
+			    "pre-receive,"
+			    "update,"
+			    "post-receive,"
+			    "post-update,"
+			    "push-to-checkout,"
+			    "pre-auto-gc,"
+			    "post-rewrite,"
+			    "sendemail-validate,"
+			    "fsmonitor-watchman,"
+			    "p4-pre-submit,"
+			    "post-index-changex";
+	struct string_list hooks_list = STRING_LIST_INIT_DUP;
+	struct string_list_item *iter = NULL;
+
+
+	if (nongit) {
+		strbuf_addstr(hook_info,
+			"not run from a git repository - no hooks to show\n");
+		return;
+	}
+
+	string_list_split(&hooks_list, hooks, ',', -1);
+
+	for_each_string_list_item(iter, &hooks_list) {
+		if (find_hook(iter->string)) {
+			strbuf_addstr(hook_info, iter->string);
+			strbuf_complete_line(hook_info);
+		}
+	}
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -166,6 +216,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Safelisted Config Info");
 	get_safelisted_config(&buffer);
 
+	get_header(&buffer, "Enabled Hooks");
+	get_populated_hooks(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v7 12/15] bugreport: count loose objects
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (10 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 11/15] bugreport: collect list of populated hooks Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 13/15] bugreport: add packed object summary Emily Shaffer
                         ` (3 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer, Johannes Schindelin

The number of unpacked objects in a user's repository may help us
understand the root of the problem they're seeing, especially if a
command is running unusually slowly.

Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 52 +++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 779c9c7fd9..7e1ed44cb4 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -32,6 +32,7 @@ The following information is captured automatically:
  - $SHELL
  - Selected config values
  - A list of enabled hooks
+ - The number of loose objects in the repository
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 7e6147a935..f5201ce416 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -10,6 +10,7 @@
 #include "bugreport-config-safelist.h"
 #include "khash.h"
 #include "run-command.h"
+#include "object-store.h"
 
 static void get_git_remote_https_version_info(struct strbuf *version_info)
 {
@@ -128,6 +129,54 @@ static void get_populated_hooks(struct strbuf *hook_info, int nongit)
 	}
 }
 
+static int loose_object_cb(const struct object_id *oid, const char *path,
+			   void *data) {
+	int *loose_object_count = data;
+
+	if (loose_object_count) {
+		(*loose_object_count)++;
+		return 0;
+	}
+
+	return 1;
+}
+
+static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
+
+	int local_loose_object_count = 0, total_loose_object_count = 0;
+	int local_count_questionable = 0, total_count_questionable = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	local_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&local_loose_object_count,
+					FOR_EACH_OBJECT_LOCAL_ONLY);
+
+	total_count_questionable = for_each_loose_object(
+					loose_object_cb,
+					&total_loose_object_count,
+					0);
+
+	strbuf_addf(obj_info, "%d local loose objects%s\n",
+		    local_loose_object_count,
+		    local_count_questionable ? " (problem during count)" : "");
+
+	strbuf_addf(obj_info, "%d alternate loose objects%s\n",
+		    total_loose_object_count - local_loose_object_count,
+		    (local_count_questionable || total_count_questionable)
+			? " (problem during count)"
+			: "");
+
+	strbuf_addf(obj_info, "%d total loose objects%s\n",
+		    total_loose_object_count,
+		    total_count_questionable ? " (problem during count)" : "");
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -219,6 +268,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Enabled Hooks");
 	get_populated_hooks(&buffer, nongit_ok);
 
+	get_header(&buffer, "Loose Object Counts");
+	get_loose_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v7 13/15] bugreport: add packed object summary
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (11 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 12/15] bugreport: count loose objects Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14  1:53       ` [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
                         ` (2 subsequent siblings)
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Alongside the loose object counts, it can be useful to show the number
of packs and packed objects. This way we can check whether the repo has
an appropriate ratio of packed to loose objects to help determine
whether it's behaving correctly.

Add a utility to easily traverse all packfiles in a given repository.
Use it in packfile.c and remove a redundant call to
prepare_packed_git(), which is already called in get_all_packs().

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 30 ++++++++++++++++++++++++++++++
 object-store.h                  |  6 ++++++
 packfile.c                      |  3 +--
 4 files changed, 38 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 7e1ed44cb4..309b34fc42 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -33,6 +33,7 @@ The following information is captured automatically:
  - Selected config values
  - A list of enabled hooks
  - The number of loose objects in the repository
+ - The number of packs and packed objects in the repository
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index f5201ce416..580de13ddc 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -177,6 +177,33 @@ static void get_loose_object_summary(struct strbuf *obj_info, int nongit) {
 		    total_count_questionable ? " (problem during count)" : "");
 }
 
+static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
+{
+	struct packed_git *pack = NULL;
+	int pack_count = 0;
+	int object_count = 0;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - no objects to show\n");
+		return;
+	}
+
+	for_each_pack(the_repository, pack) {
+		pack_count++;
+		/*
+		 * To accurately count how many objects are packed, look inside
+		 * the packfile's index.
+		 */
+		open_pack_index(pack);
+		object_count += pack->num_objects;
+	}
+
+	strbuf_addf(obj_info, "%d total packs (%d objects)\n", pack_count,
+		    object_count);
+
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -271,6 +298,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Loose Object Counts");
 	get_loose_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Packed Object Summary");
+	get_packed_object_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
diff --git a/object-store.h b/object-store.h
index 55ee639350..518c73469d 100644
--- a/object-store.h
+++ b/object-store.h
@@ -6,6 +6,7 @@
 #include "list.h"
 #include "sha1-array.h"
 #include "strbuf.h"
+#include "packfile.h"
 
 struct object_directory {
 	struct object_directory *next;
@@ -403,4 +404,9 @@ int for_each_object_in_pack(struct packed_git *p,
 int for_each_packed_object(each_packed_object_fn, void *,
 			   enum for_each_object_flags flags);
 
+#define for_each_pack(repo, pack) 		\
+		for (pack = get_all_packs(repo);\
+		     pack;			\
+		     pack = pack->next)
+
 #endif /* OBJECT_STORE_H */
diff --git a/packfile.c b/packfile.c
index 7e7c04e4d8..6873827fb5 100644
--- a/packfile.c
+++ b/packfile.c
@@ -2061,8 +2061,7 @@ int for_each_packed_object(each_packed_object_fn cb, void *data,
 	int r = 0;
 	int pack_errors = 0;
 
-	prepare_packed_git(the_repository);
-	for (p = get_all_packs(the_repository); p; p = p->next) {
+	for_each_pack(the_repository, p) {
 		if ((flags & FOR_EACH_OBJECT_LOCAL_ONLY) && !p->pack_local)
 			continue;
 		if ((flags & FOR_EACH_OBJECT_PROMISOR_ONLY) &&
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (12 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 13/15] bugreport: add packed object summary Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14 17:04         ` Junio C Hamano
  2020-02-14  1:53       ` [PATCH v7 15/15] bugreport: summarize contents of alternates file Emily Shaffer
  2020-02-14 17:32       ` [PATCH v7 00/15] add git-bugreport tool Junio C Hamano
  15 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

Miscellaneous information used about the object store can end up in
.git/objects/info; this can help us understand what may be going on with
the object store when the user is reporting a bug. Otherwise, it could
be difficult to track down what is going wrong with an object which
isn't kept locally to .git/objects/ or .git/objects/pack. Having some
understanding of where the user's objects may be kept can save us some
hops during the bug reporting process.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 54 +++++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index 309b34fc42..a21d081616 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -34,6 +34,7 @@ The following information is captured automatically:
  - A list of enabled hooks
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
+ - A list of the contents of .git/objects/info (or equivalent)
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 580de13ddc..1c67b55598 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -204,6 +204,57 @@ static void get_packed_object_summary(struct strbuf *obj_info, int nongit)
 
 }
 
+static void list_contents_of_dir_recursively(struct strbuf *contents,
+				      	     struct strbuf *dirpath)
+{
+	struct dirent *d;
+	DIR *dir;
+	size_t path_len;
+
+	dir = opendir(dirpath->buf);
+	if (!dir)
+		return;
+
+	strbuf_complete(dirpath, '/');
+	path_len = dirpath->len;
+
+	while ((d = readdir(dir))) {
+		if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
+			continue;
+
+		strbuf_addbuf(contents, dirpath);
+		strbuf_addstr(contents, d->d_name);
+		strbuf_complete_line(contents);
+
+		if (d->d_type == DT_DIR) {
+			strbuf_addstr(dirpath, d->d_name);
+			list_contents_of_dir_recursively(contents, dirpath);
+		}
+		strbuf_setlen(dirpath, path_len);
+	}
+
+	closedir(dir);
+}
+
+static void get_object_info_summary(struct strbuf *obj_info, int nongit)
+{
+	struct strbuf dirpath = STRBUF_INIT;
+
+	if (nongit) {
+		strbuf_addstr(obj_info,
+			"not run from a git repository - object info unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&dirpath, get_object_directory());
+	strbuf_complete(&dirpath, '/');
+	strbuf_addstr(&dirpath, "info/");
+
+	list_contents_of_dir_recursively(obj_info, &dirpath);
+
+	strbuf_release(&dirpath);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -301,6 +352,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Packed Object Summary");
 	get_packed_object_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Object Info Summary");
+	get_object_info_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* [PATCH v7 15/15] bugreport: summarize contents of alternates file
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (13 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
@ 2020-02-14  1:53       ` Emily Shaffer
  2020-02-14 17:32       ` [PATCH v7 00/15] add git-bugreport tool Junio C Hamano
  15 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14  1:53 UTC (permalink / raw)
  To: git; +Cc: Emily Shaffer

In some cases, it could be that the user is having a problem with an
object which isn't present in their normal object directory. We can get
a hint that that might be the case by examining the list of alternates
where their object may be stored instead. Since paths to alternates may
be sensitive, we'll instead count how many alternates have been
specified and note how many of them exist or are broken.

While object-cache.h describes a function "foreach_alt_odb()", this
function does not provide information on broken alternates, which are
skipped over in "link_alt_odb_entry()". Since the goal is to identify
missing alternates, we can gather the contents of
.git/objects/info/alternates manually.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-bugreport.txt |  1 +
 bugreport.c                     | 45 +++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)

diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
index a21d081616..e870900c80 100644
--- a/Documentation/git-bugreport.txt
+++ b/Documentation/git-bugreport.txt
@@ -35,6 +35,7 @@ The following information is captured automatically:
  - The number of loose objects in the repository
  - The number of packs and packed objects in the repository
  - A list of the contents of .git/objects/info (or equivalent)
+ - The number of valid and invalid alternates
 
 OPTIONS
 -------
diff --git a/bugreport.c b/bugreport.c
index 1c67b55598..3bc8cb3579 100644
--- a/bugreport.c
+++ b/bugreport.c
@@ -255,6 +255,48 @@ static void get_object_info_summary(struct strbuf *obj_info, int nongit)
 	strbuf_release(&dirpath);
 }
 
+static void get_alternates_summary(struct strbuf *alternates_info, int nongit)
+{
+	struct strbuf alternates_path = STRBUF_INIT;
+	struct strbuf alternate = STRBUF_INIT;
+	FILE *file;
+	size_t exists = 0, broken = 0;
+
+	if (nongit) {
+		strbuf_addstr(alternates_info,
+			"not run from a git repository - alternates unavailable\n");
+		return;
+	}
+
+	strbuf_addstr(&alternates_path, get_object_directory());
+	strbuf_complete(&alternates_path, '/');
+	strbuf_addstr(&alternates_path, "info/alternates");
+
+	file = fopen(alternates_path.buf, "r");
+	if (!file) {
+		strbuf_addstr(alternates_info, "No alternates file found.\n");
+		strbuf_release(&alternates_path);
+		return;
+	}
+
+	while (strbuf_getline(&alternate, file) != EOF) {
+		if (!access(alternate.buf, F_OK))
+			exists++;
+		else
+			broken++;
+	}
+
+	strbuf_addf(alternates_info,
+		    "%zd alternates found (%zd working, %zd broken)\n",
+		    exists + broken,
+		    exists,
+		    broken);
+
+	fclose(file);
+	strbuf_release(&alternate);
+	strbuf_release(&alternates_path);
+}
+
 static const char * const bugreport_usage[] = {
 	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
 	NULL
@@ -355,6 +397,9 @@ int cmd_main(int argc, const char **argv)
 	get_header(&buffer, "Object Info Summary");
 	get_object_info_summary(&buffer, nongit_ok);
 
+	get_header(&buffer, "Alternates");
+	get_alternates_summary(&buffer, nongit_ok);
+
 	report = fopen_for_writing(report_path.buf);
 
 	if (report == NULL) {
-- 
2.25.0.265.gbab2e86ba0-goog


^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info
  2020-02-14  1:53       ` [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
@ 2020-02-14 17:04         ` Junio C Hamano
  2020-02-18 23:59           ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2020-02-14 17:04 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> Miscellaneous information used about the object store can end up in
> .git/objects/info; this can help us understand what may be going on with
> the object store when the user is reporting a bug. Otherwise, it could
> be difficult to track down what is going wrong with an object which
> isn't kept locally to .git/objects/ or .git/objects/pack. Having some
> understanding of where the user's objects may be kept can save us some
> hops during the bug reporting process.

This step seems to have a new whitespace breakage that did not exist
in the previous round.

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-14  1:53       ` [PATCH v7 03/15] bugreport: add tool to generate debugging info Emily Shaffer
@ 2020-02-14 17:25         ` Junio C Hamano
  2020-02-15  1:57           ` Emily Shaffer
  2020-02-19 14:18         ` Johannes Schindelin
  1 sibling, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2020-02-14 17:25 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> +	switch (safe_create_leading_directories(report_path.buf)) {

This helper is about creating paths in the working tree and Git
repository, hence it has a call to adjust_shared_perm() which in
turn calls get_shared_repo(), i.e. requiring a repository.

I thought I read somewhere that this tool is meant to be usable
outside a repository?  If that is not the case, then the use of this
helper is OK.  If not, we may want to make sure that it will stay to
be safe to use the helper (I think it happens to be OK right now,
but if the reason why the user is trying to run the tool is because
the user broke Git by writing garbage into .git/config, we may
die("your configuration file is broken") before this helper returns).

Thanks.



^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 00/15] add git-bugreport tool
  2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
                         ` (14 preceding siblings ...)
  2020-02-14  1:53       ` [PATCH v7 15/15] bugreport: summarize contents of alternates file Emily Shaffer
@ 2020-02-14 17:32       ` Junio C Hamano
  2020-02-14 22:00         ` Emily Shaffer
  15 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2020-02-14 17:32 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Derrick Stolee, Johannes Schindelin, Martin Ågren,
	Aaron Schrab, Danh Doan, Eric Sunshine, SZEDER Gábor,
	Andreas Schwab

Emily Shaffer <emilyshaffer@google.com> writes:

>    present(patch 9/15). This now uses POSIX character classes and {}
>    notation instead of + and has been tested on OSX; I'd love to hear if

I'd rather not to see unnecessary uses of POSIX character classes.

The interdiff of this step between the previous and this round looks
to me more like "I used it, just because POSIX says I *can* use it",
not "I did so because I needed to do, and it should be OK on POSIX
platforms."

Instead of overly long

's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/  "\1",/p'

just limiting ourselves to SP and saying

's/^\([^ ]*\)  *annotate:bugreport\[include\].* ::$/  "\1",/p'

would keep the result much easier to read, I would think.


^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 00/15] add git-bugreport tool
  2020-02-14 17:32       ` [PATCH v7 00/15] add git-bugreport tool Junio C Hamano
@ 2020-02-14 22:00         ` Emily Shaffer
  2020-02-14 22:30           ` Junio C Hamano
  0 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-14 22:00 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Derrick Stolee, Johannes Schindelin, Martin Ågren,
	Aaron Schrab, Danh Doan, Eric Sunshine, SZEDER Gábor,
	Andreas Schwab

On Fri, Feb 14, 2020 at 09:32:08AM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> >    present(patch 9/15). This now uses POSIX character classes and {}
> >    notation instead of + and has been tested on OSX; I'd love to hear if
> 
> I'd rather not to see unnecessary uses of POSIX character classes.
> 
> The interdiff of this step between the previous and this round looks
> to me more like "I used it, just because POSIX says I *can* use it",
> not "I did so because I needed to do, and it should be OK on POSIX
> platforms."
> 
> Instead of overly long
> 
> 's/^\([^[:blank:]]*\)[[:blank:]]\{1,\}annotate:bugreport\[include\].* ::$/  "\1",/p'
> 
> just limiting ourselves to SP and saying
> 
> 's/^\([^ ]*\)  *annotate:bugreport\[include\].* ::$/  "\1",/p'
> 
> would keep the result much easier to read, I would think.

That's fine by me. I find the [[:syntax:]] extremely ugly, but I was
worried about whether that was more portable somehow. Your proposal
seems fine to me.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 00/15] add git-bugreport tool
  2020-02-14 22:00         ` Emily Shaffer
@ 2020-02-14 22:30           ` Junio C Hamano
  0 siblings, 0 replies; 177+ messages in thread
From: Junio C Hamano @ 2020-02-14 22:30 UTC (permalink / raw)
  To: Emily Shaffer
  Cc: git, Derrick Stolee, Johannes Schindelin, Martin Ågren,
	Aaron Schrab, Danh Doan, Eric Sunshine, SZEDER Gábor,
	Andreas Schwab

Emily Shaffer <emilyshaffer@google.com> writes:

> That's fine by me. I find the [[:syntax:]] extremely ugly,...

FWIW, I find it so too.

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-14 17:25         ` Junio C Hamano
@ 2020-02-15  1:57           ` Emily Shaffer
  2020-02-15 18:24             ` Junio C Hamano
  0 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-15  1:57 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Feb 14, 2020 at 09:25:12AM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > +	switch (safe_create_leading_directories(report_path.buf)) {
> 
> This helper is about creating paths in the working tree and Git
> repository,

It's also used by cmd_format_patch() with --output-directory specified,
which is how I found it. My usual workflow is to run format-patch with
-o ~/mailed-patches/topic/ specified so I don't clutter my repo, so I'm
surprised to hear the intent of safe_create_leading_directores() is for
paths in the working tree or repo.

> hence it has a call to adjust_shared_perm() which in
> turn calls get_shared_repo(), i.e. requiring a repository.

Hmmm. I was able to run it pretty happily from a nongit dir:

  emilyshaffer@podkayne:~$ tg bugreport -o other/very/deep/path/
  Created new report at 'other/very/deep/path/git-bugreport-2020-02-14-1721.txt'.
  emilyshaffer@podkayne:~$ cat other/very/deep/path/git-bugreport-2020-02-14-1721.txt
  Thank you for filling out a Git bug report!

> I thought I read somewhere that this tool is meant to be usable
> outside a repository?  If that is not the case, then the use of this
> helper is OK.  If not, we may want to make sure that it will stay to
> be safe to use the helper (I think it happens to be OK right now,
> but if the reason why the user is trying to run the tool is because
> the user broke Git by writing garbage into .git/config, we may
> die("your configuration file is broken") before this helper returns).

Can you explain a little more about what you mean? A broken local
.git/config seems to be preventing git.c from dispatching the
'bugreport' subcommand (lots of other Git commands are broken) at all -
breakpoints in cmd_main() right after the variable declarations are not
being hit.

With junk in the .git/config, I can't run bugreport from within that
repo. With junk in ~/.gitconfig, I can't run bugreport anywhere.

(To make the configs invalid, I leaned on my keyboard and added a line
'garbaghe~*~$%)%)(@' to the bottom of the config file. Most commands
terminate early with "fatal: bad config line 37 in file .git/config".)

Do you mean there's some specific config that could be misconfigured and
prevent that utility from working?

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-15  1:57           ` Emily Shaffer
@ 2020-02-15 18:24             ` Junio C Hamano
  2020-02-18 23:46               ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2020-02-15 18:24 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Emily Shaffer <emilyshaffer@google.com> writes:

> On Fri, Feb 14, 2020 at 09:25:12AM -0800, Junio C Hamano wrote:
>> Emily Shaffer <emilyshaffer@google.com> writes:
>> 
>> > +	switch (safe_create_leading_directories(report_path.buf)) {
>> 
>> This helper is about creating paths in the working tree and Git
>> repository,
>
> It's also used by cmd_format_patch() with --output-directory specified,
> which is how I found it.

And that is an example of a good use of this helper.  What will be
written out there should be visible by the same people as those who
have access to the repository; get_shared_repo() and adjust_perm()
based on what the repository you are taking patches from is
perfectly sensible.  And as it is format-patch, we know we have
get_shared_repo() working, and we know which repository we are
working on.

Output directory for bugreport is on the same boat when we know the
users are producing a report on their use of Git within a
repository.  It is not clear if the tool is started without any
repository---it happens to do a random safe thing (i.e. I think
get_shared_repo() gives PERM_UMASK, which tells adjust_perm() not to
do anything especial) right now, but there is no guarantee that we
will keep it working like that.  Somebody may come and demaind
get_shared_perm() to die() when outside a repository, for example,
and that would break the nice property that lets bugreport working
outside a repository.

I just wanted to make sure that somebody will be keeping an eye to
remind those who propose such a change in the future.  A comment
near where get_shared_repo() happens may be something that can be
done with a minimum effort when code that depends on that property
is introduced at the same time.

>> I thought I read somewhere that this tool is meant to be usable
>> outside a repository?  If that is not the case, then the use of this
>> helper is OK.  If not, we may want to make sure that it will stay to
>> be safe to use the helper (I think it happens to be OK right now,

I am actually OK if we limit the use of this tool to "use with a
repository that is not corrupt", as coping with these kinds of
breakages that in the main Git executable we deem "needs manual
intervention" inside a single process is too painful (it would have
been easier to cope with these too if we stuck with a script that
invokes many discrete commands and acts on their errors, but that is
optimizing for rare case and not recommended).  But we should tell
users about the limitation and encourage them to ask for help in non
automatable means.

Thanks.

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-15 18:24             ` Junio C Hamano
@ 2020-02-18 23:46               ` Emily Shaffer
  2020-02-18 23:56                 ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-18 23:46 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Sat, Feb 15, 2020 at 10:24:40AM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > On Fri, Feb 14, 2020 at 09:25:12AM -0800, Junio C Hamano wrote:
> >> Emily Shaffer <emilyshaffer@google.com> writes:
> >> 
> >> > +	switch (safe_create_leading_directories(report_path.buf)) {
> >> 
> >> This helper is about creating paths in the working tree and Git
> >> repository,
> >
> > It's also used by cmd_format_patch() with --output-directory specified,
> > which is how I found it.
> 
> And that is an example of a good use of this helper.  What will be
> written out there should be visible by the same people as those who
> have access to the repository; get_shared_repo() and adjust_perm()
> based on what the repository you are taking patches from is
> perfectly sensible.  And as it is format-patch, we know we have
> get_shared_repo() working, and we know which repository we are
> working on.
> 
> Output directory for bugreport is on the same boat when we know the
> users are producing a report on their use of Git within a
> repository.  It is not clear if the tool is started without any
> repository---it happens to do a random safe thing (i.e. I think
> get_shared_repo() gives PERM_UMASK, which tells adjust_perm() not to
> do anything especial) right now, but there is no guarantee that we
> will keep it working like that.  Somebody may come and demaind
> get_shared_perm() to die() when outside a repository, for example,
> and that would break the nice property that lets bugreport working
> outside a repository.

Hm, this would break the convention of the name of
safe_create_leading_directories() too though right? As I understood it,
"safe_foo" in this codebase means "this function will not die()"?

> 
> I just wanted to make sure that somebody will be keeping an eye to
> remind those who propose such a change in the future.  A comment
> near where get_shared_repo() happens may be something that can be
> done with a minimum effort when code that depends on that property
> is introduced at the same time.

Ok. I want to make sure I understand you right. I think you're saying,
"This is OK, except if someone changes get_shared_repo() it could break,
so we need a way to warn someone that it could break." It sounds like
you suggested leaving a comment in the
safe_create_leading_directories() helper. My own preference is to write
a test so that it's explicit that we depend on that behavior, instead -
it's easy to gloss over a comment or read a different part of the
codebase, but it's hard to ignore a breaking test. It'd be trivial to
add one, so I will in v8 - unless I misunderstood you.

> 
> >> I thought I read somewhere that this tool is meant to be usable
> >> outside a repository?  If that is not the case, then the use of this
> >> helper is OK.  If not, we may want to make sure that it will stay to
> >> be safe to use the helper (I think it happens to be OK right now,
> 
> I am actually OK if we limit the use of this tool to "use with a
> repository that is not corrupt", as coping with these kinds of
> breakages that in the main Git executable we deem "needs manual
> intervention" inside a single process is too painful (it would have
> been easier to cope with these too if we stuck with a script that
> invokes many discrete commands and acts on their errors, but that is
> optimizing for rare case and not recommended).  But we should tell
> users about the limitation and encourage them to ask for help in non
> automatable means.

I think you're saying, "Mention this drawback in the manpage for
git-bugreport." Sounds like a good idea to me, so I'll add it for v8.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-18 23:46               ` Emily Shaffer
@ 2020-02-18 23:56                 ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-18 23:56 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Tue, Feb 18, 2020 at 03:46:28PM -0800, Emily Shaffer wrote:
> On Sat, Feb 15, 2020 at 10:24:40AM -0800, Junio C Hamano wrote:
> > I am actually OK if we limit the use of this tool to "use with a
> > repository that is not corrupt", as coping with these kinds of
> > breakages that in the main Git executable we deem "needs manual
> > intervention" inside a single process is too painful (it would have
> > been easier to cope with these too if we stuck with a script that
> > invokes many discrete commands and acts on their errors, but that is
> > optimizing for rare case and not recommended).  But we should tell
> > users about the limitation and encourage them to ask for help in non
> > automatable means.
> 
> I think you're saying, "Mention this drawback in the manpage for
> git-bugreport." Sounds like a good idea to me, so I'll add it for v8.

I'm pretty unsure about how you wanted this to sound; rather than
sending a v8 only to revise it a lot, I'll send the paragraph for
wordsmithing beforehand. Is this what you meant?

  This tool is invoked via the typical Git setup process, which means that in some
  cases, it might not be able to launch - for example, if a relevant config file
  is unreadable. In this kind of scenario, it may be helpful to manually gather
  the kind of information listed above when manually asking for help.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info
  2020-02-14 17:04         ` Junio C Hamano
@ 2020-02-18 23:59           ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-18 23:59 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

On Fri, Feb 14, 2020 at 09:04:05AM -0800, Junio C Hamano wrote:
> Emily Shaffer <emilyshaffer@google.com> writes:
> 
> > Miscellaneous information used about the object store can end up in
> > .git/objects/info; this can help us understand what may be going on with
> > the object store when the user is reporting a bug. Otherwise, it could
> > be difficult to track down what is going wrong with an object which
> > isn't kept locally to .git/objects/ or .git/objects/pack. Having some
> > understanding of where the user's objects may be kept can save us some
> > hops during the bug reporting process.
> 
> This step seems to have a new whitespace breakage that did not exist
> in the previous round.

Hmph. I saw it warned at some point during the rebase-and-format-patch
workflow for v7 but thought I had quashed it. Very sorry.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-14  1:53       ` [PATCH v7 03/15] bugreport: add tool to generate debugging info Emily Shaffer
  2020-02-14 17:25         ` Junio C Hamano
@ 2020-02-19 14:18         ` Johannes Schindelin
  2020-02-19 16:55           ` Junio C Hamano
  1 sibling, 1 reply; 177+ messages in thread
From: Johannes Schindelin @ 2020-02-19 14:18 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Thu, 13 Feb 2020, Emily Shaffer wrote:

> diff --git a/bugreport.c b/bugreport.c
> new file mode 100644
> index 0000000000..a9398e6a2a
> --- /dev/null
> +++ b/bugreport.c
> @@ -0,0 +1,94 @@
> +#include "builtin.h"
> +#include "parse-options.h"
> +#include "stdio.h"
> +#include "strbuf.h"
> +#include "time.h"
> +
> +static const char * const bugreport_usage[] = {
> +	N_("git bugreport [-o|--output-directory <file>] [-s|--suffix <format>]"),
> +	NULL
> +};
> +
> +static int get_bug_template(struct strbuf *template)
> +{
> +	const char template_text[] = N_(
> +"Thank you for filling out a Git bug report!\n"
> +"Please answer the following questions to help us understand your issue.\n"
> +"\n"
> +"What did you do before the bug happened? (Steps to reproduce your issue)\n"
> +"\n"
> +"What did you expect to happen? (Expected behavior)\n"
> +"\n"
> +"What happened instead? (Actual behavior)\n"
> +"\n"
> +"What's different between what you expected and what actually happened?\n"
> +"\n"
> +"Anything else you want to add:\n"
> +"\n"
> +"Please review the rest of the bug report below.\n"
> +"You can delete any lines you don't wish to share.\n");
> +
> +	strbuf_addstr(template, template_text);
> +	return 0;
> +}
> +
> +int cmd_main(int argc, const char **argv)
> +{
> +	struct strbuf buffer = STRBUF_INIT;
> +	struct strbuf report_path = STRBUF_INIT;
> +	FILE *report;
> +	time_t now = time(NULL);
> +	char *option_output = NULL;
> +	char *option_suffix = "%F-%H%M";
> +	struct stat statbuf;
> +
> +	const struct option bugreport_options[] = {
> +		OPT_STRING('o', "output-directory", &option_output, N_("path"),
> +			   N_("specify a destination for the bugreport file")),
> +		OPT_STRING('s', "suffix", &option_suffix, N_("format"),
> +			   N_("specify a strftime format suffix for the filename")),
> +		OPT_END()
> +	};
> +	argc = parse_options(argc, argv, "", bugreport_options,
> +			     bugreport_usage, 0);
> +
> +	if (option_output) {
> +		strbuf_addstr(&report_path, option_output);
> +		strbuf_complete(&report_path, '/');
> +	}
> +
> +
> +	strbuf_addstr(&report_path, "git-bugreport-");
> +	strbuf_addftime(&report_path, option_suffix, localtime(&now), 0, 0);
> +	strbuf_addstr(&report_path, ".txt");
> +
> +	if (!stat(report_path.buf, &statbuf))
> +		die("'%s' already exists", report_path.buf);
> +
> +	switch (safe_create_leading_directories(report_path.buf)) {
> +	case SCLD_OK:
> +	case SCLD_EXISTS:
> +		break;
> +	default:
> +		die(_("could not create leading directories for '%s'"),
> +		    report_path.buf);
> +	}
> +
> +	get_bug_template(&buffer);
> +
> +	report = fopen_for_writing(report_path.buf);
> +
> +	if (report == NULL) {
> +		strbuf_release(&report_path);
> +		die("couldn't open '%s' for writing", report_path.buf);
> +	}
> +
> +	strbuf_write(&buffer, report);
> +	fclose(report);
> +
> +	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
> +
> +	UNLEAK(buffer);
> +	UNLEAK(report_path);
> +	return -launch_editor(report_path.buf, NULL, NULL);

This would be the first time (at least that _I_ know of) that we use `-`
in this way. We seem to use `!!` a lot more often. And now I wonder
whether there is a reason for that `-` that I missed?

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 06/15] bugreport: add compiler info
  2020-02-14  1:53       ` [PATCH v7 06/15] bugreport: add compiler info Emily Shaffer
@ 2020-02-19 14:23         ` Johannes Schindelin
  2020-02-19 22:45           ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Johannes Schindelin @ 2020-02-19 14:23 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Thu, 13 Feb 2020, Emily Shaffer wrote:

> To help pinpoint the source of a regression, it is useful to know some
> info about the compiler which the user's Git client was built with. By
> adding a generic get_compiler_info() in 'compat/' we can choose which
> relevant information to share per compiler; to get started, let's
> demonstrate the version of glibc if the user built with 'gcc'.

I agree with the need for the compiler information, but in the patch I
only see information about glibc being printed out. Shouldn't we use
`__GNUC__` and `__GNUC_MINOR__` here?

Don't get me wrong, the glibc version is good and all, but the compiler
information might be even more crucial. Git for Windows had to hold back
compiling with GCC v8.x for a while, for example, because the stacksmasher
was broken. Similar issues are not unheard of, and could help pinpoint
compiler-related problems a lot quicker.

Thanks,
Dscho

>
> Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
> ---
>  Documentation/git-bugreport.txt |  1 +
>  bugreport.c                     |  5 +++++
>  compat/compiler.h               | 24 ++++++++++++++++++++++++
>  3 files changed, 30 insertions(+)
>  create mode 100644 compat/compiler.h
>
> diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt
> index 4dd72c60f5..8bbc4c960c 100644
> --- a/Documentation/git-bugreport.txt
> +++ b/Documentation/git-bugreport.txt
> @@ -27,6 +27,7 @@ The following information is captured automatically:
>
>   - 'git version --build-options'
>   - uname sysname, release, version, and machine strings
> + - Compiler-specific info string
>
>  OPTIONS
>  -------
> diff --git a/bugreport.c b/bugreport.c
> index b76a1dfb2a..4f9101caeb 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -4,6 +4,7 @@
>  #include "strbuf.h"
>  #include "time.h"
>  #include "help.h"
> +#include "compat/compiler.h"
>
>  static void get_system_info(struct strbuf *sys_info)
>  {
> @@ -24,6 +25,10 @@ static void get_system_info(struct strbuf *sys_info)
>  			    uname_info.release,
>  			    uname_info.version,
>  			    uname_info.machine);
> +
> +	strbuf_addstr(sys_info, "compiler info: ");
> +	get_compiler_info(sys_info);
> +	strbuf_complete_line(sys_info);
>  }
>
>  static const char * const bugreport_usage[] = {
> diff --git a/compat/compiler.h b/compat/compiler.h
> new file mode 100644
> index 0000000000..bda5098e1b
> --- /dev/null
> +++ b/compat/compiler.h
> @@ -0,0 +1,24 @@
> +#ifndef COMPILER_H
> +#define COMPILER_H
> +
> +#include "git-compat-util.h"
> +#include "strbuf.h"
> +
> +#ifdef __GLIBC__
> +#include <gnu/libc-version.h>
> +
> +static inline void get_compiler_info(struct strbuf *info)
> +{
> +	strbuf_addf(info, "glibc: %s", gnu_get_libc_version());
> +}
> +
> +#else
> +
> +static inline void get_compiler_info(struct strbuf *info)
> +{
> +	strbuf_addstr(info, "get_compiler_info() not implemented");
> +}
> +
> +#endif
> +
> +#endif /* COMPILER_H */
> --
> 2.25.0.265.gbab2e86ba0-goog
>
>

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 07/15] bugreport: add git-remote-https version
  2020-02-14  1:53       ` [PATCH v7 07/15] bugreport: add git-remote-https version Emily Shaffer
@ 2020-02-19 14:28         ` Johannes Schindelin
  2020-02-19 22:28           ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Johannes Schindelin @ 2020-02-19 14:28 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: git

Hi Emily,

On Thu, 13 Feb 2020, Emily Shaffer wrote:

> diff --git a/bugreport.c b/bugreport.c
> index 4f9101caeb..bfdff33368 100644
> --- a/bugreport.c
> +++ b/bugreport.c
> @@ -5,6 +5,18 @@
>  #include "time.h"
>  #include "help.h"
>  #include "compat/compiler.h"
> +#include "run-command.h"
> +
> +static void get_git_remote_https_version_info(struct strbuf *version_info)
> +{
> +	struct child_process cp = CHILD_PROCESS_INIT;
> +
> +	argv_array_push(&cp.args, "git");
> +	argv_array_push(&cp.args, "remote-https");
> +	argv_array_push(&cp.args, "--build-info");
> +	if (capture_command(&cp, version_info, 0))

Let's use RUN_GIT_CMD instead of adding `"git"` explicitly; It documents
that we're interested in a Git command, and if we ever build a
single-binary version of Git (as some Git for Windows users already asked
for), it will make things easier.

> +	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
> +}
>
>  static void get_system_info(struct strbuf *sys_info)
>  {
> diff --git a/remote-curl.c b/remote-curl.c
> index 350d92a074..c590fbfae3 100644
> --- a/remote-curl.c
> +++ b/remote-curl.c
> @@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
>  	string_list_init(&options.deepen_not, 1);
>  	string_list_init(&options.push_options, 1);
>
> +	if (!strcmp("--build-info", argv[1])) {

The context does not say this, but at this point, we already verified that
`argc` is larger than 1. Good.

Also, in keeping with the existing code, we would need to use
`--build-options` here (this is what `git version` calls the equivalent
mode).

_However_.

I like your `--build-info` a lot more than `--build-options` (because the
latter is very misleading: the commit and the date of the build are not
"options" at all).

Thanks,
Dscho

> +		printf("git-http-fetch version: %s\n", git_version_string);
> +		printf("built from commit: %s\n", git_built_from_commit_string);
> +		printf("curl version: %s\n", curl_version());
> +		return 0;
> +	}
> +
>  	/*
>  	 * Just report "remote-curl" here (folding all the various aliases
>  	 * ("git-remote-http", "git-remote-https", and etc.) here since they
> --
> 2.25.0.265.gbab2e86ba0-goog
>
>

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-19 14:18         ` Johannes Schindelin
@ 2020-02-19 16:55           ` Junio C Hamano
  2020-02-19 21:52             ` Emily Shaffer
  0 siblings, 1 reply; 177+ messages in thread
From: Junio C Hamano @ 2020-02-19 16:55 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Emily Shaffer, git

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> +int cmd_main(int argc, const char **argv)
>> +{
>> +...
>> +	if (report == NULL) {
>> +		strbuf_release(&report_path);
>> +		die("couldn't open '%s' for writing", report_path.buf);
>> +	}
>> +
>> +	strbuf_write(&buffer, report);
>> +	fclose(report);
>> +
>> +	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
>> +
>> +	UNLEAK(buffer);
>> +	UNLEAK(report_path);
>> +	return -launch_editor(report_path.buf, NULL, NULL);
>
> This would be the first time (at least that _I_ know of) that we use `-`
> in this way. We seem to use `!!` a lot more often. And now I wonder
> whether there is a reason for that `-` that I missed?

In general, our preferred way to report an error from API functions
is to return a negative number and a success is reported by
returning zero.

The argument to exit(3), which the return value from the main()
function essentially is, on the other hand, is expected to be a
small non-negative integer.  As long as we are in tight control of
the range of the returned value from launch_editor() (i.e. it must
return a small non-positive integer whose negation is suitable to be
fed to exit(3)), the above is fine.

The idiom "return !!fn();" is to canonicalize the value to 0 or 1
for the caller who asked "so, did fn() give us 0 or non-zero?",
which can also be used as the value given to exit(3), and could be
more appropriate under some conditions:

 - we MUST know launch_editor() returns zero if and only if it is
   successful.

 - we do not have to be confident to be in tight control of the
   range of the returned value from launch_editor() (e.g. it could
   return a positive upon an error).

 - we MUST NOT care to differenciate different error codes returned
   from launch_editor().  IOW, we must be fine to give the invoker
   of the program only 0 (success) or 1 (unspecified failure).

Use of "return !!fn();" that is not to give to exit(3) is probably
more common in our codebase.  See apply.c::try_create_file() and the
comment before the function about its possible return values for
example.

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-19 16:55           ` Junio C Hamano
@ 2020-02-19 21:52             ` Emily Shaffer
  2020-02-19 22:09               ` Junio C Hamano
  0 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-19 21:52 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Johannes Schindelin, git

On Wed, Feb 19, 2020 at 08:55:04AM -0800, Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> +int cmd_main(int argc, const char **argv)
> >> +{
> >> +...
> >> +	if (report == NULL) {
> >> +		strbuf_release(&report_path);
> >> +		die("couldn't open '%s' for writing", report_path.buf);
> >> +	}
> >> +
> >> +	strbuf_write(&buffer, report);
> >> +	fclose(report);
> >> +
> >> +	fprintf(stderr, _("Created new report at '%s'.\n"), report_path.buf);
> >> +
> >> +	UNLEAK(buffer);
> >> +	UNLEAK(report_path);
> >> +	return -launch_editor(report_path.buf, NULL, NULL);
> >
> > This would be the first time (at least that _I_ know of) that we use `-`
> > in this way. We seem to use `!!` a lot more often. And now I wonder
> > whether there is a reason for that `-` that I missed?
> 
> In general, our preferred way to report an error from API functions
> is to return a negative number and a success is reported by
> returning zero.
> 
> The argument to exit(3), which the return value from the main()
> function essentially is, on the other hand, is expected to be a
> small non-negative integer.  As long as we are in tight control of
> the range of the returned value from launch_editor() (i.e. it must
> return a small non-positive integer whose negation is suitable to be
> fed to exit(3)), the above is fine.
> 
> The idiom "return !!fn();" is to canonicalize the value to 0 or 1
> for the caller who asked "so, did fn() give us 0 or non-zero?",
> which can also be used as the value given to exit(3), and could be
> more appropriate under some conditions:

In editor.c, launch_editor() returns launch_specified_editor() without
altering the return code.

> 
>  - we MUST know launch_editor() returns zero if and only if it is
>    successful.

launch_specified_editor() has a handful of exit points, of three kinds:
 1. return error(something)
 2. raise(sigsomething)
 3. return 0
    a. when the editor process closed happily, but the user supplied
       NULL instead of a buffer. That is, the user didn't want the
       contents of the editor given back to them in a strbuf.
    b. when the editor process closed happily and the user's supplied
       buffer was filled with the file's contents with no issue.

So I think we can check "yes" here.

> 
>  - we do not have to be confident to be in tight control of the
>    range of the returned value from launch_editor() (e.g. it could
>    return a positive upon an error).

According to 'man 3 raise' (POSIX), raise() exits the current thread. If it
can't exit itself(?) it seems it returns "nonzero for failure". We've
also got a mingw_raise() in compat/mingw.h which could be used instead;
this one seems to return -1 or the result of raise(), presumably the one
from the MSC runtime[1], which also returns "a nonzero value" if not
successful.

So it's true that we aren't confident that launch_editor() returns a
positive or negative value.

>  - we MUST NOT care to differenciate different error codes returned
>    from launch_editor().  IOW, we must be fine to give the invoker
>    of the program only 0 (success) or 1 (unspecified failure).

This part is a little sticking point for me; I think I'd rather give the
user some hint of what's broken than no hint at all, especially in this
scenario, where it's conceivable someone could say "I tried to run
'git-bugreport' and it crashed, here is the return code and I have
manually collected some relevant info too".

The uncertainty coming from raise() (POSIX and MSCR both) leads me to
believe if I do want to return the error on exit, I should at least
check the sign before I flip it. Anybody else have opinions?

 - Emily

[1] https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/raise?view=vs-2019

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 03/15] bugreport: add tool to generate debugging info
  2020-02-19 21:52             ` Emily Shaffer
@ 2020-02-19 22:09               ` Junio C Hamano
  0 siblings, 0 replies; 177+ messages in thread
From: Junio C Hamano @ 2020-02-19 22:09 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Johannes Schindelin, git

Emily Shaffer <emilyshaffer@google.com> writes:

> launch_specified_editor() has a handful of exit points, of three kinds:
>  1. return error(something)
>  2. raise(sigsomething)
>  3. return 0
>     a. when the editor process closed happily, but the user supplied
>        NULL instead of a buffer. That is, the user didn't want the
>        contents of the editor given back to them in a strbuf.
>     b. when the editor process closed happily and the user's supplied
>        buffer was filled with the file's contents with no issue.
>
> So I think we can check "yes" here.

Heh.  If we raised a signal to kill ourselves, then we won't be
returning a value from launch_editor() anyway.  That case won't
affect the "between returning negation or !!, which is more
appropriate?" discussion, I think.

>>  - we MUST NOT care to differenciate different error codes returned
>>    from launch_editor().  IOW, we must be fine to give the invoker
>>    of the program only 0 (success) or 1 (unspecified failure).

I actually think this holds for the codepath.  A failure from
start_command() returns error(), and finish_command() that waits for
the spawned editor process to complete yields the exit status from
the editor, but unless we re-raise the signal that killed the editor
process to ourselves, we just turn any non-zero exit into "return
error()", so it is safe to say launch_editor() can return either 0
or -1 and nothing else.  Would we later want to tell callers of
launch_editor() how/why the editor session failed by returning
something else?  I do not offhand think of any---we do not even
differenciate between failure to start (e.g. misspelt command name
for the editor) and other failures WITH the return value and
consider it sufficient to tell the user with different error
message right now.

So in practice returning -launch_editor() and !!launch_editor()
would not make any difference, I would think.


^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 07/15] bugreport: add git-remote-https version
  2020-02-19 14:28         ` Johannes Schindelin
@ 2020-02-19 22:28           ` Emily Shaffer
  2020-02-19 22:33             ` Junio C Hamano
  0 siblings, 1 reply; 177+ messages in thread
From: Emily Shaffer @ 2020-02-19 22:28 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git

On Wed, Feb 19, 2020 at 03:28:35PM +0100, Johannes Schindelin wrote:
> Hi Emily,
> 
> On Thu, 13 Feb 2020, Emily Shaffer wrote:
> 
> > diff --git a/bugreport.c b/bugreport.c
> > index 4f9101caeb..bfdff33368 100644
> > --- a/bugreport.c
> > +++ b/bugreport.c
> > @@ -5,6 +5,18 @@
> >  #include "time.h"
> >  #include "help.h"
> >  #include "compat/compiler.h"
> > +#include "run-command.h"
> > +
> > +static void get_git_remote_https_version_info(struct strbuf *version_info)
> > +{
> > +	struct child_process cp = CHILD_PROCESS_INIT;
> > +
> > +	argv_array_push(&cp.args, "git");
> > +	argv_array_push(&cp.args, "remote-https");
> > +	argv_array_push(&cp.args, "--build-info");
> > +	if (capture_command(&cp, version_info, 0))
> 
> Let's use RUN_GIT_CMD instead of adding `"git"` explicitly; It documents
> that we're interested in a Git command, and if we ever build a
> single-binary version of Git (as some Git for Windows users already asked
> for), it will make things easier.

Hm. RUN_GIT_CMD is an argument used for the run_command_v_opt* family of
calls, but it seems that setting child_process.git_cmd has the same
effect. Done.

> 
> > +	    strbuf_addstr(version_info, "'git-remote-https --build-info' not supported\n");
> > +}
> >
> >  static void get_system_info(struct strbuf *sys_info)
> >  {
> > diff --git a/remote-curl.c b/remote-curl.c
> > index 350d92a074..c590fbfae3 100644
> > --- a/remote-curl.c
> > +++ b/remote-curl.c
> > @@ -1374,6 +1375,13 @@ int cmd_main(int argc, const char **argv)
> >  	string_list_init(&options.deepen_not, 1);
> >  	string_list_init(&options.push_options, 1);
> >
> > +	if (!strcmp("--build-info", argv[1])) {
> 
> The context does not say this, but at this point, we already verified that
> `argc` is larger than 1. Good.
> 
> Also, in keeping with the existing code, we would need to use
> `--build-options` here (this is what `git version` calls the equivalent
> mode).
> 
> _However_.
> 
> I like your `--build-info` a lot more than `--build-options` (because the
> latter is very misleading: the commit and the date of the build are not
> "options" at all).

Sure. Thanks for saying so.

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 07/15] bugreport: add git-remote-https version
  2020-02-19 22:28           ` Emily Shaffer
@ 2020-02-19 22:33             ` Junio C Hamano
  0 siblings, 0 replies; 177+ messages in thread
From: Junio C Hamano @ 2020-02-19 22:33 UTC (permalink / raw)
  To: Emily Shaffer; +Cc: Johannes Schindelin, git

Emily Shaffer <emilyshaffer@google.com> writes:

>> Also, in keeping with the existing code, we would need to use
>> `--build-options` here (this is what `git version` calls the equivalent
>> mode).
>> 
>> _However_.
>> 
>> I like your `--build-info` a lot more than `--build-options` (because the
>> latter is very misleading: the commit and the date of the build are not
>> "options" at all).
>
> Sure. Thanks for saying so.

I don't think anybody would mind introducing --build-info to "git
version" as a synonym and deprecate --build-options from it ;-)

^ permalink raw reply	[flat|nested] 177+ messages in thread

* Re: [PATCH v7 06/15] bugreport: add compiler info
  2020-02-19 14:23         ` Johannes Schindelin
@ 2020-02-19 22:45           ` Emily Shaffer
  0 siblings, 0 replies; 177+ messages in thread
From: Emily Shaffer @ 2020-02-19 22:45 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: git

On Wed, Feb 19, 2020 at 03:23:34PM +0100, Johannes Schindelin wrote:
> Hi Emily,
> 
> On Thu, 13 Feb 2020, Emily Shaffer wrote:
> 
> > To help pinpoint the source of a regression, it is useful to know some
> > info about the compiler which the user's Git client was built with. By
> > adding a generic get_compiler_info() in 'compat/' we can choose which
> > relevant information to share per compiler; to get started, let's
> > demonstrate the version of glibc if the user built with 'gcc'.
> 
> I agree with the need for the compiler information, but in the patch I
> only see information about glibc being printed out. Shouldn't we use
> `__GNUC__` and `__GNUC_MINOR__` here?
> 
> Don't get me wrong, the glibc version is good and all, but the compiler
> information might be even more crucial. Git for Windows had to hold back
> compiling with GCC v8.x for a while, for example, because the stacksmasher
> was broken. Similar issues are not unheard of, and could help pinpoint
> compiler-related problems a lot quicker.

Hm, sure. Good point - thanks.

This does make me start to wonder, though - does it really make sense to
have ifdef-gated redefinitions of the whole get_compiler_info() method
like I do now? I wonder if it makes more sense to have only one
definition, so we can write down everything we know regardless of which
pieces are put together. My thinking is something like this - what if I
am using glibc, but not a GNU compiler? (The GNU docs on __GCC__
indicate this is a situation that might occur - "a non-GCC compiler that
claims to accept the GNU C dialects") Is there some striking reason not
to implement this compat command thusly instead:

  #ifdef __GLIBC__
  #include <gnu/libc-version.h>
  #endif

  static inline void get_compiler_info(struct strbuf *info)
  {
  	#ifdef __GLIBC__
	strbuf_addf(info, "glibc: %s\n", gnu_get_libc_version());
	#endif

	#ifdef __GNUC__
	strbuf_addf(info, "gnuc: %d.%d\n", __GNUC__, __GNUC_MINOR__);
	#endif

	#ifdef _MSC_VER
	strbuf_addf(info, "msc runtime: %s\n", some_msc_info());
	#endif
  }

The thinking being - this way if I decide to use, say, LLVM + glibc,
then I don't need to reimplement this command with all the glibc
diagnostics again. Or, if someone else already wrote diagnostics for
LLVM with some other libc, then it even Just Works for me and my new
combination.

That said, I'm reasoning about these combinations of compilers and libcs
and whatever else from an inexperienced viewpoint, so maybe this isn't
necessary?

 - Emily

^ permalink raw reply	[flat|nested] 177+ messages in thread

end of thread, back to index

Thread overview: 177+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-12-13  0:42 [PATCH v4 00/15] add git-bugreport tool Emily Shaffer
2019-12-13  0:42 ` [PATCH v4 01/15] bugreport: add tool to generate debugging info Emily Shaffer
2019-12-13  0:42 ` [PATCH v4 02/15] help: move list_config_help to builtin/help Emily Shaffer
2019-12-13 20:51   ` Junio C Hamano
2019-12-16 21:36     ` Emily Shaffer
2019-12-16 22:19       ` Junio C Hamano
2019-12-16 22:34         ` Emily Shaffer
2019-12-13  0:43 ` [PATCH v4 03/15] bugreport: gather git version and build info Emily Shaffer
2019-12-13 21:06   ` Junio C Hamano
2019-12-20  1:46     ` Emily Shaffer
2019-12-17 18:45   ` Johannes Schindelin
2019-12-17 20:34     ` Junio C Hamano
2019-12-20  1:25       ` Emily Shaffer
2019-12-13  0:43 ` [PATCH v4 04/15] help: add shell-path to --build-options Emily Shaffer
2019-12-13  0:43 ` [PATCH v4 05/15] bugreport: add uname info Emily Shaffer
2019-12-13 21:12   ` Junio C Hamano
2020-01-10  2:05   ` Aaron Schrab
2019-12-13  0:43 ` [PATCH v4 06/15] bugreport: add glibc version Emily Shaffer
2019-12-13 21:18   ` Junio C Hamano
2019-12-16 22:39     ` Emily Shaffer
2019-12-13  0:43 ` [PATCH v4 07/15] bugreport: add curl version Emily Shaffer
2019-12-13 21:27   ` Junio C Hamano
2019-12-16 22:49     ` Emily Shaffer
2019-12-17 18:47   ` Johannes Schindelin
2019-12-13  0:43 ` [PATCH v4 08/15] bugreport: include user interactive shell Emily Shaffer
2019-12-13 21:38   ` Junio C Hamano
2019-12-13  0:43 ` [PATCH v4 09/15] bugreport: generate config safelist based on docs Emily Shaffer
2019-12-13 22:57   ` Junio C Hamano
2019-12-16 23:01     ` Emily Shaffer
2019-12-17  0:41       ` Emily Shaffer
2019-12-15 20:17   ` Johannes Schindelin
2019-12-16 22:52     ` Emily Shaffer
2019-12-17 18:38   ` Johannes Schindelin
2019-12-13  0:43 ` [PATCH v4 10/15] bugreport: add config values from safelist Emily Shaffer
2019-12-13 21:45   ` Junio C Hamano
2019-12-16 23:40     ` Emily Shaffer
2019-12-17 17:43       ` Junio C Hamano
2020-01-24  3:29         ` Emily Shaffer
2019-12-29 20:17   ` Johannes Schindelin
2019-12-13  0:43 ` [PATCH v4 11/15] bugreport: collect list of populated hooks Emily Shaffer
2019-12-13 21:47   ` Junio C Hamano
2019-12-16 23:51     ` Emily Shaffer
2019-12-13  0:43 ` [PATCH v4 12/15] bugreport: count loose objects Emily Shaffer
2019-12-13 21:51   ` Junio C Hamano
2019-12-16 23:54     ` Emily Shaffer
2019-12-13  0:43 ` [PATCH v4 13/15] bugreport: add packed object summary Emily Shaffer
2019-12-13 21:56   ` Junio C Hamano
2019-12-16 23:56     ` Emily Shaffer
2019-12-13  0:43 ` [PATCH v4 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
2019-12-13  0:43 ` [PATCH v4 15/15] bugreport: summarize contents of alternates file Emily Shaffer
2020-01-24  3:34 ` [PATCH v5 00/15] add git-bugreport tool emilyshaffer
2020-01-24  3:34   ` [PATCH v5 01/15] bugreport: add tool to generate debugging info emilyshaffer
2020-01-30 22:18     ` Martin Ågren
2020-02-04 22:00       ` Emily Shaffer
2020-01-24  3:34   ` [PATCH v5 02/15] help: move list_config_help to builtin/help emilyshaffer
2020-01-30 22:19     ` Martin Ågren
2020-02-04  0:53       ` Emily Shaffer
2020-01-24  3:34   ` [PATCH v5 03/15] bugreport: gather git version and build info emilyshaffer
2020-01-30 22:19     ` Martin Ågren
2020-02-04 22:21       ` Emily Shaffer
2020-01-24  3:34   ` [PATCH v5 04/15] help: add shell-path to --build-options emilyshaffer
2020-01-30 22:21     ` Martin Ågren
2020-01-24  3:34   ` [PATCH v5 05/15] bugreport: add uname info emilyshaffer
2020-01-24  3:34   ` [PATCH v5 06/15] bugreport: add compiler info emilyshaffer
2020-01-30 22:21     ` Martin Ågren
2020-02-04 22:51       ` Emily Shaffer
2020-02-05 19:47         ` Martin Ågren
2020-01-24  3:34   ` [PATCH v5 07/15] bugreport: add curl version emilyshaffer
2020-01-30 22:27     ` Martin Ågren
2020-02-04 22:54       ` Emily Shaffer
2020-01-24  3:34   ` [PATCH v5 08/15] bugreport: include user interactive shell emilyshaffer
2020-01-30 22:28     ` Martin Ågren
2020-02-04 23:16       ` Emily Shaffer
2020-02-05 20:06       ` Junio C Hamano
2020-02-05 20:14         ` Martin Ågren
2020-01-24  3:34   ` [PATCH v5 09/15] bugreport: generate config safelist based on docs emilyshaffer
2020-01-30 22:34     ` Martin Ågren
2020-02-05  0:44       ` Emily Shaffer
2020-02-05 19:53         ` Martin Ågren
2020-01-31 21:20     ` Martin Ågren
2020-02-05  0:30       ` Emily Shaffer
2020-02-05  0:52         ` Emily Shaffer
2020-01-24  3:34   ` [PATCH v5 10/15] bugreport: add config values from safelist emilyshaffer
2020-01-30 22:36     ` Martin Ågren
2020-02-05  1:34       ` Emily Shaffer
2020-01-31 21:25     ` Martin Ågren
2020-02-05  2:31       ` Emily Shaffer
2020-02-05 20:12         ` Martin Ågren
2020-01-24  3:34   ` [PATCH v5 11/15] bugreport: collect list of populated hooks emilyshaffer
2020-02-04 18:44     ` Junio C Hamano
2020-02-05  2:48       ` Emily Shaffer
2020-02-05  3:00         ` Emily Shaffer
2020-01-24  3:34   ` [PATCH v5 12/15] bugreport: count loose objects emilyshaffer
2020-02-04 18:48     ` Junio C Hamano
2020-02-05  2:50       ` Emily Shaffer
2020-01-24  3:34   ` [PATCH v5 13/15] bugreport: add packed object summary emilyshaffer
2020-02-04 19:00     ` Junio C Hamano
2020-02-05  3:15       ` Emily Shaffer
2020-02-04 19:03     ` Junio C Hamano
2020-02-05  3:09       ` Emily Shaffer
2020-01-24  3:34   ` [PATCH v5 14/15] bugreport: list contents of $OBJDIR/info emilyshaffer
2020-01-24  3:34   ` [PATCH v5 15/15] bugreport: summarize contents of alternates file emilyshaffer
2020-01-24  3:38   ` [PATCH v5 00/15] add git-bugreport tool Emily Shaffer
2020-01-28 23:04   ` Jonathan Tan
2020-01-28 23:26     ` Emily Shaffer
2020-01-30 22:15   ` Martin Ågren
2020-02-04  0:07     ` Emily Shaffer
2020-02-06  0:40   ` [PATCH v6 " Emily Shaffer
2020-02-06  0:40     ` [PATCH v6 01/15] help: move list_config_help to builtin/help Emily Shaffer
2020-02-06  1:35       ` Danh Doan
2020-02-13 22:58         ` Emily Shaffer
2020-02-13 23:07           ` Eric Sunshine
2020-02-13 23:24             ` Junio C Hamano
2020-02-13 23:29               ` Eric Sunshine
2020-02-14  1:20                 ` Emily Shaffer
2020-02-06  0:40     ` [PATCH v6 02/15] help: add shell-path to --build-options Emily Shaffer
2020-02-06  0:40     ` [PATCH v6 03/15] bugreport: add tool to generate debugging info Emily Shaffer
2020-02-07 14:18       ` SZEDER Gábor
2020-02-07 18:51         ` Junio C Hamano
2020-02-11 22:40           ` Emily Shaffer
2020-02-07 14:54       ` SZEDER Gábor
2020-02-12 18:06       ` Junio C Hamano
2020-02-12 22:36         ` Emily Shaffer
2020-02-06  0:40     ` [PATCH v6 04/15] bugreport: gather git version and build info Emily Shaffer
2020-02-06  0:40     ` [PATCH v6 05/15] bugreport: add uname info Emily Shaffer
2020-02-06  0:40     ` [PATCH v6 06/15] bugreport: add compiler info Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 07/15] bugreport: add git-remote-https version Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 08/15] bugreport: include user interactive shell Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 09/15] bugreport: generate config safelist based on docs Emily Shaffer
2020-02-07 15:30       ` SZEDER Gábor
2020-02-13 23:14         ` Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 10/15] bugreport: add config values from safelist Emily Shaffer
2020-02-07 14:47       ` SZEDER Gábor
2020-02-07 15:08         ` SZEDER Gábor
2020-02-07 16:24           ` Eric Sunshine
2020-02-07 16:51             ` Andreas Schwab
2020-02-13 22:02               ` Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 11/15] bugreport: collect list of populated hooks Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 12/15] bugreport: count loose objects Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 13/15] bugreport: add packed object summary Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
2020-02-06  0:41     ` [PATCH v6 15/15] bugreport: summarize contents of alternates file Emily Shaffer
2020-02-14  1:53     ` [PATCH v7 00/15] add git-bugreport tool Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 01/15] help: move list_config_help to builtin/help Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 02/15] help: add shell-path to --build-options Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 03/15] bugreport: add tool to generate debugging info Emily Shaffer
2020-02-14 17:25         ` Junio C Hamano
2020-02-15  1:57           ` Emily Shaffer
2020-02-15 18:24             ` Junio C Hamano
2020-02-18 23:46               ` Emily Shaffer
2020-02-18 23:56                 ` Emily Shaffer
2020-02-19 14:18         ` Johannes Schindelin
2020-02-19 16:55           ` Junio C Hamano
2020-02-19 21:52             ` Emily Shaffer
2020-02-19 22:09               ` Junio C Hamano
2020-02-14  1:53       ` [PATCH v7 04/15] bugreport: gather git version and build info Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 05/15] bugreport: add uname info Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 06/15] bugreport: add compiler info Emily Shaffer
2020-02-19 14:23         ` Johannes Schindelin
2020-02-19 22:45           ` Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 07/15] bugreport: add git-remote-https version Emily Shaffer
2020-02-19 14:28         ` Johannes Schindelin
2020-02-19 22:28           ` Emily Shaffer
2020-02-19 22:33             ` Junio C Hamano
2020-02-14  1:53       ` [PATCH v7 08/15] bugreport: include user interactive shell Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 09/15] bugreport: generate config safelist based on docs Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 10/15] bugreport: add config values from safelist Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 11/15] bugreport: collect list of populated hooks Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 12/15] bugreport: count loose objects Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 13/15] bugreport: add packed object summary Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 14/15] bugreport: list contents of $OBJDIR/info Emily Shaffer
2020-02-14 17:04         ` Junio C Hamano
2020-02-18 23:59           ` Emily Shaffer
2020-02-14  1:53       ` [PATCH v7 15/15] bugreport: summarize contents of alternates file Emily Shaffer
2020-02-14 17:32       ` [PATCH v7 00/15] add git-bugreport tool Junio C Hamano
2020-02-14 22:00         ` Emily Shaffer
2020-02-14 22:30           ` Junio C Hamano

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.io/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git