git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
From: "Jeff Hostetler via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: gitster@pobox.com, peff@peff.net, jrnieder@gmail.com,
	steadmon@google.com, avarab@gmail.com,
	Junio C Hamano <gitster@pobox.com>
Subject: [PATCH v2 0/7] trace2: load trace2 settings from system config
Date: Fri, 29 Mar 2019 10:04:24 -0700 (PDT)
Message-ID: <pull.169.v2.git.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.169.git.gitgitgadget@gmail.com>

Here is version 2. It addresses most the V1 comments WRT the system config
changes.

It also addresses the format and uniqueness of the SID as discussed in [1].
The SID now containes: the UTC date/time, part of SHA1 of the hostname, and
the PID and is formatted to make it safe for filenames.

It also contains (a somewhat unrelated platform-specific) commit to report
total process memory usage at exit. This is helpful when looking for
problematic commands that might have scaling problems.

[1] 
https://public-inbox.org/git/51e88650-8667-df1f-13ef-4537f2e70346@jeffhostetler.com/T/#m6b4e6f2b0374d5ba88de8d0350ce6bf51b28d7da


----------------------------------------------------------------------------

Teach git to load default Trace2 settings from the system config (usually
"/etc/gitconfig"). The existing GIT_TR2_* environment variables can be used
to override the new system defaults. It also includes a little startup
refactoring.

Note: I found interactive testing of this feature to be awkward on some
platforms because of the use of prefix- or runtime-prefix-relative locations
for the system configuration. It was easy to accidentally use an officially
installed version of git to set a system config variable in the official
system config directory; and then when testing with the test version of git,
that value would not be seen because it was looking for the system config
file in a different directory.

Jeff Hostetler (7):
  trace2: refactor setting process starting time
  trace2: add absolute elapsed time to start event
  trace2: find exec-dir before trace2 initialization
  trace2: use system config for default trace2 settings
  trace2: report peak memory usage of the process
  trace2: clarify UTC datetime formatting
  trace2: make SIDs more unique

 Documentation/technical/api-trace2.txt   |  66 +++++++++---
 Makefile                                 |   1 +
 common-main.c                            |   8 +-
 compat/mingw.c                           |   2 +
 compat/win32/trace2_win32_process_info.c |  50 ++++++++-
 t/t0210-trace2-normal.sh                 |  41 +++++++-
 t/t0211-trace2-perf.sh                   |  53 ++++++++--
 t/t0212-trace2-event.sh                  |  52 ++++++++-
 trace2.c                                 |  21 +++-
 trace2.h                                 |  43 ++++++--
 trace2/tr2_cfg.c                         |   7 +-
 trace2/tr2_dst.c                         |  24 ++---
 trace2/tr2_dst.h                         |   3 +-
 trace2/tr2_sid.c                         |  39 ++++++-
 trace2/tr2_sysenv.c                      | 128 +++++++++++++++++++++++
 trace2/tr2_sysenv.h                      |  36 +++++++
 trace2/tr2_tbuf.c                        |  20 +++-
 trace2/tr2_tbuf.h                        |   5 +-
 trace2/tr2_tgt.h                         |   1 +
 trace2/tr2_tgt_event.c                   |  53 +++++-----
 trace2/tr2_tgt_normal.c                  |  19 ++--
 trace2/tr2_tgt_perf.c                    |  23 ++--
 trace2/tr2_tls.c                         |  38 ++++---
 trace2/tr2_tls.h                         |   8 +-
 24 files changed, 604 insertions(+), 137 deletions(-)
 create mode 100644 trace2/tr2_sysenv.c
 create mode 100644 trace2/tr2_sysenv.h


base-commit: 041f5ea1cf987a4068ef5f39ba0a09be85952064
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-169%2Fjeffhostetler%2Fcore-tr2-startup-and-sysenv-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-169/jeffhostetler/core-tr2-startup-and-sysenv-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/169

Range-diff vs v1:

 1:  7f1a347322 = 1:  7f1a347322 trace2: refactor setting process starting time
 2:  c15f783c26 = 2:  c15f783c26 trace2: add absolute elapsed time to start event
 3:  d4023acde7 = 3:  d4023acde7 trace2: find exec-dir before trace2 initialization
 4:  7e0d4e20fb ! 4:  d048f3ffb8 trace2: use system config for default trace2 settings
     @@ -332,11 +332,16 @@
        *
      - * The environment variable "GIT_TR2_CONFIG_PARAMS" can be set to a
      - * list of patterns considered important.  For example:
     -+ * Use the TR2_SYSENV_CFG_PARAM setting to register a list of patterns
     -+ * configured important.  For example:
     -  *
     -  *    GIT_TR2_CONFIG_PARAMS="core.*,remote.*.url"
     +- *
     +- *    GIT_TR2_CONFIG_PARAMS="core.*,remote.*.url"
     ++ * Use the TR2_SYSENV_CFG_PARAM setting to register a comma-separated
     ++ * list of patterns configured important.  For example:
     ++ *     git config --system trace2.configParams 'core.*,remote.*.url'
     ++ * or:
     ++ *     GIT_TR2_CONFIG_PARAMS=core.*,remote.*.url"
        *
     +  * Note: this routine does a read-only iteration on the config data
     +  * (using read_early_config()), so it must not be called until enough
      
       diff --git a/trace2/tr2_cfg.c b/trace2/tr2_cfg.c
       --- a/trace2/tr2_cfg.c
     @@ -527,6 +532,9 @@
      +{
      +	int k;
      +
     ++	if (!starts_with(key, "trace2."))
     ++		return 0;
     ++
      +	for (k = 0; k < ARRAY_SIZE(tr2_sysenv_settings); k++) {
      +		if (!strcmp(key, tr2_sysenv_settings[k].git_config_name)) {
      +			free(tr2_sysenv_settings[k].value);
     @@ -673,7 +681,8 @@
      + * region details in the event target.
        */
      -#define TR2_ENVVAR_EVENT_NESTING "GIT_TR2_EVENT_NESTING"
     - static int tr2env_event_nesting_wanted = 2;
     +-static int tr2env_event_nesting_wanted = 2;
     ++static int tr2env_event_max_nesting_levels = 2;
       
       /*
      - * Set this environment variable to true to omit the <time>, <file>, and
     @@ -681,12 +690,14 @@
        * <line> fields from most events.
        */
      -#define TR2_ENVVAR_EVENT_BRIEF "GIT_TR2_EVENT_BRIEF"
     - static int tr2env_event_brief;
     +-static int tr2env_event_brief;
     ++static int tr2env_event_be_brief;
       
       static int fn_init(void)
     -@@
     + {
       	int want = tr2_dst_trace_want(&tr2dst_event);
     - 	int want_nesting;
     +-	int want_nesting;
     ++	int max_nesting;
       	int want_brief;
      -	char *nesting;
      -	char *brief;
     @@ -697,15 +708,74 @@
       		return want;
       
      -	nesting = getenv(TR2_ENVVAR_EVENT_NESTING);
     +-	if (nesting && ((want_nesting = atoi(nesting)) > 0))
     +-		tr2env_event_nesting_wanted = want_nesting;
      +	nesting = tr2_sysenv_get(TR2_SYSENV_EVENT_NESTING);
     - 	if (nesting && ((want_nesting = atoi(nesting)) > 0))
     - 		tr2env_event_nesting_wanted = want_nesting;
     ++	if (nesting && *nesting && ((max_nesting = atoi(nesting)) > 0))
     ++		tr2env_event_max_nesting_levels = max_nesting;
       
      -	brief = getenv(TR2_ENVVAR_EVENT_BRIEF);
     +-	if (brief && ((want_brief = atoi(brief)) > 0))
     +-		tr2env_event_brief = want_brief;
      +	brief = tr2_sysenv_get(TR2_SYSENV_EVENT_BRIEF);
     - 	if (brief && ((want_brief = atoi(brief)) > 0))
     - 		tr2env_event_brief = want_brief;
     ++	if (brief && *brief &&
     ++	    ((want_brief = git_parse_maybe_bool(brief)) != -1))
     ++		tr2env_event_be_brief = want_brief;
       
     + 	return want;
     + }
     +@@
     + 	/*
     + 	 * In brief mode, only emit <time> on these 2 event types.
     + 	 */
     +-	if (!tr2env_event_brief || !strcmp(event_name, "version") ||
     ++	if (!tr2env_event_be_brief || !strcmp(event_name, "version") ||
     + 	    !strcmp(event_name, "atexit")) {
     + 		tr2_tbuf_utc_time(&tb_now);
     + 		jw_object_string(jw, "time", tb_now.buf);
     + 	}
     + 
     +-	if (!tr2env_event_brief && file && *file) {
     ++	if (!tr2env_event_be_brief && file && *file) {
     + 		jw_object_string(jw, "file", file);
     + 		jw_object_intmax(jw, "line", line);
     + 	}
     +@@
     + {
     + 	const char *event_name = "region_enter";
     + 	struct tr2tls_thread_ctx *ctx = tr2tls_get_self();
     +-	if (ctx->nr_open_regions <= tr2env_event_nesting_wanted) {
     ++	if (ctx->nr_open_regions <= tr2env_event_max_nesting_levels) {
     + 		struct json_writer jw = JSON_WRITER_INIT;
     + 
     + 		jw_object_begin(&jw, 0);
     +@@
     + {
     + 	const char *event_name = "region_leave";
     + 	struct tr2tls_thread_ctx *ctx = tr2tls_get_self();
     +-	if (ctx->nr_open_regions <= tr2env_event_nesting_wanted) {
     ++	if (ctx->nr_open_regions <= tr2env_event_max_nesting_levels) {
     + 		struct json_writer jw = JSON_WRITER_INIT;
     + 		double t_rel = (double)us_elapsed_region / 1000000.0;
     + 
     +@@
     + {
     + 	const char *event_name = "data";
     + 	struct tr2tls_thread_ctx *ctx = tr2tls_get_self();
     +-	if (ctx->nr_open_regions <= tr2env_event_nesting_wanted) {
     ++	if (ctx->nr_open_regions <= tr2env_event_max_nesting_levels) {
     + 		struct json_writer jw = JSON_WRITER_INIT;
     + 		double t_abs = (double)us_elapsed_absolute / 1000000.0;
     + 		double t_rel = (double)us_elapsed_region / 1000000.0;
     +@@
     + {
     + 	const char *event_name = "data_json";
     + 	struct tr2tls_thread_ctx *ctx = tr2tls_get_self();
     +-	if (ctx->nr_open_regions <= tr2env_event_nesting_wanted) {
     ++	if (ctx->nr_open_regions <= tr2env_event_max_nesting_levels) {
     + 		struct json_writer jw = JSON_WRITER_INIT;
     + 		double t_abs = (double)us_elapsed_absolute / 1000000.0;
     + 		double t_rel = (double)us_elapsed_region / 1000000.0;
      
       diff --git a/trace2/tr2_tgt_normal.c b/trace2/tr2_tgt_normal.c
       --- a/trace2/tr2_tgt_normal.c
     @@ -730,9 +800,11 @@
        * Unit tests may want to use this to help with testing.
        */
      -#define TR2_ENVVAR_NORMAL_BRIEF "GIT_TR2_BRIEF"
     - static int tr2env_normal_brief;
     +-static int tr2env_normal_brief;
     ++static int tr2env_normal_be_brief;
       
       #define TR2FMT_NORMAL_FL_WIDTH (50)
     + 
      @@
       {
       	int want = tr2_dst_trace_want(&tr2dst_normal);
     @@ -747,7 +819,20 @@
      +	brief = tr2_sysenv_get(TR2_SYSENV_NORMAL_BRIEF);
       	if (brief && *brief &&
       	    ((want_brief = git_parse_maybe_bool(brief)) != -1))
     - 		tr2env_normal_brief = want_brief;
     +-		tr2env_normal_brief = want_brief;
     ++		tr2env_normal_be_brief = want_brief;
     + 
     + 	return want;
     + }
     +@@
     + {
     + 	strbuf_setlen(buf, 0);
     + 
     +-	if (!tr2env_normal_brief) {
     ++	if (!tr2env_normal_be_brief) {
     + 		struct tr2_tbuf tb_now;
     + 
     + 		tr2_tbuf_local_time(&tb_now);
      
       diff --git a/trace2/tr2_tgt_perf.c b/trace2/tr2_tgt_perf.c
       --- a/trace2/tr2_tgt_perf.c
     @@ -772,9 +857,11 @@
        * Unit tests may want to use this to help with testing.
        */
      -#define TR2_ENVVAR_PERF_BRIEF "GIT_TR2_PERF_BRIEF"
     - static int tr2env_perf_brief;
     +-static int tr2env_perf_brief;
     ++static int tr2env_perf_be_brief;
       
       #define TR2FMT_PERF_FL_WIDTH (50)
     + #define TR2FMT_PERF_MAX_EVENT_NAME (12)
      @@
       {
       	int want = tr2_dst_trace_want(&tr2dst_perf);
     @@ -791,4 +878,17 @@
      +	brief = tr2_sysenv_get(TR2_SYSENV_PERF_BRIEF);
       	if (brief && *brief &&
       	    ((want_brief = git_parse_maybe_bool(brief)) != -1))
     - 		tr2env_perf_brief = want_brief;
     +-		tr2env_perf_brief = want_brief;
     ++		tr2env_perf_be_brief = want_brief;
     + 
     + 	return want;
     + }
     +@@
     + 
     + 	strbuf_setlen(buf, 0);
     + 
     +-	if (!tr2env_perf_brief) {
     ++	if (!tr2env_perf_be_brief) {
     + 		struct tr2_tbuf tb_now;
     + 
     + 		tr2_tbuf_local_time(&tb_now);
 -:  ---------- > 5:  86d8fe28cf trace2: report peak memory usage of the process
 -:  ---------- > 6:  ae11cc4186 trace2: clarify UTC datetime formatting
 -:  ---------- > 7:  4352952677 trace2: make SIDs more unique

-- 
gitgitgadget

  parent reply index

Thread overview: 68+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-03-28 13:30 [PATCH 0/4] " Jeff Hostetler via GitGitGadget
2019-03-28 13:30 ` [PATCH 1/4] trace2: refactor setting process starting time Jeff Hostetler via GitGitGadget
2019-03-28 13:31 ` [PATCH 2/4] trace2: add absolute elapsed time to start event Jeff Hostetler via GitGitGadget
2019-03-28 13:31 ` [PATCH 3/4] trace2: find exec-dir before trace2 initialization Jeff Hostetler via GitGitGadget
2019-03-28 13:31 ` [PATCH 4/4] trace2: use system config for default trace2 settings Jeff Hostetler via GitGitGadget
2019-03-28 14:36   ` Ævar Arnfjörð Bjarmason
2019-03-28 18:50     ` Jeff Hostetler
2019-03-28 21:28   ` Josh Steadmon
2019-03-29 17:04 ` Jeff Hostetler via GitGitGadget [this message]
2019-03-29 17:04   ` [PATCH v2 1/7] trace2: refactor setting process starting time Jeff Hostetler via GitGitGadget
2019-03-29 17:04   ` [PATCH v2 2/7] trace2: add absolute elapsed time to start event Jeff Hostetler via GitGitGadget
2019-03-29 17:04   ` [PATCH v2 3/7] trace2: find exec-dir before trace2 initialization Jeff Hostetler via GitGitGadget
2019-03-29 17:04   ` [PATCH v2 4/7] trace2: use system config for default trace2 settings Jeff Hostetler via GitGitGadget
2019-04-01 21:00     ` Josh Steadmon
2019-04-01 21:06       ` Jeff Hostetler
2019-04-03  0:01       ` Jonathan Nieder
2019-04-03  0:00     ` Jonathan Nieder
2019-04-09 15:58       ` Jeff Hostetler
2019-03-29 17:04   ` [PATCH v2 5/7] trace2: report peak memory usage of the process Jeff Hostetler via GitGitGadget
2019-03-29 22:16     ` Ævar Arnfjörð Bjarmason
2019-04-01 21:05       ` Jeff Hostetler
2019-03-29 17:04   ` [PATCH v2 6/7] trace2: clarify UTC datetime formatting Jeff Hostetler via GitGitGadget
2019-03-29 17:04   ` [PATCH v2 7/7] trace2: make SIDs more unique Jeff Hostetler via GitGitGadget
2019-03-29 22:12     ` Ævar Arnfjörð Bjarmason
2019-04-01 21:16       ` Jeff Hostetler
2019-04-01 21:02   ` [PATCH v2 0/7] trace2: load trace2 settings from system config Josh Steadmon
2019-04-11 15:18   ` [PATCH v3 00/10] " Jeff Hostetler via GitGitGadget
2019-04-11 15:18     ` [PATCH v3 01/10] config: initialize opts structure in repo_read_config() Jeff Hostetler via GitGitGadget
2019-04-12  3:52       ` Jonathan Nieder
2019-04-15 14:34         ` Johannes Schindelin
2019-04-11 15:18     ` [PATCH v3 02/10] trace2: refactor setting process starting time Jeff Hostetler via GitGitGadget
2019-04-11 15:18     ` [PATCH v3 03/10] trace2: add absolute elapsed time to start event Jeff Hostetler via GitGitGadget
2019-04-11 15:18     ` [PATCH v3 04/10] trace2: find exec-dir before trace2 initialization Jeff Hostetler via GitGitGadget
2019-04-11 15:18     ` [PATCH v3 05/10] config: add read_very_early_config() Jeff Hostetler via GitGitGadget
2019-04-11 15:18     ` [PATCH v3 06/10] trace2: use system/global config for default trace2 settings Jeff Hostetler via GitGitGadget
2019-04-11 15:18     ` [PATCH v3 07/10] trace2: report peak memory usage of the process Jeff Hostetler via GitGitGadget
2019-04-11 15:18     ` [PATCH v3 08/10] trace2: clarify UTC datetime formatting Jeff Hostetler via GitGitGadget
2019-04-11 15:18     ` [PATCH v3 10/10] trace2: update docs to describe system/global config settings Jeff Hostetler via GitGitGadget
2019-04-11 15:18     ` [PATCH v3 09/10] trace2: make SIDs more unique Jeff Hostetler via GitGitGadget
2019-04-12  2:29     ` [PATCH v3 00/10] trace2: load trace2 settings from system config Junio C Hamano
2019-04-12 13:47       ` Jeff Hostetler
2019-04-15 20:39     ` [PATCH v4 " Jeff Hostetler via GitGitGadget
2019-04-15 20:39       ` [PATCH v4 01/10] config: initialize opts structure in repo_read_config() Jeff Hostetler via GitGitGadget
2019-04-15 20:39       ` [PATCH v4 02/10] trace2: refactor setting process starting time Jeff Hostetler via GitGitGadget
2019-04-15 20:39       ` [PATCH v4 03/10] trace2: add absolute elapsed time to start event Jeff Hostetler via GitGitGadget
2019-04-15 20:39       ` [PATCH v4 04/10] trace2: find exec-dir before trace2 initialization Jeff Hostetler via GitGitGadget
2019-04-15 20:39       ` [PATCH v4 05/10] config: add read_very_early_config() Jeff Hostetler via GitGitGadget
2019-04-15 20:39       ` [PATCH v4 06/10] trace2: use system/global config for default trace2 settings Jeff Hostetler via GitGitGadget
2019-04-27 13:43         ` SZEDER Gábor
2019-04-29 19:03           ` Jeff Hostetler
2019-04-15 20:39       ` [PATCH v4 07/10] trace2: report peak memory usage of the process Jeff Hostetler via GitGitGadget
2019-04-15 20:39       ` [PATCH v4 08/10] trace2: clarify UTC datetime formatting Jeff Hostetler via GitGitGadget
2019-04-15 20:39       ` [PATCH v4 09/10] trace2: make SIDs more unique Jeff Hostetler via GitGitGadget
2019-04-15 20:39       ` [PATCH v4 10/10] trace2: update docs to describe system/global config settings Jeff Hostetler via GitGitGadget
2019-04-29 20:14       ` [PATCH v5 00/11] trace2: load trace2 settings from system config Jeff Hostetler via GitGitGadget
2019-04-29 20:14         ` [PATCH v5 02/11] trace2: refactor setting process starting time Jeff Hostetler via GitGitGadget
2019-04-29 20:14         ` [PATCH v5 01/11] config: initialize opts structure in repo_read_config() Jeff Hostetler via GitGitGadget
2019-04-29 20:14         ` [PATCH v5 03/11] trace2: add absolute elapsed time to start event Jeff Hostetler via GitGitGadget
2019-04-29 20:14         ` [PATCH v5 04/11] trace2: find exec-dir before trace2 initialization Jeff Hostetler via GitGitGadget
2019-04-29 20:14         ` [PATCH v5 05/11] config: add read_very_early_config() Jeff Hostetler via GitGitGadget
2019-04-29 20:14         ` [PATCH v5 06/11] trace2: use system/global config for default trace2 settings Jeff Hostetler via GitGitGadget
2019-04-29 20:14         ` [PATCH v5 07/11] trace2: report peak memory usage of the process Jeff Hostetler via GitGitGadget
2019-04-29 20:14         ` [PATCH v5 09/11] trace2: make SIDs more unique Jeff Hostetler via GitGitGadget
2019-04-29 20:14         ` [PATCH v5 08/11] trace2: clarify UTC datetime formatting Jeff Hostetler via GitGitGadget
2019-04-29 20:14         ` [PATCH v5 10/11] trace2: update docs to describe system/global config settings Jeff Hostetler via GitGitGadget
2019-04-29 20:14         ` [PATCH v5 11/11] trace2: fixup access problem on /etc/gitconfig in read_very_early_config Jeff Hostetler via GitGitGadget
2019-04-29 20:21         ` [PATCH v5 00/11] trace2: load trace2 settings from system config Jeff Hostetler
2019-05-07  1:18           ` Junio C Hamano

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=pull.169.v2.git.gitgitgadget@gmail.com \
    --to=gitgitgadget@gmail.com \
    --cc=avarab@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jrnieder@gmail.com \
    --cc=peff@peff.net \
    --cc=steadmon@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox