What's cooking in git.git (Jan 2020, #04; Wed, 22)

What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed with
'-' are only in 'pu' (proposed updates) while commits prefixed with
'+' are in 'next'.  The ones marked with '.' do not appear in any of
the integration branches, but I am still holding onto them.

Git 2.25 is out.  The tip of 'next' has been rewound and a handful
of topics have been rebased to avoid the premature merge of
ra/rebase-i-more-options which has been reverted.

You can find the changes described here in the integration branches
of the repositories listed at

    http://git-blame.blogspot.com/p/git-public-repositories.html

--------------------------------------------------
[Graduated to "master"]

* do/gitweb-typofix-in-comments (2020-01-04) 1 commit
  (merged to 'next' on 2020-01-06 at 66ce6539c4)
 + gitweb: fix a couple spelling errors in comments

 Typofix.


* ds/graph-assert-fix (2020-01-08) 2 commits
  (merged to 'next' on 2020-01-08 at 4b896fb9b5)
 + graph: fix lack of color in horizontal lines
 + graph: drop assert() for merge with two collapsing parents
 (this branch is used by ds/graph-horizontal-edges.)

 Since recent updates to the log graph rendering code, drawing
 certain merges started triggering an assert on a condition that
 would no longer hold true, which has been corrected.


* jb/doc-multi-pack-idx-fix (2020-01-04) 1 commit
  (merged to 'next' on 2020-01-06 at f19f7d1016)
 + multi-pack-index: correct configuration in documentation

 Typofix.


* js/mingw-loosen-overstrict-tree-entry-checks (2020-01-10) 1 commit
  (merged to 'next' on 2020-01-10 at f43f0fe74b)
 + mingw: safeguard better against backslashes in file names

 Further tweak to a "no backslash in indexed paths" for Windows port
 we applied earlier.


* ma/config-advice-markup-fix (2020-01-08) 1 commit
  (merged to 'next' on 2020-01-09 at 1c4b540795)
 + config/advice.txt: fix description list separator

 Documentation markup fix.


* pm/am-in-body-header-doc-update (2020-01-04) 1 commit
  (merged to 'next' on 2020-01-06 at 73b0a3a49c)
 + am: document that Date: can appear as an in-body header

 Doc update.


* tm/doc-submodule-absorb-fix (2020-01-06) 1 commit
  (merged to 'next' on 2020-01-07 at cee89422db)
 + doc: submodule: fix typo for command absorbgitdirs

 Typofix.

--------------------------------------------------
[New Topics]

* en/simplify-check-updates-in-unpack-trees (2020-01-07) 1 commit
  (merged to 'next' on 2020-01-15 at 586c055b69)
 + unpack-trees: exit check_updates() early if updates are not wanted

 Originally merged to 'next' on 2020-01-09

 Code simplification.

 Will merge to 'master'.


* en/string-list-can-be-custom-sorted (2020-01-07) 1 commit
  (merged to 'next' on 2020-01-15 at 2afe9536e6)
 + string-list: note in docs that callers can specify sorting function

 Originally merged to 'next' on 2020-01-09

 API-doc update.

 Will merge to 'master'.


* am/checkout-file-and-ref-ref-ambiguity (2020-01-07) 2 commits
 - checkout: don't revert file on ambiguous tracking branches
 - parse_branchname_arg(): extract part as new function

 "git checkout X" did not correctly fail when X is not a local
 branch but could name more than one remote-tracking branches
 (i.e. to be dwimmed as the starting point to create a corresponding
 local branch), which has been corrected.

 Will merge to 'next'.


* am/update-pathspec-f-f-tests (2020-01-15) 2 commits
 - t: directly test parse_pathspec_file()
 - t: fix quotes tests for --pathspec-from-file

 Test updates.

 Will merge to 'next'.


* bc/run-command-nullness-after-free-fix (2020-01-07) 1 commit
  (merged to 'next' on 2020-01-15 at 56b3148fee)
 + run-command: avoid undefined behavior in exists_in_PATH

 Originally merged to 'next' on 2020-01-09

 C pedantry ;-) fix.

 Will merge to 'master'.


* kw/fsmonitor-watchman-racefix (2020-01-13) 4 commits
 - fsmonitor: update documentation for hook version and watchman hooks
 - fsmonitor: add fsmonitor hook scripts for version 2
 - fsmonitor: handle version 2 of the hooks that will use opaque token
 - fsmonitor: change last update timestamp on the index_state to opaque token

 A new version of fsmonitor-watchman hook has been introduced, to
 avoid races.

 Will merge to 'next'.


* es/unpack-trees-oob-fix (2020-01-08) 1 commit
  (merged to 'next' on 2020-01-15 at 832ecf4366)
 + unpack-trees: watch for out-of-range index position

 Originally merged to 'next' on 2020-01-09

 The code that tries to skip over the entries for the paths in a
 single directory using the cache-tree was not careful enough
 against corrupt index file.

 Will merge to 'master'.


* hw/advice-add-nothing (2020-01-15) 1 commit
 - add: use advise function to display hints

 Two help messages given when "git add" notices the user gave it
 nothing to add have been updated to use advise() API.

 Will merge to 'next'.


* hw/tutorial-favor-switch-over-checkout (2020-01-08) 1 commit
  (merged to 'next' on 2020-01-15 at 25e4fca9ec)
 + doc/gitcore-tutorial: fix prose to match example command

 Originally merged to 'next' on 2020-01-09

 Complete an update to tutorial that encourages "git switch" over
 "git checkout" that was done only half-way.

 Will merge to 'master'.


* jk/no-flush-upon-disconnecting-slrpc-transport (2020-01-08) 1 commit
  (merged to 'next' on 2020-01-15 at 5014feacb0)
 + transport: don't flush when disconnecting stateless-rpc helper

 Originally merged to 'next' on 2020-01-09

 Reduce unnecessary round-trip when running "ls-remote" over the
 stateless RPC mechanism.

 Will merge to 'master'.


* nd/switch-and-restore (2020-01-08) 1 commit
  (merged to 'next' on 2020-01-15 at ffd0b1e54e)
 + restore: invalidate cache-tree when removing entries with --staged

 Originally merged to 'next' on 2020-01-09

 "git restore --staged" did not correctly update the cache-tree
 structure, resulting in bogus trees to be written afterwards, which
 has been corrected.

 Will merge to 'master'.


* ds/graph-horizontal-edges (2020-01-15) 2 commits
 - graph: fix collapse of multiple edges
 - graph: add test to demonstrate horizontal line bug

 Rendering by "git log --graph" of ancestry lines leading to a merge
 commit were made suboptimal to waste vertical space a bit with a
 recent update, which has been corrected.

 Will merge to 'next'.


* ds/sparse-cone (2020-01-10) 1 commit
 - unpack-trees: correctly compute result count

 The code recently added in this release to move to the entry beyond
 the ones in the same directory in the index in the sparse-cone mode
 did not count the number of entries to skip over incorrectly, which
 has been corrected.

 Will merge to 'next'.


* km/submodule-add-errmsg (2020-01-15) 1 commit
 - submodule add: show 'add --dry-run' stderr when aborting

 Improve error message generation for "git submodule add".

 Will merge to 'next'.


* en/fill-directory-fixes-more (2020-01-16) 4 commits
 - dir: point treat_leading_path() warning to the right place
 - dir: restructure in a way to avoid passing around a struct dirent
 - dir: treat_leading_path() and read_directory_recursive(), round 2
 - clean: demonstrate a bug with pathspecs

 Corner case bugs in "git clean" that stems from a (necessarily for
 performance reasons) awkward calling convention in the directory
 enumeration API has been corrected.

 Will merge to 'next'.


* es/fetch-show-failed-submodules-atend (2020-01-17) 1 commit
 - fetch: emphasize failure during submodule fetch

 A fetch that is told to recursively fetch updates in submodules
 inevitably produces reams of output, and it becomes hard to spot
 error messages.  The command has been taught to enumerate
 submodules that had errors at the end of the operation.

 Will merge to 'next'.


* jk/asan-build-fix (2020-01-16) 1 commit
 - Makefile: use compat regex with SANITIZE=address

 Work around test breakages caused by custom regex engine used in
 libasan, when address sanitizer is used with more recent versions
 of gcc and clang.

 Will merge to 'next'.


* jk/test-fixes (2020-01-16) 2 commits
 - t7800: don't rely on reuse_worktree_file()
 - t4018: drop "debugging" cat from hunk-header tests

 Test fixes.

 Will merge to 'next'.


* js/builtin-add-i-cmds (2020-01-16) 2 commits
 - built-in add -i: accept open-ended ranges again
 - built-in add -i: do not try to `patch`/`diff` an empty list of files

 Minor bugfixes to "git add -i" that has recently been rewritten in C.

 Will merge to 'next'.


* rt/submodule-i18n (2020-01-16) 1 commit
 - submodule.c: mark more strings for translation

 Comments update.

 Will merge to 'next'.


* am/pathspec-f-f-more (2020-01-21) 8 commits
 - stash push: support the --pathspec-from-file option
 - stash: eliminate crude option parsing
 - doc: stash: synchronize <pathspec> description
 - doc: stash: document more options
 - doc: stash: split options from description (2)
 - doc: stash: split options from description (1)
 - rm: support the --pathspec-from-file option
 - doc: rm: synchronize <pathspec> description

 "git rm" and "git stash" learns the new "--pathspec-from-file"
 option.


* bc/actualmente (2020-01-21) 1 commit
 - docs: use "currently" for the present time

 Doc grammo fix.

 Will merge to 'next'.


* bc/author-committer-doc (2020-01-22) 3 commits
 - doc: provide guidance on user.name format
 - docs: expand on possible and recommended user config options
 - doc: move author and committer information to git-commit(1)

 Clarify documentation on committer/author identities.

 Will merge to 'next'.


* bc/misconception-doc (2020-01-22) 2 commits
 - docs: mention when increasing http.postBuffer is valuable
 - doc: dissuade users from trying to ignore tracked files

 Doc updates.

 Will merge to 'next'.


* ds/refmap-doc (2020-01-21) 1 commit
 - fetch: document and test --refmap=""

 "git fetch --refmap=" option has got a better documentation.

 Will merge to 'next'.


* js/rebase-i-with-colliding-hash (2020-01-21) 3 commits
 - rebase -i: also avoid SHA-1 collisions with missingCommitsCheck
 - rebase -i: re-fix short SHA-1 collision
 - parse_insn_line(): improve error message when parsing failed


* lh/bool-to-type-bool (2020-01-21) 1 commit
 - templates: fix deprecated type option `--bool`

 Replace "git config --bool" calls with "git config --type=bool" in
 sample templates.

 Will merge to 'next'.


* pb/recurse-submodule-in-worktree-fix (2020-01-22) 4 commits
 - submodule.c: use get_git_dir() instead of get_git_common_dir()
 - t2405: clarify test descriptions and simplify test
 - t2405: use git -C and test_commit -C instead of subshells
 - t7410: rename to t2405-worktree-submodule.sh

 The "--recurse-submodules" option of various subcommands did not
 work well when run in an alternate worktree, which has been
 corrected.

 Will merge to 'next'.


* ss/t6025-modernize (2020-01-21) 2 commits
 - t6025: use helpers to replace test -f <path>
 - t6025: modernize style

 Test style updates.

 Will merge to 'next'.

--------------------------------------------------
[Stalled]

* ag/edit-todo-drop-check (2019-12-06) 2 commits
 - rebase-interactive: warn if commit is dropped with `rebase --edit-todo'
 - sequencer: move check_todo_list_from_file() to rebase-interactive.c

 Allow the rebase.missingCommitsCheck configuration to kick in when
 "rebase --edit-todo" and "rebase --continue" restarts the procedure.

 Broken.
 cf. <64aa4049-ee35-df4c-1e6c-80707f4f9070@gmail.com>


* es/pathspec-f-f-grep (2020-01-13) 1 commit
 - grep: support the --pathspec-from-file option

 "git grep" learned the "--pathspec-from-file" command line
 option.

 Getting tired of waiting for review responses.  Will discard.
 cf. <20191204203911.237056-1-emilyshaffer@google.com>


* at/rebase-fork-point-regression-fix (2019-12-09) 1 commit
 - rebase: fix --fork-point with short refname

 The "--fork-point" mode of "git rebase" regressed when the command
 was rewritten in C back in 2.20 era, which has been corrected.

 Was waiting for discussion to settle.
 cf. <CAPig+cQ-3Ds41hr91fRo_GvuFMTP7zNVJtaSqi-Yccq4Pk-8Qg@mail.gmail.com>


* ma/config-bool-valex (2019-11-14) 8 commits
 - builtin/config: die if "value_regex" doesn't canonicalize as boolean
 - builtin/config: warn if "value_regex" doesn't canonicalize as boolean
 - builtin/config: canonicalize "value_regex" with `--type=bool-or-int`
 - builtin/config: canonicalize "value_regex" with `--type=bool`
 - builtin/config: collect "value_regexp" data in a struct
 - builtin/config: extract `handle_value_regex()` from `get_value()`
 - t1300: modernize part of script
 - config: make `git_parse_maybe_bool_text()` public

 "git config" can be told to affect the existing entries that
 "match" the given value via its value_regex argument.  It learned
 to normalize the value set in the configuration and the value given
 from the command line before computing they "match", e.g. "true" in
 the configuration file can now match with "yes" given from the
 command line.

 Needs a bit more work?
 cf. <CAN0heSrtwi9V607vBX9PMSfNLQ8iGcno6_iGuR4Fs8ndGxqh8A@mail.gmail.com>


* ds/fsmonitor-testing (2019-12-09) 8 commits
 - test-lib: clear watchman watches at test completion
 - t7519: disable external GIT_TEST_FSMONITOR variable
 - t7063: disable fsmonitor with status cache
 - tests: disable fsmonitor in submodule tests
 - t3030-merge-recursive.sh: disable fsmonitor when tweaking GIT_WORK_TREE
 - t1301-shared-repo.sh: disable FSMONITOR
 - fsmonitor: do not output to stderr for tests
 - fsmonitor: disable in a bare repo

 Updates around testing fsmoitor integration.

 cf. <pull.466.v2.git.1575907804.gitgitgadget@gmail.com>


* vn/reset-deleted-ita (2019-07-26) 1 commit
 - reset: unstage empty deleted ita files

 "git reset HEAD [<pathspec>]" did not reset an empty file that was
 added with the intent-to-add bit.

 Expecting a reroll.


* jn/unknown-index-extensions (2018-11-21) 2 commits
 - index: offer advice for unknown index extensions
 - index: do not warn about unrecognized extensions

 A bit too alarming warning given when unknown index extensions
 exist is getting revamped.

 Expecting a reroll.


* jc/format-patch-delay-message-id (2019-04-05) 1 commit
 - format-patch: move message-id and related headers to the end

 The location "git format-patch --thread" adds the Message-Id:
 header in the series of header fields has been moved down, which
 may help working around a suspected bug in GMail MSA, reported at
 <CAHk-=whP1stFZNAaJiMi5eZ9rj0MRt20Y_yHVczZPH+O01d+sA@mail.gmail.com>

 Waiting for feedback to see if it truly helps.
 Needs tests.


* js/protocol-advertise-multi (2018-12-28) 1 commit
 - protocol: advertise multiple supported versions

 The transport layer has been updated so that the protocol version
 used can be negotiated between the parties, by the initiator
 listing the protocol versions it is willing to talk, and the other
 side choosing from one of them.

 Expecting a reroll.
 cf. <CANq=j3u-zdb_FvNJGPCmygNMScseav63GhVvBX3NcVS4f7TejA@mail.gmail.com>


* mk/use-size-t-in-zlib (2018-10-15) 1 commit
 - zlib.c: use size_t for size

 The wrapper to call into zlib followed our long tradition to use
 "unsigned long" for sizes of regions in memory, which have been
 updated to use "size_t".

--------------------------------------------------
[Cooking]

* mt/threaded-grep-in-object-store (2020-01-17) 12 commits
 - grep: use no. of cores as the default no. of threads
 - grep: move driver pre-load out of critical section
 - grep: re-enable threads in non-worktree case
 - grep: protect packed_git [re-]initialization
 - grep: allow submodule functions to run in parallel
 - submodule-config: add skip_if_read option to repo_read_gitmodules()
 - grep: replace grep_read_mutex by internal obj read lock
 - object-store: allow threaded access to object reading
 - replace-object: make replace operations thread-safe
 - grep: fix racy calls in grep_objects()
 - grep: fix race conditions at grep_submodule()
 - grep: fix race conditions on userdiff calls

 Traditionally, we avoided threaded grep while searching in objects
 (as opposed to files in the working tree) as accesses to the object
 layer is not thread-safe.  This limitation is getting lifted.


* hi/indent-text-with-tabs-in-editorconfig (2020-01-06) 1 commit
 - editorconfig: indent text files with tabs

 Tell .editorconfig that in this project, *.txt files are indented
 with tabs.

 Will merge to 'next'.


* jn/pretend-object-doc (2020-01-06) 1 commit
 - sha1-file: document how to use pretend_object_file

 Warn programmers about pretend_object_file() that allows the code
 to tentatively use in-core objects.


* en/unpack-trees-check-updates-simplify (2020-01-04) 1 commit
 - unpack-trees: exit check_updates() early if updates are not wanted

 Code simplification.

 Will merge to 'next'.


* dl/merge-autostash (2020-01-13) 17 commits
 - pull: pass --autostash to merge
 - t5520: make test_pull_autostash() accept expect_parent_num
 - merge: teach --autostash option
 - sequencer: unlink autostash in apply_autostash()
 - sequencer: extract perform_autostash() from rebase
 - rebase: generify create_autostash()
 - rebase: extract create_autostash()
 - reset: extract reset_head() from rebase
 - rebase: generify reset_head()
 - rebase: use apply_autostash() from sequencer.c
 - sequencer: make apply_rebase() accept a path
 - rebase: use read_oneliner()
 - sequencer: make read_oneliner() extern
 - sequencer: configurably warn on non-existent files
 - sequencer: use file strbuf for read_oneliner()
 - t7600: use test_write_lines()
 - Makefile: alphabetically sort += lists

 "git merge" learns the "--autostash" option.

 What's the status of this one?  Are people happy with the shape of
 the code?


* dl/test-must-fail-fixes-2 (2020-01-07) 16 commits
 - t4124: only mark git command with test_must_fail
 - t3507: use test_path_is_missing()
 - t3507: fix indentation
 - t3504: do check for conflict marker after failed cherry-pick
 - t3419: stop losing return code of git command
 - t3415: increase granularity of test_auto_{fixup,squash}()
 - t3415: stop losing return codes of git commands
 - t3310: extract common notes_merge_files_gone()
 - t3030: use test_path_is_missing()
 - t2018: replace "sha" with "oid"
 - t2018: don't lose return code of git commands
 - t2018: teach do_checkout() to accept `!` arg
 - t2018: use test_expect_code for failing git commands
 - t2018: improve style of if-statement
 - t2018: add space between function name and ()
 - t2018: remove trailing space from test description

 Test updates.

 Will merge to 'next'.


* jn/promote-proto2-to-default (2020-01-15) 5 commits
 - fetch: default to protocol version 2
 - protocol test: let protocol.version override GIT_TEST_PROTOCOL_VERSION
 - test: request GIT_TEST_PROTOCOL_VERSION=0 when appropriate
 - config doc: protocol.version is not experimental
 - fetch test: use more robust test for filtered objects
 (this branch uses jn/test-lint-one-shot-export-to-shell-function.)

 The transport protocol version 2 becomes the default one.

 Will merge to 'next'.


* am/test-pathspec-f-f-error-cases (2020-01-15) 1 commit
 - t: add tests for error conditions with --pathspec-from-file

 More tests.

 Will merge to 'next'.


* jt/sha1-file-remove-oi-skip-cached (2020-01-02) 1 commit
  (merged to 'next' on 2020-01-15 at 4feaff54f3)
 + sha1-file: remove OBJECT_INFO_SKIP_CACHED

 Originally merged to 'next' on 2020-01-04

 has_object_file() said "no" given an object registered to the
 system via pretend_object_file(), making it inconsistent with
 read_object_file(), causing lazy fetch to attempt fetching an
 empty tree from promisor remotes.

 Will merge to 'master'.


* hw/commit-advise-while-rejecting (2019-12-19) 1 commit
  (merged to 'next' on 2020-01-15 at 4f16e5a3b6)
 + commit: honor advice.statusHints when rejecting an empty commit

 Originally merged to 'next' on 2019-12-30

 "git commit" gives output similar to "git status" when there is
 nothing to commit, but without honoring the advise.statusHints
 configuration variable, which has been corrected.

 Will merge to 'master'.


* yz/p4-py3 (2020-01-15) 14 commits
 - ci: also run linux-gcc pipeline with python3.5 environment
 - git-p4: use python3's input() everywhere
 - git-p4: simplify regex pattern generation for parsing diff-tree
 - git-p4: use dict.items() iteration for python3 compatibility
 - git-p4: use functools.reduce instead of reduce
 - git-p4: fix freezing while waiting for fast-import progress
 - git-p4: use marshal format version 2 when sending to p4
 - git-p4: open .gitp4-usercache.txt in text mode
 - git-p4: convert path to unicode before processing them
 - git-p4: encode/decode communication with git for python3
 - git-p4: encode/decode communication with p4 for python3
 - git-p4: remove string type aliasing
 - git-p4: change the expansion test from basestring to list
 - git-p4: make python2.7 the oldest supported version

 Update "git p4" to work with Python 3.

 Will merge to 'next'.


* hi/gpg-mintrustlevel (2020-01-15) 1 commit
 - gpg-interface: add minTrustLevel as a configuration option

 gpg.minTrustLevel configuration variable has been introduced to
 tell various signature verification codepaths the required minimum
 trust level.

 Will merge to 'next'.


* sg/completion-worktree (2020-01-15) 6 commits
 - completion: list paths and refs for 'git worktree add'
 - completion: list existing working trees for 'git worktree' subcommands
 - completion: simplify completing 'git worktree' subcommands and options
 - completion: return the index of found word from __git_find_on_cmdline()
 - completion: clean up the __git_find_on_cmdline() helper function
 - t9902-completion: add tests for the __git_find_on_cmdline() helper

 The command line completion (in contrib/) learned to complete
 subcommands and arguments to "git worktree".

 Will merge to 'next'.


* dl/credential-netrc (2019-12-20) 2 commits
  (merged to 'next' on 2020-01-15 at 768fa1c364)
 + contrib/credential/netrc: work outside a repo
 + contrib/credential/netrc: make PERL_PATH configurable

 Originally merged to 'next' on 2019-12-25

 Sample credential helper for using .netrc has been updated to work
 out of the box.

 Will merge to 'master'.


* dl/test-must-fail-fixes (2019-12-20) 15 commits
 - t1507: inline full_name()
 - t1507: run commands within test_expect_success
 - t1507: stop losing return codes of git commands
 - t1501: remove use of `test_might_fail cp`
 - t1409: use test_path_is_missing()
 - t1409: let sed open its own input file
 - t1307: reorder `nongit test_must_fail`
 - t1306: convert `test_might_fail rm` to `rm -f`
 - t0020: use ! check_packed_refs_marked
 - t0020: don't use `test_must_fail has_cr`
 - t0003: don't use `test_must_fail attr_check`
 - t0003: use test_must_be_empty()
 - t0003: use named parameters in attr_check()
 - t0000: replace test_must_fail with run_sub_test_lib_test_err()
 - t/lib-git-p4: use test_path_is_missing()

 Test clean-up.

 Will merge to 'next'.


* en/rebase-backend (2020-01-17) 19 commits
 - rebase: change the default backend from "am" to "merge"
 - rebase: make the backend configurable via config setting
 - rebase tests: repeat some tests using the merge backend instead of am
 - rebase tests: mark tests specific to the am-backend with --am
 - rebase: drop '-i' from the reflog for interactive-based rebases
 - git-prompt: change the prompt for interactive-based rebases
 - rebase: add an --am option
 - rebase: move incompatibility checks between backend options a bit earlier
 - git-rebase.txt: add more details about behavioral differences of backends
 - rebase: allow more types of rebases to fast-forward
 - t3432: make these tests work with either am or merge backends
 - rebase: fix handling of restrict_revision
 - rebase: make sure to pass along the quiet flag to the sequencer
 - rebase, sequencer: remove the broken GIT_QUIET handling
 - t3406: simplify an already simple test
 - rebase (interactive-backend): fix handling of commits that become empty
 - rebase (interactive-backend): make --keep-empty the default
 - t3404: directly test the behavior of interest
 - git-rebase.txt: update description of --allow-empty-message

 "git rebase" has learned to use the sequencer backend by default,
 while allowing "--am" option to go back to the traditional "am"
 backend.


* bc/hash-independent-tests-part-7 (2020-01-15) 20 commits
 - t5604: make hash independent
 - t5601: switch into repository to hash object
 - t5562: use $ZERO_OID
 - t5540: make hash size independent
 - t5537: make hash size independent
 - t5530: compute results based on object length
 - t5512: abstract away SHA-1-specific constants
 - t5510: make hash size independent
 - t5504: make hash algorithm independent
 - t5324: make hash size independent
 - t5319: make test work with SHA-256
 - t5319: change invalid offset for SHA-256 compatibility
 - t5318: update for SHA-256
 - t4300: abstract away SHA-1-specific constants
 - t4204: make hash size independent
 - t4202: abstract away SHA-1-specific constants
 - t4200: make hash size independent
 - t4134: compute appropriate length constant
 - t4066: compute index line in diffs
 - t4054: make hash-size independent

 Preparation of test scripts for the day when the object names will
 use SHA-256 continues.

 Will merge to 'next'.


* jn/test-lint-one-shot-export-to-shell-function (2020-01-15) 3 commits
 - fetch test: mark test of "skipping" haves as v0-only
 - t/check-non-portable-shell: detect "FOO= shell_func", too
 - fetch test: avoid use of "VAR= cmd" with a shell function
 (this branch is used by jn/promote-proto2-to-default.)

 The test-lint machinery knew to check "VAR=VAL shell_function"
 construct, but did not check "VAR= shell_funciton", which has been
 corrected.

 Will merge to 'next'.


* js/add-p-leftover-bits (2020-01-15) 10 commits
 - ci: include the built-in `git add -i` in the `linux-gcc` job
 - built-in add -p: handle Escape sequences more efficiently
 - built-in add -p: handle Escape sequences in interactive.singlekey mode
 - built-in add -p: respect the `interactive.singlekey` config setting
 - terminal: add a new function to read a single keystroke
 - terminal: accommodate Git for Windows' default terminal
 - terminal: make the code of disable_echo() reusable
 - built-in add -p: handle diff.algorithm
 - built-in add -p: support interactive.diffFilter
 - t3701: adjust difffilter test
 (this branch uses js/patch-mode-in-others-in-c.)

 The final leg of rewriting "add -i/-p" in C.

 Will merge to 'next'.


* pw/advise-rebase-skip (2019-12-06) 9 commits
 - rebase -i: leave CHERRY_PICK_HEAD when there are conflicts
 - rebase: fix advice when a fixup creates an empty commit
 - commit: give correct advice for empty commit during a rebase
 - commit: encapsulate determine_whence() for sequencer
 - commit: use enum value for multiple cherry-picks
 - sequencer: write CHERRY_PICK_HEAD for reword and edit
 - cherry-pick: check commit error messages
 - cherry-pick: add test for `--skip` advice in `git commit`
 - t3404: use test_cmp_rev

 The mechanism to prevent "git commit" from making an empty commit
 or amending during an interrupted cherry-pick was broken during the
 rewrite of "git rebase" in C, which has been corrected.

 What's the status of this one?
 The tip two are still RFC.


* js/patch-mode-in-others-in-c (2019-12-21) 7 commits
 - commit --interactive: make it work with the built-in `add -i`
 - built-in add -p: implement the "worktree" patch modes
 - built-in add -p: implement the "checkout" patch modes
 - built-in stash: use the built-in `git add -p` if so configured
 - legacy stash -p: respect the add.interactive.usebuiltin setting
 - built-in add -p: implement the "stash" and "reset" patch modes
 - built-in add -p: prepare for patch modes other than "stage"
 (this branch is used by js/add-p-leftover-bits.)

 The effort to move "git-add--interactive" to C continues.

 Will merge to 'next'.


* jk/packfile-reuse-cleanup (2019-10-23) 9 commits
 - pack-objects: improve partial packfile reuse
 - builtin/pack-objects: introduce obj_is_packed()
 - pack-objects: introduce pack.allowPackReuse
 - csum-file: introduce hashfile_total()
 - pack-bitmap: introduce bitmap_walk_contains()
 - pack-bitmap: don't rely on bitmap_git->reuse_objects
 - ewah/bitmap: introduce bitmap_word_alloc()
 - packfile: expose get_delta_base()
 - builtin/pack-objects: report reused packfile objects

 The way "git pack-objects" reuses objects stored in existing pack
 to generate its result has been improved.

 Needs further clarification?
 cf. <20191115180319.113991-1-jonathantanmy@google.com>

--------------------------------------------------
[Discarded]

* js/advise-rebase-skip (2019-10-23) 3 commits
 . commit: give correct advice for empty commit during a rebase
 . sequencer: export the function to get the path of `.git/rebase-merge/`
 . cherry-pick: add test for `--skip` advice in `git commit`

 The logic used in "git commit" to give hints and errors depending
 on what operation was in progress learned to distinguish rebase and
 cherry-pick better.

 Superseded by pw/advise-rebase-skip


* bk/p4-exception-cleanup (2019-12-16) 2 commits
 . git-p4: failure because of RCS keywords should show help
 . git-p4: wrap patchRCSKeywords test to revert changes on failure

 Discarded for now without prejudice.

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Elijah Newren]

On Wed, Jan 22, 2020 at 2:19 PM Junio C Hamano <gitster@pobox.com> wrote:
> --------------------------------------------------
> [New Topics]
>
> * en/simplify-check-updates-in-unpack-trees (2020-01-07) 1 commit
>   (merged to 'next' on 2020-01-15 at 586c055b69)
>  + unpack-trees: exit check_updates() early if updates are not wanted
>
>  Originally merged to 'next' on 2020-01-09
>
>  Code simplification.
>
>  Will merge to 'master'.

This is v2 of the series; and it looks good.  But...

> * en/unpack-trees-check-updates-simplify (2020-01-04) 1 commit
>  - unpack-trees: exit check_updates() early if updates are not wanted
>
>  Code simplification.
>
>  Will merge to 'next'.

...this is v1 of the same submission.  We don't need both v1 and v2 of
the same patch to be merged (the only difference between the two is
that v2 has a more detailed commit message); luckily, the one you
already merged to next is v2.  Can you just drop the
en/unpack-trees-check-updates-simplify topic?

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

>> * en/unpack-trees-check-updates-simplify (2020-01-04) 1 commit
>>  - unpack-trees: exit check_updates() early if updates are not wanted
>>
>>  Code simplification.
>>
>>  Will merge to 'next'.
>
> ...this is v1 of the same submission.

Thanks for spotting.  Discarded.

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[SZEDER Gábor]

On Wed, Jan 22, 2020 at 02:18:05PM -0800, Junio C Hamano wrote:
> * yz/p4-py3 (2020-01-15) 14 commits
>  - ci: also run linux-gcc pipeline with python3.5 environment

I still think that this last patch needs to be reworked before this
series is merged any further.

The only Python script we have is 'git p4', so the Python version is
only relevant for 'git p4' tests ('t98*'), while the rest of Git and
the test suite couldn't care less [1].  This patch, however, not only
builds Git and runs the full test suite for each of the two Python
versions, but, worse, runs the full test suite _twice_ for each, first
as a "regular" test run and then again with all the GIT_TEST_* knobs
enabled.  Consequently, it adds ~50mins to every build's runtime.

That's just too wasteful.


[1] Well, there is 'contrib/svn-fe/svnrdump_sim.py' as well, but
    that's contrib, though it is used in 't9020-remote-svn.sh'.

>  - git-p4: use python3's input() everywhere
>  - git-p4: simplify regex pattern generation for parsing diff-tree
>  - git-p4: use dict.items() iteration for python3 compatibility
>  - git-p4: use functools.reduce instead of reduce
>  - git-p4: fix freezing while waiting for fast-import progress
>  - git-p4: use marshal format version 2 when sending to p4
>  - git-p4: open .gitp4-usercache.txt in text mode
>  - git-p4: convert path to unicode before processing them
>  - git-p4: encode/decode communication with git for python3
>  - git-p4: encode/decode communication with p4 for python3
>  - git-p4: remove string type aliasing
>  - git-p4: change the expansion test from basestring to list
>  - git-p4: make python2.7 the oldest supported version
> 
>  Update "git p4" to work with Python 3.
> 
>  Will merge to 'next'.

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Denton Liu]

Hi Junio,

Sorry, I'm back in school so I haven't had the time to keep up with my
contributions very much. Feel free to give my topics lower priority for
the next couple of months if they interfere with any other topics.

On Wed, Jan 22, 2020 at 02:18:05PM -0800, Junio C Hamano wrote:
> * dl/merge-autostash (2020-01-13) 17 commits
>  - pull: pass --autostash to merge
>  - t5520: make test_pull_autostash() accept expect_parent_num
>  - merge: teach --autostash option
>  - sequencer: unlink autostash in apply_autostash()
>  - sequencer: extract perform_autostash() from rebase
>  - rebase: generify create_autostash()
>  - rebase: extract create_autostash()
>  - reset: extract reset_head() from rebase
>  - rebase: generify reset_head()
>  - rebase: use apply_autostash() from sequencer.c
>  - sequencer: make apply_rebase() accept a path
>  - rebase: use read_oneliner()
>  - sequencer: make read_oneliner() extern
>  - sequencer: configurably warn on non-existent files
>  - sequencer: use file strbuf for read_oneliner()
>  - t7600: use test_write_lines()
>  - Makefile: alphabetically sort += lists
> 
>  "git merge" learns the "--autostash" option.
> 
>  What's the status of this one?  Are people happy with the shape of
>  the code?

I'm not quite happy with this yet. Phillip Wood pointed out that if we
do `git reset --hard` mid-merge with a stash, the stash will pop _after_
the reset, which is very surprising since it leaves a dirty tree.

I think I will have time to reroll this on the weekend.

> * dl/test-must-fail-fixes-2 (2020-01-07) 16 commits
>  - t4124: only mark git command with test_must_fail
>  - t3507: use test_path_is_missing()
>  - t3507: fix indentation
>  - t3504: do check for conflict marker after failed cherry-pick
>  - t3419: stop losing return code of git command
>  - t3415: increase granularity of test_auto_{fixup,squash}()
>  - t3415: stop losing return codes of git commands
>  - t3310: extract common notes_merge_files_gone()
>  - t3030: use test_path_is_missing()
>  - t2018: replace "sha" with "oid"
>  - t2018: don't lose return code of git commands
>  - t2018: teach do_checkout() to accept `!` arg
>  - t2018: use test_expect_code for failing git commands
>  - t2018: improve style of if-statement
>  - t2018: add space between function name and ()
>  - t2018: remove trailing space from test description
> 
>  Test updates.
> 
>  Will merge to 'next'.

Eric Sunshine sent out a reworded version of "t2018: use
test_expect_code for failing git commands"'s commit message. I'll send
out that replacement patch later this weekend as well.

Thanks,

Denton

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Junio C Hamano]

SZEDER Gábor <szeder.dev@gmail.com> writes:

> On Wed, Jan 22, 2020 at 02:18:05PM -0800, Junio C Hamano wrote:
>> * yz/p4-py3 (2020-01-15) 14 commits
>>  - ci: also run linux-gcc pipeline with python3.5 environment
>
> I still think that this last patch needs to be reworked before this
> series is merged any further.
>
> The only Python script we have is 'git p4', so the Python version is
> only relevant for 'git p4' tests ('t98*'), while the rest of Git and
> the test suite couldn't care less [1].  This patch, however, not only
> builds Git and runs the full test suite for each of the two Python
> versions, but, worse, runs the full test suite _twice_ for each, first
> as a "regular" test run and then again with all the GIT_TEST_* knobs
> enabled.  Consequently, it adds ~50mins to every build's runtime.
>
> That's just too wasteful.

Thanks for a reminder.  Yes, I do recall you raised the above point
and I agree with the assessment.

What's the ideal endgame wrt the tests?  Build with Py$N and run
full test suite once, and run full test suite again with the unusual
knobs enabled, which is what is done without this series, plus build
with Py(5-$N) and run and run only t98?? tests?

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Junio C Hamano]

Denton Liu <liu.denton@gmail.com> writes:

> Sorry, I'm back in school so I haven't had the time to keep up with my
> contributions very much. Feel free to give my topics lower priority for
> the next couple of months if they interfere with any other topics.

Thanks for a quick status report.

> On Wed, Jan 22, 2020 at 02:18:05PM -0800, Junio C Hamano wrote:
>> * dl/merge-autostash (2020-01-13) 17 commits
>> ..
>>  What's the status of this one?  Are people happy with the shape of
>>  the code?
>
> I'm not quite happy with this yet. Phillip Wood pointed out that if we
> do `git reset --hard` mid-merge with a stash, the stash will pop _after_
> the reset, which is very surprising since it leaves a dirty tree.
>
> I think I will have time to reroll this on the weekend.

OK, no need for rush.

>> * dl/test-must-fail-fixes-2 (2020-01-07) 16 commits
>> ...
>>  - t2018: remove trailing space from test description
>> 
>>  Test updates.
>> 
>>  Will merge to 'next'.
>
> Eric Sunshine sent out a reworded version of "t2018: use
> test_expect_code for failing git commands"'s commit message. I'll send
> out that replacement patch later this weekend as well.

Thanks.

Good luck with your studies and have fun at school.

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1977 bytes --]

Hi,

On Thu, 23 Jan 2020, SZEDER Gábor wrote:

> On Wed, Jan 22, 2020 at 02:18:05PM -0800, Junio C Hamano wrote:
> > * yz/p4-py3 (2020-01-15) 14 commits
> >  - ci: also run linux-gcc pipeline with python3.5 environment
>
> I still think that this last patch needs to be reworked before this
> series is merged any further.
>
> The only Python script we have is 'git p4', so the Python version is
> only relevant for 'git p4' tests ('t98*'), while the rest of Git and
> the test suite couldn't care less [1].  This patch, however, not only
> builds Git and runs the full test suite for each of the two Python
> versions, but, worse, runs the full test suite _twice_ for each, first
> as a "regular" test run and then again with all the GIT_TEST_* knobs
> enabled.  Consequently, it adds ~50mins to every build's runtime.
>
> That's just too wasteful.
>
>
> [1] Well, there is 'contrib/svn-fe/svnrdump_sim.py' as well, but
>     that's contrib, though it is used in 't9020-remote-svn.sh'.

For what it's worth, I fully support Gábor's assessment.

Ciao,
Dscho

>
> >  - git-p4: use python3's input() everywhere
> >  - git-p4: simplify regex pattern generation for parsing diff-tree
> >  - git-p4: use dict.items() iteration for python3 compatibility
> >  - git-p4: use functools.reduce instead of reduce
> >  - git-p4: fix freezing while waiting for fast-import progress
> >  - git-p4: use marshal format version 2 when sending to p4
> >  - git-p4: open .gitp4-usercache.txt in text mode
> >  - git-p4: convert path to unicode before processing them
> >  - git-p4: encode/decode communication with git for python3
> >  - git-p4: encode/decode communication with p4 for python3
> >  - git-p4: remove string type aliasing
> >  - git-p4: change the expansion test from basestring to list
> >  - git-p4: make python2.7 the oldest supported version
> >
> >  Update "git p4" to work with Python 3.
> >
> >  Will merge to 'next'.
>
>

yz/p4-py3, was Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2285 bytes --]

Hi Junio,

On Wed, 22 Jan 2020, Junio C Hamano wrote:

> SZEDER Gábor <szeder.dev@gmail.com> writes:
>
> > On Wed, Jan 22, 2020 at 02:18:05PM -0800, Junio C Hamano wrote:
> >> * yz/p4-py3 (2020-01-15) 14 commits
> >>  - ci: also run linux-gcc pipeline with python3.5 environment
> >
> > I still think that this last patch needs to be reworked before this
> > series is merged any further.
> >
> > The only Python script we have is 'git p4', so the Python version is
> > only relevant for 'git p4' tests ('t98*'), while the rest of Git and
> > the test suite couldn't care less [1].  This patch, however, not only
> > builds Git and runs the full test suite for each of the two Python
> > versions, but, worse, runs the full test suite _twice_ for each, first
> > as a "regular" test run and then again with all the GIT_TEST_* knobs
> > enabled.  Consequently, it adds ~50mins to every build's runtime.
> >
> > That's just too wasteful.
>
> Thanks for a reminder.  Yes, I do recall you raised the above point
> and I agree with the assessment.
>
> What's the ideal endgame wrt the tests?  Build with Py$N and run
> full test suite once, and run full test suite again with the unusual
> knobs enabled, which is what is done without this series, plus build
> with Py(5-$N) and run and run only t98?? tests?

Should we declare `t98xx` to be the namespace for the Python-based
scripts, or alternatively declare that we won't ever include another
Python script but `git-p4`?

But yes, I think that we should probably "tack on" the Python 3.x tests to
the `linux-gcc` job.

Or maybe finally split this job into three: one job that does what
`linux-gcc` suggests, a `linux-gcc-knobs` one that sets all those `GIT_*`
variables, and a python3x one that only runs t98*.sh.

The reason to split it off is this: on rare occasion, I have to restart
the `linux-gcc` job because _one_ of those `git-p4` tests failed due to
some reason or other, probably timing-related, I did not have time to
investigate this. Having to re-run the entire test suite, twice, just to
work around those flaky tests is rather wasteful.

That would actually be my prereference. If people agree, I will revive
https://github.com/gitgitgadget/git/pull/266.

Ciao,
Dscho

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[SZEDER Gábor]

On Wed, Jan 22, 2020 at 10:06:19PM -0800, Junio C Hamano wrote:
> SZEDER Gábor <szeder.dev@gmail.com> writes:
> 
> > On Wed, Jan 22, 2020 at 02:18:05PM -0800, Junio C Hamano wrote:
> >> * yz/p4-py3 (2020-01-15) 14 commits
> >>  - ci: also run linux-gcc pipeline with python3.5 environment
> >
> > I still think that this last patch needs to be reworked before this
> > series is merged any further.
> >
> > The only Python script we have is 'git p4', so the Python version is
> > only relevant for 'git p4' tests ('t98*'), while the rest of Git and
> > the test suite couldn't care less [1].  This patch, however, not only
> > builds Git and runs the full test suite for each of the two Python
> > versions, but, worse, runs the full test suite _twice_ for each, first
> > as a "regular" test run and then again with all the GIT_TEST_* knobs
> > enabled.  Consequently, it adds ~50mins to every build's runtime.
> >
> > That's just too wasteful.
> 
> Thanks for a reminder.  Yes, I do recall you raised the above point
> and I agree with the assessment.
> 
> What's the ideal endgame wrt the tests?  Build with Py$N and run
> full test suite once, and run full test suite again with the unusual
> knobs enabled, which is what is done without this series, plus build
> with Py(5-$N) and run and run only t98?? tests?

Running the 'linux-clang' job with Python 2 and the 'linux-gcc' job
with Python 3 would be the simplest and cheapest, I'd think.  We'd
only need to add the appropriate 'PYTHON_PATH=...' to out MAKEFLAGS.
As far as Travis CI is concerned, their Xenial image (i.e. the Linux
image we're using) comes with both 'python2' and 'python3' in PATH, at
versions v2.7 and v3.5, with the former being the default.

Perhaps we could do the same with the OSX Clang and GCC jobs as well,
dunno.  Travis CI's OSX image, too, comes with both 'python2' and
'python3' in PATH, though Python 3 is already at v3.7, but still v2.7
is the default.

(Note that 'contrib/svn-fe/svnrdump_sim.py' is not added to
SCRIPT_PYTHON in our Makefile, so it is not affected by the
PYTHON_PATH we'd set in MAKEFLAGS, and it's shebang line remains
'#!/usr/bin/env python'.)

I think the choice of compiler to build Git and the choice of Python
version to run 'git p4' are completely independent, and it's not worth
to check all their possible combinations.

Furthermore, to re-run only a subset of the test suite with 'prove'
we'd need to tweak our $GIT_PROVE_OPTS, because the '--state=' option
that we have in there, will cause us troubles:

  $ cd t/
  # I have the prove state file from the last full test run:
  $ ls -l .prove
  -rw-r--r-- 1 szeder szeder 188758 Jan 23 14:02 .prove
  # Let's try to run only a 'git p4' test script with the default
  # '--state=' option from our $GIT_PROVE_OPTS:
  $ make DEFAULT_TEST_TARGET=prove \
    GIT_PROVE_OPTS=--state=failed,slow,save T=t9800-git-p4-basic.sh 
  rm -f -r 'test-results'
  *** prove ***
  t9001-send-email.sh ................................ 23/?
  <... snip ...>
  # Uh-oh, it proceeds to run all the test scripts that are recorded
  # in the prove state file, i.e. the whole test suite.
  
  # Now let's try that without the '--state=' option:
  $ make DEFAULT_TEST_TARGET=prove GIT_PROVE_OPTS= T=t9800-git-p4-basic.sh 
  rm -f -r 'test-results'
  *** prove ***
  t9800-git-p4-basic.sh .. ok    
  All tests successful.
  Files=1, Tests=21, 12 wallclock secs ( 0.02 usr  0.01 sys +  3.46 cusr
  2.04 csys =  5.53 CPU)
  Result: PASS

I couldn't find a set of 'prove' options that allow us to run slowest
tests first, but run only the tests that are explicitly specified as
cmdline arguments.

So we'd need to tweak how $GIT_PROVE_OPTS is used in our CI builds to
drop the '--state=' option but still keep all other options
('--jobs'!) when re-running only the 'git p4' tests.

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Christian Couder]

On Wed, Jan 22, 2020 at 11:19 PM Junio C Hamano <gitster@pobox.com> wrote:

> * jk/packfile-reuse-cleanup (2019-10-23) 9 commits
>  - pack-objects: improve partial packfile reuse
>  - builtin/pack-objects: introduce obj_is_packed()
>  - pack-objects: introduce pack.allowPackReuse
>  - csum-file: introduce hashfile_total()
>  - pack-bitmap: introduce bitmap_walk_contains()
>  - pack-bitmap: don't rely on bitmap_git->reuse_objects
>  - ewah/bitmap: introduce bitmap_word_alloc()
>  - packfile: expose get_delta_base()
>  - builtin/pack-objects: report reused packfile objects
>
>  The way "git pack-objects" reuses objects stored in existing pack
>  to generate its result has been improved.
>
>  Needs further clarification?
>  cf. <20191115180319.113991-1-jonathantanmy@google.com>

Peff replied to Jonathan's questions and reviewed the above patch V3 series

See for example:

https://public-inbox.org/git/20191209081129.GA1546451@coredump.intra.peff.net/
https://public-inbox.org/git/20191209061853.GA38588@coredump.intra.peff.net/

I then sent a V4 based on Peff's comments:

https://public-inbox.org/git/20191218112547.4974-1-chriscool@tuxfamily.org/

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[SZEDER Gábor]

On Thu, Jan 23, 2020 at 03:16:26PM +0100, SZEDER Gábor wrote:
> > What's the ideal endgame wrt the tests?

> Running the 'linux-clang' job with Python 2 and the 'linux-gcc' job
> with Python 3 would be the simplest and cheapest, I'd think.  We'd
> only need to add the appropriate 'PYTHON_PATH=...' to out MAKEFLAGS.
> As far as Travis CI is concerned, their Xenial image (i.e. the Linux
> image we're using) comes with both 'python2' and 'python3' in PATH, at
> versions v2.7 and v3.5, with the former being the default.
> 
> Perhaps we could do the same with the OSX Clang and GCC jobs as well,
> dunno.  Travis CI's OSX image, too, comes with both 'python2' and
> 'python3' in PATH, though Python 3 is already at v3.7, but still v2.7
> is the default.

Replacing that last patch of the series with the diff below works both
on Linux and macOS and both on Travis CI and Azure Pipelines.

linux-clang with Python 2:
  https://travis-ci.org/szeder/git/jobs/640912453#L499
  https://dev.azure.com/gitgitgadget/git/_build/results?buildId=27690&view=logs&j=8f20da19-31b7-5cef-4813-95b8788bd086&t=56027f08-fde3-50ad-0c9a-5ec7df432ed0&l=615

linux-gcc with Python 3:
  https://travis-ci.org/szeder/git/jobs/640912454#L606
  https://dev.azure.com/gitgitgadget/git/_build/results?buildId=27690&view=logs&j=275f1d19-1bd8-5591-b06b-07d489ea915a&t=33e5d3ec-87e7-5f80-0281-074c6962cb44&l=652

osx-clang with Python 2:
  https://travis-ci.org/szeder/git/jobs/640912455#L272
  https://dev.azure.com/gitgitgadget/git/_build/results?buildId=27690&view=logs&j=b80c90c8-f62d-51c1-0986-3bb8359d9b6f&t=f8b92b00-54c3-55aa-48a6-84ec793cfb94&l=365

osx-gcc with Python 3:
  https://travis-ci.org/szeder/git/jobs/640912456#L283
  https://dev.azure.com/gitgitgadget/git/_build/results?buildId=27690&view=logs&j=cfa20e98-6997-523c-4233-f0a7302c929f&t=3de1ae02-4adb-5138-54da-65cec5dd3141&l=394


 --- >8 ---

diff --git a/ci/lib.sh b/ci/lib.sh
index a90d0dc0fd..c3a8cd2104 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -162,6 +162,9 @@ linux-clang|linux-gcc)
 	if [ "$jobname" = linux-gcc ]
 	then
 		export CC=gcc-8
+		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
+	else
+		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
 	fi
 
 	export GIT_TEST_HTTPD=true
@@ -182,6 +185,9 @@ osx-clang|osx-gcc)
 	if [ "$jobname" = osx-gcc ]
 	then
 		export CC=gcc-9
+		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
+	else
+		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
 	fi
 
 	# t9810 occasionally fails on Travis CI OS X

Re: yz/p4-py3, was Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Yang Zhao]

On Thu, Jan 23, 2020 at 4:11 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Should we declare `t98xx` to be the namespace for the Python-based
> scripts, or alternatively declare that we won't ever include another
> Python script but `git-p4`?

Seeing how git-p4 has de facto taken over t98xx, declaring the former
makes sense. I intend to introduce more features into git-p4 so it'll
probably be practical to reserve more of the space for it, probably up
to t9850.

On declaring `git-p4` the sole approved use of python, I'm personally
OK with it if there's no appetite to better integrate Python on
Windows.

> But yes, I think that we should probably "tack on" the Python 3.x tests to
> the `linux-gcc` job.
>
> Or maybe finally split this job into three: ...
>
> The reason to split it off is this: on rare occasion, I have to restart
> the `linux-gcc` job because _one_ of those `git-p4` tests failed due to
> some reason or other, probably timing-related, I did not have time to
> investigate this. Having to re-run the entire test suite, twice, just to
> work around those flaky tests is rather wasteful.

Agreed.

-- 
yang

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Junio C Hamano]

Christian Couder <christian.couder@gmail.com> writes:

> Peff replied to Jonathan's questions and reviewed the above patch V3 series
>
> See for example:
>
> https://public-inbox.org/git/20191209081129.GA1546451@coredump.intra.peff.net/
> https://public-inbox.org/git/20191209061853.GA38588@coredump.intra.peff.net/
>
> I then sent a V4 based on Peff's comments:
>
> https://public-inbox.org/git/20191218112547.4974-1-chriscool@tuxfamily.org/

I don't think I saw any review on the latest round---let me pick it
up and replace with the older round that is on 'pu'.

Thanks.

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Junio C Hamano]

SZEDER Gábor <szeder.dev@gmail.com> writes:

> Replacing that last patch of the series with the diff below works both
> on Linux and macOS and both on Travis CI and Azure Pipelines.
>...

Yang, care to do the honors to wrap it up with summary of the
decisions as a replacement patch for the last step?

Thanks all for helping to tie loose ends.

>  --- >8 ---
>
> diff --git a/ci/lib.sh b/ci/lib.sh
> index a90d0dc0fd..c3a8cd2104 100755
> --- a/ci/lib.sh
> +++ b/ci/lib.sh
> @@ -162,6 +162,9 @@ linux-clang|linux-gcc)
>  	if [ "$jobname" = linux-gcc ]
>  	then
>  		export CC=gcc-8
> +		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
> +	else
> +		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
>  	fi
>  
>  	export GIT_TEST_HTTPD=true
> @@ -182,6 +185,9 @@ osx-clang|osx-gcc)
>  	if [ "$jobname" = osx-gcc ]
>  	then
>  		export CC=gcc-9
> +		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
> +	else
> +		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
>  	fi
>  
>  	# t9810 occasionally fails on Travis CI OS X

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2954 bytes --]

Hi Gábor,

On Thu, 23 Jan 2020, SZEDER Gábor wrote:

> On Thu, Jan 23, 2020 at 03:16:26PM +0100, SZEDER Gábor wrote:
> > > What's the ideal endgame wrt the tests?
>
> > Running the 'linux-clang' job with Python 2 and the 'linux-gcc' job
> > with Python 3 would be the simplest and cheapest, I'd think.  We'd
> > only need to add the appropriate 'PYTHON_PATH=...' to out MAKEFLAGS.
> > As far as Travis CI is concerned, their Xenial image (i.e. the Linux
> > image we're using) comes with both 'python2' and 'python3' in PATH, at
> > versions v2.7 and v3.5, with the former being the default.
> >
> > Perhaps we could do the same with the OSX Clang and GCC jobs as well,
> > dunno.  Travis CI's OSX image, too, comes with both 'python2' and
> > 'python3' in PATH, though Python 3 is already at v3.7, but still v2.7
> > is the default.
>
> Replacing that last patch of the series with the diff below works both
> on Linux and macOS and both on Travis CI and Azure Pipelines.
>
> linux-clang with Python 2:
>   https://travis-ci.org/szeder/git/jobs/640912453#L499
>   https://dev.azure.com/gitgitgadget/git/_build/results?buildId=27690&view=logs&j=8f20da19-31b7-5cef-4813-95b8788bd086&t=56027f08-fde3-50ad-0c9a-5ec7df432ed0&l=615
>
> linux-gcc with Python 3:
>   https://travis-ci.org/szeder/git/jobs/640912454#L606
>   https://dev.azure.com/gitgitgadget/git/_build/results?buildId=27690&view=logs&j=275f1d19-1bd8-5591-b06b-07d489ea915a&t=33e5d3ec-87e7-5f80-0281-074c6962cb44&l=652
>
> osx-clang with Python 2:
>   https://travis-ci.org/szeder/git/jobs/640912455#L272
>   https://dev.azure.com/gitgitgadget/git/_build/results?buildId=27690&view=logs&j=b80c90c8-f62d-51c1-0986-3bb8359d9b6f&t=f8b92b00-54c3-55aa-48a6-84ec793cfb94&l=365
>
> osx-gcc with Python 3:
>   https://travis-ci.org/szeder/git/jobs/640912456#L283
>   https://dev.azure.com/gitgitgadget/git/_build/results?buildId=27690&view=logs&j=cfa20e98-6997-523c-4233-f0a7302c929f&t=3de1ae02-4adb-5138-54da-65cec5dd3141&l=394
>
>
>  --- >8 ---
>
> diff --git a/ci/lib.sh b/ci/lib.sh
> index a90d0dc0fd..c3a8cd2104 100755
> --- a/ci/lib.sh
> +++ b/ci/lib.sh
> @@ -162,6 +162,9 @@ linux-clang|linux-gcc)
>  	if [ "$jobname" = linux-gcc ]
>  	then
>  		export CC=gcc-8
> +		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
> +	else
> +		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
>  	fi
>
>  	export GIT_TEST_HTTPD=true
> @@ -182,6 +185,9 @@ osx-clang|osx-gcc)
>  	if [ "$jobname" = osx-gcc ]
>  	then
>  		export CC=gcc-9
> +		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
> +	else
> +		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
>  	fi
>
>  	# t9810 occasionally fails on Travis CI OS X

My only worry is that this makes it even more obscure what purpose which
job has. Nothing in the name `osx-gcc` shouts loudly "I want to use Python
3.x!" to me.

Other than that, it is sensible.

Ciao,
Dscho

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[SZEDER Gábor]

On Thu, Jan 23, 2020 at 10:39:12PM +0100, Johannes Schindelin wrote:
> > diff --git a/ci/lib.sh b/ci/lib.sh
> > index a90d0dc0fd..c3a8cd2104 100755
> > --- a/ci/lib.sh
> > +++ b/ci/lib.sh
> > @@ -162,6 +162,9 @@ linux-clang|linux-gcc)
> >  	if [ "$jobname" = linux-gcc ]
> >  	then
> >  		export CC=gcc-8
> > +		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
> > +	else
> > +		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
> >  	fi
> >
> >  	export GIT_TEST_HTTPD=true
> > @@ -182,6 +185,9 @@ osx-clang|osx-gcc)
> >  	if [ "$jobname" = osx-gcc ]
> >  	then
> >  		export CC=gcc-9
> > +		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
> > +	else
> > +		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
> >  	fi
> >
> >  	# t9810 occasionally fails on Travis CI OS X
> 
> My only worry is that this makes it even more obscure what purpose which
> job has. Nothing in the name `osx-gcc` shouts loudly "I want to use Python
> 3.x!" to me.

Do they have to shout that loudly in the name?

We could rename these jobs to e.g. 'linux-clang-py2' and the like, but
I think it would bring little benefit, if any.  In our Travis CI
builds these Linux/OSX Clang/GCC jobs come from the build matrix,
therefore the jobname is not visible on the Travis CI web interface or
API, only in the build logs.  There are some pages on Azure Pipelines
that do show the jobname (and some that could, but hide it instead),
but it's just too convoluted (or sometimes even impossible, well, for
me anyway) to get there.

And if the requested Python binary can't be found, which will
eventually happen with 'python2', then the non-zero exit code of
'which' will abort the build, no matter how the job is called.

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Yang Zhao]

On Thu, Jan 23, 2020 at 12:52 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> SZEDER Gábor <szeder.dev@gmail.com> writes:
>
> > Replacing that last patch of the series with the diff below works both
> > on Linux and macOS and both on Travis CI and Azure Pipelines.
> >...
>
> Yang, care to do the honors to wrap it up with summary of the
> decisions as a replacement patch for the last step?

I've done a new rebase on master, did the CI change as discussed, and
pushed the changes to the GitHub PR
(https://github.com/git/git/pull/673). osx-gcc test seems to have
failed something unrelated to these changes, but everything else still
passes.

The changes I've had to make to make it merge were minimal. I can send
a new revision to the list later today if that's more convenient.

Thanks,
-- 
yang

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1261 bytes --]

Hi yang,

On Fri, 24 Jan 2020, Yang Zhao wrote:

> On Thu, Jan 23, 2020 at 12:52 PM Junio C Hamano <gitster@pobox.com> wrote:
> >
> > SZEDER Gábor <szeder.dev@gmail.com> writes:
> >
> > > Replacing that last patch of the series with the diff below works both
> > > on Linux and macOS and both on Travis CI and Azure Pipelines.
> > >...
> >
> > Yang, care to do the honors to wrap it up with summary of the
> > decisions as a replacement patch for the last step?
>
> I've done a new rebase on master, did the CI change as discussed, and
> pushed the changes to the GitHub PR
> (https://github.com/git/git/pull/673). osx-gcc test seems to have
> failed something unrelated to these changes, but everything else still
> passes.

This is our good old friend, the flaky test case

	t5516.81 deny fetch unreachable SHA1, allowtipsha1inwant=false

It has been discussed before, and it seems that Gábor has a patch that
works, but that he is not completely happy with, yet.

I restarted the job, it should turn green in about 20-25 minutes.

Thanks,
Dscho

>
> The changes I've had to make to make it merge were minimal. I can send
> a new revision to the list later today if that's more convenient.
>
> Thanks,
> --
> yang
>

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2142 bytes --]

Hi Gábor,

On Fri, 24 Jan 2020, SZEDER Gábor wrote:

> On Thu, Jan 23, 2020 at 10:39:12PM +0100, Johannes Schindelin wrote:
> > > diff --git a/ci/lib.sh b/ci/lib.sh
> > > index a90d0dc0fd..c3a8cd2104 100755
> > > --- a/ci/lib.sh
> > > +++ b/ci/lib.sh
> > > @@ -162,6 +162,9 @@ linux-clang|linux-gcc)
> > >  	if [ "$jobname" = linux-gcc ]
> > >  	then
> > >  		export CC=gcc-8
> > > +		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
> > > +	else
> > > +		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
> > >  	fi
> > >
> > >  	export GIT_TEST_HTTPD=true
> > > @@ -182,6 +185,9 @@ osx-clang|osx-gcc)
> > >  	if [ "$jobname" = osx-gcc ]
> > >  	then
> > >  		export CC=gcc-9
> > > +		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
> > > +	else
> > > +		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python2)"
> > >  	fi
> > >
> > >  	# t9810 occasionally fails on Travis CI OS X
> >
> > My only worry is that this makes it even more obscure what purpose which
> > job has. Nothing in the name `osx-gcc` shouts loudly "I want to use Python
> > 3.x!" to me.
>
> Do they have to shout that loudly in the name?
>
> We could rename these jobs to e.g. 'linux-clang-py2' and the like, but
> I think it would bring little benefit, if any.  In our Travis CI
> builds these Linux/OSX Clang/GCC jobs come from the build matrix,
> therefore the jobname is not visible on the Travis CI web interface or
> API, only in the build logs.  There are some pages on Azure Pipelines
> that do show the jobname (and some that could, but hide it instead),
> but it's just too convoluted (or sometimes even impossible, well, for
> me anyway) to get there.
>
> And if the requested Python binary can't be found, which will
> eventually happen with 'python2', then the non-zero exit code of
> 'which' will abort the build, no matter how the job is called.

I am mostly worried about contributors whose PRs break for "magic"
reasons. If it is not clear where the difference between `linux-gcc` and
`linux-clang` lies, that can cause unintended frustration, and I do not
want to cause that.

Ciao,
Dscho

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[SZEDER Gábor]

On Sat, Jan 25, 2020 at 01:13:52AM +0100, Johannes Schindelin wrote:
> > I've done a new rebase on master, did the CI change as discussed, and
> > pushed the changes to the GitHub PR
> > (https://github.com/git/git/pull/673). osx-gcc test seems to have
> > failed something unrelated to these changes, but everything else still
> > passes.
> 
> This is our good old friend, the flaky test case
> 
> 	t5516.81 deny fetch unreachable SHA1, allowtipsha1inwant=false
> 
> It has been discussed before, and it seems that Gábor has a patch that
> works, but that he is not completely happy with, yet.

Hehh.  Do I have a what?! :)

This is what you mean?

  https://public-inbox.org/git/20190830121005.GI8571@szeder.dev/

Gah, this is not how I wanted to start my Saturday morning ;)

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1019 bytes --]

Hi Gábor,

On Sat, 25 Jan 2020, SZEDER Gábor wrote:

> On Sat, Jan 25, 2020 at 01:13:52AM +0100, Johannes Schindelin wrote:
> > > I've done a new rebase on master, did the CI change as discussed, and
> > > pushed the changes to the GitHub PR
> > > (https://github.com/git/git/pull/673). osx-gcc test seems to have
> > > failed something unrelated to these changes, but everything else still
> > > passes.
> >
> > This is our good old friend, the flaky test case
> >
> > 	t5516.81 deny fetch unreachable SHA1, allowtipsha1inwant=false
> >
> > It has been discussed before, and it seems that Gábor has a patch that
> > works, but that he is not completely happy with, yet.
>
> Hehh.  Do I have a what?! :)
>
> This is what you mean?
>
>   https://public-inbox.org/git/20190830121005.GI8571@szeder.dev/

Yep, that's the patch I was talking about.

> Gah, this is not how I wanted to start my Saturday morning ;)

Sorry... FWIW I would be in favor of the reader approach ;-)

Ciao,
Dscho

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Denton Liu]

Hi Junio,

On Wed, Jan 22, 2020 at 02:18:05PM -0800, Junio C Hamano wrote:
> * ds/sparse-cone (2020-01-10) 1 commit
>  - unpack-trees: correctly compute result count
> 
>  The code recently added in this release to move to the entry beyond
>  the ones in the same directory in the index in the sparse-cone mode
>  did not count the number of entries to skip over incorrectly, which
>  has been corrected.
> 
>  Will merge to 'next'.

This commit has incorrect authorship information for Stolee. The author
is listed as "Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>"
due to a bug (which has been fixed) in GGG. We should fix the authorship
information before merging to 'next'.

On the topic of faulty authorship information, I sent along two mailmap
changes that were dropped[1][2]. Could you please pick these up?

Thanks,

Denton

[1]: https://public-inbox.org/git/20200114024938.GA17003@generichostname/
[2]: https://public-inbox.org/git/21b8a0d08764c31de12ef7661667eb1117d41ac4.1578972215.git.liu.denton@gmail.com/

Re: yz/p4-py3, was Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[SZEDER Gábor]

On Thu, Jan 23, 2020 at 01:11:52PM +0100, Johannes Schindelin wrote:
> on rare occasion, I have to restart
> the `linux-gcc` job because _one_ of those `git-p4` tests failed due to
> some reason or other, probably timing-related, I did not have time to
> investigate this.

I'm only aware of one timing-related issue in 'git p4' tests and
reported what I believe to be the cause of it a while ago.  Luke said
he'll have a look, but, alas, not much has happened since then, so
maybe you can poke him a bit as well?

https://public-inbox.org/git/CAE5ih78MV1qGTHBmCaN5k+VGv8Cy-RnFwOU0yuJBPEyn37C_4w@mail.gmail.com/


Or...  Yang Zhao mentioned somewhere planning further improvements to
'git p4', so he knows how it works.  Maybe we can trick him into
fixing this issue? ;)

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Junio C Hamano]

Denton Liu <liu.denton@gmail.com> writes:

> On Wed, Jan 22, 2020 at 02:18:05PM -0800, Junio C Hamano wrote:
>> * ds/sparse-cone (2020-01-10) 1 commit
>>  - unpack-trees: correctly compute result count
>> ...
> This commit has incorrect authorship information for Stolee. The author
> is listed as "Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>"
> due to a bug (which has been fixed) in GGG. We should fix the authorship
> information before merging to 'next'.

Ouch, but that was a bit too late X-<.

[PATCH] .mailmap: fix erroneous authorship for Derrick Stolee

[Denton Liu]

In 4c6c7971e0 (unpack-trees: correctly compute result count,
2020-01-10), the commit author is listed as "Derrick Stolee via
GitGitGadget <gitgitgadget@gmail.com>", however this authorship is
erroneous. Fix the authorship by mapping the erroneous authorship to
Derrick Stolee's canonical authorship information.

Signed-off-by: Denton Liu <liu.denton@gmail.com>
---
Whoops, I didn't realise that the branch had already been merged into
'next'. Anyway, we can apply this mailmap patch on the tip of
'ds/sparse-cone' to fix that issue.

 .mailmap | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.mailmap b/.mailmap
index 14fa041043..cf3f68ecaf 100644
--- a/.mailmap
+++ b/.mailmap
@@ -59,6 +59,7 @@ David S. Miller <davem@davemloft.net>
 David Turner <novalis@novalis.org> <dturner@twopensource.com>
 David Turner <novalis@novalis.org> <dturner@twosigma.com>
 Derrick Stolee <dstolee@microsoft.com> <stolee@gmail.com>
+Derrick Stolee <dstolee@microsoft.com> Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
 Deskin Miller <deskinm@umich.edu>
 Dirk Süsserott <newsletter@dirk.my1.cc>
 Eric Blake <eblake@redhat.com> <ebb9@byu.net>
-- 
2.25.0.rc1.180.g49a268d3eb

Re: [PATCH] .mailmap: fix erroneous authorship for Derrick Stolee

[Junio C Hamano]

Denton Liu <liu.denton@gmail.com> writes:

> In 4c6c7971e0 (unpack-trees: correctly compute result count,
> 2020-01-10), the commit author is listed as "Derrick Stolee via
> GitGitGadget <gitgitgadget@gmail.com>", however this authorship is
> erroneous. Fix the authorship by mapping the erroneous authorship to
> Derrick Stolee's canonical authorship information.
>
> Signed-off-by: Denton Liu <liu.denton@gmail.com>
> ---
> Whoops, I didn't realise that the branch had already been merged into
> 'next'. Anyway, we can apply this mailmap patch on the tip of
> 'ds/sparse-cone' to fix that issue.

Thanks, but that was also a bit too late, in that I queued an
equivalent already.

Thanks for reading our log with extra care.

>  .mailmap | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/.mailmap b/.mailmap
> index 14fa041043..cf3f68ecaf 100644
> --- a/.mailmap
> +++ b/.mailmap
> @@ -59,6 +59,7 @@ David S. Miller <davem@davemloft.net>
>  David Turner <novalis@novalis.org> <dturner@twopensource.com>
>  David Turner <novalis@novalis.org> <dturner@twosigma.com>
>  Derrick Stolee <dstolee@microsoft.com> <stolee@gmail.com>
> +Derrick Stolee <dstolee@microsoft.com> Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
>  Deskin Miller <deskinm@umich.edu>
>  Dirk Süsserott <newsletter@dirk.my1.cc>
>  Eric Blake <eblake@redhat.com> <ebb9@byu.net>

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Denton Liu]

Hi Junio,

On Sun, Jan 26, 2020 at 03:07:28PM -0500, Denton Liu wrote:
> This commit has incorrect authorship information for Stolee. The author
> is listed as "Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>"
> due to a bug (which has been fixed) in GGG. We should fix the authorship
> information before merging to 'next'.
> 
> On the topic of faulty authorship information, I sent along two mailmap
> changes that were dropped[1][2]. Could you please pick these up?

I see that you picked up the patch for Dscho[1] but not the patch for
Yi-Jyun Pan[2]. Was this a mistake or was there some other reason? The
patch got a "sort of ack" from the original author[3].

> 
> Thanks,
> 
> Denton
> 
> [1]: https://public-inbox.org/git/20200114024938.GA17003@generichostname/
> [2]: https://public-inbox.org/git/21b8a0d08764c31de12ef7661667eb1117d41ac4.1578972215.git.liu.denton@gmail.com/
[3]: https://github.com/git-l10n/git-po/pull/408#issuecomment-573991430

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> Do they have to shout that loudly in the name?
>>
>> We could rename these jobs to e.g. 'linux-clang-py2' and the like, but
>> I think it would bring little benefit, if any.  In our Travis CI
>> builds these Linux/OSX Clang/GCC jobs come from the build matrix,
>> therefore the jobname is not visible on the Travis CI web interface or
>> API, only in the build logs.  There are some pages on Azure Pipelines
>> that do show the jobname (and some that could, but hide it instead),
>> but it's just too convoluted (or sometimes even impossible, well, for
>> me anyway) to get there.
>>
>> And if the requested Python binary can't be found, which will
>> eventually happen with 'python2', then the non-zero exit code of
>> 'which' will abort the build, no matter how the job is called.
>
> I am mostly worried about contributors whose PRs break for "magic"
> reasons. If it is not clear where the difference between `linux-gcc` and
> `linux-clang` lies, that can cause unintended frustration, and I do not
> want to cause that.

So, what, if any, decision have we reached?

If linux-gcc and linux-clang labels are not visible, linux-clang-py2
and osx-py3 would not be, either, so...

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[SZEDER Gábor]

On Wed, Feb 05, 2020 at 01:01:50PM -0800, Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> Do they have to shout that loudly in the name?
> >>
> >> We could rename these jobs to e.g. 'linux-clang-py2' and the like, but
> >> I think it would bring little benefit, if any.  In our Travis CI
> >> builds these Linux/OSX Clang/GCC jobs come from the build matrix,
> >> therefore the jobname is not visible on the Travis CI web interface or
> >> API, only in the build logs.  There are some pages on Azure Pipelines
> >> that do show the jobname (and some that could, but hide it instead),
> >> but it's just too convoluted (or sometimes even impossible, well, for
> >> me anyway) to get there.
> >>
> >> And if the requested Python binary can't be found, which will
> >> eventually happen with 'python2', then the non-zero exit code of
> >> 'which' will abort the build, no matter how the job is called.
> >
> > I am mostly worried about contributors whose PRs break for "magic"
> > reasons. If it is not clear where the difference between `linux-gcc` and
> > `linux-clang` lies, that can cause unintended frustration, and I do not
> > want to cause that.

I'm not worried about that.  If a contributor doesn't touch any of our
Python scripts, then I don't see why using a different Python version
in the build would cause any issues.  And if they do modify one of the
Python scripts, then they should make sure that their modifications
work both with Python 2 and 3 in the first place.

> So, what, if any, decision have we reached?
> 
> If linux-gcc and linux-clang labels are not visible, linux-clang-py2
> and osx-py3 would not be, either, so...

The 'linux-gcc' and 'linux-clang' labels are not visible on Travis CI,
because those jobs as part of the build matrix, and, consequently, we
can't set the a 'jobname' environment variable for them in
'.travis.yml'.  If we were to include additional jobs for the Python
scripts, then for those we can (and should!) set
'jobname=linux-python' or something, and that would be visible on the
Travis CI web interface, just like e.g. 'jobname=StaticAnalysis'.

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 7146 bytes --]

Hi Gábor,

On Thu, 6 Feb 2020, SZEDER Gábor wrote:

> On Wed, Feb 05, 2020 at 01:01:50PM -0800, Junio C Hamano wrote:
> > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >
> > >> Do they have to shout that loudly in the name?
> > >>
> > >> We could rename these jobs to e.g. 'linux-clang-py2' and the like, but
> > >> I think it would bring little benefit, if any.  In our Travis CI
> > >> builds these Linux/OSX Clang/GCC jobs come from the build matrix,
> > >> therefore the jobname is not visible on the Travis CI web interface or
> > >> API, only in the build logs.  There are some pages on Azure Pipelines
> > >> that do show the jobname (and some that could, but hide it instead),
> > >> but it's just too convoluted (or sometimes even impossible, well, for
> > >> me anyway) to get there.
> > >>
> > >> And if the requested Python binary can't be found, which will
> > >> eventually happen with 'python2', then the non-zero exit code of
> > >> 'which' will abort the build, no matter how the job is called.
> > >
> > > I am mostly worried about contributors whose PRs break for "magic"
> > > reasons. If it is not clear where the difference between `linux-gcc` and
> > > `linux-clang` lies, that can cause unintended frustration, and I do not
> > > want to cause that.
>
> I'm not worried about that.  If a contributor doesn't touch any of our
> Python scripts, then I don't see why using a different Python version
> in the build would cause any issues.  And if they do modify one of the
> Python scripts, then they should make sure that their modifications
> work both with Python 2 and 3 in the first place.

If the frequent problems with downloading the Perforce binariers taught me
anything, it is that the most likely explanation for failures in the
linux-gcc job is that Perforce, once again, updated their binaries,
uploaded them _to the exact same URL as before_, and that there is nothing
wrong in the PR or the patches.

That _is_ the most likely explanation, given our record.

So what are contributors supposed to do with that? Nothing in the name
`linux-gcc` cries out loud: Hey, this is a Homebrew problem, there is most
likely already a PR up to fix it, and the job needs to be re-run once that
PR is merged, that's all, please ignore for a day.

Now, with the log that we currently have, it is still somewhat easy to
figure out what is going wrong. Somewhere along the lines it spits out an
error that talks about some sort of package and about some sort of
checksum. Most developers deduce from that message that it's not their
fault and move on.

So this is _already_ an annoying problem, but it gets worse: every once in
a while, a build is "green" _except_ in linux-gcc. The contributor runs
the test locally, it passes, so they conclude that the test must be flaky
or at least that it is not their fault.

How on this dear planet should they know to run the test again, but this
time with `GIT_TEST_SPLIT_INDEX=yes GIT_TEST_FULL_IN_PACK_ARRAY=true
GIT_TEST_OE_SIZE=10 GIT_TEST_OE_DELTA_SIZE=5 GIT_TEST_COMMIT_GRAPH=1
GIT_TEST_MULTI_PACK_INDEX=1 GIT_TEST_ADD_I_USE_BUILTIN=1`?

You know that they should do that. I know that they should do that. They
don't. And why don't they know that? Because _we make it hard for them to
know that_.

I agree that your changes make sense, from a lot of points of view. Except
one. The one where a contributor has to spend an unnecessarily long time
to figure out how to proceed given a test failure in `linux-clang` and
`osx-clang` that they never saw in their development (e.g. because they
only run with a non-EOL-ed Python).

What we need, therefore, is a way to let the users know _precisely_ what
is going on and even more importantly _what they should do now_.

Simply tacking on the Python3 stuff onto -clang (or -gcc? I forgot which
one, see, it even confuses _me_) is not going to do that.

Granted, this is not at all a new problem, it is related to that "let's
pile another test run with all kinds of `GIT_TEST_*` knobs turned to the
non-default settings onto, well, let's see, how about `linux-gcc`?"
problem.

In this light, I kind of agree that it is not the responsibility of the
py2 vs py3 changes you proposed to fix this. But they make the problem
even worse.

Ideally, I would prefer a new job into which the second half of
`linux-gcc` is moved, just like I proposed many moons ago. This would also
help the problem where flaky tests require a re-run of that insanely
long-running job.

Of course, you might find a clever way to enhance the failed test's log such
that it makes it obvious that this was run with special options (similar in
spirit to ffe1afe67c0 (tests: show the test name and number at the start of
verbose output, 2019-08-05)), and then tacking on the py3 thing onto -clang
(or was it -gcc? I am _still_ confused about that) would still be okay.

But then, I would contend that we should do the same for
`GIT_TEST_SPLIT_INDEX=yes GIT_TEST_FULL_IN_PACK_ARRAY=true
GIT_TEST_OE_SIZE=10 GIT_TEST_OE_DELTA_SIZE=5 GIT_TEST_COMMIT_GRAPH=1
GIT_TEST_MULTI_PACK_INDEX=1 GIT_TEST_ADD_I_USE_BUILTIN=1`. Only use them in
`-clang` (or was it `-gcc`? I still cannot remember), and run the other with
default settings only.

> > So, what, if any, decision have we reached?
> >
> > If linux-gcc and linux-clang labels are not visible, linux-clang-py2
> > and osx-py3 would not be, either, so...
>
> The 'linux-gcc' and 'linux-clang' labels are not visible on Travis CI,
> because those jobs as part of the build matrix, and, consequently, we
> can't set the a 'jobname' environment variable for them in
> '.travis.yml'.  If we were to include additional jobs for the Python
> scripts, then for those we can (and should!) set
> 'jobname=linux-python' or something, and that would be visible on the
> Travis CI web interface, just like e.g. 'jobname=StaticAnalysis'.

I think we can see that jobname very well, though. If you direct your web
browser to
https://travis-ci.org/git/git/builds/646646192?utm_source=github_status&utm_medium=notification
you will see something like this:

    Build jobs		View config

! 5281.1 AMD64		Compiler: clang Xcode: xcode10.1 C	no environment variables set	8 min 20 sec
! 5281.2 AMD64		Compiler: gcc Xcode: xcode10.1 C	no environment variables set	8 min 23 sec
X 5281.3 AMD64		Compiler: clang Xcode: xcode10.1 C	no environment variables set	1 min 57 sec
X 5281.4 AMD64		Compiler: gcc Xcode: xcode10.1 C	no environment variables set	2 min 41 sec
! 5281.5 AMD64		Xcode: xcode10.1 C			jobname=GIT_TEST_GETTEXT_POISON	5 min 14 sec
X 5281.6 AMD64		Xcode: xcode10.1 C			jobname=linux-gcc-4.8		1 min 13 sec
! 5281.7 AMD64		Xcode: xcode10.1 C			jobname=Linux32			6 min 50 sec
✓ 5281.8 AMD64		Xcode: xcode10.1 C			jobname=StaticAnalysis		10 min 56 sec
✓ 5281.9 AMD64		Xcode: xcode10.1 C			jobname=Documentation		6 min 15 sec

Never mind that it is somewhat dubious to see that the Linux32 job is run with
Xcode... but you definitely see the jobname, loud and clear.

Ciao,
Dscho

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[SZEDER Gábor]

On Thu, Feb 06, 2020 at 09:57:51AM +0100, Johannes Schindelin wrote:
> Hi Gábor,
> 
> On Thu, 6 Feb 2020, SZEDER Gábor wrote:
> 
> > On Wed, Feb 05, 2020 at 01:01:50PM -0800, Junio C Hamano wrote:
> > > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> > >
> > > >> Do they have to shout that loudly in the name?
> > > >>
> > > >> We could rename these jobs to e.g. 'linux-clang-py2' and the like, but
> > > >> I think it would bring little benefit, if any.  In our Travis CI
> > > >> builds these Linux/OSX Clang/GCC jobs come from the build matrix,
> > > >> therefore the jobname is not visible on the Travis CI web interface or
> > > >> API, only in the build logs.  There are some pages on Azure Pipelines
> > > >> that do show the jobname (and some that could, but hide it instead),
> > > >> but it's just too convoluted (or sometimes even impossible, well, for
> > > >> me anyway) to get there.
> > > >>
> > > >> And if the requested Python binary can't be found, which will
> > > >> eventually happen with 'python2', then the non-zero exit code of
> > > >> 'which' will abort the build, no matter how the job is called.
> > > >
> > > > I am mostly worried about contributors whose PRs break for "magic"
> > > > reasons. If it is not clear where the difference between `linux-gcc` and
> > > > `linux-clang` lies, that can cause unintended frustration, and I do not
> > > > want to cause that.
> >
> > I'm not worried about that.  If a contributor doesn't touch any of our
> > Python scripts, then I don't see why using a different Python version
> > in the build would cause any issues.  And if they do modify one of the
> > Python scripts, then they should make sure that their modifications
> > work both with Python 2 and 3 in the first place.
> 
> If the frequent problems with downloading the Perforce binariers taught me
> anything, it is that the most likely explanation for failures in the
> linux-gcc job is that Perforce, once again, updated their binaries,
> uploaded them _to the exact same URL as before_, and that there is nothing
> wrong in the PR or the patches.
> 
> That _is_ the most likely explanation, given our record.
> 
> So what are contributors supposed to do with that? Nothing in the name
> `linux-gcc` cries out loud: Hey, this is a Homebrew problem, there is most

Yes, because the 'linux-gcc' job doesn't run Homebrew...

> > > So, what, if any, decision have we reached?
> > >
> > > If linux-gcc and linux-clang labels are not visible, linux-clang-py2
> > > and osx-py3 would not be, either, so...
> >
> > The 'linux-gcc' and 'linux-clang' labels are not visible on Travis CI,
> > because those jobs as part of the build matrix, and, consequently, we
> > can't set the a 'jobname' environment variable for them in
> > '.travis.yml'.  If we were to include additional jobs for the Python
> > scripts, then for those we can (and should!) set
> > 'jobname=linux-python' or something, and that would be visible on the
> > Travis CI web interface, just like e.g. 'jobname=StaticAnalysis'.
> 
> I think we can see that jobname very well, though. If you direct your web
> browser to
> https://travis-ci.org/git/git/builds/646646192?utm_source=github_status&utm_medium=notification
> you will see something like this:
> 
>     Build jobs		View config
> 
> ! 5281.1 AMD64		Compiler: clang Xcode: xcode10.1 C	no environment variables set	8 min 20 sec
> ! 5281.2 AMD64		Compiler: gcc Xcode: xcode10.1 C	no environment variables set	8 min 23 sec
> X 5281.3 AMD64		Compiler: clang Xcode: xcode10.1 C	no environment variables set	1 min 57 sec
> X 5281.4 AMD64		Compiler: gcc Xcode: xcode10.1 C	no environment variables set	2 min 41 sec
> ! 5281.5 AMD64		Xcode: xcode10.1 C			jobname=GIT_TEST_GETTEXT_POISON	5 min 14 sec
> X 5281.6 AMD64		Xcode: xcode10.1 C			jobname=linux-gcc-4.8		1 min 13 sec
> ! 5281.7 AMD64		Xcode: xcode10.1 C			jobname=Linux32			6 min 50 sec
> ✓ 5281.8 AMD64		Xcode: xcode10.1 C			jobname=StaticAnalysis		10 min 56 sec
> ✓ 5281.9 AMD64		Xcode: xcode10.1 C			jobname=Documentation		6 min 15 sec

I don't see any 'linux-gcc' and 'linux-clang' jobnames.

Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 2255 bytes --]

Hi Gábor,

On Thu, 6 Feb 2020, SZEDER Gábor wrote:

> On Thu, Feb 06, 2020 at 09:57:51AM +0100, Johannes Schindelin wrote:
> >
> > On Thu, 6 Feb 2020, SZEDER Gábor wrote:
> >
> > > On Wed, Feb 05, 2020 at 01:01:50PM -0800, Junio C Hamano wrote:
> > > >
> > > > If linux-gcc and linux-clang labels are not visible, linux-clang-py2
> > > > and osx-py3 would not be, either, so...
> > >
> > > The 'linux-gcc' and 'linux-clang' labels are not visible on Travis CI,
> > > because those jobs as part of the build matrix, and, consequently, we
> > > can't set the a 'jobname' environment variable for them in
> > > '.travis.yml'.  If we were to include additional jobs for the Python
> > > scripts, then for those we can (and should!) set
> > > 'jobname=linux-python' or something, and that would be visible on the
> > > Travis CI web interface, just like e.g. 'jobname=StaticAnalysis'.
> >
> > I think we can see that jobname very well, though. If you direct your web
> > browser to
> > https://travis-ci.org/git/git/builds/646646192?utm_source=github_status&utm_medium=notification
> > you will see something like this:
> >
> >     Build jobs		View config
> >
> > ! 5281.1 AMD64		Compiler: clang Xcode: xcode10.1 C	no environment variables set	8 min 20 sec
> > ! 5281.2 AMD64		Compiler: gcc Xcode: xcode10.1 C	no environment variables set	8 min 23 sec
> > X 5281.3 AMD64		Compiler: clang Xcode: xcode10.1 C	no environment variables set	1 min 57 sec
> > X 5281.4 AMD64		Compiler: gcc Xcode: xcode10.1 C	no environment variables set	2 min 41 sec
> > ! 5281.5 AMD64		Xcode: xcode10.1 C			jobname=GIT_TEST_GETTEXT_POISON	5 min 14 sec
> > X 5281.6 AMD64		Xcode: xcode10.1 C			jobname=linux-gcc-4.8		1 min 13 sec
> > ! 5281.7 AMD64		Xcode: xcode10.1 C			jobname=Linux32			6 min 50 sec
> > ✓ 5281.8 AMD64		Xcode: xcode10.1 C			jobname=StaticAnalysis		10 min 56 sec
> > ✓ 5281.9 AMD64		Xcode: xcode10.1 C			jobname=Documentation		6 min 15 sec
>
> I don't see any 'linux-gcc' and 'linux-clang' jobnames.

Ah, that's what you meant. Indeed, those are not displayed because we set
them at the wrong layer, they would need to be defined in `.travis.yml`.
You probably _can_ use variables in the `jobname`.

Ciao,
Dscho