[ANNOUNCE] Git v2.21.0-rc2

[ANNOUNCE] Git v2.21.0-rc2

[Junio C Hamano]

A release candidate Git v2.21.0-rc2 is now available for testing
at the usual places.  It is comprised of 474 non-merge commits
since v2.20.0, contributed by 61 people, 16 of which are new faces.

The tarballs are found at:

    https://www.kernel.org/pub/software/scm/git/testing/

The following public repositories all have a copy of the
'v2.21.0-rc2' tag and the 'master' branch that the tag points at:

  url = https://kernel.googlesource.com/pub/scm/git/git
  url = git://repo.or.cz/alt-git.git
  url = https://github.com/gitster/git

New contributors whose contributions weren't in v2.20.0 are as follows.
Welcome to the Git development community!

  Arti Zirk, Brandon Richardson, Chayoung You, Denis Ovsienko,
  Emilio Cobos Álvarez, Erin Dahlgren, Force Charlie, Frank Dana,
  Issac Trotts, Katrin Leinweber, Laura Abbott, Patrick Hogg,
  Peter Osterlund, Shahzad Lone, Slavica Djukic, and Tanushree
  Tumane.

Returning contributors who helped this release are as follows.
Thanks for your continued support.

  Ævar Arnfjörð Bjarmason, Ben Peart, Brandon Williams, brian
  m. carlson, Carlo Marcelo Arenas Belón, Christian Couder,
  David Turner, Derrick Stolee, Elijah Newren, Eric Sunshine,
  Eric Wong, Jean-Noël Avila, Jeff King, Johannes Schindelin,
  Jonathan Nieder, Jonathan Tan, Josh Steadmon, Junio C Hamano,
  Kevin Daudt, Kim Gybels, Kyle Meyer, Linus Torvalds, Luke
  Diamand, Martin Ågren, Masaya Suzuki, Matthew DeVore, Matthieu
  Moy, Max Kirillov, Nguyễn Thái Ngọc Duy, Olga Telezhnaya,
  Orgad Shaneh, Phillip Wood, Pranit Bauva, Ramsay Jones,
  Randall S. Becker, René Scharfe, Sebastian Staudt, Sergey
  Organov, Stefan Beller, Stephen P. Smith, Sven van Haastregt,
  SZEDER Gábor, Thomas Braun, Thomas Gummerer, Todd Zullinger,
  and Torsten Bögershausen.

----------------------------------------------------------------

Git 2.21 Release Notes (draft)
==============================

Backward Compatibility Notes
----------------------------

 * Historically, the "-m" (mainline) option can only be used for "git
   cherry-pick" and "git revert" when working with a merge commit.
   This version of Git no longer warns or errors out when working with
   a single-parent commit, as long as the argument to the "-m" option
   is 1 (i.e. it has only one parent, and the request is to pick or
   revert relative to that first parent).  Scripts that relied on the
   behaviour may get broken with this change.


Updates since v2.20
-------------------

UI, Workflows & Features

 * The "http.version" configuration variable can be used with recent
   enough versions of cURL library to force the version of HTTP used
   to talk when fetching and pushing.

 * Small fixes and features for fast-export and fast-import, mostly on
   the fast-export side has been made.

 * "git push $there $src:$dst" rejects when $dst is not a fully
   qualified refname and it is not clear what the end user meant.  The
   codepath has been taught to give a clearer error message, and also
   guess where the push should go by taking the type of the pushed
   object into account (e.g. a tag object would want to go under
   refs/tags/).

 * "git checkout [<tree-ish>] path..." learned to report the number of
   paths that have been checked out of the index or the tree-ish,
   which gives it the same degree of noisy-ness as the case in which
   the command checks out a branch.  "git checkout -m <pathspec>" to
   undo conflict resolution gives a similar message.

 * "git quiltimport" learned "--keep-non-patch" option.

 * "git worktree remove" and "git worktree move" refused to work when
   there is a submodule involved.  This has been loosened to ignore
   uninitialized submodules.

 * "git cherry-pick -m1" was forbidden when picking a non-merge
   commit, even though there _is_ parent number 1 for such a commit.
   This was done to avoid mistakes back when "cherry-pick" was about
   picking a single commit, but is no longer useful with "cherry-pick"
   that can pick a range of commits.  Now the "-m$num" option is
   allowed when picking any commit, as long as $num names an existing
   parent of the commit.

 * Update "git multimail" from the upstream.

 * "git p4" update.

 * The "--format=<placeholder>" option of for-each-ref, branch and tag
   learned to show a few more traits of objects that can be learned by
   the object_info API.

 * "git rebase -i" learned to re-execute a command given with 'exec'
   to run after it failed the last time.

 * "git diff --color-moved-ws" updates.

 * Custom userformat "log --format" learned %S atom that stands for
   the tip the traversal reached the commit from, i.e. --source.

 * "git instaweb" learned to drive http.server that comes with
   "batteries included" Python installation (both Python2 & 3).

 * A new encoding UTF-16LE-BOM has been invented to force encoding to
   UTF-16 with BOM in little endian byte order, which cannot be directly
   generated by using iconv.

 * A new date format "--date=human" that morphs its output depending
   on how far the time is from the current time has been introduced.
   "--date=auto:human" can be used to use this new format (or any
   existing format) when the output is going to the pager or to the
   terminal, and otherwise the default format.


Performance, Internal Implementation, Development Support etc.

 * Code clean-up with optimization for the codepath that checks
   (non-)existence of loose objects.

 * More codepaths have become aware of working with in-core repository
   instances other than the default "the_repository".

 * The "strncat()" function is now among the banned functions.

 * Portability updates for the HPE NonStop platform.

 * Earlier we added "-Wformat-security" to developer builds, assuming
   that "-Wall" (which includes "-Wformat" which in turn is required
   to use "-Wformat-security") is always in effect.  This is not true
   when config.mak.autogen is in use, unfortunately.  This has been
   fixed by unconditionally adding "-Wall" to developer builds.

 * The loose object cache used to optimize existence look-up has been
   updated.

 * Flaky tests can now be repeatedly run under load with the
   "--stress" option.

 * Documentation/Makefile is getting prepared for manpage
   localization.

 * "git fetch-pack" now can talk the version 2 protocol.

 * sha-256 hash has been added and plumbed through the code to allow
   building Git with the "NewHash".

 * Debugging help for http transport.

 * "git fetch --deepen=<more>" has been corrected to work over v2
   protocol.

 * The code to walk tree objects has been taught that we may be
   working with object names that are not computed with SHA-1.

 * The in-core repository instances are passed through more codepaths.

 * Update the protocol message specification to allow only the limited
   use of scaled quantities.  This is to ensure potential compatibility
   issues will not get out of hand.

 * Micro-optimize the code that prepares commit objects to be walked
   by "git rev-list" when the commit-graph is available.

 * "git fetch" and "git upload-pack" learned to send all exchanges over
   the sideband channel while talking the v2 protocol.

 * The codepath to write out commit-graph has been optimized by
   following the usual pattern of visiting objects in in-pack order.

 * The codepath to show progress meter while writing out commit-graph
   file has been improved.

 * Cocci rules have been updated to encourage use of strbuf_addbuf().

 * "git rebase --merge" has been reimplemented by reusing the internal
   machinery used for "git rebase -i".

 * More code in "git bisect" has been rewritten in C.

 * Instead of going through "git-rebase--am" scriptlet to use the "am"
   backend, the built-in version of "git rebase" learned to drive the
   "am" backend directly.

 * The assumption to work on the single "in-core index" instance has
   been reduced from the library-ish part of the codebase.

 * The test lint learned to catch non-portable "sed" options.

 * "git pack-objects" learned another algorithm to compute the set of
   objects to send, that trades the resulting packfile off to save
   traversal cost to favor small pushes.

 * The travis CI scripts have been corrected to build Git with the
   compiler(s) of our choice.

 * "git submodule update" learned to abort early when core.worktree
   for the submodule is not set correctly to prevent spreading damage.

 * Test suite has been adjusted to run on Azure Pipeline.

 * Running "Documentation/doc-diff x" from anywhere other than the
   top-level of the working tree did not show the usage string
   correctly, which has been fixed.

 * Use of the sparse tool got easier to customize from the command
   line to help developers.

 * A new target "coverage-prove" to run the coverage test under
   "prove" has been added.

 * A flakey "p4" test has been removed.

 * The code and tests assume that the system supplied iconv() would
   always use BOM in its output when asked to encode to UTF-16 (or
   UTF-32), but apparently some implementations output big-endian
   without BOM.  A compile-time knob has been added to help such
   systems (e.g. NonStop) to add BOM to the output to increase
   portability.


Fixes since v2.20
-----------------

 * Updates for corner cases in merge-recursive.
   (merge cc4cb0902c en/merge-path-collision later to maint).

 * "git checkout frotz" (without any double-dash) avoids ambiguity by
   making sure 'frotz' cannot be interpreted as a revision and as a
   path at the same time.  This safety has been updated to check also
   a unique remote-tracking branch 'frotz' in a remote, when dwimming
   to create a local branch 'frotz' out of a remote-tracking branch
   'frotz' from a remote.
   (merge be4908f103 nd/checkout-dwim-fix later to maint).

 * Refspecs configured with "git -c var=val clone" did not propagate
   to the resulting repository, which has been corrected.
   (merge 7eae4a3ac4 sg/clone-initial-fetch-configuration later to maint).

 * A properly configured username/email is required under
   user.useConfigOnly in order to create commits; now "git stash"
   (even though it creates commit objects to represent stash entries)
   command is exempt from the requirement.
   (merge 3bc2111fc2 sd/stash-wo-user-name later to maint).

 * The http-backend CGI process did not correctly clean up the child
   processes it spawns to run upload-pack etc. when it dies itself,
   which has been corrected.
   (merge 02818a98d7 mk/http-backend-kill-children-before-exit later to maint).

 * "git rev-list --exclude-promisor-objects" had to take an object
   that does not exist locally (and is lazily available) from the
   command line without barfing, but the code dereferenced NULL.
   (merge 4cf67869b2 md/list-lazy-objects-fix later to maint).

 * The traversal over tree objects has learned to honor
   ":(attr:label)" pathspec match, which has been implemented only for
   enumerating paths on the filesystem.
   (merge 5a0b97b34c nd/attr-pathspec-in-tree-walk later to maint).

 * BSD port updates.
   (merge 4e3ecbd439 cb/openbsd-allows-reading-directory later to maint).
   (merge b6bdc2a0f5 cb/t5004-empty-tar-archive-fix later to maint).
   (merge 82cbc8cde2 cb/test-lint-cp-a later to maint).

 * Lines that begin with a certain keyword that come over the wire, as
   well as lines that consist only of one of these keywords, ought to
   be painted in color for easier eyeballing, but the latter was
   broken ever since the feature was introduced in 2.19, which has
   been corrected.
   (merge 1f67290450 hn/highlight-sideband-keywords later to maint).

 * "git log -G<regex>" looked for a hunk in the "git log -p" patch
   output that contained a string that matches the given pattern.
   Optimize this code to ignore binary files, which by default will
   not show any hunk that would match any pattern (unless textconv or
   the --text option is in effect, that is).
   (merge e0e7cb8080 tb/log-G-binary later to maint).

 * "git submodule update" ought to use a single job unless asked, but
   by mistake used multiple jobs, which has been fixed.
   (merge e3a9d1aca9 sb/submodule-fetchjobs-default-to-one later to maint).

 * "git stripspace" should be usable outside a git repository, but
   under the "-s" or "-c" mode, it didn't.
   (merge 957da75802 jn/stripspace-wo-repository later to maint).

 * Some of the documentation pages formatted incorrectly with
   Asciidoctor, which have been fixed.
   (merge b62eb1d2f4 ma/asciidoctor later to maint).

 * The core.worktree setting in a submodule repository should not be
   pointing at a directory when the submodule loses its working tree
   (e.g. getting deinit'ed), but the code did not properly maintain
   this invariant.

 * With zsh, "git cmd path<TAB>" was completed to "git cmd path name"
   when the completed path has a special character like SP in it,
   without any attempt to keep "path name" a single filename.  This
   has been fixed to complete it to "git cmd path\ name" just like
   Bash completion does.

 * The test suite tried to see if it is run under bash, but the check
   itself failed under some other implementations of shell (notably
   under NetBSD).  This has been corrected.
   (merge 54ea72f09c sg/test-bash-version-fix later to maint).

 * "git gc" and "git repack" did not close the open packfiles that
   they found unneeded before removing them, which didn't work on a
   platform incapable of removing an open file.  This has been
   corrected.
   (merge 5bdece0d70 js/gc-repack-close-before-remove later to maint).

 * The code to drive GIT_EXTERNAL_DIFF command relied on the string
   returned from getenv() to be non-volatile, which is not true, that
   has been corrected.
   (merge 6776a84dae kg/external-diff-save-env later to maint).

 * There were many places the code relied on the string returned from
   getenv() to be non-volatile, which is not true, that have been
   corrected.
   (merge 0da0e9268b jk/save-getenv-result later to maint).

 * The v2 upload-pack protocol implementation failed to honor
   hidden-ref configuration, which has been corrected.
   (merge e20b4192a3 jk/proto-v2-hidden-refs-fix later to maint).

 * "git fetch --recurse-submodules" may not fetch the necessary commit
   that is bound to the superproject, which is getting corrected.
   (merge be76c21282 sb/submodule-recursive-fetch-gets-the-tip later to maint).

 * "git rebase" internally runs "checkout" to switch between branches,
   and the command used to call the post-checkout hook, but the
   reimplementation stopped doing so, which is getting fixed.

 * "git add -e" got confused when the change it wants to let the user
   edit is smaller than the previous change that was left over in a
   temporary file.
   (merge fa6f225e01 js/add-e-clear-patch-before-stating later to maint).

 * "git p4" failed to update a shelved change when there were moved
   files, which has been corrected.
   (merge 7a10946ab9 ld/git-p4-shelve-update-fix later to maint).

 * The codepath to read from the commit-graph file attempted to read
   past the end of it when the file's table-of-contents was corrupt.

 * The compat/obstack code had casts that -Wcast-function-type
   compilation option found questionable.
   (merge 764473d257 sg/obstack-cast-function-type-fix later to maint).

 * An obvious typo in an assertion error message has been fixed.
   (merge 3c27e2e059 cc/test-ref-store-typofix later to maint).

 * In Git for Windows, "git clone \\server\share\path" etc. that uses
   UNC paths from command line had bad interaction with its shell
   emulation.

 * "git add --ignore-errors" did not work as advertised and instead
   worked as an unintended synonym for "git add --renormalize", which
   has been fixed.
   (merge e2c2a37545 jk/add-ignore-errors-bit-assignment-fix later to maint).

 * On a case-insensitive filesystem, we failed to compare the part of
   the path that is above the worktree directory in an absolute
   pathname, which has been corrected.

 * Asking "git check-attr" about a macro (e.g. "binary") on a specific
   path did not work correctly, even though "git check-attr -a" listed
   such a macro correctly.  This has been corrected.
   (merge 7b95849be4 jk/attr-macro-fix later to maint).

 * "git pack-objects" incorrectly used uninitialized mutex, which has
   been corrected.
   (merge edb673cf10 ph/pack-objects-mutex-fix later to maint).

 * "git checkout -b <new> [HEAD]" to create a new branch from the
   current commit and check it out ought to be a no-op in the index
   and the working tree in normal cases, but there are corner cases
   that do require updates to the index and the working tree.  Running
   it immediately after "git clone --no-checkout" is one of these
   cases that an earlier optimization kicked in incorrectly, which has
   been fixed.
   (merge 8424bfd45b bp/checkout-new-branch-optim later to maint).

 * "git diff --color-moved --cc --stat -p" did not work well due to
   funny interaction between a bug in color-moved and the rest, which
   has been fixed.
   (merge dac03b5518 jk/diff-cc-stat-fixes later to maint).

 * When GIT_SEQUENCE_EDITOR is set, the command was incorrectly
   started when modes of "git rebase" that implicitly uses the
   machinery for the interactive rebase are run, which has been
   corrected.
   (merge 891d4a0313 pw/no-editor-in-rebase-i-implicit later to maint).

 * The commit-graph facility did not work when in-core objects that
   are promoted from unknown type to commit (e.g. a commit that is
   accessed via a tag that refers to it) were involved, which has been
   corrected.
   (merge 4468d4435c sg/object-as-type-commit-graph-fix later to maint).

 * "git fetch" output cleanup.
   (merge dc40b24df4 nd/fetch-compact-update later to maint).

 * "git cat-file --batch" reported a dangling symbolic link by
   mistake, when it wanted to report that a given name is ambiguous.

 * Documentation around core.crlf has been updated.
   (merge c9446f0504 jk/autocrlf-overrides-eol-doc later to maint).

 * The documentation of "git commit-tree" said that the command
   understands "--gpg-sign" in addition to "-S", but the command line
   parser did not know about the longhand, which has been corrected.

 * "git rebase -x $cmd" did not reject multi-line command, even though
   the command is incapable of handling such a command.  It now is
   rejected upfront.
   (merge c762aada1a pw/rebase-x-sanity-check later to maint).

 * Output from "git help" was not correctly aligned, which has been
   fixed.
   (merge 6195a76da4 nd/help-align-command-desc later to maint).

 * The "git submodule summary" subcommand showed shortened commit
   object names by mechanically truncating them at 7-hexdigit, which
   has been improved to let "rev-parse --short" scale the length of
   the abbreviation with the size of the repository.
   (merge 0586a438f6 sh/submodule-summary-abbrev-fix later to maint).

 * The way the OSX build jobs updates its build environment used the
   "--quiet" option to "brew update" command, but it wasn't all that
   quiet to be useful.  The use of the option has been replaced with
   an explicit redirection to the /dev/null (which incidentally would
   have worked around a breakage by recent updates to homebrew, which
   has fixed itself already).
   (merge a1ccaedd62 sg/travis-osx-brew-breakage-workaround later to maint).

 * "git --work-tree=$there --git-dir=$here describe --dirty" did not
   work correctly as it did not pay attention to the location of the
   worktree specified by the user by mistake, which has been
   corrected.
   (merge c801170b0c ss/describe-dirty-in-the-right-directory later to maint).

 * "git fetch" over protocol v2 that needs to make a second connection
   to backfill tags did not clear a variable that holds shallow
   repository information correctly, leading to an access of freed
   piece of memory.

 * Some errors from the other side coming over smart HTTP transport
   were not noticed, which has been corrected.

 * Code cleanup, docfix, build fix, etc.
   (merge 89ba9a79ae hb/t0061-dot-in-path-fix later to maint).
   (merge d173e799ea sb/diff-color-moved-config-option-fixup later to maint).
   (merge a8f5a59067 en/directory-renames-nothanks-doc-update later to maint).
   (merge ec36c42a63 nd/indentation-fix later to maint).
   (merge f116ee21cd do/gitweb-strict-export-conf-doc later to maint).
   (merge 112ea42663 fd/gitweb-snapshot-conf-doc-fix later to maint).
   (merge 1cadad6f65 tb/use-common-win32-pathfuncs-on-cygwin later to maint).
   (merge 57e9dcaa65 km/rebase-doc-typofix later to maint).
   (merge b8b4cb27e6 ds/gc-doc-typofix later to maint).
   (merge 3b3357626e nd/style-opening-brace later to maint).
   (merge b4583d5595 es/doc-worktree-guessremote-config later to maint).
   (merge cce99cd8c6 ds/commit-graph-assert-missing-parents later to maint).
   (merge 0650614982 cy/completion-typofix later to maint).
   (merge 6881925ef5 rs/sha1-file-close-mapped-file-on-error later to maint).
   (merge bd8d6f0def en/show-ref-doc-fix later to maint).
   (merge 1747125e2c cc/partial-clone-doc-typofix later to maint).
   (merge e01378753d cc/fetch-error-message-fix later to maint).
   (merge 54e8c11215 jk/remote-insteadof-cleanup later to maint).
   (merge d609615f48 js/test-git-installed later to maint).
   (merge ba170517be ja/doc-style-fix later to maint).
   (merge 86fb1c4e77 km/init-doc-typofix later to maint).
   (merge 5cfd4a9d10 nd/commit-doc later to maint).
   (merge 9fce19a431 ab/diff-tree-doc-fix later to maint).
   (merge 2e285e7803 tz/gpg-test-fix later to maint).
   (merge 5427de960b kl/pretty-doc-markup-fix later to maint).
   (merge 3815f64b0d js/mingw-host-cpu later to maint).
   (merge 5fe81438b5 rj/sequencer-sign-off-header-static later to maint).
   (merge 18a4f6be6b nd/fileno-may-be-macro later to maint).
   (merge 99e9ab54ab kd/t0028-octal-del-is-377-not-777 later to maint).

RE: [ANNOUNCE] Git v2.21.0-rc2

[Randall S. Becker]

On February 19, 2019 18:29, Junio C Hamano wrote:
> A release candidate Git v2.21.0-rc2 is now available for testing at the usual
> places.  It is comprised of 474 non-merge commits since v2.20.0, contributed
> by 61 people, 16 of which are new faces.

Thanks. t5562 works properly on NonStop (3 tests under various loads). I am starting a full regression test now - see you in two days or so with a full report.

Regards,
Randall

Re: [ANNOUNCE] Git v2.21.0-rc2

[Junio C Hamano]

"Randall S. Becker" <rsbecker@nexbridge.com> writes:

> On February 19, 2019 18:29, Junio C Hamano wrote:
>> A release candidate Git v2.21.0-rc2 is now available for testing at the usual
>> places.  It is comprised of 474 non-merge commits since v2.20.0, contributed
>> by 61 people, 16 of which are new faces.
>
> Thanks. t5562 works properly on NonStop (3 tests under various
> loads). I am starting a full regression test now - see you in two
> days or so with a full report.

Thanks.  In the end I ended up doing the following:

 - kept most of your "feed NULs generated from a helper program via
   pipe, instead of reading from /dev/zero" patch,

 - replaced the helper program used in the above with the test-tool
   one by Dscho,

 - used Max's "do not reuse the output file---the background writer
   about to die may still have it open when the next test starts",

 - used Max's "when testing the program with overly large input
   declared with content-length, do not have to feed from /dev/zero;
   /dev/null suffices as a mistaken program attempting to read would
   die when seeing EOF anyway" patch to reduce one mention of
   /dev/zero (or one pipe from the helper output) further.

Hopefully Dscho's Azure thing would also be happy with one less use
of Perl script.

As I said earlier, a few places in t/ still mention /dev/zero and I
think it is a real bug on platforms without /dev/zero in t4153:

    $ git grep -h dev/zero t/
            test_must_fail test_terminal git am --3way </dev/zero &&
            test_must_fail test_terminal git am --reject </dev/zero &&

Curiously, applying the attached patch and running the test on Linux
would notice that we do not have such a device file, and the use of
test_must_fail does *not* make it a-OK [*].  I am not sure why you
are not getting hit by the same issue.

	Side note: test_must_fail is to ensure that the command
	exits in a controlled way without segfault, so at the first
	glance, replacing /dev/zero with something nonsensical and
	nonexistent ought to make the command pass, but that is not
	the case.  The shell notices redirection failure and aborts
	the &&-chain.  This behaviour matches what we want, so I
	won't complain ;-)

Another mention of /dev/zero appears in t/helper/test-sha1.sh (not
to be confused with t/helper/test-sha1.c).  This seems to be run
only with an explicit "make -C t/helper check-sha1" request, so
perhaps nobody on your platform ran it to get hit by it.  I wonder
if anybody runs this on any platform, to be honest, though.  Duy
Cc'ed as the last person to touch that particular target---yes, I
know I was guilty of introducing it in b65bc21e ("Makefile: add
framework to verify and bench sha1 implementations.", 2006-06-24).

 t/t4153-am-resume-override-opts.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/t4153-am-resume-override-opts.sh b/t/t4153-am-resume-override-opts.sh
index 8ea22d1bcb..29ef22c94f 100755
--- a/t/t4153-am-resume-override-opts.sh
+++ b/t/t4153-am-resume-override-opts.sh
@@ -40,7 +40,7 @@ test_expect_success TTY '--3way overrides --no-3way' '
 
 	# Applying side1 with am --3way will succeed due to the threeway-merge.
 	# Applying side2 will fail as --3way does not apply to it.
-	test_must_fail test_terminal git am --3way </dev/zero &&
+	test_must_fail test_terminal git am --3way </dev/zarro &&
 	test_path_is_dir .git/rebase-apply &&
 	test side1 = "$(cat file2)"
 '
@@ -94,7 +94,7 @@ test_expect_success TTY '--reject overrides --no-reject' '
 	test_path_is_dir .git/rebase-apply &&
 	test_path_is_missing file.rej &&
 
-	test_must_fail test_terminal git am --reject </dev/zero &&
+	test_must_fail test_terminal git am --reject </dev/zarro &&
 	test_path_is_dir .git/rebase-apply &&
 	test_path_is_file file.rej
 '

Re: [ANNOUNCE] Git v2.21.0-rc2

[Johannes Schindelin]

Hi Junio,

On Wed, 20 Feb 2019, Junio C Hamano wrote:

> Hopefully Dscho's Azure thing would also be happy with one less use
> of Perl script.

My "Azure thing" does not time out anymore, indeed!

Thank you so much, Junio,
Dscho

Git for Windows v2.21.0-rc2, was Re: [ANNOUNCE] Git v2.21.0-rc2

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 23761 bytes --]

Team,

the Git for Windows v2.21.0-rc pre-release followed suite last night:
https://github.com/git-for-windows/git/releases/tag/v2.21.0-rc2.windows.1

Please test (and don't worry when I don't reply during the next few days,
please, I'll reply, no worries).

Ciao,
Johannes

On Tue, 19 Feb 2019, Junio C Hamano wrote:

> A release candidate Git v2.21.0-rc2 is now available for testing
> at the usual places.  It is comprised of 474 non-merge commits
> since v2.20.0, contributed by 61 people, 16 of which are new faces.
> 
> The tarballs are found at:
> 
>     https://www.kernel.org/pub/software/scm/git/testing/
> 
> The following public repositories all have a copy of the
> 'v2.21.0-rc2' tag and the 'master' branch that the tag points at:
> 
>   url = https://kernel.googlesource.com/pub/scm/git/git
>   url = git://repo.or.cz/alt-git.git
>   url = https://github.com/gitster/git
> 
> New contributors whose contributions weren't in v2.20.0 are as follows.
> Welcome to the Git development community!
> 
>   Arti Zirk, Brandon Richardson, Chayoung You, Denis Ovsienko,
>   Emilio Cobos Álvarez, Erin Dahlgren, Force Charlie, Frank Dana,
>   Issac Trotts, Katrin Leinweber, Laura Abbott, Patrick Hogg,
>   Peter Osterlund, Shahzad Lone, Slavica Djukic, and Tanushree
>   Tumane.
> 
> Returning contributors who helped this release are as follows.
> Thanks for your continued support.
> 
>   Ævar Arnfjörð Bjarmason, Ben Peart, Brandon Williams, brian
>   m. carlson, Carlo Marcelo Arenas Belón, Christian Couder,
>   David Turner, Derrick Stolee, Elijah Newren, Eric Sunshine,
>   Eric Wong, Jean-Noël Avila, Jeff King, Johannes Schindelin,
>   Jonathan Nieder, Jonathan Tan, Josh Steadmon, Junio C Hamano,
>   Kevin Daudt, Kim Gybels, Kyle Meyer, Linus Torvalds, Luke
>   Diamand, Martin Ågren, Masaya Suzuki, Matthew DeVore, Matthieu
>   Moy, Max Kirillov, Nguyễn Thái Ngọc Duy, Olga Telezhnaya,
>   Orgad Shaneh, Phillip Wood, Pranit Bauva, Ramsay Jones,
>   Randall S. Becker, René Scharfe, Sebastian Staudt, Sergey
>   Organov, Stefan Beller, Stephen P. Smith, Sven van Haastregt,
>   SZEDER Gábor, Thomas Braun, Thomas Gummerer, Todd Zullinger,
>   and Torsten Bögershausen.
> 
> ----------------------------------------------------------------
> 
> Git 2.21 Release Notes (draft)
> ==============================
> 
> Backward Compatibility Notes
> ----------------------------
> 
>  * Historically, the "-m" (mainline) option can only be used for "git
>    cherry-pick" and "git revert" when working with a merge commit.
>    This version of Git no longer warns or errors out when working with
>    a single-parent commit, as long as the argument to the "-m" option
>    is 1 (i.e. it has only one parent, and the request is to pick or
>    revert relative to that first parent).  Scripts that relied on the
>    behaviour may get broken with this change.
> 
> 
> Updates since v2.20
> -------------------
> 
> UI, Workflows & Features
> 
>  * The "http.version" configuration variable can be used with recent
>    enough versions of cURL library to force the version of HTTP used
>    to talk when fetching and pushing.
> 
>  * Small fixes and features for fast-export and fast-import, mostly on
>    the fast-export side has been made.
> 
>  * "git push $there $src:$dst" rejects when $dst is not a fully
>    qualified refname and it is not clear what the end user meant.  The
>    codepath has been taught to give a clearer error message, and also
>    guess where the push should go by taking the type of the pushed
>    object into account (e.g. a tag object would want to go under
>    refs/tags/).
> 
>  * "git checkout [<tree-ish>] path..." learned to report the number of
>    paths that have been checked out of the index or the tree-ish,
>    which gives it the same degree of noisy-ness as the case in which
>    the command checks out a branch.  "git checkout -m <pathspec>" to
>    undo conflict resolution gives a similar message.
> 
>  * "git quiltimport" learned "--keep-non-patch" option.
> 
>  * "git worktree remove" and "git worktree move" refused to work when
>    there is a submodule involved.  This has been loosened to ignore
>    uninitialized submodules.
> 
>  * "git cherry-pick -m1" was forbidden when picking a non-merge
>    commit, even though there _is_ parent number 1 for such a commit.
>    This was done to avoid mistakes back when "cherry-pick" was about
>    picking a single commit, but is no longer useful with "cherry-pick"
>    that can pick a range of commits.  Now the "-m$num" option is
>    allowed when picking any commit, as long as $num names an existing
>    parent of the commit.
> 
>  * Update "git multimail" from the upstream.
> 
>  * "git p4" update.
> 
>  * The "--format=<placeholder>" option of for-each-ref, branch and tag
>    learned to show a few more traits of objects that can be learned by
>    the object_info API.
> 
>  * "git rebase -i" learned to re-execute a command given with 'exec'
>    to run after it failed the last time.
> 
>  * "git diff --color-moved-ws" updates.
> 
>  * Custom userformat "log --format" learned %S atom that stands for
>    the tip the traversal reached the commit from, i.e. --source.
> 
>  * "git instaweb" learned to drive http.server that comes with
>    "batteries included" Python installation (both Python2 & 3).
> 
>  * A new encoding UTF-16LE-BOM has been invented to force encoding to
>    UTF-16 with BOM in little endian byte order, which cannot be directly
>    generated by using iconv.
> 
>  * A new date format "--date=human" that morphs its output depending
>    on how far the time is from the current time has been introduced.
>    "--date=auto:human" can be used to use this new format (or any
>    existing format) when the output is going to the pager or to the
>    terminal, and otherwise the default format.
> 
> 
> Performance, Internal Implementation, Development Support etc.
> 
>  * Code clean-up with optimization for the codepath that checks
>    (non-)existence of loose objects.
> 
>  * More codepaths have become aware of working with in-core repository
>    instances other than the default "the_repository".
> 
>  * The "strncat()" function is now among the banned functions.
> 
>  * Portability updates for the HPE NonStop platform.
> 
>  * Earlier we added "-Wformat-security" to developer builds, assuming
>    that "-Wall" (which includes "-Wformat" which in turn is required
>    to use "-Wformat-security") is always in effect.  This is not true
>    when config.mak.autogen is in use, unfortunately.  This has been
>    fixed by unconditionally adding "-Wall" to developer builds.
> 
>  * The loose object cache used to optimize existence look-up has been
>    updated.
> 
>  * Flaky tests can now be repeatedly run under load with the
>    "--stress" option.
> 
>  * Documentation/Makefile is getting prepared for manpage
>    localization.
> 
>  * "git fetch-pack" now can talk the version 2 protocol.
> 
>  * sha-256 hash has been added and plumbed through the code to allow
>    building Git with the "NewHash".
> 
>  * Debugging help for http transport.
> 
>  * "git fetch --deepen=<more>" has been corrected to work over v2
>    protocol.
> 
>  * The code to walk tree objects has been taught that we may be
>    working with object names that are not computed with SHA-1.
> 
>  * The in-core repository instances are passed through more codepaths.
> 
>  * Update the protocol message specification to allow only the limited
>    use of scaled quantities.  This is to ensure potential compatibility
>    issues will not get out of hand.
> 
>  * Micro-optimize the code that prepares commit objects to be walked
>    by "git rev-list" when the commit-graph is available.
> 
>  * "git fetch" and "git upload-pack" learned to send all exchanges over
>    the sideband channel while talking the v2 protocol.
> 
>  * The codepath to write out commit-graph has been optimized by
>    following the usual pattern of visiting objects in in-pack order.
> 
>  * The codepath to show progress meter while writing out commit-graph
>    file has been improved.
> 
>  * Cocci rules have been updated to encourage use of strbuf_addbuf().
> 
>  * "git rebase --merge" has been reimplemented by reusing the internal
>    machinery used for "git rebase -i".
> 
>  * More code in "git bisect" has been rewritten in C.
> 
>  * Instead of going through "git-rebase--am" scriptlet to use the "am"
>    backend, the built-in version of "git rebase" learned to drive the
>    "am" backend directly.
> 
>  * The assumption to work on the single "in-core index" instance has
>    been reduced from the library-ish part of the codebase.
> 
>  * The test lint learned to catch non-portable "sed" options.
> 
>  * "git pack-objects" learned another algorithm to compute the set of
>    objects to send, that trades the resulting packfile off to save
>    traversal cost to favor small pushes.
> 
>  * The travis CI scripts have been corrected to build Git with the
>    compiler(s) of our choice.
> 
>  * "git submodule update" learned to abort early when core.worktree
>    for the submodule is not set correctly to prevent spreading damage.
> 
>  * Test suite has been adjusted to run on Azure Pipeline.
> 
>  * Running "Documentation/doc-diff x" from anywhere other than the
>    top-level of the working tree did not show the usage string
>    correctly, which has been fixed.
> 
>  * Use of the sparse tool got easier to customize from the command
>    line to help developers.
> 
>  * A new target "coverage-prove" to run the coverage test under
>    "prove" has been added.
> 
>  * A flakey "p4" test has been removed.
> 
>  * The code and tests assume that the system supplied iconv() would
>    always use BOM in its output when asked to encode to UTF-16 (or
>    UTF-32), but apparently some implementations output big-endian
>    without BOM.  A compile-time knob has been added to help such
>    systems (e.g. NonStop) to add BOM to the output to increase
>    portability.
> 
> 
> Fixes since v2.20
> -----------------
> 
>  * Updates for corner cases in merge-recursive.
>    (merge cc4cb0902c en/merge-path-collision later to maint).
> 
>  * "git checkout frotz" (without any double-dash) avoids ambiguity by
>    making sure 'frotz' cannot be interpreted as a revision and as a
>    path at the same time.  This safety has been updated to check also
>    a unique remote-tracking branch 'frotz' in a remote, when dwimming
>    to create a local branch 'frotz' out of a remote-tracking branch
>    'frotz' from a remote.
>    (merge be4908f103 nd/checkout-dwim-fix later to maint).
> 
>  * Refspecs configured with "git -c var=val clone" did not propagate
>    to the resulting repository, which has been corrected.
>    (merge 7eae4a3ac4 sg/clone-initial-fetch-configuration later to maint).
> 
>  * A properly configured username/email is required under
>    user.useConfigOnly in order to create commits; now "git stash"
>    (even though it creates commit objects to represent stash entries)
>    command is exempt from the requirement.
>    (merge 3bc2111fc2 sd/stash-wo-user-name later to maint).
> 
>  * The http-backend CGI process did not correctly clean up the child
>    processes it spawns to run upload-pack etc. when it dies itself,
>    which has been corrected.
>    (merge 02818a98d7 mk/http-backend-kill-children-before-exit later to maint).
> 
>  * "git rev-list --exclude-promisor-objects" had to take an object
>    that does not exist locally (and is lazily available) from the
>    command line without barfing, but the code dereferenced NULL.
>    (merge 4cf67869b2 md/list-lazy-objects-fix later to maint).
> 
>  * The traversal over tree objects has learned to honor
>    ":(attr:label)" pathspec match, which has been implemented only for
>    enumerating paths on the filesystem.
>    (merge 5a0b97b34c nd/attr-pathspec-in-tree-walk later to maint).
> 
>  * BSD port updates.
>    (merge 4e3ecbd439 cb/openbsd-allows-reading-directory later to maint).
>    (merge b6bdc2a0f5 cb/t5004-empty-tar-archive-fix later to maint).
>    (merge 82cbc8cde2 cb/test-lint-cp-a later to maint).
> 
>  * Lines that begin with a certain keyword that come over the wire, as
>    well as lines that consist only of one of these keywords, ought to
>    be painted in color for easier eyeballing, but the latter was
>    broken ever since the feature was introduced in 2.19, which has
>    been corrected.
>    (merge 1f67290450 hn/highlight-sideband-keywords later to maint).
> 
>  * "git log -G<regex>" looked for a hunk in the "git log -p" patch
>    output that contained a string that matches the given pattern.
>    Optimize this code to ignore binary files, which by default will
>    not show any hunk that would match any pattern (unless textconv or
>    the --text option is in effect, that is).
>    (merge e0e7cb8080 tb/log-G-binary later to maint).
> 
>  * "git submodule update" ought to use a single job unless asked, but
>    by mistake used multiple jobs, which has been fixed.
>    (merge e3a9d1aca9 sb/submodule-fetchjobs-default-to-one later to maint).
> 
>  * "git stripspace" should be usable outside a git repository, but
>    under the "-s" or "-c" mode, it didn't.
>    (merge 957da75802 jn/stripspace-wo-repository later to maint).
> 
>  * Some of the documentation pages formatted incorrectly with
>    Asciidoctor, which have been fixed.
>    (merge b62eb1d2f4 ma/asciidoctor later to maint).
> 
>  * The core.worktree setting in a submodule repository should not be
>    pointing at a directory when the submodule loses its working tree
>    (e.g. getting deinit'ed), but the code did not properly maintain
>    this invariant.
> 
>  * With zsh, "git cmd path<TAB>" was completed to "git cmd path name"
>    when the completed path has a special character like SP in it,
>    without any attempt to keep "path name" a single filename.  This
>    has been fixed to complete it to "git cmd path\ name" just like
>    Bash completion does.
> 
>  * The test suite tried to see if it is run under bash, but the check
>    itself failed under some other implementations of shell (notably
>    under NetBSD).  This has been corrected.
>    (merge 54ea72f09c sg/test-bash-version-fix later to maint).
> 
>  * "git gc" and "git repack" did not close the open packfiles that
>    they found unneeded before removing them, which didn't work on a
>    platform incapable of removing an open file.  This has been
>    corrected.
>    (merge 5bdece0d70 js/gc-repack-close-before-remove later to maint).
> 
>  * The code to drive GIT_EXTERNAL_DIFF command relied on the string
>    returned from getenv() to be non-volatile, which is not true, that
>    has been corrected.
>    (merge 6776a84dae kg/external-diff-save-env later to maint).
> 
>  * There were many places the code relied on the string returned from
>    getenv() to be non-volatile, which is not true, that have been
>    corrected.
>    (merge 0da0e9268b jk/save-getenv-result later to maint).
> 
>  * The v2 upload-pack protocol implementation failed to honor
>    hidden-ref configuration, which has been corrected.
>    (merge e20b4192a3 jk/proto-v2-hidden-refs-fix later to maint).
> 
>  * "git fetch --recurse-submodules" may not fetch the necessary commit
>    that is bound to the superproject, which is getting corrected.
>    (merge be76c21282 sb/submodule-recursive-fetch-gets-the-tip later to maint).
> 
>  * "git rebase" internally runs "checkout" to switch between branches,
>    and the command used to call the post-checkout hook, but the
>    reimplementation stopped doing so, which is getting fixed.
> 
>  * "git add -e" got confused when the change it wants to let the user
>    edit is smaller than the previous change that was left over in a
>    temporary file.
>    (merge fa6f225e01 js/add-e-clear-patch-before-stating later to maint).
> 
>  * "git p4" failed to update a shelved change when there were moved
>    files, which has been corrected.
>    (merge 7a10946ab9 ld/git-p4-shelve-update-fix later to maint).
> 
>  * The codepath to read from the commit-graph file attempted to read
>    past the end of it when the file's table-of-contents was corrupt.
> 
>  * The compat/obstack code had casts that -Wcast-function-type
>    compilation option found questionable.
>    (merge 764473d257 sg/obstack-cast-function-type-fix later to maint).
> 
>  * An obvious typo in an assertion error message has been fixed.
>    (merge 3c27e2e059 cc/test-ref-store-typofix later to maint).
> 
>  * In Git for Windows, "git clone \\server\share\path" etc. that uses
>    UNC paths from command line had bad interaction with its shell
>    emulation.
> 
>  * "git add --ignore-errors" did not work as advertised and instead
>    worked as an unintended synonym for "git add --renormalize", which
>    has been fixed.
>    (merge e2c2a37545 jk/add-ignore-errors-bit-assignment-fix later to maint).
> 
>  * On a case-insensitive filesystem, we failed to compare the part of
>    the path that is above the worktree directory in an absolute
>    pathname, which has been corrected.
> 
>  * Asking "git check-attr" about a macro (e.g. "binary") on a specific
>    path did not work correctly, even though "git check-attr -a" listed
>    such a macro correctly.  This has been corrected.
>    (merge 7b95849be4 jk/attr-macro-fix later to maint).
> 
>  * "git pack-objects" incorrectly used uninitialized mutex, which has
>    been corrected.
>    (merge edb673cf10 ph/pack-objects-mutex-fix later to maint).
> 
>  * "git checkout -b <new> [HEAD]" to create a new branch from the
>    current commit and check it out ought to be a no-op in the index
>    and the working tree in normal cases, but there are corner cases
>    that do require updates to the index and the working tree.  Running
>    it immediately after "git clone --no-checkout" is one of these
>    cases that an earlier optimization kicked in incorrectly, which has
>    been fixed.
>    (merge 8424bfd45b bp/checkout-new-branch-optim later to maint).
> 
>  * "git diff --color-moved --cc --stat -p" did not work well due to
>    funny interaction between a bug in color-moved and the rest, which
>    has been fixed.
>    (merge dac03b5518 jk/diff-cc-stat-fixes later to maint).
> 
>  * When GIT_SEQUENCE_EDITOR is set, the command was incorrectly
>    started when modes of "git rebase" that implicitly uses the
>    machinery for the interactive rebase are run, which has been
>    corrected.
>    (merge 891d4a0313 pw/no-editor-in-rebase-i-implicit later to maint).
> 
>  * The commit-graph facility did not work when in-core objects that
>    are promoted from unknown type to commit (e.g. a commit that is
>    accessed via a tag that refers to it) were involved, which has been
>    corrected.
>    (merge 4468d4435c sg/object-as-type-commit-graph-fix later to maint).
> 
>  * "git fetch" output cleanup.
>    (merge dc40b24df4 nd/fetch-compact-update later to maint).
> 
>  * "git cat-file --batch" reported a dangling symbolic link by
>    mistake, when it wanted to report that a given name is ambiguous.
> 
>  * Documentation around core.crlf has been updated.
>    (merge c9446f0504 jk/autocrlf-overrides-eol-doc later to maint).
> 
>  * The documentation of "git commit-tree" said that the command
>    understands "--gpg-sign" in addition to "-S", but the command line
>    parser did not know about the longhand, which has been corrected.
> 
>  * "git rebase -x $cmd" did not reject multi-line command, even though
>    the command is incapable of handling such a command.  It now is
>    rejected upfront.
>    (merge c762aada1a pw/rebase-x-sanity-check later to maint).
> 
>  * Output from "git help" was not correctly aligned, which has been
>    fixed.
>    (merge 6195a76da4 nd/help-align-command-desc later to maint).
> 
>  * The "git submodule summary" subcommand showed shortened commit
>    object names by mechanically truncating them at 7-hexdigit, which
>    has been improved to let "rev-parse --short" scale the length of
>    the abbreviation with the size of the repository.
>    (merge 0586a438f6 sh/submodule-summary-abbrev-fix later to maint).
> 
>  * The way the OSX build jobs updates its build environment used the
>    "--quiet" option to "brew update" command, but it wasn't all that
>    quiet to be useful.  The use of the option has been replaced with
>    an explicit redirection to the /dev/null (which incidentally would
>    have worked around a breakage by recent updates to homebrew, which
>    has fixed itself already).
>    (merge a1ccaedd62 sg/travis-osx-brew-breakage-workaround later to maint).
> 
>  * "git --work-tree=$there --git-dir=$here describe --dirty" did not
>    work correctly as it did not pay attention to the location of the
>    worktree specified by the user by mistake, which has been
>    corrected.
>    (merge c801170b0c ss/describe-dirty-in-the-right-directory later to maint).
> 
>  * "git fetch" over protocol v2 that needs to make a second connection
>    to backfill tags did not clear a variable that holds shallow
>    repository information correctly, leading to an access of freed
>    piece of memory.
> 
>  * Some errors from the other side coming over smart HTTP transport
>    were not noticed, which has been corrected.
> 
>  * Code cleanup, docfix, build fix, etc.
>    (merge 89ba9a79ae hb/t0061-dot-in-path-fix later to maint).
>    (merge d173e799ea sb/diff-color-moved-config-option-fixup later to maint).
>    (merge a8f5a59067 en/directory-renames-nothanks-doc-update later to maint).
>    (merge ec36c42a63 nd/indentation-fix later to maint).
>    (merge f116ee21cd do/gitweb-strict-export-conf-doc later to maint).
>    (merge 112ea42663 fd/gitweb-snapshot-conf-doc-fix later to maint).
>    (merge 1cadad6f65 tb/use-common-win32-pathfuncs-on-cygwin later to maint).
>    (merge 57e9dcaa65 km/rebase-doc-typofix later to maint).
>    (merge b8b4cb27e6 ds/gc-doc-typofix later to maint).
>    (merge 3b3357626e nd/style-opening-brace later to maint).
>    (merge b4583d5595 es/doc-worktree-guessremote-config later to maint).
>    (merge cce99cd8c6 ds/commit-graph-assert-missing-parents later to maint).
>    (merge 0650614982 cy/completion-typofix later to maint).
>    (merge 6881925ef5 rs/sha1-file-close-mapped-file-on-error later to maint).
>    (merge bd8d6f0def en/show-ref-doc-fix later to maint).
>    (merge 1747125e2c cc/partial-clone-doc-typofix later to maint).
>    (merge e01378753d cc/fetch-error-message-fix later to maint).
>    (merge 54e8c11215 jk/remote-insteadof-cleanup later to maint).
>    (merge d609615f48 js/test-git-installed later to maint).
>    (merge ba170517be ja/doc-style-fix later to maint).
>    (merge 86fb1c4e77 km/init-doc-typofix later to maint).
>    (merge 5cfd4a9d10 nd/commit-doc later to maint).
>    (merge 9fce19a431 ab/diff-tree-doc-fix later to maint).
>    (merge 2e285e7803 tz/gpg-test-fix later to maint).
>    (merge 5427de960b kl/pretty-doc-markup-fix later to maint).
>    (merge 3815f64b0d js/mingw-host-cpu later to maint).
>    (merge 5fe81438b5 rj/sequencer-sign-off-header-static later to maint).
>    (merge 18a4f6be6b nd/fileno-may-be-macro later to maint).
>    (merge 99e9ab54ab kd/t0028-octal-del-is-377-not-777 later to maint).
> 
> -- 
> You received this message because you are subscribed to the Google Groups "git-packagers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to git-packagers+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/git-packagers/xmqq8sybz7b2.fsf%40gitster-ct.c.googlers.com.
> For more options, visit https://groups.google.com/d/optout.
> 

Re: [ANNOUNCE] Git v2.21.0-rc2

[Duy Nguyen]

On Thu, Feb 21, 2019 at 2:41 AM Junio C Hamano <gitster@pobox.com> wrote:
> Another mention of /dev/zero appears in t/helper/test-sha1.sh (not
> to be confused with t/helper/test-sha1.c).  This seems to be run
> only with an explicit "make -C t/helper check-sha1" request, so
> perhaps nobody on your platform ran it to get hit by it.  I wonder
> if anybody runs this on any platform, to be honest, though.

I vaguely remember Brian added something to test SHA-512 performance
but I can't find it. Maybe when he adds something (if it's not there
already) we can retire this script, or update it to check sha-512 too.

> Duy Cc'ed as the last person to touch that particular target---yes, I
> know I was guilty of introducing it in b65bc21e ("Makefile: add
> framework to verify and bench sha1 implementations.", 2006-06-24).
-- 
Duy

RE: [ANNOUNCE] Git v2.21.0-rc2

[Randall S. Becker]

On February 20, 2019 14:42, Junio C Hamano wrote:
> "Randall S. Becker" <rsbecker@nexbridge.com> writes:
> 
> > On February 19, 2019 18:29, Junio C Hamano wrote:
> >> A release candidate Git v2.21.0-rc2 is now available for testing at
> >> the usual places.  It is comprised of 474 non-merge commits since
> >> v2.20.0, contributed by 61 people, 16 of which are new faces.
> >
> > Thanks. t5562 works properly on NonStop (3 tests under various loads).
> > I am starting a full regression test now - see you in two days or so
> > with a full report.
> 
> Thanks.  In the end I ended up doing the following:
> 
>  - kept most of your "feed NULs generated from a helper program via
>    pipe, instead of reading from /dev/zero" patch,
> 
>  - replaced the helper program used in the above with the test-tool
>    one by Dscho,
> 
>  - used Max's "do not reuse the output file---the background writer
>    about to die may still have it open when the next test starts",
> 
>  - used Max's "when testing the program with overly large input
>    declared with content-length, do not have to feed from /dev/zero;
>    /dev/null suffices as a mistaken program attempting to read would
>    die when seeing EOF anyway" patch to reduce one mention of
>    /dev/zero (or one pipe from the helper output) further.
> 
> Hopefully Dscho's Azure thing would also be happy with one less use of
Perl
> script.
> 
> As I said earlier, a few places in t/ still mention /dev/zero and I think
it is a
> real bug on platforms without /dev/zero in t4153:
> 
>     $ git grep -h dev/zero t/
>             test_must_fail test_terminal git am --3way </dev/zero &&
>             test_must_fail test_terminal git am --reject </dev/zero &&
> 
> Curiously, applying the attached patch and running the test on Linux would
> notice that we do not have such a device file, and the use of
test_must_fail
> does *not* make it a-OK [*].  I am not sure why you are not getting hit by
> the same issue.
> 
> 	Side note: test_must_fail is to ensure that the command
> 	exits in a controlled way without segfault, so at the first
> 	glance, replacing /dev/zero with something nonsensical and
> 	nonexistent ought to make the command pass, but that is not
> 	the case.  The shell notices redirection failure and aborts
> 	the &&-chain.  This behaviour matches what we want, so I
> 	won't complain ;-)
> 
> Another mention of /dev/zero appears in t/helper/test-sha1.sh (not to be
> confused with t/helper/test-sha1.c).  This seems to be run only with an
> explicit "make -C t/helper check-sha1" request, so perhaps nobody on your
> platform ran it to get hit by it.  I wonder if anybody runs this on any
platform,
> to be honest, though.  Duy Cc'ed as the last person to touch that
particular
> target---yes, I know I was guilty of introducing it in b65bc21e
("Makefile: add
> framework to verify and bench sha1 implementations.", 2006-06-24).
> 
>  t/t4153-am-resume-override-opts.sh | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/t/t4153-am-resume-override-opts.sh b/t/t4153-am-resume-
> override-opts.sh
> index 8ea22d1bcb..29ef22c94f 100755
> --- a/t/t4153-am-resume-override-opts.sh
> +++ b/t/t4153-am-resume-override-opts.sh
> @@ -40,7 +40,7 @@ test_expect_success TTY '--3way overrides --no-3way' '
> 
>  	# Applying side1 with am --3way will succeed due to the threeway-
> merge.
>  	# Applying side2 will fail as --3way does not apply to it.
> -	test_must_fail test_terminal git am --3way </dev/zero &&
> +	test_must_fail test_terminal git am --3way </dev/zarro &&
>  	test_path_is_dir .git/rebase-apply &&
>  	test side1 = "$(cat file2)"
>  '
> @@ -94,7 +94,7 @@ test_expect_success TTY '--reject overrides --no-reject'
> '
>  	test_path_is_dir .git/rebase-apply &&
>  	test_path_is_missing file.rej &&
> 
> -	test_must_fail test_terminal git am --reject </dev/zero &&
> +	test_must_fail test_terminal git am --reject </dev/zarro &&
>  	test_path_is_dir .git/rebase-apply &&
>  	test_path_is_file file.rej
>  '

Here's what we get with this patch (and without it):

/home/git/git/t: sh t4153-am-resume-override-opts.sh
ok 1 - setup
ok 2 # skip --3way overrides --no-3way (missing TTY)
ok 3 - --no-quiet overrides --quiet
ok 4 - --signoff overrides --no-signoff
ok 5 # skip --reject overrides --no-reject (missing TTY)
# passed all 5 test(s)
1..5

Technically, we have TTY, but not that dev. The TTYs are not under
/dev/tty*, however.

Regards,
Randall

[PATCH 0/2] BSD portability fixes for 2.21.0-rc2

[Ævar Arnfjörð Bjarmason]

Last-minute fixes for two issues new in 2.21, not in 2.20.

This fixes a NetBSD portability issue I noted in
https://public-inbox.org/git/87d0nm5x1d.fsf@evledraar.gmail.com/
thanks to SZEDER for the help.

Then there's a another fix for a test breaking on OpenBSD due to
recently merged unportable regex syntax.

Ævar Arnfjörð Bjarmason (2):
  tests: fix unportable "\?" and "\+" regex syntax
  commit-graph tests: fix cryptic unportable "dd" invocation

 t/t5318-commit-graph.sh | 3 ++-
 t/t6120-describe.sh     | 8 ++++----
 2 files changed, 6 insertions(+), 5 deletions(-)

-- 
2.21.0.rc0.258.g878e2cd30e

[PATCH 1/2] tests: fix unportable "\?" and "\+" regex syntax

[Ævar Arnfjörð Bjarmason]

Fix widely supported but non-POSIX basic regex syntax introduced in
[1] and [2]. On GNU, NetBSD and FreeBSD the following works:

    $ echo xy >f
    $ grep 'xy\?' f; echo $?
    xy
    0

The same goes for "\+". The "?" and "+" syntax is not in the BRE
syntax, just in ERE, but on some implementations it can be invoked by
prefixing the meta-operator with "\", but not on OpenBSD:

    $ uname -a
    OpenBSD obsd.my.domain 6.2 GENERIC#132 amd64
    $ grep --version
    grep version 0.9
    $ grep 'xy\?' f; echo $?
    1

Let's fix this by moving to ERE syntax instead, where "?" and "+" are
universally supported:

    $ grep -E 'xy?' f; echo $?
    xy
    0

1. 2ed5c8e174 ("describe: setup working tree for --dirty", 2019-02-03)
2. c801170b0c ("t6120: test for describe with a bare repository",
   2019-02-03)

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t6120-describe.sh | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/t/t6120-describe.sh b/t/t6120-describe.sh
index ee5b03ee18..2b883d8174 100755
--- a/t/t6120-describe.sh
+++ b/t/t6120-describe.sh
@@ -146,7 +146,7 @@ check_describe A-* HEAD
 test_expect_success 'describe works from outside repo using --git-dir' '
 	git clone --bare "$TRASH_DIRECTORY" "$TRASH_DIRECTORY/bare" &&
 	git --git-dir "$TRASH_DIRECTORY/bare" describe >out &&
-	grep "^A-[1-9][0-9]\?-g[0-9a-f]\+$" out
+	grep -E "^A-[1-9][0-9]?-g[0-9a-f]+$" out
 '
 
 check_describe "A-*[0-9a-f]" --dirty
@@ -156,7 +156,7 @@ test_expect_success 'describe --dirty with --work-tree' '
 		cd "$TEST_DIRECTORY" &&
 		git --git-dir "$TRASH_DIRECTORY/.git" --work-tree "$TRASH_DIRECTORY" describe --dirty >"$TRASH_DIRECTORY/out"
 	) &&
-	grep "^A-[1-9][0-9]\?-g[0-9a-f]\+$" out
+	grep -E "^A-[1-9][0-9]?-g[0-9a-f]+$" out
 '
 
 test_expect_success 'set-up dirty work tree' '
@@ -170,7 +170,7 @@ test_expect_success 'describe --dirty with --work-tree (dirty)' '
 		cd "$TEST_DIRECTORY" &&
 		git --git-dir "$TRASH_DIRECTORY/.git" --work-tree "$TRASH_DIRECTORY" describe --dirty >"$TRASH_DIRECTORY/out"
 	) &&
-	grep "^A-[1-9][0-9]\?-g[0-9a-f]\+-dirty$" out
+	grep -E "^A-[1-9][0-9]?-g[0-9a-f]+-dirty$" out
 '
 
 check_describe "A-*[0-9a-f].mod" --dirty=.mod
@@ -180,7 +180,7 @@ test_expect_success 'describe --dirty=.mod with --work-tree (dirty)' '
 		cd "$TEST_DIRECTORY" &&
 		git --git-dir "$TRASH_DIRECTORY/.git" --work-tree "$TRASH_DIRECTORY" describe --dirty=.mod >"$TRASH_DIRECTORY/out"
 	) &&
-	grep "^A-[1-9][0-9]\?-g[0-9a-f]\+.mod$" out
+	grep -E "^A-[1-9][0-9]?-g[0-9a-f]+.mod$" out
 '
 
 test_expect_success 'describe --dirty HEAD' '
-- 
2.21.0.rc0.258.g878e2cd30e

[PATCH 2/2] commit-graph tests: fix cryptic unportable "dd" invocation

[Ævar Arnfjörð Bjarmason]

Change an unportable invocation of "dd" that truncated the
commit-graph to call Perl's truncate() function instead.

In POSIX it is unspecified what happens when count=0 is
provided[1]. The NetBSD "dd" behavior differs from GNU (and seemingly
other BSDs), which as left this test broken since
d2b86fbaa1 ("commit-graph: fix buffer read-overflow", 2019-01-15).

In POSIX the truncate(2) and ftruncate(2) functions are
portable. We've used the latter since 271421cd34 ("Update partial HTTP
transfers.", 2005-09-30), but the truncate(1) command-line tool is
GNU-specific. Thus let's use Perl's version of it. We could also just
introduce a "test-tool truncate" in the future if we wanted to avoid
shelling out to perl.

On Linux and NetBSD we don't need the "if -s $ARGV[0] > $ARGV[1]"
condition I'm adding. We never have a $zero_pos longer than the file
being truncated. But let's have that condition to future-proof the
code, and because "the behavior is undefined if LENGTH is greater than
the length of the file" (perldoc -f truncate).

1. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t5318-commit-graph.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
index d4bd1522fe..d99bea6cce 100755
--- a/t/t5318-commit-graph.sh
+++ b/t/t5318-commit-graph.sh
@@ -382,7 +382,8 @@ corrupt_graph_and_verify() {
 	test_when_finished mv commit-graph-backup $objdir/info/commit-graph &&
 	cp $objdir/info/commit-graph commit-graph-backup &&
 	printf "$data" | dd of="$objdir/info/commit-graph" bs=1 seek="$pos" conv=notrunc &&
-	dd of="$objdir/info/commit-graph" bs=1 seek="$zero_pos" count=0 &&
+	perl -we 'truncate $ARGV[0], $ARGV[1] if -s $ARGV[0] > $ARGV[1]' \
+		$objdir/info/commit-graph $zero_pos &&
 	generate_zero_bytes $(($orig_size - $zero_pos)) >>"$objdir/info/commit-graph" &&
 	test_must_fail git commit-graph verify 2>test_err &&
 	grep -v "^+" test_err >err &&
-- 
2.21.0.rc0.258.g878e2cd30e

RE: [ANNOUNCE] Git v2.21.0-rc2

[Randall S. Becker]

On February 21, 2019 10:55, I wrote:
> On February 20, 2019 14:42, Junio C Hamano wrote:
> > "Randall S. Becker" <rsbecker@nexbridge.com> writes:
> >
> > > On February 19, 2019 18:29, Junio C Hamano wrote:
> > >> A release candidate Git v2.21.0-rc2 is now available for testing at
> > >> the usual places.  It is comprised of 474 non-merge commits since
> > >> v2.20.0, contributed by 61 people, 16 of which are new faces.
> > >
> > > Thanks. t5562 works properly on NonStop (3 tests under various loads).
> > > I am starting a full regression test now - see you in two days or so
> > > with a full report.
> >
> > Thanks.  In the end I ended up doing the following:
> >
> >  - kept most of your "feed NULs generated from a helper program via
> >    pipe, instead of reading from /dev/zero" patch,
> >
> >  - replaced the helper program used in the above with the test-tool
> >    one by Dscho,
> >
> >  - used Max's "do not reuse the output file---the background writer
> >    about to die may still have it open when the next test starts",
> >
> >  - used Max's "when testing the program with overly large input
> >    declared with content-length, do not have to feed from /dev/zero;
> >    /dev/null suffices as a mistaken program attempting to read would
> >    die when seeing EOF anyway" patch to reduce one mention of
> >    /dev/zero (or one pipe from the helper output) further.
> >
> > Hopefully Dscho's Azure thing would also be happy with one less use of
> > Perl script.
> >
> > As I said earlier, a few places in t/ still mention /dev/zero and I
> > think it is a real bug on platforms without /dev/zero in t4153:
> >
> >     $ git grep -h dev/zero t/
> >             test_must_fail test_terminal git am --3way </dev/zero &&
> >             test_must_fail test_terminal git am --reject </dev/zero &&
> >
> > Curiously, applying the attached patch and running the test on Linux
> > would notice that we do not have such a device file, and the use of
> > test_must_fail does *not* make it a-OK [*].  I am not sure why you are
> > not getting hit by the same issue.
> >
> > 	Side note: test_must_fail is to ensure that the command
> > 	exits in a controlled way without segfault, so at the first
> > 	glance, replacing /dev/zero with something nonsensical and
> > 	nonexistent ought to make the command pass, but that is not
> > 	the case.  The shell notices redirection failure and aborts
> > 	the &&-chain.  This behaviour matches what we want, so I
> > 	won't complain ;-)
> >
> > Another mention of /dev/zero appears in t/helper/test-sha1.sh (not to
> > be confused with t/helper/test-sha1.c).  This seems to be run only
> > with an explicit "make -C t/helper check-sha1" request, so perhaps
> > nobody on your platform ran it to get hit by it.  I wonder if anybody
> > runs this on any platform, to be honest, though.  Duy Cc'ed as the
> > last person to touch that particular target---yes, I know I was guilty
> > of introducing it in b65bc21e ("Makefile: add framework to verify and
> bench sha1 implementations.", 2006-06-24).
> >
> >  t/t4153-am-resume-override-opts.sh | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/t/t4153-am-resume-override-opts.sh b/t/t4153-am-resume-
> > override-opts.sh index 8ea22d1bcb..29ef22c94f 100755
> > --- a/t/t4153-am-resume-override-opts.sh
> > +++ b/t/t4153-am-resume-override-opts.sh
> > @@ -40,7 +40,7 @@ test_expect_success TTY '--3way overrides --no-
> 3way' '
> >
> >  	# Applying side1 with am --3way will succeed due to the threeway-
> > merge.
> >  	# Applying side2 will fail as --3way does not apply to it.
> > -	test_must_fail test_terminal git am --3way </dev/zero &&
> > +	test_must_fail test_terminal git am --3way </dev/zarro &&
> >  	test_path_is_dir .git/rebase-apply &&
> >  	test side1 = "$(cat file2)"
> >  '
> > @@ -94,7 +94,7 @@ test_expect_success TTY '--reject overrides --no-
> reject'
> > '
> >  	test_path_is_dir .git/rebase-apply &&
> >  	test_path_is_missing file.rej &&
> >
> > -	test_must_fail test_terminal git am --reject </dev/zero &&
> > +	test_must_fail test_terminal git am --reject </dev/zarro &&
> >  	test_path_is_dir .git/rebase-apply &&
> >  	test_path_is_file file.rej
> >  '
> 
> Here's what we get with this patch (and without it):
> 
> /home/git/git/t: sh t4153-am-resume-override-opts.sh ok 1 - setup ok 2 #
> skip --3way overrides --no-3way (missing TTY) ok 3 - --no-quiet overrides
--
> quiet ok 4 - --signoff overrides --no-signoff ok 5 # skip --reject
overrides --
> no-reject (missing TTY) # passed all 5 test(s)
> 1..5
> 
> Technically, we have TTY, but not that dev. The TTYs are not under
/dev/tty*,
> however.

Final NonStop test results on 2.21.0-rc2:
t0021 subtest 15 failed - apparently flakey test case
t9001 subtests 33,81,117,118,145 failed - Not new. This is unsurprising as
this never worked. It might be nice to fix, but $DAYJOB.
t9020 subtests 1,2,5,6 failed - Not new. unsurprising as there is no SVN or
perl with SVN module on platform. It might be useful to have a detection to
skip of Perl SVN is not present.

Overall, pass and much better than prior releases.

Cheers,
Randall

Re: [ANNOUNCE] Git v2.21.0-rc2

[Junio C Hamano]

"Randall S. Becker" <rsbecker@nexbridge.com> writes:

> Here's what we get with this patch (and without it):
>
> /home/git/git/t: sh t4153-am-resume-override-opts.sh
> ok 1 - setup
> ok 2 # skip --3way overrides --no-3way (missing TTY)
> ok 3 - --no-quiet overrides --quiet
> ok 4 - --signoff overrides --no-signoff
> ok 5 # skip --reject overrides --no-reject (missing TTY)
> # passed all 5 test(s)
> 1..5
>
> Technically, we have TTY, but not that dev. The TTYs are not under
> /dev/tty*, however.

Ah, OK, that explains it.

Re: [PATCH 2/2] commit-graph tests: fix cryptic unportable "dd" invocation

[SZEDER Gábor]

On Thu, Feb 21, 2019 at 08:28:49PM +0100, Ævar Arnfjörð Bjarmason wrote:
> Change an unportable invocation of "dd" that truncated the
> commit-graph to call Perl's truncate() function instead.
> 
> In POSIX it is unspecified what happens when count=0 is
> provided[1]. The NetBSD "dd" behavior differs from GNU (and seemingly
> other BSDs), which as left this test broken since
> d2b86fbaa1 ("commit-graph: fix buffer read-overflow", 2019-01-15).
> 
> In POSIX the truncate(2) and ftruncate(2) functions are
> portable. We've used the latter since 271421cd34 ("Update partial HTTP
> transfers.", 2005-09-30), but the truncate(1) command-line tool is
> GNU-specific. Thus let's use Perl's version of it. We could also just
> introduce a "test-tool truncate" in the future if we wanted to avoid
> shelling out to perl.
> 
> On Linux and NetBSD we don't need the "if -s $ARGV[0] > $ARGV[1]"
> condition I'm adding. We never have a $zero_pos longer than the file
> being truncated. But let's have that condition to future-proof the
> code, and because "the behavior is undefined if LENGTH is greater than
> the length of the file" (perldoc -f truncate).
> 
> 1. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html
> 
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  t/t5318-commit-graph.sh | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
> index d4bd1522fe..d99bea6cce 100755
> --- a/t/t5318-commit-graph.sh
> +++ b/t/t5318-commit-graph.sh
> @@ -382,7 +382,8 @@ corrupt_graph_and_verify() {
>  	test_when_finished mv commit-graph-backup $objdir/info/commit-graph &&
>  	cp $objdir/info/commit-graph commit-graph-backup &&
>  	printf "$data" | dd of="$objdir/info/commit-graph" bs=1 seek="$pos" conv=notrunc &&
> -	dd of="$objdir/info/commit-graph" bs=1 seek="$zero_pos" count=0 &&
> +	perl -we 'truncate $ARGV[0], $ARGV[1] if -s $ARGV[0] > $ARGV[1]' \
> +		$objdir/info/commit-graph $zero_pos &&

This will make Dscho unhappy :)

Is there a problem with:

  dd if=/dev/null of="$objdir/info/commit-graph" bs=1 seek="$zero_pos"

?

To my understanding of the specs it's well-defined what it should do,
even when $zero_pos is larget than the file size,  it's shorter,
simpler, and doesn't introduce yet another Perl dependency.


>  	generate_zero_bytes $(($orig_size - $zero_pos)) >>"$objdir/info/commit-graph" &&
>  	test_must_fail git commit-graph verify 2>test_err &&
>  	grep -v "^+" test_err >err &&
> -- 
> 2.21.0.rc0.258.g878e2cd30e
> 

Re: [ANNOUNCE] Git v2.21.0-rc2

[SZEDER Gábor]

On Thu, Feb 21, 2019 at 10:54:31AM -0500, Randall S. Becker wrote:
> /home/git/git/t: sh t4153-am-resume-override-opts.sh
> ok 1 - setup
> ok 2 # skip --3way overrides --no-3way (missing TTY)
> ok 3 - --no-quiet overrides --quiet
> ok 4 - --signoff overrides --no-signoff
> ok 5 # skip --reject overrides --no-reject (missing TTY)
> # passed all 5 test(s)
> 1..5
> 
> Technically, we have TTY, but not that dev. The TTYs are not under
> /dev/tty*, however.

The TTY prereq is not about /dev/tty, but about whether
't/test-terminal.perl' can simulate a pseudo-terminal, i.e. the
necessary Perl modules are installed and working appropriately.

RE: [ANNOUNCE] Git v2.21.0-rc2

[Randall S. Becker]

On February 21, 2019 16:07, SZEDER Gábor:
> On Thu, Feb 21, 2019 at 10:54:31AM -0500, Randall S. Becker wrote:
> > /home/git/git/t: sh t4153-am-resume-override-opts.sh ok 1 - setup ok 2
> > # skip --3way overrides --no-3way (missing TTY) ok 3 - --no-quiet
> > overrides --quiet ok 4 - --signoff overrides --no-signoff ok 5 # skip
> > --reject overrides --no-reject (missing TTY) # passed all 5 test(s)
> > 1..5
> >
> > Technically, we have TTY, but not that dev. The TTYs are not under
> > /dev/tty*, however.
> 
> The TTY prereq is not about /dev/tty, but about whether 't/test-terminal.perl'
> can simulate a pseudo-terminal, i.e. the necessary Perl modules are installed
> and working appropriately.

Good point. We don't have IO:Pty on the platform.

Re: [PATCH 2/2] commit-graph tests: fix cryptic unportable "dd" invocation

[Ævar Arnfjörð Bjarmason]

On Thu, Feb 21 2019, SZEDER Gábor wrote:

> On Thu, Feb 21, 2019 at 08:28:49PM +0100, Ævar Arnfjörð Bjarmason wrote:
>> Change an unportable invocation of "dd" that truncated the
>> commit-graph to call Perl's truncate() function instead.
>>
>> In POSIX it is unspecified what happens when count=0 is
>> provided[1]. The NetBSD "dd" behavior differs from GNU (and seemingly
>> other BSDs), which as left this test broken since
>> d2b86fbaa1 ("commit-graph: fix buffer read-overflow", 2019-01-15).
>>
>> In POSIX the truncate(2) and ftruncate(2) functions are
>> portable. We've used the latter since 271421cd34 ("Update partial HTTP
>> transfers.", 2005-09-30), but the truncate(1) command-line tool is
>> GNU-specific. Thus let's use Perl's version of it. We could also just
>> introduce a "test-tool truncate" in the future if we wanted to avoid
>> shelling out to perl.
>>
>> On Linux and NetBSD we don't need the "if -s $ARGV[0] > $ARGV[1]"
>> condition I'm adding. We never have a $zero_pos longer than the file
>> being truncated. But let's have that condition to future-proof the
>> code, and because "the behavior is undefined if LENGTH is greater than
>> the length of the file" (perldoc -f truncate).
>>
>> 1. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html
>>
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> ---
>>  t/t5318-commit-graph.sh | 3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
>> index d4bd1522fe..d99bea6cce 100755
>> --- a/t/t5318-commit-graph.sh
>> +++ b/t/t5318-commit-graph.sh
>> @@ -382,7 +382,8 @@ corrupt_graph_and_verify() {
>>  	test_when_finished mv commit-graph-backup $objdir/info/commit-graph &&
>>  	cp $objdir/info/commit-graph commit-graph-backup &&
>>  	printf "$data" | dd of="$objdir/info/commit-graph" bs=1 seek="$pos" conv=notrunc &&
>> -	dd of="$objdir/info/commit-graph" bs=1 seek="$zero_pos" count=0 &&
>> +	perl -we 'truncate $ARGV[0], $ARGV[1] if -s $ARGV[0] > $ARGV[1]' \
>> +		$objdir/info/commit-graph $zero_pos &&
>
> This will make Dscho unhappy :)

Sorry Dscho :)

Although this is a one-off in one test, as opposed to a new "perl -e" in
test-lib-functions.sh

> Is there a problem with:
>
>   dd if=/dev/null of="$objdir/info/commit-graph" bs=1 seek="$zero_pos"
>
> ?
>
> To my understanding of the specs it's well-defined what it should do,
> even when $zero_pos is larget than the file size,  it's shorter,
> simpler, and doesn't introduce yet another Perl dependency.

I tried that as a one-off and it indeed works as a "truncate" on NetBSD
& GNU.

My reading of POSIX "dd" and "lseek" docs is that we'd need some similar
guard if we're going to be paranoid about a $zero_pos value past the end
of the file. It doesn't look like that's portable, my assumption from
reading the docs is that the seek=* will devolve without a stat() check
on some "dd" implementations to an "lseek".

I'm not going to submit a re-roll of this because it works, and I'd
still trust Perl's truncate(...) portability over dd.

But more importantly because it takes me *ages* to fully re-test
anything on the slow BSD VMs I have access to, and I already tore town
my one-off hacking env there after testing these patches...

>>  	generate_zero_bytes $(($orig_size - $zero_pos)) >>"$objdir/info/commit-graph" &&
>>  	test_must_fail git commit-graph verify 2>test_err &&
>>  	grep -v "^+" test_err >err &&
>> --
>> 2.21.0.rc0.258.g878e2cd30e
>>

Re: [ANNOUNCE] Git v2.21.0-rc2

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1066 bytes --]

On Thu, Feb 21, 2019 at 08:10:00PM +0700, Duy Nguyen wrote:
> On Thu, Feb 21, 2019 at 2:41 AM Junio C Hamano <gitster@pobox.com> wrote:
> > Another mention of /dev/zero appears in t/helper/test-sha1.sh (not
> > to be confused with t/helper/test-sha1.c).  This seems to be run
> > only with an explicit "make -C t/helper check-sha1" request, so
> > perhaps nobody on your platform ran it to get hit by it.  I wonder
> > if anybody runs this on any platform, to be honest, though.
> 
> I vaguely remember Brian added something to test SHA-512 performance
> but I can't find it. Maybe when he adds something (if it's not there
> already) we can retire this script, or update it to check sha-512 too.

Doing git blame on that script seems to show that it was written mostly
by Junio. I think you may be thinking of test-tool hash-speed, which can
be used to benchmark a hash implementation. That's what I used for most
of my testing.

We may not need both anymore, though.
-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 868 bytes --]

Re: [PATCH 0/2] BSD portability fixes for 2.21.0-rc2

[Josh Steadmon]

On 2019.02.21 20:28, Ævar Arnfjörð Bjarmason wrote:
> Last-minute fixes for two issues new in 2.21, not in 2.20.
> 
> This fixes a NetBSD portability issue I noted in
> https://public-inbox.org/git/87d0nm5x1d.fsf@evledraar.gmail.com/
> thanks to SZEDER for the help.
> 
> Then there's a another fix for a test breaking on OpenBSD due to
> recently merged unportable regex syntax.
> 
> Ævar Arnfjörð Bjarmason (2):
>   tests: fix unportable "\?" and "\+" regex syntax
>   commit-graph tests: fix cryptic unportable "dd" invocation
> 
>  t/t5318-commit-graph.sh | 3 ++-
>  t/t6120-describe.sh     | 8 ++++----
>  2 files changed, 6 insertions(+), 5 deletions(-)
> 
> -- 
> 2.21.0.rc0.258.g878e2cd30e
> 

These both look good to me. Thanks!

Reviewed-by: Josh Steadmon <steadmon@google.com>

Re: [PATCH 1/2] tests: fix unportable "\?" and "\+" regex syntax

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> Fix widely supported but non-POSIX basic regex syntax introduced in
> [1] and [2]. On GNU, NetBSD and FreeBSD the following works:
>
>     $ echo xy >f
>     $ grep 'xy\?' f; echo $?
>     xy
>     0
>
> The same goes for "\+". The "?" and "+" syntax is not in the BRE
> syntax, just in ERE, but on some implementations it can be invoked by
> prefixing the meta-operator with "\", but not on OpenBSD:

Yup, thanks for testing, spotting and fixing.  I do recall finding
one of these during my review during recent couple of cycles, but
apparently eyeballing will not always catch everything X-<.

Re: [ANNOUNCE] Git v2.21.0-rc2

[Duy Nguyen]

On Fri, Feb 22, 2019 at 6:55 AM brian m. carlson
<sandals@crustytoothpaste.net> wrote:
>
> On Thu, Feb 21, 2019 at 08:10:00PM +0700, Duy Nguyen wrote:
> > On Thu, Feb 21, 2019 at 2:41 AM Junio C Hamano <gitster@pobox.com> wrote:
> > > Another mention of /dev/zero appears in t/helper/test-sha1.sh (not
> > > to be confused with t/helper/test-sha1.c).  This seems to be run
> > > only with an explicit "make -C t/helper check-sha1" request, so
> > > perhaps nobody on your platform ran it to get hit by it.  I wonder
> > > if anybody runs this on any platform, to be honest, though.
> >
> > I vaguely remember Brian added something to test SHA-512 performance
> > but I can't find it. Maybe when he adds something (if it's not there
> > already) we can retire this script, or update it to check sha-512 too.
>
> Doing git blame on that script seems to show that it was written mostly
> by Junio. I think you may be thinking of test-tool hash-speed, which can
> be used to benchmark a hash implementation. That's what I used for most
> of my testing.
>
> We may not need both anymore, though.

Well since you're basically killing sha-1 everywhere. I'll leave it to
you to delete check-sha1.sh if you delete hash-speed too :)
-- 
Duy

Re: [PATCH 2/2] commit-graph tests: fix cryptic unportable "dd" invocation

[SZEDER Gábor]

On Thu, Feb 21, 2019 at 11:26:26PM +0100, Ævar Arnfjörð Bjarmason wrote:
> >> -	dd of="$objdir/info/commit-graph" bs=1 seek="$zero_pos" count=0 &&
> >> +	perl -we 'truncate $ARGV[0], $ARGV[1] if -s $ARGV[0] > $ARGV[1]' \
> >> +		$objdir/info/commit-graph $zero_pos &&
> >
> > This will make Dscho unhappy :)
> 
> Sorry Dscho :)
> 
> Although this is a one-off in one test, as opposed to a new "perl -e" in
> test-lib-functions.sh
> 
> > Is there a problem with:
> >
> >   dd if=/dev/null of="$objdir/info/commit-graph" bs=1 seek="$zero_pos"
> >
> > ?
> >
> > To my understanding of the specs it's well-defined what it should do,
> > even when $zero_pos is larget than the file size,  it's shorter,
> > simpler, and doesn't introduce yet another Perl dependency.
> 
> I tried that as a one-off and it indeed works as a "truncate" on NetBSD
> & GNU.
> 
> My reading of POSIX "dd" and "lseek" docs is that we'd need some similar
> guard if we're going to be paranoid about a $zero_pos value past the end
> of the file. It doesn't look like that's portable, my assumption from
> reading the docs is that the seek=* will devolve without a stat() check
> on some "dd" implementations to an "lseek".

Could you point to the part of the specs where your assumption comes
from?  The specs are quite clear on what should happen:

  If the size of the seek plus the size of the input file is less than
  the previous size of the output file, the output file shall be
  shortened by the copy. If the input file is empty and either the
  size of the seek is greater than the previous size of the output
  file or the output file did not previously exist, the size of the
  output file shall be set to the file offset after the seek.

IOW no such guard is necessary.

I checked the man pages of FreeBSD's, NetBSD's, OpenBSD's and Solaris'
'dd', and they are clearly following the specs in this respect.  I
tried NetBSD 6.0's and 8.0's 'dd', and both behave as advertised.

And using 'dd' doesn't add a condition after statement...

> I'm not going to submit a re-roll of this because it works, and I'd
> still trust Perl's truncate(...) portability over dd.
>
> But more importantly because it takes me *ages* to fully re-test
> anything on the slow BSD VMs I have access to, and I already tore town
> my one-off hacking env there after testing these patches...
> 
> >>  	generate_zero_bytes $(($orig_size - $zero_pos)) >>"$objdir/info/commit-graph" &&
> >>  	test_must_fail git commit-graph verify 2>test_err &&
> >>  	grep -v "^+" test_err >err &&
> >> --
> >> 2.21.0.rc0.258.g878e2cd30e
> >>

Re: [PATCH 2/2] commit-graph tests: fix cryptic unportable "dd" invocation

[Ævar Arnfjörð Bjarmason]

On Fri, Feb 22 2019, SZEDER Gábor wrote:

> On Thu, Feb 21, 2019 at 11:26:26PM +0100, Ævar Arnfjörð Bjarmason wrote:
>> >> -	dd of="$objdir/info/commit-graph" bs=1 seek="$zero_pos" count=0 &&
>> >> +	perl -we 'truncate $ARGV[0], $ARGV[1] if -s $ARGV[0] > $ARGV[1]' \
>> >> +		$objdir/info/commit-graph $zero_pos &&
>> >
>> > This will make Dscho unhappy :)
>>
>> Sorry Dscho :)
>>
>> Although this is a one-off in one test, as opposed to a new "perl -e" in
>> test-lib-functions.sh
>>
>> > Is there a problem with:
>> >
>> >   dd if=/dev/null of="$objdir/info/commit-graph" bs=1 seek="$zero_pos"
>> >
>> > ?
>> >
>> > To my understanding of the specs it's well-defined what it should do,
>> > even when $zero_pos is larget than the file size,  it's shorter,
>> > simpler, and doesn't introduce yet another Perl dependency.
>>
>> I tried that as a one-off and it indeed works as a "truncate" on NetBSD
>> & GNU.
>>
>> My reading of POSIX "dd" and "lseek" docs is that we'd need some similar
>> guard if we're going to be paranoid about a $zero_pos value past the end
>> of the file. It doesn't look like that's portable, my assumption from
>> reading the docs is that the seek=* will devolve without a stat() check
>> on some "dd" implementations to an "lseek".
>
> Could you point to the part of the specs where your assumption comes
> from?  The specs are quite clear on what should happen:
>
>   If the size of the seek plus the size of the input file is less than
>   the previous size of the output file, the output file shall be
>   shortened by the copy. If the input file is empty and either the
>   size of the seek is greater than the previous size of the output
>   file or the output file did not previously exist, the size of the
>   output file shall be set to the file offset after the seek.
>
> IOW no such guard is necessary.

It was my reading of the seek=* section ("the implementation shall seek
to the specified offset"). I didn't spot that bit covered in of=*. Yeah,
I see that's defined & safe after reading that.

> I checked the man pages of FreeBSD's, NetBSD's, OpenBSD's and Solaris'
> 'dd', and they are clearly following the specs in this respect.  I
> tried NetBSD 6.0's and 8.0's 'dd', and both behave as advertised.
>
> And using 'dd' doesn't add a condition after statement...

>> I'm not going to submit a re-roll of this because it works, and I'd
>> still trust Perl's truncate(...) portability over dd.
>>
>> But more importantly because it takes me *ages* to fully re-test
>> anything on the slow BSD VMs I have access to, and I already tore town
>> my one-off hacking env there after testing these patches...
>>
>> >>  	generate_zero_bytes $(($orig_size - $zero_pos)) >>"$objdir/info/commit-graph" &&
>> >>  	test_must_fail git commit-graph verify 2>test_err &&
>> >>  	grep -v "^+" test_err >err &&
>> >> --
>> >> 2.21.0.rc0.258.g878e2cd30e
>> >>

Re: [PATCH 2/2] commit-graph tests: fix cryptic unportable "dd" invocation

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> It was my reading of the seek=* section ("the implementation shall seek
> to the specified offset"). I didn't spot that bit covered in of=*. Yeah,
> I see that's defined & safe after reading that.

OK, so...

-- >8 --
From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Thu, 21 Feb 2019 20:28:49 +0100
Subject: [PATCH] commit-graph tests: fix cryptic unportable "dd" invocation

Change an unportable invocation of "dd" with count=0, that wanted to
truncate the commit-graph file.  In POSIX it is unspecified what
happens when count=0 is provided[1]. The NetBSD "dd" behavior
differs from GNU (and seemingly other BSDs), which as left this test
broken since d2b86fbaa1 ("commit-graph: fix buffer read-overflow",
2019-01-15).

Copying from /dev/null would seek/truncate to seek=$zero_pos and
stop immediately after that (without being able to copy anything),
which is the right way to truncate the file.

1. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/t5318-commit-graph.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
index d4bd1522fe..561796f280 100755
--- a/t/t5318-commit-graph.sh
+++ b/t/t5318-commit-graph.sh
@@ -382,7 +382,7 @@ corrupt_graph_and_verify() {
 	test_when_finished mv commit-graph-backup $objdir/info/commit-graph &&
 	cp $objdir/info/commit-graph commit-graph-backup &&
 	printf "$data" | dd of="$objdir/info/commit-graph" bs=1 seek="$pos" conv=notrunc &&
-	dd of="$objdir/info/commit-graph" bs=1 seek="$zero_pos" count=0 &&
+	dd of="$objdir/info/commit-graph" bs=1 seek="$zero_pos" if=/dev/null &&
 	generate_zero_bytes $(($orig_size - $zero_pos)) >>"$objdir/info/commit-graph" &&
 	test_must_fail git commit-graph verify 2>test_err &&
 	grep -v "^+" test_err >err &&
-- 
2.21.0-rc2

Re: [PATCH 2/2] commit-graph tests: fix cryptic unportable "dd" invocation

[Todd Zullinger]

Junio C Hamano wrote:
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> 
>> It was my reading of the seek=* section ("the implementation shall seek
>> to the specified offset"). I didn't spot that bit covered in of=*. Yeah,
>> I see that's defined & safe after reading that.
> 
> OK, so...
> 
> -- >8 --
> From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> Date: Thu, 21 Feb 2019 20:28:49 +0100
> Subject: [PATCH] commit-graph tests: fix cryptic unportable "dd" invocation
> 
> Change an unportable invocation of "dd" with count=0, that wanted to
> truncate the commit-graph file.  In POSIX it is unspecified what
> happens when count=0 is provided[1]. The NetBSD "dd" behavior
> differs from GNU (and seemingly other BSDs), which as left this test

s/ as/ has/ ?

> broken since d2b86fbaa1 ("commit-graph: fix buffer read-overflow",
> 2019-01-15).
> 
> Copying from /dev/null would seek/truncate to seek=$zero_pos and
> stop immediately after that (without being able to copy anything),
> which is the right way to truncate the file.
> 
> 1. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html
> 
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  t/t5318-commit-graph.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
> index d4bd1522fe..561796f280 100755
> --- a/t/t5318-commit-graph.sh
> +++ b/t/t5318-commit-graph.sh
> @@ -382,7 +382,7 @@ corrupt_graph_and_verify() {
>  	test_when_finished mv commit-graph-backup $objdir/info/commit-graph &&
>  	cp $objdir/info/commit-graph commit-graph-backup &&
>  	printf "$data" | dd of="$objdir/info/commit-graph" bs=1 seek="$pos" conv=notrunc &&
> -	dd of="$objdir/info/commit-graph" bs=1 seek="$zero_pos" count=0 &&
> +	dd of="$objdir/info/commit-graph" bs=1 seek="$zero_pos" if=/dev/null &&
>  	generate_zero_bytes $(($orig_size - $zero_pos)) >>"$objdir/info/commit-graph" &&
>  	test_must_fail git commit-graph verify 2>test_err &&
>  	grep -v "^+" test_err >err &&

-- 
Todd

[PATCH v3] commit-graph tests: fix unportable "dd" invocation

[Junio C Hamano]

Todd Zullinger <tmz@pobox.com> writes:

> s/ as/ has/ ?

Oops and thanks.  Here is my second attempt (which makes it v3).

-- >8 --
From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Thu, 21 Feb 2019 20:28:49 +0100

Change an unportable invocation of "dd" with count=0, that wanted to
truncate the commit-graph file.  In POSIX it is unspecified what
happens when count=0 is provided[1]. The NetBSD "dd" behavior
differs from GNU (and seemingly other BSDs), which has left this test
broken since d2b86fbaa1 ("commit-graph: fix buffer read-overflow",
2019-01-15).

Copying from /dev/null would seek/truncate to seek=$zero_pos and
stop immediately after that (without being able to copy anything),
which is the right way to truncate the file.

1. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Helped-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 t/t5318-commit-graph.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
index d4bd1522fe..561796f280 100755
--- a/t/t5318-commit-graph.sh
+++ b/t/t5318-commit-graph.sh
@@ -382,7 +382,7 @@ corrupt_graph_and_verify() {
 	test_when_finished mv commit-graph-backup $objdir/info/commit-graph &&
 	cp $objdir/info/commit-graph commit-graph-backup &&
 	printf "$data" | dd of="$objdir/info/commit-graph" bs=1 seek="$pos" conv=notrunc &&
-	dd of="$objdir/info/commit-graph" bs=1 seek="$zero_pos" count=0 &&
+	dd of="$objdir/info/commit-graph" bs=1 seek="$zero_pos" if=/dev/null &&
 	generate_zero_bytes $(($orig_size - $zero_pos)) >>"$objdir/info/commit-graph" &&
 	test_must_fail git commit-graph verify 2>test_err &&
 	grep -v "^+" test_err >err &&
-- 
2.21.0-rc2