What's cooking in git.git (May 2012, #06; Wed, 23)

What's cooking in git.git (May 2012, #06; Wed, 23)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed with '-' are
only in 'pu' (proposed updates) while commits prefixed with '+' are in 'next'.

You can find the changes described here in the integration branches of the
repositories listed at

    http://git-blame.blogspot.com/p/git-public-repositories.html

--------------------------------------------------
[Graduated to "master"]

* bp/diff-no-index-strbuf-fix (2012-05-16) 2 commits
  (merged to 'next' on 2012-05-18 at b2c26bb)
 + diff --no-index: don't leak buffers in queue_diff
 + diff --no-index: reset temporary buffer lengths on directory iteration

Fix regressions to "git diff --no-index" when it recurses down.

* hv/submodule-alt-odb (2012-05-14) 1 commit
  (merged to 'next' on 2012-05-18 at 2f17ac8)
 + teach add_submodule_odb() to look for alternates

When peeking into object stores of submodules, the code forgot that they
might borrow objects from alternate object stores on their own.

* js/rebase-i-p-test-fix (2012-05-18) 1 commit
  (merged to 'next' on 2012-05-18 at b7482fc)
 + Fix t3411.3 to actually rebase something

* js/rev-parse-doc-fix (2012-05-18) 1 commit
  (merged to 'next' on 2012-05-18 at c16f1c7)
 + rev-parse doc: --git-dir does not always show a relative path

* rs/archive-tree-in-tip-simplify (2012-05-18) 2 commits
  (merged to 'next' on 2012-05-18 at a8ca614)
 + archive-tar: keep const in checksum calculation
 + archive: simplify refname handling

--------------------------------------------------
[New Topics]

* fc/git-prompt-script (2012-05-22) 5 commits
 - completion: split __git_ps1 into a separate script
 - completion: remove executable mode
 - Merge branch 'fc/git-complete-helper' into fc/git-prompt-script
 - tests: add tests for the bash prompt functions in the completion script
 - tests: move code to run tests under bash into a helper library
 (this branch is tangled with sg/bash-prompt.)

Looked reasonable from my point of view, but I am not familiar with the
direction in which larger "bash completion" project folks are going, so
people more knowledgeable than myself need to help me guiding this topic.

* jk/fetch-pack-remove-dups-optim (2012-05-22) 5 commits
  (merged to 'next' on 2012-05-23 at 6057c21)
 + fetch-pack: avoid quadratic loop in filter_refs
 + fetch-pack: sort the list of incoming refs
 + add sorting infrastructure for list refs
 + fetch-pack: avoid quadratic behavior in remove_duplicates
 + fetch-pack: sort incoming heads

The way "fetch-pack" taht is given multiple references to fetch tried to
remove duplicates was very inefficient.

Will merge to master.

* jk/ident-split-fix (2012-05-22) 1 commit
  (merged to 'next' on 2012-05-23 at 811c9ec)
 + fix off-by-one error in split_ident_line

An author/committer name that is a single character was mishandled as an
invalid name by mistake.

Will merge to master and later to maint.

* jk/pretty-commit-header-incomplete-line (2012-05-22) 1 commit
  (merged to 'next' on 2012-05-23 at f788433)
 + avoid segfault when reading header of malformed commits

Will merge to master and later to maint.

* mh/fetch-pack-constness (2012-05-22) 4 commits
  (merged to 'next' on 2012-05-23 at be95121)
 + cmd_fetch_pack(): respect constness of argv parameter
 + cmd_fetch_pack(): combine the loop termination conditions
 + cmd_fetch_pack(): handle non-option arguments outside of the loop
 + cmd_fetch_pack(): declare dest to be const

Tighten constness of some local variables in a callchain.

* rs/refs-string-slice (2012-05-22) 4 commits
  (merged to 'next' on 2012-05-23 at d224de1)
 + refs: do not create ref_entry when searching
 + refs: use strings directly in find_containing_dir()
 + refs: convert parameter of create_dir_entry() to length-limited string
 + refs: convert parameter of search_ref_dir() to length-limited string

Avoid unnecessary temporary allocations while looking for matching refs
inside refs API.

* cr/persistent-https (2012-05-23) 1 commit
 - Add persistent-https to contrib

A remote helper that acts as a proxy that caches ssl session for the
https:// transport is added to the contrib/ area.

* jk/format-person-part-buffer-limit (2012-05-22) 1 commit
  (merged to 'next' on 2012-05-23 at ef025b0)
 + pretty: avoid buffer overflow in format_person_part

Will merge to master and later to maint.

* nd/stream-index-pack (2012-05-23) 6 commits
 - sha1_loose_object_info: do not complain out loud on non-existent objects
 - index-pack: avoid collision test when verifying in-repo pack
 - index-pack: use streaming interface for collision test on large blobs
 - index-pack: factor out unpack core from get_data_from_pack
 - index-pack: use streaming interface on large blobs (most of the time)
 - index-pack: hash non-delta objects while reading from stream

Use streaming API to read from the object store to avoid having to hold
a large blob object in-core while running index-pack.

The later 3 patches in the series looked a bit suspect.

* va/git-p4-test (2012-05-23) 2 commits
 - git-p4: Verify detection of "empty" branch creation
 - git-p4: Test changelists touching two branches

--------------------------------------------------
[Stalled]

* nl/http-proxy-more (2012-05-11) 2 commits
 - http: rename HTTP_REAUTH to HTTP_AUTH_RETRY
 - http: Avoid limit of retrying request only twice

I queued only the later two patches from this series, even though they do
not make much sense without the first one that seems to need a bit more
work, so that we won't forget.

* sg/bash-prompt (2012-05-09) 4 commits
 - completion: respect $GIT_DIR
 - completion: use __gitdir() in _git_log()
 - tests: add tests for the bash prompt functions in the completion script
 - tests: move code to run tests under bash into a helper library
 (this branch is tangled with fc/git-prompt-script.)

This is only the "correction" bits taken from the beginning of a larger
series that is to be rerolled.  The first two are tangled with Felipe's
topic so a reroll, if comes, should build on top of them.

* jc/apply-3way (2012-05-16) 12 commits
 - WIP: the message is bogus in apply:::check_patch()
 - apply: refactor "previous patch" logic
 - apply: a bit more comments on PATH_TO_BE_DELETED
 - apply: document --3way option
 - apply: allow rerere() upon --3way results
 - apply: register conflicted stages to the index
 - apply: plug the three-way merge logic in
 - apply: fall back on three-way merge
 - apply: accept -3/--3way command line option
 - apply: split load_preimage() helper function out
 - apply: refactor read_file_or_gitlink()
 - apply: clear_image() clears things a bit more
 (this branch is tangled with jc/wip-apply-3way.)

"git apply" learns to wiggle the base version and perform three-way merge
when a patch does not exactly apply to the version you have.

It turns out that it is somewhat unpleasant to handle add/add conflicts in
this code, but it seems necessary if we want to use "apply -3" to replace
the use of "apply --build-fake-ancestor" followed by the slow "merge" in
"am -3".

* jc/maint-push-refs-all (2012-05-04) 2 commits
 - get_fetch_map(): tighten checks on dest refs
 - fetch/push: allow refs/*:refs/*

Allows pushing and fetching refs/stash.
Not ready.
There still seem to be other bugs hiding (e.g. try pushing twice).

* jc/run-hook-env-1 (2012-03-11) 1 commit
 - run_hook(): enhance the interface to pass arbitrary environment

Not urgent.

Updates run_hook() API to be much less specific to "commit".  It would
only be useful if people start doing more interesting things with hooks.

* jc/split-blob (2012-04-03) 6 commits
 - chunked-object: streaming checkout
 - chunked-object: fallback checkout codepaths
 - bulk-checkin: support chunked-object encoding
 - bulk-checkin: allow the same data to be multiply hashed
 - new representation types in the packstream
 - packfile: use varint functions

Not ready.

I finished the streaming checkout codepath, but as explained in
127b177 (bulk-checkin: support chunked-object encoding, 2011-11-30),
these are still early steps of a long and painful journey. At least
pack-objects and fsck need to learn the new encoding for the series
to be usable locally, and then index-pack/unpack-objects needs to
learn it to be used remotely.

Given that I heard a lot of noise that people want large files, and
that I was asked by somebody at GitTogether'11 privately for an
advice on how to pay developers (not me) to help adding necessary
support, I am somewhat dissapointed that the original patch series
that was sent almost two months ago still remains here without much
comments and updates from the developer community. I even made the
interface to the logic that decides where to split chunks easily
replaceable, and I deliberately made the logic in the original patch
extremely stupid to entice others, especially the "bup" fanboys, to
come up with a better logic, thinking that giving people an easy
target to shoot for, they may be encouraged to help out. The plan is
not working :-(.

--------------------------------------------------
[Cooking]

* jk/ident-gecos-strbuf (2012-05-22) 15 commits
  (merged to 'next' on 2012-05-23 at 6fce816)
 + format-patch: refactor get_patch_filename
 + ident: trim whitespace from default name/email
 + ident: use a dynamic strbuf in fmt_ident
 + ident: use full dns names to generate email addresses
 + ident: report passwd errors with a more friendly message
 + drop length limitations on gecos-derived names and emails
 + ident: don't write fallback username into git_default_name
 + fmt_ident: drop IDENT_WARN_ON_NO_NAME code
 + format-patch: use default email for generating message ids
 + ident: trim trailing newline from /etc/mailname
 + move git_default_* variables to ident.c
 + move identity config parsing to ident.c
 + fmt-merge-msg: don't use static buffer in record_person
 + http-push: do not access git_default_email directly
 + ident: split setup_ident into separate functions

Fixes quite a lot of brokenness when ident information needs to be taken
from the system and cleans up the code.

Will merge to master.

* js/submodule-relative (2012-05-23) 2 commits
 - submodule: fix handling of relative superproject origin URLs
 - submodule: document handling of relative superproject origin URLs

* mh/ref-api-lazy-loose (2012-05-20) 1 commit
  (merged to 'next' on 2012-05-23 at c41a30a)
 + free_ref_entry(): do not trigger reading of loose refs

The code to lazily read loose refs by mistake unnecessarily read the refs
in a subhierarchy when we free the data for the subhierarchy.

Will merge to master.

* mh/test-keep-prove-cache (2012-05-20) 1 commit
  (merged to 'next' on 2012-05-23 at 870d1f9)
 + t/Makefile: retain cache t/.prove across prove runs

* ng/pack-objects-cleanup (2012-05-18) 2 commits
  (merged to 'next' on 2012-05-23 at f90b1ba)
 + pack-objects: refactor write_object() into helper functions
 + pack-objects, streaming: turn "xx >= big_file_threshold" to ".. > .."

* rs/maint-grep-F (2012-05-21) 4 commits
  (merged to 'next' on 2012-05-23 at b24143c)
 + grep: stop leaking line strings with -f
 + grep: support newline separated pattern list
 + grep: factor out do_append_grep_pat()
 + grep: factor out create_grep_pat()

"git grep -F", unlike the case where it reads from a file, did not treat
individual lines in the given pattern argument as separate patterns as it
should.

Will merge to master and later to maint.

* rs/xdiff-fast-hash-fix (2012-05-23) 3 commits
  (merged to 'next' on 2012-05-23 at 8df26a5)
 + xdiff: import new 32-bit version of count_masked_bytes()
 + xdiff: avoid more compiler warnings with XDL_FAST_HASH on 32-bit machines
 + xdiff: avoid compiler warnings with XDL_FAST_HASH on 32-bit machines

Will merge to master.

* sp/sh-windows-pwd (2012-05-16) 1 commit
  (merged to 'next' on 2012-05-23 at 63004c3)
 + git-sh-setup: define workaround wrappers before they are used

Will merge to master.

* fc/git-complete-helper-fix (2012-05-20) 2 commits
  (merged to 'next' on 2012-05-23 at 6124da2)
 + completion: add support for backwards compatibility
 + completion: rename internal helpers _git and _gitk

Will merge to master.

* rs/dir-strbuf-read-recursive-fix (2012-05-11) 2 commits
  (merged to 'next' on 2012-05-23 at 2e22549)
 + dir: simplify fill_directory()
 + dir: respect string length argument of read_directory_recursive()

Will merge to master.

* mm/push-default-switch-warning (2012-04-26) 2 commits
 - t5541: warning message is given even with --quiet
 - push: start warning upcoming default change for push.default

Will squash the two, but this has to wait for a few release cycles.

--------------------------------------------------
[Discarded]

* cb/daemon-test-race-fix (2012-04-27) 2 commits
  (merged to 'next' on 2012-04-27 at 84bbcf8)
 + Revert "git-daemon wrapper to wait until daemon is ready"
  (merged to 'next' on 2012-04-24 at d5c30be)
 + git-daemon wrapper to wait until daemon is ready

Reverted from 'next' to replace it with js/daemon-test-race-fix.

Re: What's cooking in git.git (May 2012, #06; Wed, 23)

[René Scharfe]

Am 24.05.2012 00:24, schrieb Junio C Hamano:
> * rs/maint-grep-F (2012-05-21) 4 commits
>    (merged to 'next' on 2012-05-23 at b24143c)
>   + grep: stop leaking line strings with -f
>   + grep: support newline separated pattern list
>   + grep: factor out do_append_grep_pat()
>   + grep: factor out create_grep_pat()
>
> "git grep -F", unlike the case where it reads from a file, did not treat
> individual lines in the given pattern argument as separate patterns as it
> should.

This is not specific to -F; grep(1) accepts newline-separated pattern 
lists with -E etc. as well, as does git grep with the patches above.

René

Re: What's cooking in git.git (May 2012, #06; Wed, 23)

[Johannes Sixt]

Am 5/24/2012 7:36, schrieb René Scharfe:
> Am 24.05.2012 00:24, schrieb Junio C Hamano:
>> * rs/maint-grep-F (2012-05-21) 4 commits
>>    (merged to 'next' on 2012-05-23 at b24143c)
>>   + grep: stop leaking line strings with -f
>>   + grep: support newline separated pattern list
>>   + grep: factor out do_append_grep_pat()
>>   + grep: factor out create_grep_pat()
>>
>> "git grep -F", unlike the case where it reads from a file, did not treat
>> individual lines in the given pattern argument as separate patterns as it
>> should.
> 
> This is not specific to -F; grep(1) accepts newline-separated pattern
> lists with -E etc. as well, as does git grep with the patches above.

Shouldn't we worry that this change breaks existing users?

Consider a script that generates a pattern that sometimes contains NL.
Yes, it would not match anywhere, but what if that is a deliberate choice
of the script writer? With this change, the script would now observe
spurious matches where earlier there were no matches.

-- Hannes

Re: What's cooking in git.git (May 2012, #06; Wed, 23)

[Junio C Hamano]

René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:

> Am 24.05.2012 00:24, schrieb Junio C Hamano:
>> * rs/maint-grep-F (2012-05-21) 4 commits
>>    (merged to 'next' on 2012-05-23 at b24143c)
>>   + grep: stop leaking line strings with -f
>>   + grep: support newline separated pattern list
>>   + grep: factor out do_append_grep_pat()
>>   + grep: factor out create_grep_pat()
>>
>> "git grep -F", unlike the case where it reads from a file, did not treat
>> individual lines in the given pattern argument as separate patterns as it
>> should.
>
> This is not specific to -F; grep(1) accepts newline-separated pattern
> lists with -E etc. as well, as does git grep with the patches above.

Yeah, I noticed that "unlike where reading from a file" should contrast
against "where params come from the command line", not with any particular
way to interpret individual pattern. after sending it out.

Thanks.

Re: What's cooking in git.git (May 2012, #06; Wed, 23)

[Junio C Hamano]

Johannes Sixt <j.sixt@viscovery.net> writes:

> Am 5/24/2012 7:36, schrieb René Scharfe:
>> Am 24.05.2012 00:24, schrieb Junio C Hamano:
>>> * rs/maint-grep-F (2012-05-21) 4 commits
>>>    (merged to 'next' on 2012-05-23 at b24143c)
>>>   + grep: stop leaking line strings with -f
>>>   + grep: support newline separated pattern list
>>>   + grep: factor out do_append_grep_pat()
>>>   + grep: factor out create_grep_pat()
>>>
>>> "git grep -F", unlike the case where it reads from a file, did not treat
>>> individual lines in the given pattern argument as separate patterns as it
>>> should.
>> 
>> This is not specific to -F; grep(1) accepts newline-separated pattern
>> lists with -E etc. as well, as does git grep with the patches above.
>
> Shouldn't we worry that this change breaks existing users?
>
> Consider a script that generates a pattern that sometimes contains NL.
> Yes, it would not match anywhere, but what if that is a deliberate choice
> of the script writer? With this change, the script would now observe
> spurious matches where earlier there were no matches.

While I prefer to be conservative, and the scenario above may be
theoretically possible, I personally find this particular "breakage" of
backward compatibility is very well defensible, as the original behaviour
is clearly buggy and unintuitive.