From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: Jonathan Tan <jonathantanmy@google.com>
Cc: git@vger.kernel.org
Subject: Re: [PATCH 0/2] negotiator: improve recent behavior + docs
Date: Thu, 27 Sep 2018 22:41:29 +0200 [thread overview]
Message-ID: <87o9ciisg6.fsf@evledraar.gmail.com> (raw)
In-Reply-To: <20180927194125.8380-1-jonathantanmy@google.com>
On Thu, Sep 27 2018, Jonathan Tan wrote:
>> > If you wanted to do this, it seems better to me to just declare a "null"
>> > negotiation algorithm that does not perform any negotiation at all.
>>
>> I think such an algorithm is a good idea in general, especially for
>> testing, and yeah, maybe that's the best way out of this, i.e. to do:
>>
>> if git rev-parse {}/HEAD 2>/dev/null
>> then
>> git fetch --negotiation-tip={}/HEAD {}
>> else
>> git -c fetch.negotiationAlgorithm=null fetch {}
>> fi
>>
>> Would such an algorithm be added by overriding default.c's add_tip
>> function to never add anything by calling default_negotiator_init()
>> followed by null_negotiator_init(), which would only override add_tip?
>> (yay C OO)
>>
>> If so from fetch-pack.c it looks like there may be the limitation on the
>> interface that the negotiator can't exit early (in
>> fetch-pack.c:mark_tips). But I've just skimmed this, so maybe I've
>> missed something.
>
> (I was reminded to reply to this offlist - sorry for the late reply.)
>
> I think too many things need to be replaced (known_common, add_tip, and
> ack all need to do nothing), so it's best to start from scratch. That
> way, we also don't need to deal with the subtleties of C OO :-)
>
>> Also, looks like because of the current interface =null and
>> --negotiation-tip=* would (somewhat confusingly) do a "real" negotiation
>> if done that way, since it'll bypass the API and insert tips for it to
>> negotiate, but it looks like overriding next() will get around that.
>
> If you do it as I suggest (in particular, add_tip doing nothing) then
> there is the opposite problem that it won't be easy to inform the user
> that --negotiation-tip does nothing in this case. Maybe there needs to
> be an "accepts_tips" field in struct fetch_negotiator that, if false,
> means that custom tips (or any tips) are not accepted, allowing the
> caller of the negotiator to print a warning message in this case.
Thanks, yeah it seems the interface would need to be tweaked for such a
"null" negotiator.
Some more general questions (which I can turn into docs once I
understand this). If I run this, as a testcase for two random repos
where I "fetch" an unrelated one and use the first ever commit to
git.git as an alias for this "null" negotiatior, i.e. "just present this
one commit":
(
rm -rf /tmp/git &&
git clone https://github.com/git/git.git /tmp/git &&
cd /tmp/git &&
git remote add gitlab-shell https://github.com/cr-marcstevens/sha1collisiondetection &&
GIT_TRACE_PACKET=/tmp/git/packet.trace git fetch --negotiation-tip=$(git log --reverse|head -n 1|cut -d ' ' -f2) gitlab-shell &&
grep -c "fetch-pack> have" /tmp/git/packet.trace
)
I get:
warning: Ignoring --negotiation-tip because the protocol does not support it.
And the grep -c shows we tried to present 55170 commits in "have" lines
to the server. Now, change that to SSH and all is well:
(
rm -rf /tmp/git &&
git clone git@github.com:git/git.git /tmp/git &&
cd /tmp/git &&
git remote add gitlab-shell git@github.com:cr-marcstevens/sha1collisiondetection &&
GIT_TRACE_PACKET=/tmp/git/packet.trace git fetch --negotiation-tip=$(git log --reverse|head -n 1|cut -d ' ' -f2) gitlab-shell &&
grep -c "fetch-pack> have" /tmp/git/packet.trace
)
I don't understand this limitation. With the SSH version we skip
straight to saying we "want" with just the 1 "have" line of
"e83c5163316f89bfbde7d9ab23ca2e25604af290".
Why aren't we doing the same over http? I don't get how protocol support
is needed, it's us who decide to send over the "have" lines. Some
variant of this does work over "skipping":
(
rm -rf /tmp/git &&
git clone https://github.com/git/git.git /tmp/git &&
cd /tmp/git &&
git remote add gitlab-shell https://github.com/cr-marcstevens/sha1collisiondetection &&
GIT_TRACE_PACKET=/tmp/git/packet.trace git -c fetch.negotiationAlgorithm=skipping fetch gitlab-shell &&
grep -c "fetch-pack> have" /tmp/git/packet.trace
)
There we send 14002 "have" lines, which seems expected, but then with
the same thing over SSH we don't send any:
(
rm -rf /tmp/git &&
git clone git@github.com:git/git.git /tmp/git &&
cd /tmp/git &&
git remote add gitlab-shell git@github.com:cr-marcstevens/sha1collisiondetection &&
GIT_TRACE_PACKET=/tmp/git/packet.trace git -c fetch.negotiationAlgorithm=skipping fetch gitlab-shell &&
grep -c "fetch-pack> have" /tmp/git/packet.trace
)
So that seems like another bug, and as an aside, a "skipping"
implementation that sends ~1/4 of the commits in the repo seems way less
aggressive than it should be. I was expecting something that would
gradually "ramp up" from the tips. Where say starting at master/next/pu
we present every 100th commit as a "have" until the 1000th commit, then
every 1000 commits until 10k and quickly after that step up the size
rapidly.
next prev parent reply other threads:[~2018-09-27 20:41 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-16 18:44 [PATCH] negotiator/skipping: skip commits during fetch Jonathan Tan
2018-07-16 23:02 ` Junio C Hamano
2018-07-26 10:36 ` Johannes Schindelin
2018-07-26 14:14 ` Johannes Schindelin
2018-07-26 19:16 ` Jonathan Tan
2018-07-27 15:48 ` Johannes Schindelin
2018-08-03 13:07 ` Johannes Schindelin
2018-07-31 15:02 ` Ævar Arnfjörð Bjarmason
2018-07-31 18:02 ` Jonathan Tan
2018-08-01 15:18 ` [PATCH 0/2] negotiator: improve recent behavior + docs Ævar Arnfjörð Bjarmason
2018-08-01 20:25 ` Jonathan Tan
2018-08-01 21:13 ` Ævar Arnfjörð Bjarmason
2018-09-27 19:41 ` Jonathan Tan
2018-09-27 20:41 ` Ævar Arnfjörð Bjarmason [this message]
2018-09-27 22:46 ` Jonathan Tan
2018-08-01 15:18 ` [PATCH 1/2] negotiator: unknown fetch.negotiationAlgorithm should error out Ævar Arnfjörð Bjarmason
2018-08-01 15:18 ` [PATCH 2/2] fetch doc: cross-link two new negotiation options Ævar Arnfjörð Bjarmason
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87o9ciisg6.fsf@evledraar.gmail.com \
--to=avarab@gmail.com \
--cc=git@vger.kernel.org \
--cc=jonathantanmy@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).