Re: [PATCHSET] git-reverse-trailer-xrefs: Reverse map cherry-picks and other cross-references

[Junio C Hamano <gitster@pobox.com>]

Tejun Heo <tj@kernel.org> writes:

> Some trailers refer to other commits.  Let's call them xrefs
> (cross-references).  For example, a cherry pick trailer points to the
> source commit.  It is sometimes useful to build a reverse map of these
> xrefs - ie. source -> cherry-pick instead of cherry-pick -> source.
>
> This, e.g, can answer which releases a commit ended up in on
> repositories which cherry-picks fixes back from the development
> branch.  Let's say the repository looks like the following.
>
> 	    D'---E'' release-B
> 	   /
> 	  C'      E' release-D
>        /       /
>   A---B---C---D---E master
>
> where the followings cherry-picks took place.
>
>   C -> C'
>   D -> D'
>   E -> E' -> E''
>
> Using the reverse-mapping in this patchset, we can get the following
> annotated log which succinctly shows which commit ended up where.
> ...

Thanks.  A few comments, after skimming the patches enough to get
the general feel of the design but before deeply reading them to see
how complete the implementation is.

The topic introduces two new hooks: one to run immediately after
cherry-picking so that the reverse mapping for a single src->dst
pair can be added, and another to run immediately after fetching so
that all new commits that have appeared over the wire can be scanned
to see if any of them is a cherry-pick of other commits.

They are good starting points, but for a complete solution I'd
imagine that you'd need to hook to many other places in the
workflow.  For example, you'd need to deal with the case where a
commit created by cherry-picking an original is further rewritten
with "commit --amend" or "rebase".  They may already trigger the
post rewrite hook, so you may not have to add a new hook, but in
[3/5], you'd need to describe in the documentaiton for the new
reverse-trailer command full set of hooks you expect the command to
be used to maintain the reverse mapping, as the hooks you give them
as examples in [5/5] are merely a part of a complete solution.

It also is not immediately obvious to me what your general strategy
to maintain this reverse mapping is, when new ways and codepaths to
cause new commits with "cherry-picked-from" trailer appear.  Do we
keep piling on new hooks?  Or is the reverse mapping information
allowed to be a bit stale and be completed immediately before it
gets used?  A totally different approach could be to record list of
commits, all commits behind which have been scanned for reverse
mapping, in the tip of the notes history, perhaps in the commit log
message (which is machine generated anyway).  Then, before you need
the up-to-date-to-the-last-second reverse mapping, you could run

	git rev-list --all --not $these_tips_recorded

and scan the commits, just like you scan what you fetched.  And when
you update the reverse mapping notes tree, the commit to record that
notes update can record the tip of the above traverseal.

I also wonder how this topic and the topic Stefan Xenos has been
working on (cf. <20181201005240.113210-1-sxenos@google.com>) can
benefit from each other by cross pollination.  Stefan's topic also
needs to answer, given a commit, what other commits were created out
of it by cherry-picking and then further amending the results, so
that when the original commit itself gets amended, the cherry-picked
ones that were created from the original can be easily identified
and get updated in the same way as necessary.  

The motivating workflow your topic's cover letter gives us is to
maintain the release branch that goes only forward, so in that
context, how a commit on the release branch that was created by
cherry-picking an original gets updated when the original commit
gets amended would be different (I am assuming that you cannot
affored to rebase the release branch to update a deeply buried
commit that was an earlier cherry-pick), but I would imagine that
both topics share the need for a good data structure to keep track
of (1) the relationship between the original and copy of the
cherry-pick and (2) the fact that the original of such a cherry-pick
is now stale and the copy might want to get updated.

Thanks.