Re: [PATCHSET] git-reverse-trailer-xrefs: Reverse map cherry-picks and other cross-references

[Stefan Xenos <sxenos@google.com>]

I've just uploaded a new evolve proposal that includes a spec for the
"hiddenmetas" namespace, where we can store historical cherry-pick
information.

On Tue, Dec 18, 2018 at 6:40 AM Stefan Xenos <sxenos@google.com> wrote:
>
> Heya, Tejun!
>
> Thanks for getting in touch. I agree with Junio that we probably
> shouldn't be tracking the same information in two ways if we can think
> of something that does both... so let's see if we can think of
> something! The evolve proposal suggests having a metas/ namespace to
> track ongoing changes is intended as a way to track the changes a user
> is actively working on. If we were to use it to retroactively store
> every historical cherry-pick in a large repository, I'm concerned it
> may get too cluttered to use for its originally-intended purpose. I'm
> also not sure how well things would perform with a huge number of
> refs. The remainder of the proposal (using metacommits to store the
> relationships) would work for the xref use-case, but we'd probably
> want to tweak the way we store the root refs in order to do a good job
> with the large number of inactive cherry-picks he probably wants. Any
> code that is looking for cross-reference metadata could look in both
> namespaces.
>
> Conversely, if we were to tweak the xrefs proposal by adding the
> obsolescence information that evolve needs, we'd be missing a place to
> store the user's ongoing changes, a way to push and pull specific
> changes, a way to describe alternate histories for a commit, and a
> mechanism for preventing the commits needed by evolve from getting
> garbage collected.
>
> All the problems with both approaches are solve-able, though.
>
> I spent a few hours discussing this with Stefan Beller last week and I
> think we've come up with a variation on the evolve proposal that would
> cover both our use-cases. Let me know what you think. To address the
> cluttering of the metas namespace, we could introduce a new
> "hiddenmetas" namespace. It would contain the same type of data
> recorded in the metas namespace, but it would normally be hidden from
> the user when they list their changes, etc. It would also be immune
> from the automatic deletions that are applied to the "metas"
> namespace. From your point of view, the critical bit is that it would
> contain information about cherry-picks. That would address the
> "user-visible clutter" problem. Utility methods that want to search
> for cherry-picks would iterate over both namespaces.
>
> For the performance problem, I think we could just live with it
> temporarily and wait for the currently-ongoing reftable work since the
> reftable proposal would address exactly this performance issue (by
> allowing us to store and manipulate large numbers of refs
> efficiently).
>
> A nice characteristic of this approach is that it would also address a
> problem with the evolve proposal that had concerned me: how to deal
> with the filter-branch command, which would have created pretty much
> the same problems regarding the large number of metadata refs for
> changes the user isn't actively working on.
>
> It might be nice to also consider and some alternatives, but I haven't
> had enough time to think up more of them (and I haven't digested the
> xrefs proposal sufficiently to suggest an enhanced version of it yet).
> If anyone else has ideas for combining the xrefs and evolve use-cases,
> having more alternatives to choose from is always better!
>
> If you're okay with the "hiddenmetas" approach in principle, I'll
> update the evolve proposal doc to include it. Once we work out how the
> storage format will work, we can coordinate our implementation work.
>
>   - Stefan
>
>
>
>
> On Wed, Dec 12, 2018 at 7:46 PM Tejun Heo <tj@kernel.org> wrote:
> >
> > Hello, Junio, Stefan.
> >
> > On Thu, Dec 13, 2018 at 12:09:39PM +0900, Junio C Hamano wrote:
> > > Please do not take the above as "don't do notes/xref-; instead read
> > > from the 'meta commits'".  I do not have a preference between the
> > > two proposed implementations.  The important thing is that we won't
> > > end up with having to maintain two separate mechanisms that want to
> > > keep track of essentially the same class of information.  FWIW I'd
> > > be perfectly fine if the unification goes the other way, as long as
> > > goals of both parties are met, and for that, I'd like to see you two
> > > work together, or at least be aware of what each other is doing and
> > > see if cross-pollination would result in a mutually better solution.
> >
> > So, from my POV, the only thing I want is being able to easily tell
> > which commit got cherry picked where.  I really don't care how that's
> > achieved.  Stefan, if there's anything I can do to push it forward,
> > let me know and if you see anything useful in my patchset, please feel
> > free to incorporate them in any way.
> >
> > Thanks.
> >
> > --
> > tejun