Feature request: flagging “volatile” branches for integration/development

Feature request: flagging “volatile” branches for integration/development

[Herbert, Marc]

(Thanks to Josh Triplett[*] for contributing to this message)

Hi,

We often work with development/integration branches that regularly
rebase, in addition to stable branches that do not. Git is used to share
two different types of branches:
  1. Pull requests and merged code with final SHA1s
  2. Work in progress with volatile SHA1s.

We’d like to have a consistent way to distinguish these two types by
advertising a branch as “volatile”. Such a branch supports shared
development on work-in-progress code (not yet ready to merge, or still
being emailed as PATCHv{2,3,4,...}), or an integration/testing branch
for a combination of such development branches.  Branch naming
conventions could help a bit here, but a large and varied set of naming
conventions already exist, none of which provide machine-readable
information that git itself can rely on. So the only thing available is
tribal knowledge or out-of-band documentation at best, e.g.:
https://01.org/linuxgraphics/gfx-docs/maintainer-tools/drm-intel.html

Such a “volatile” flag would instantly warn that code is not ready to
re-use:
https://www.chromium.org/chromium-os/chromiumos-design-docs/upstream-first

Another common issue with volatile branches is their commits being
misunderstood as immutable and then their transient, top SHA1
being reported as bogus and unusable version control information in bugs
(on a non-volatile branch, a single SHA1 captures the entire and exact
snapshot and history).

Humans would be the initial consumers of this flag but I can imagine git
itself also using it. For instance, cherry-pick could have a “smart”
setting for -x, that doesn’t bother recording transient commit hashes. A
merge could print an optional warning when pulling in changes from a
volatile branch, and a rebase could similarly print a warning when
rebasing on top of such a branch. A git server could be configured to
treat non-fast forward forced pushes differently depending on the
“volatility” of the target branch. A fancy user interface could
color volatile SHA1s differently to discourage copy/paste. Etc.

Maybe this has already been discussed (or implemented even), and I
couldn’t find the right search keywords; in this case please help me cut
this discussion short. We’d appreciate any feedback you might have,
either on the idea itself, or on other ways to solve the same problem.

[ ] “send patches”
[ ] use some other existing mechanism to solve this
[ ] will never work because of X and Y; don’t even bother

-- 
Marc

PS: please keep me in Cc:, thanks.

[*] on a related topic: https://github.com/git-series/git-series

Re: Feature request: flagging “volatile” branches for integration/development

[Duy Nguyen]

On Wed, Feb 1, 2017 at 12:12 AM, Herbert, Marc <marc.herbert@intel.com> wrote:
> (Thanks to Josh Triplett[*] for contributing to this message)
>
> Hi,
>
> We often work with development/integration branches that regularly
> rebase, in addition to stable branches that do not. Git is used to share
> two different types of branches:
>   1. Pull requests and merged code with final SHA1s
>   2. Work in progress with volatile SHA1s.
>
> We’d like to have a consistent way to distinguish these two types by
> advertising a branch as “volatile”.

I don't think we have branch metadata (besides reflog). The closet one
is probably config variable branch.<name>.description, which can be
picked up by format-patch to create cover letters. We could do
something similar, e.g. new config branch.<name>.volatile. The
commands can learn about it and apply special treatments if wanted.

But that would be local information only. We don't have ways to
transfer branch metadata (and we definitely don't want to just share
.git/config file with everybody). I suppose extending git protocol for
this is not hard (e.g. appending metadata in the "have" lines). The
hard part may be policy (e.g. what if the user does not want a branch
to be treated volatile by various commands even if it receives such
flag from a git server).
-- 
Duy

Re: Feature request: flagging “volatile” branches for integration/development

[Herbert, Marc]

[apologies for the accidental "smart" quotes and the resulting UTF8
encoding of the subject]

On 04/02/2017 06:01, Duy Nguyen wrote:
> 
> But that would be local information only. We don't have ways to
> transfer branch metadata (and we definitely don't want to just share
> .git/config file with everybody). I suppose extending git protocol for
> this is not hard (e.g. appending metadata in the "have" lines).

Thanks Duy. So did you mean:

[ X ] send (big!) patches ?

> The hard part may be policy (e.g. what if the user does not want a branch
> to be treated volatile by various commands even if it receives such
> flag from a git server).

There would be instant, human-readable value in such a new "volatile"
flag. Machine use and policies can be discussed later. These will be
easier to prototype, experiment and refine once the flag exists.

  ----

Interestingly, it was pointed to me (thanks Calvin) that GitLab has
implemented this volatile flag exactly. It's called... "work in progress":
https://docs.gitlab.com/ee/user/project/merge_requests/work_in_progress_merge_requests.html

I'm not familiar with GitHub, however browsing its documentation the
(in)existence of a pull request seems equivalent to a (non-)volatile
flag. Just like a pull request by email without the need to find and search
a mailing-list.

I'm familiar with Gerrit and there's no strict equivalent to a volatile
flag, however it's:
- totally obvious when the commit has been accepted and merged - hence
  its SHA1 final.
- usually fairly clear whether the code is still WIP or near the
  "pull request" stage based on: how the code review is going, approvals
  and other metadata.

Long story short: to integrate code reviews and source control these
systems overload git with a ton of metadata so it's no surprise to
always find in them something that more or less looks like a "volatile"
flag. I guess this leads to the more general question of core git possibly
implementing some generic metadata/property system (key,value pairs?
everything is a ref?) to better support code review and other
git-based software... Now I bet this on the other hand must have been
discussed (and rejected?) before, any pointer?


Marc

Re: Feature request: flagging “volatile” branches for integration/development

[Junio C Hamano]

"Herbert, Marc" <marc.herbert@intel.com> writes:

>> The hard part may be policy (e.g. what if the user does not want a branch
>> to be treated volatile by various commands even if it receives such
>> flag from a git server).
>
> There would be instant, human-readable value in such a new "volatile"
> flag. Machine use and policies can be discussed later. These will be
> easier to prototype, experiment and refine once the flag exists.

We tend to avoid adding random cruft to the system whose semantics
is not well defined, so that we can avoid having to support an ill
defined feature forever.

> ... Now I bet this on the other hand must have been
> discussed (and rejected?) before, any pointer?

I suspect that people may have expressed vague wish from time to
time, but I do not think we saw a proposal that outlines the design
at a concrete enough level to let us rejecting in the past ;-)

Let me list some things that needs to be designed that comes to my
mind offhand:

 * How would a user mark a ref as "volatile"?  I am assuming that
   anybody do so in her own local repository, but how does one mark
   a ref as "volatile" at a hosting site, and who is allowed to do
   so (one possibile design is "new option to 'git push' will do so,
   and anybody who can push to the ref is allowed to", and I am fine
   with that design, but you have to spell that out in a proposal)?

 * How would a user learn if a ref is "volatile"?  Does "ls-remote"
   show that information?

 * Does volatile-ness of a ref at the remote site propagate to your
   remote-tracking ref that corresponds to it?  What does it mean
   that refs/remotes/origin/pu is marked as volatile in your local
   repository?  You cannot "checkout -b" based on it?  Does "branch"
   based on it need to be forbidden as well?

 * Can a local ref be "volatile"?  What does it mean (the same
   subquestions as above)?

 * If your local branch myA is set to build on a remote-tracking
   branch A and push back to branch A at the remote, i.e.

    $ git checkout -t -b myA refs/remotes/origin/A
    $ ... work work work ...
    $ git push

   is set to result in their branch A updated with what you built in
   myA, and if the branch A at the remote is marked as "volatile",
   does it make your "myA" also "volatile"?  How is the volatile-ness 
   inherited?  From their A to your remotes/origin/A and then to
   your myA?  Any other useful rule that defines the propagation?

 * Do we only care about "volatile"?  If we are extending the system
   to allow setting and propagating this new bit per ref (I am
   blindly assuming that you do not have a strong reason to limit
   this to branches), we may as well just design this extension to
   allow the projects to assign arbitrary set of bits to refs.  Some
   projects may want to have different degree of volatile-ness and
   have "volatile" refs, "unstable" refs and "iffy" refs, for
   example.

   Side note: even if we were to go with "any random label can be
   assigned and the meaning for the labels can be defined by the
   project convention" approach, it does not necessarily mean we are
   adding a random cruft whose semantics is ill-defined.  "Git does
   not do anything special to a ref based on what labels it has--it
   just remembers what labels the user told it to attach to the ref,
   and shows what labels the ref has when asked" can be very well
   defined semantics.