Google Doc about the Contributors' Summit

Google Doc about the Contributors' Summit

[Johannes Schindelin]

Hi team,

I just started typing stuff up in a Google Doc, and made it editable to
everyone, feel free to help and add things:

https://docs.google.com/document/d/1KDoSn4btbK5VJCVld32he29U0pUeFGhpFxyx9ZJDDB0/edit?usp=sharing

Ciao,
Johannes

Re: Google Doc about the Contributors' Summit

[Johannes Schindelin]

Hi team,

On Thu, 2 Feb 2017, Johannes Schindelin wrote:

> I just started typing stuff up in a Google Doc, and made it editable to
> everyone, feel free to help and add things:
> 
> https://docs.google.com/document/d/1KDoSn4btbK5VJCVld32he29U0pUeFGhpFxyx9ZJDDB0/edit?usp=sharing

I am terribly sorry... yesterday I simply tried to restrict editing so
that nobody would just spam the document, but in my haste I even disabled
viewing.

The link is functional again, sorry for that.

Thanks, Tim, for reporting the problem!

Ciao,
Dscho

Re: Google Doc about the Contributors' Summit

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> I just started typing stuff up in a Google Doc, and made it editable to
> everyone, feel free to help and add things:
>
> https://docs.google.com/document/d/1KDoSn4btbK5VJCVld32he29U0pUeFGhpFxyx9ZJDDB0/edit?usp=sharing

Thanks for a write-up, Dscho.

List of bullet points just make non-attendees envious, imagining
that attendees had all the fun discussing what is behind these
bullet points, without being able to know what was discussed, if
they reached consensus and what the consensus was, but it is OK ;-)

A few items caught my attention, not because I found them more
important than other items on the page, but because they seem to
want my input without directly asking me ;-)

* If Junio would accept patches by replying to the sender with an
  ID and/or a patch name. He picks this (branch) name when he gets
  your patch.

I am not sure what exactly I am asked to "accept" here.  I sometimes
forget to respond with "Thanks, will queue." to the patch message
and whoever said this wants me to consistently do so?  I never say
"... will queue as js/difftool-builtin topic." mainly because I do
not know what the name of the topic branch should be at the point of
reading the patch.  All I know is that decided that it may be worth
queuing, so it is a bit harder to arrange.  But I can certainly try
if it makes the lives of contributors easier.


* Junio has a script for the todo branch which we can use to
  generate the what's cooking branch. Perhaps we could continuously
  generate this onto a webpage.

I have no objection, but I doubt that people find such an auto
generated version all that useful, as "git log --first-parent
origin/master..origin/pu" would tell exactly the same story.  It
will lack the "topic summary" comment I have yet to write, if the
final 'pu' of the day that was pushed out was before my local update
to the draft of the next issue of "What's cooking" report [*1*], and
would not have any update on the next action (e.g. "Will merge to
...") relative to the latest issue of "What's cooking" report.  IOW,
such an auto-generated report lacks all the added value over "git log"
output.


* Making the actual workflow more publicly known, e.g. document how
  to generate the cooking email, to learn about the state of a
  generation.

The exact mechanics of "how to generate" may be of less importance
than "how the information contained therein relates to their own
work" to the contributors, and I think MaintNotes that is sent out
at key milestones more or less covers the mechanics, but here is how
the sausage is made these days:

- I find a patch series on the list that is in good enough shape to
  be in 'pu'.  Perhaps it was already discussed and redone a few
  times without hitting my tree.  Or it may be the first attempt of
  a new topic.  I come up with a topic name, decide where the topic
  should fork at [*2*], create the topic branch and queue the
  patches.  I may or may not test the topic in isolation at this
  point.

- I may find an updated patch series of what has been queued.  I go
  to the existing topic branch and replace it (I try to keep it
  forked at the same commit) after inspecting what got updated.
  "git tbdiff" is a useful program to help this step.  I may or may
  not test the topic in isolation at this point.

- I repeat the above two for various topics during the day, and at
  some point between 14:00-15:00 my time, stop taking new patches.
  The day's integration cycle starts.

- If there are topics that have cooked long enough in 'next' and
  planned to graduate to 'master', merge [*3*] them to 'master',
  update the draft Release notes.  Otherwise skip this step.

- If there are topics that have cooked long enough in 'pu' and
  planned to graduate to 'next', merge them to 'next'.  Otherwise
  skip this step.

- If I updated 'master', merge its tip to 'next' (this should update
  the draft release notes and nothing else, unless I took something
  directly to 'master').

- Then I recreate 'jch', which is a point between 'master' and 'pu'.
  This is the version I use for my own work, contains all topics in
  'next' but a bit more.  "git checkout -B jch master" begins it,
  and then the topics that were in 'jch' are merged on top.  The
  latest version of updated topics that were already in 'jch' are
  incorporated at this point, and "git diff jch@{20.hours}" would
  show the effect of their interdiff (plus RelNotes update, if
  'master' was updated during this integration cycle).

- I ran "git branch --no-merged pu --sort=-committerdate" to see the
  topics that are new; the top of this list shows new topics and
  updated topics (note that I just updated 'jch' but not 'pu' yet at
  this point).  Among them, I pick the ones that I am willing to be
  a guinea-pig for before they hit 'next' and merge them to 'jch'.
  Other topics that used to be in 'pu' may also be merged at this
  point, when they turn out to be "interesting" enough.

- 'pu' is recreated on top of 'jch' [*4*].  "git checkout -B pu jch"
  begins it, the topics that were already in 'pu' are merged on top
  (some of them may have been merged to updated 'jch' in the above
  step), together with the new topics we acquired.

- All integration branches are then tested (and installed for my own
  use) at this point.

- While the tests are running, the draft "What's cooking" report is
  updated.  Here is where I write topic summary for new topics and
  update topic summary for updated topics.  The next action
  (e.g. "Will merge to ...") may also be updated at this point.

- If the test notices problems, I go back to redo the day's merges
  [*5*].  The first thing to do is to rewind the integration
  branches back to the state before the day's integration started.
  Trivial breakages are fixed-up in place with "rebase -i" or
  queuing "SQUASH???" fix on top of the offending topic before
  merging them back to 'master', 'next', 'jch' or 'pu'.  A more
  involved ones may force me to punt and eject the topic from the
  integration branches.  Or I may leave it in 'pu' so that others
  who are more clueful in the area the offending topic touches can
  notice the breakage and send in fix-up patches [*6*].

- If I have time, I redo 'jch' and 'pu' at this point once again,
  because the merge commit for new topics we just tested do not have
  their merge log message in sync with the topic summary in the
  draft "What's cooking" report at this point [*6*].  I run the
  tests for integration branches again, but the tree objects at the
  tips of the integration branches are expected to be the same as
  what was tested earlier, so it becomes a no-op.

- I push out the integration branches to the usual places [*7*].

- Then I go back to the very beginning.  I start taking patches for
  the next day's integration cycle.

Interested people are welcome to condense the above to update
MaintNotes in my 'todo' branch.


[Footnote]

*1* Merges that appear early in the "--first-parent master..pu"
    carry the topic summary I have in the draft of "What's cooking"
    report as its merge message.

*2* Sometimes a maint-worthy patch series is done in such a way that
    it only applies to 'next'.  I try to tweak such a patch series
    to apply 'maint' and either use the result of my tweak, or ask
    the submitter to base it on 'maint', depending on how confident
    I am with the tweaked result.

*3* When I do this and any other "merge" of a topic branch into an
    integration branch, the topic description in the draft "What's
    cooking" report is used as the merge log message, if one already
    exists.

*4* Readers would notice that 'jch' and 'pu' are not decendants of
    'next'; this is very much deliberate, and it has helped me to
    spot mismerges of tricky topics when they graduate to 'master' a
    few times.

*5* IOW, I go back to the step "around 14:00-15:00".

*6* This is why the tip of 'pu' sometimes may not even build.  IOW,
    a broken 'pu' is an invitation for help, not complaint.

*7* Remember *3* above and also the fact that "What's cooking" draft
    is updated only after everything is merged to 'pu'; there is a
    small chicken and egg problem here and redoing 'jch/pu' will fix
    it.

*8* Broken-out individual topics are sent to github.com/gitster/git

Re: Google Doc about the Contributors' Summit

[Johannes Schindelin]

Hi Junio,

On Thu, 9 Feb 2017, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > I just started typing stuff up in a Google Doc, and made it editable to
> > everyone, feel free to help and add things:
> >
> > https://docs.google.com/document/d/1KDoSn4btbK5VJCVld32he29U0pUeFGhpFxyx9ZJDDB0/edit?usp=sharing
> 
> Thanks for a write-up, Dscho.

Technically, it is not a write-up, and I never meant it to be that. I
intended this document to help me remember what had been discussed, and I
doubt it is useful at all to anybody who has not been there.

I abused the Git mailing list to share that link, what I really should
have done is to use an URL shortener and jot the result down on the
whiteboard.

Very sorry for that,
Johannes

Re: Google Doc about the Contributors' Summit

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Technically, it is not a write-up, and I never meant it to be that. I
> intended this document to help me remember what had been discussed, and I
> doubt it is useful at all to anybody who has not been there.
>
> I abused the Git mailing list to share that link, what I really should
> have done is to use an URL shortener and jot the result down on the
> whiteboard.
>
> Very sorry for that,

Heh, no need to apologize.

I saw your <alpine.DEB.2.20.1702071248430.3496@virtualbox> that was
sent to the list long after the event, which obviously no longer
meant for collaborative note taking and thought that you are
inviting others to read the result of that note taking, and that is
why I commented on that.  I've hopefully touched some "ask Junio
what he thinks of this" items and the whole thing was not wasted ;-)

Clarification about "Better tooling for reviews", was Re: Google Doc about the Contributors' Summit

[Johannes Schindelin]

Hi Junio,

On Fri, 10 Feb 2017, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > Technically, it is not a write-up, and I never meant it to be that. I
> > intended this document to help me remember what had been discussed,
> > and I doubt it is useful at all to anybody who has not been there.
> >
> > I abused the Git mailing list to share that link, what I really should
> > have done is to use an URL shortener and jot the result down on the
> > whiteboard.
> >
> > Very sorry for that,
> 
> Heh, no need to apologize.

Well, you clearly misunderstood the purpose of the document, and that was
my fault, as I had not made that clear.

> I saw your <alpine.DEB.2.20.1702071248430.3496@virtualbox> that was
> sent to the list long after the event, which obviously no longer
> meant for collaborative note taking and thought that you are
> inviting others to read the result of that note taking, and that is
> why I commented on that.

Of course you are free to read it, and to guess from the sparse notes
around what the discussions revolved. I do not think that you'll get much
out of the notes, though.

> I've hopefully touched some "ask Junio what he thinks of this" items and
> the whole thing was not wasted ;-)

I am afraid that it was quite clear to everybody in the room "what you
think of this".

And I do not think that your clarifications how you review code had any
direct relation with the discussions in particular about better tooling
for the review process.

To open the discussion of that particular part of the Contributors'
Summit, I mentioned a couple of pain points, and the remainder of the
discussion really revolved around those, constructively so, I want to add:

- we actively keep potential contributors out by insisting that email
  communication is the most open and inclusive, when right off the bat,
  without any notice to anybody, our mailing list rejects mails sent both
  by the most popular desktop mail client as well as by the most popular
  web mail client.

- developers who really, really want to contribute may switch email
  clients or try to configure their existing email clients to skip the
  HTML part of their emails, only to be met with the reply "your patch is
  white-space corrupted" which cannot fail to sound harsh.

- the few contributors not deterred by this problem, and persistent enough
  to try until they manage to get through, often drop the ball after being
  met with suggestions that would ideally be addressed by automation so
  that humans can focus on problems only humans can solve (every time I
  read "this line is too long" in a review I want to cry: how is this
  worth the time of contributor or reviewer? There are *tools* for that).

- discussions often veer into completely tangential topics so that the
  actual review of the patches is drowned out (and subsequently
  forgotten).

- for any given patch series, it requires a good amount of manual digging
  to figure out what its current state is: what reviewers' comments are
  still unaddressed? Is the patch series in pu/next/master yet? Is the
  *correct* iteration of the patch series in pu/next/master? How does the
  version in pu/next/master differ from what the contributor has in their
  local repository? etc

- the closest thing to "this is the state of that patch series" is the
  What's Cooking email that neither CC:s the original contributors nor
  does it bear any reliable link to the original patch mail, let alone the
  original commit(s) in the contributor's repository.

I really do not think that any of your descriptions of your workflow and
of your review priorities could possibly be expected to fix any of these.

I have an additional pain point, of course, in that your priorities in
patch review (let's admit that it is not a code review, but a patch review
instead, and as such limited in other ways than just the lack of focus on
avoiding potential bugs) are unfortunate in my opinion. But that is not
your problem.

It is clear to me that these pain points only affect potential
contributors (and some of them only frequent contributors who are as
uncomfortable with the sheer amount of tedious "where is that mail that
contained that patch? Oh, and what was the latest reply to this one? Okay,
and in which worktree do I have those changes again?" type of things that
really should not by *my* burden, given that we are trying to develop an
application that helps relieve developers of tedious burden by automating
recurring tasks).

That is why I did not call session "Let's criticize Junio for something
that does not even bother him", but instead "Better tooling for reviews".

The only way forward, as I see it, is for other like-minded contributors
(one of the GitMerge talks even dedicated a good chunk of time to the
description of the pitfalls of the Git mailing list, and home-grown tools
to work around them, so I am definitely not alone) to come together and
try to consolidate and develop the tools to work around the problems
incurred from the choice of using the Git mailing list as the only vehicle
for code contributions [*1*].

When (or if?) those tools become polished and convenient enough to make a
difference, who knows? Maybe even you or Peff find them useful enough to
switch. After all, even The Linus switched from tarballs and patches to
BitKeeper (and I read that it took only a substantial amount of time and
effort and money on Larry McVoy's part[*2*]).

But nothing of that comes as news to you. You knew all that about my
opinions and my pains. I actually did not expect there to be any benefit
in the "Better tools for review" session for you, specifically, and it
seems I was correct in that assumption.

Ciao,
Johannes

Footnote *1*: git-svn, gitk and git-gui Pull Requests and subsequent
merges do not count, as it is clear that they are not *really* part of the
Git source code, and therefore not *really* reviewed on the Git mailing
list.

Footnote *2*: Some posts I read about it are unfortuntely gone, but here
is an archived one that still works:
https://web.archive.org/web/20120414183625/http://kerneltrap.org/node/222