Re: Fwd: git-reviewed: linking commits to review discussion in git

[Peter C Rigby <peter.rigby@concordia.ca>]

On Tue, Feb 18, 2014 at 12:28 AM, Jeff King <peff@peff.net> wrote:
[snip]
>
>   3. How do we present the emails to the user (including showing
>      threads, letting them dig deeper, etc)?

We created a website, you enter a commit and it shows you the reviews:
http://cesel.encs.concordia.ca/process_request.php?repo=git

The site will either display the message id for the review or redirect
you the mailing list archive.


On Sat, Feb 22, 2014 at 11:47 AM, Thomas Rast <tr@thomasrast.ch> wrote:
> Jeff King <peff@peff.net> writes:
>
>> On Mon, Feb 17, 2014 at 03:12:48PM -0500, Murtuza Mukadam wrote:

Very sorry for the delayed response. We wanted compare our linking
technique with yours, see below.

>>
>>> We have linked peer review discussions on
>>> git@vger.kernel.org to their respective commits within the main
>>> git.git repository. You can view the linked reviews from 2012
>>> until present in the GitHub repo at:
>>> https://github.com/mmukadam/git/tree/review
>>
>> Neat. We've experimented in the past with mapping commits back to
>> mailing list discussions.  Thomas (cc'd) has a script that creates
>> git-notes trees mapping commits to the relevant message-id, which can
>> then be found in the list archive.
>>
>> To me, the interesting bits of such a project are:
>>
>>   1. How do we decide which messages led to which commits? There is
>>      definitely some room for heuristics here, as patches are sometimes
>>      tweaked in transit, or come in multiple stages (e.g., the original
>>      patch, then somebody suggests a fixup on top). You might want to
>>      compare your work with the script from Thomas here:
>>
>>        http://repo.or.cz/w/trackgit.git
>
> Eh, or don't.  My script nowadays uses Junio's suggestion of matching on
> (author, authordate) with a little bit of tweaking in case there is no
> match.  The name/date match works for most cases even in slightly
> tweaked forms.

In our technique, we take each email patch, eliminate white space and
hash each line. We then compare the lines with those in commits to the
same files. The commit that changes the same files and has the largest
number of matching lines is considered to be the reviewed commit.

We compared Junio's (author, authordate) and our technique on a
manually created benchmark of 30 messages from the linux, postgres,
and git mailing lists. We found that Junio's simple technique
performed equally well to our more complicated technique (see results
at end of email).

On Tue, Feb 18, 2014 at 12:28 AM, Jeff King <peff@peff.net> wrote:
>   2. How do we store the mapping? I think git-notes are a natural fit
>      here, but you don't seem to use them. Is there a reason?

We wanted a way to store each review as its own blob, so we put the
reviews on a detached 'review' branch and then wrote some simple
scripts to access and display these reviews on the command line (\eg
git reviewed --show <commit_hash>).

However, given the previous discussion on this list
(http://git.661346.n2.nabble.com/RFC-RFH-Fun-things-with-git-notes-or-patch-tracking-backwards-td2297330.html),
we agree that just putting the msg_id as a note is probably more
useful. One additional problem we ran into when storing all the
reviews, is for something like Linux, the reviews took up a massive
amount of storage.

> It would be interesting to apply some kind of clustering algorithm that
> automatically determines the messages related to a commit, including
> both the patch but also any discussion leading up to it. I realize that
> may be getting far afield of your original goals, but hey, you said you
> wanted feedback. I can reach for the stars. :)

It would be interesting to be able to tie in other discussion, perhaps
related bugs. A future project.

Thanks,
Peter

Evaluation of linking commits with email reviews on 30 messages from
Linux, postgres, and git mailing lists

Linux - Files and lines changed:

Perfect: 87%
No Match: 13%
Incorrect: 0%

Linux - Thomas/Junio  (author, authordate)

Perfect: 87%
No Match: 13%
Incorrect: 0%

Git - Files and lines changed:

Perfect: 74 %
No Match  23  %
Incorrect: 3 %

Git - Thomas/Junio  (author, authordate)

Perfect: 77%
No Match: 23%
Incorrect: 0%

Postgres - Files and lines changed:

Perfect: 57 %
No Match  36  %
Incorrect: 7 %

Postgres - Thomas/Junio  (author, authordate)

Perfect: 50%
No Match: 37%
Incorrect: 13%

-- 
http://users.encs.concordia.ca/~pcr/