difflame

difflame

[Edmundo Carmona Antoranz]

Hi!

For a very long time I had wanted to get the output of diff to include
blame information as well (to see when something was added/removed).

I just created a very small (and rough) tool for that purpose. It's
written in python and if it gets to something better than a small
tool, I think it could be worth to be added into git main (perhaps
into contrib?).

If you want to give ir a try:
https://github.com/eantoranz/difflame

Just provide the two treeishs you would like to diff (no more
parameters are accepted at the time) and you will get the diff along
with blame. Running it right now on the project itself:

✔ ~/difflame [master L|⚑ 1]
23:21 $ ./difflame.py HEAD~3 HEAD~
diff --git a/README.txt b/README.txt
new file mode 100644
index 0000000..a82aa27
--- /dev/null
+++ b/README.txt
@@ -0,0 +1,11 @@
+3d426842 (Edmundo Carmona Antoranz 2017-01-17 22:26:18 -0600  1) difflame
+3d426842 (Edmundo Carmona Antoranz 2017-01-17 22:26:18 -0600  2)
+3d426842 (Edmundo Carmona Antoranz 2017-01-17 22:26:18 -0600  3)
Copyright 2017 Edmundo Carmona Antoranz
+3d426842 (Edmundo Carmona Antoranz 2017-01-17 22:26:18 -0600  4)
Released under the terms of GPLv2
+3d426842 (Edmundo Carmona Antoranz 2017-01-17 22:26:18 -0600  5)
+3d426842 (Edmundo Carmona Antoranz 2017-01-17 22:26:18 -0600  6) Show
the output of diff with the additional information of blame.
+3d426842 (Edmundo Carmona Antoranz 2017-01-17 22:26:18 -0600  7)
+3d426842 (Edmundo Carmona Antoranz 2017-01-17 22:26:18 -0600  8)
Lines that remain the same or that were added will indicate when those
lines were 'added' to the file
+3d426842 (Edmundo Carmona Antoranz 2017-01-17 22:26:18 -0600  9)
Lines that were removed will display the last revision where those
lines were _present_ on the file (as provided by blame --re
verse)
+3d426842 (Edmundo Carmona Antoranz 2017-01-17 22:26:18 -0600 10)
+3d426842 (Edmundo Carmona Antoranz 2017-01-17 22:26:18 -0600 11) At
the moment, only two parameters need to be provided: two treeishs to
get the diff from
diff --git a/difflame.py b/difflame.py
index f6e879b..06bfc03 100755
--- a/difflame.py
+++ b/difflame.py
@@ -112,16 +112,20 @@ def process_file_from_diff_output(output_lines,
starting_line):
c661286f (Edmundo Carmona Antoranz 2017-01-17 20:10:07 -0600 112)
diff_line = output_lines[i].split()
c661286f (Edmundo Carmona Antoranz 2017-01-17 20:10:07 -0600 113)
if diff_line[0] != "diff":
c661286f (Edmundo Carmona Antoranz 2017-01-17 20:10:07 -0600 114)
   raise Exception("Doesn't seem to exist a 'diff' line at line " +
str(i + 1) + ": " + output_lines[i])
-3d426842 (Edmundo Carmona Antoranz 2017-01-17 22:26:18 -0600 115)
original_name = diff_line[1]
-3d426842 (Edmundo Carmona Antoranz 2017-01-17 22:26:18 -0600 116)
final_name = diff_line[2]
+f135bf04 (Edmundo Carmona Antoranz 2017-01-17 22:50:50 -0600 115)
original_name = diff_line[2]
+f135bf04 (Edmundo Carmona Antoranz 2017-01-17 22:50:50 -0600 116)
final_name = diff_line[3]
c661286f (Edmundo Carmona Antoranz 2017-01-17 20:10:07 -0600 117)
print output_lines[i]; i+=1
.
.
.


Hope you find it useful

Best regards!

Re: difflame

[Jeff King]

On Tue, Jan 17, 2017 at 11:24:02PM -0600, Edmundo Carmona Antoranz wrote:

> For a very long time I had wanted to get the output of diff to include
> blame information as well (to see when something was added/removed).

This is something I've wanted, too. The trickiest part, though, is
blaming deletions, because git-blame only tracks the origin of content,
not the origin of a change.

For example, try this case:

  git init
  for i in $(seq 1 10); do
    echo $i >>file
    git add file
    git commit -m "add $i"
  done
  sed 4d <file >tmp && mv tmp file
  git commit -am "drop 4"

Running "difflame HEAD~5 HEAD" produces this output:

  diff --git a/file b/file
  index b414108..051c298 100644
  --- a/file
  +++ b/file
  @@ -1,6 +1,9 @@
   ^2b028ff (Jeff King 2017-01-27 22:44:10 -0500 1) 1
   ed056366 (Jeff King 2017-01-27 22:44:10 -0500 2) 2
   771030d8 (Jeff King 2017-01-27 22:44:10 -0500 3) 3
  -89c09c82 (Jeff King 2017-01-27 22:44:10 -0500 4) 4
   b619039c (Jeff King 2017-01-27 22:44:10 -0500 4) 5
   6a7aa0e5 (Jeff King 2017-01-27 22:44:10 -0500 5) 6
  +39bc9dc4 (Jeff King 2017-01-27 22:44:10 -0500 6) 7
  +f253cc8f (Jeff King 2017-01-27 22:44:10 -0500 7) 8
  +85c10f46 (Jeff King 2017-01-27 22:44:10 -0500 8) 9
  +89c09c82 (Jeff King 2017-01-27 22:44:10 -0500 9) 10

The last 4 lines are right; they correspond to the addition commits. But
the line taking away 4 is wrong. You can see even without looking at its
patch, because it is blamed to the same commit that added "10", which
is wrong.

Sorry I don't have a solution. I think it's an open problem with
git-blame, though you could probably script something around "git blame
--reverse". See the commit message of 85af7929e (git-blame --reverse,
2008-04-02) for some discussion.

-Peff

Re: difflame

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Tue, Jan 17, 2017 at 11:24:02PM -0600, Edmundo Carmona Antoranz wrote:
>
>> For a very long time I had wanted to get the output of diff to include
>> blame information as well (to see when something was added/removed).
>
> This is something I've wanted, too. The trickiest part, though, is
> blaming deletions, because git-blame only tracks the origin of content,
> not the origin of a change.

Hmph, this is a comment without looking at what difflame does
internally, so you can ignore me if I am misunderstood what problem
you are pointing out, but I am not sure how "tracks the origin of
content" could be a problem.

If output from "git show" says this:

	--- a/file
	+++ b/file
	@@ -1,5 +1,6 @@
	 a
	 b
	-c
	+C
	+D
	 d
	 e

in order to annotate lines 'a', 'b', 'd', and 'e' for their origin,
you would run 'blame' on the commit the above output was taken from
(or its parent---they are in the context so either would be OK).

You know where 'C' and 'D' came from already.  It's the commit you
are feeding "git show".

In order to run blame to find where 'c' came from, you need to start
at the _parent_ of the commit the above output came from, and the
hunk header shows which line range to find the final 'c'.

So...

Re: difflame

[Jeff King]

On Mon, Jan 30, 2017 at 01:08:41PM -0800, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > On Tue, Jan 17, 2017 at 11:24:02PM -0600, Edmundo Carmona Antoranz wrote:
> >
> >> For a very long time I had wanted to get the output of diff to include
> >> blame information as well (to see when something was added/removed).
> >
> > This is something I've wanted, too. The trickiest part, though, is
> > blaming deletions, because git-blame only tracks the origin of content,
> > not the origin of a change.
> 
> Hmph, this is a comment without looking at what difflame does
> internally, so you can ignore me if I am misunderstood what problem
> you are pointing out, but I am not sure how "tracks the origin of
> content" could be a problem.
> 
> If output from "git show" says this:
> 
> 	--- a/file
> 	+++ b/file
> 	@@ -1,5 +1,6 @@
> 	 a
> 	 b
> 	-c
> 	+C
> 	+D
> 	 d
> 	 e
> 
> in order to annotate lines 'a', 'b', 'd', and 'e' for their origin,
> you would run 'blame' on the commit the above output was taken from
> (or its parent---they are in the context so either would be OK).
> 
> You know where 'C' and 'D' came from already.  It's the commit you
> are feeding "git show".

I think the point (or at least as I understand it) is that the diff is
not "git show" for a particular commit. It could be part of a much
larger diff that covers many commits.

As a non-hypothetical instance, I have a fork of git.git that has
various enhancements. I want to feed those enhancements upstream. I need
to know which commits should be cherry-picked to get those various
enhancements.

Looking at "log origin..fork" tells me too many commits, because it
includes ones which aren't useful anymore. Either because they already
went upstream, or because they were cherry-picked to the fork and their
upstream counterparts merged (or even equivalent commits made upstream
that obsoleted the features).

Looking at "git diff origin fork" tells me what the actual differences
are, but it doesn't show me which commits are responsible for them.

I can "git blame" each individual line of the diff (starting with "fork"
as the tip), but that doesn't work for lines that no longer exist (i.e.,
when the interesting change is a deletion).

> In order to run blame to find where 'c' came from, you need to start
> at the _parent_ of the commit the above output came from, and the
> hunk header shows which line range to find the final 'c'.

So perhaps that explains my comment more. "blame" is not good for
finding which commit took away a line. I've tried using "blame
--reverse", but it shows you the parent of the commit you are looking
for, which is slightly annoying. :)

"git log -S" is probably a better tool for finding that.

-Peff

Re: difflame

[Edmundo Carmona Antoranz]

I'm thinking of something like this:

Say I just discovered a problem in a file.... I want to see who worked
on it since some revision that I know is working fine (or even
something as generic as HEAD~100..). It could be a number of people
with different revisions. I would diff to see what changed.... and
blame the added lines (blame reverse to try to get as close as
possible with a single command in case I want to see what happened
with something that was deleted). If I could get blame information of
added/deleted lines in a single run, that would help a lot.

Lo and behold: difflame.



On Mon, Jan 30, 2017 at 5:26 PM, Jeff King <peff@peff.net> wrote:
> On Mon, Jan 30, 2017 at 01:08:41PM -0800, Junio C Hamano wrote:
>
>> Jeff King <peff@peff.net> writes:
>>
>> > On Tue, Jan 17, 2017 at 11:24:02PM -0600, Edmundo Carmona Antoranz wrote:
>> >
>> >> For a very long time I had wanted to get the output of diff to include
>> >> blame information as well (to see when something was added/removed).
>> >
>> > This is something I've wanted, too. The trickiest part, though, is
>> > blaming deletions, because git-blame only tracks the origin of content,
>> > not the origin of a change.
>>
>> Hmph, this is a comment without looking at what difflame does
>> internally, so you can ignore me if I am misunderstood what problem
>> you are pointing out, but I am not sure how "tracks the origin of
>> content" could be a problem.
>>
>> If output from "git show" says this:
>>
>>       --- a/file
>>       +++ b/file
>>       @@ -1,5 +1,6 @@
>>        a
>>        b
>>       -c
>>       +C
>>       +D
>>        d
>>        e
>>
>> in order to annotate lines 'a', 'b', 'd', and 'e' for their origin,
>> you would run 'blame' on the commit the above output was taken from
>> (or its parent---they are in the context so either would be OK).
>>
>> You know where 'C' and 'D' came from already.  It's the commit you
>> are feeding "git show".
>
> I think the point (or at least as I understand it) is that the diff is
> not "git show" for a particular commit. It could be part of a much
> larger diff that covers many commits.
>
> As a non-hypothetical instance, I have a fork of git.git that has
> various enhancements. I want to feed those enhancements upstream. I need
> to know which commits should be cherry-picked to get those various
> enhancements.
>
> Looking at "log origin..fork" tells me too many commits, because it
> includes ones which aren't useful anymore. Either because they already
> went upstream, or because they were cherry-picked to the fork and their
> upstream counterparts merged (or even equivalent commits made upstream
> that obsoleted the features).
>
> Looking at "git diff origin fork" tells me what the actual differences
> are, but it doesn't show me which commits are responsible for them.
>
> I can "git blame" each individual line of the diff (starting with "fork"
> as the tip), but that doesn't work for lines that no longer exist (i.e.,
> when the interesting change is a deletion).
>
>> In order to run blame to find where 'c' came from, you need to start
>> at the _parent_ of the commit the above output came from, and the
>> hunk header shows which line range to find the final 'c'.
>
> So perhaps that explains my comment more. "blame" is not good for
> finding which commit took away a line. I've tried using "blame
> --reverse", but it shows you the parent of the commit you are looking
> for, which is slightly annoying. :)
>
> "git log -S" is probably a better tool for finding that.
>
> -Peff

Re: difflame

[Edmundo Carmona Antoranz]

Maybe a little work on blame to get the actual revision where some
lines were "deleted"?

Something like git blame --blame-deletion that could be based on --reverse?

On Mon, Jan 30, 2017 at 7:35 PM, Edmundo Carmona Antoranz
<eantoranz@gmail.com> wrote:
> I'm thinking of something like this:
>
> Say I just discovered a problem in a file.... I want to see who worked
> on it since some revision that I know is working fine (or even
> something as generic as HEAD~100..). It could be a number of people
> with different revisions. I would diff to see what changed.... and
> blame the added lines (blame reverse to try to get as close as
> possible with a single command in case I want to see what happened
> with something that was deleted). If I could get blame information of
> added/deleted lines in a single run, that would help a lot.
>
> Lo and behold: difflame.
>
>
>
> On Mon, Jan 30, 2017 at 5:26 PM, Jeff King <peff@peff.net> wrote:
>> On Mon, Jan 30, 2017 at 01:08:41PM -0800, Junio C Hamano wrote:
>>
>>> Jeff King <peff@peff.net> writes:
>>>
>>> > On Tue, Jan 17, 2017 at 11:24:02PM -0600, Edmundo Carmona Antoranz wrote:
>>> >
>>> >> For a very long time I had wanted to get the output of diff to include
>>> >> blame information as well (to see when something was added/removed).
>>> >
>>> > This is something I've wanted, too. The trickiest part, though, is
>>> > blaming deletions, because git-blame only tracks the origin of content,
>>> > not the origin of a change.
>>>
>>> Hmph, this is a comment without looking at what difflame does
>>> internally, so you can ignore me if I am misunderstood what problem
>>> you are pointing out, but I am not sure how "tracks the origin of
>>> content" could be a problem.
>>>
>>> If output from "git show" says this:
>>>
>>>       --- a/file
>>>       +++ b/file
>>>       @@ -1,5 +1,6 @@
>>>        a
>>>        b
>>>       -c
>>>       +C
>>>       +D
>>>        d
>>>        e
>>>
>>> in order to annotate lines 'a', 'b', 'd', and 'e' for their origin,
>>> you would run 'blame' on the commit the above output was taken from
>>> (or its parent---they are in the context so either would be OK).
>>>
>>> You know where 'C' and 'D' came from already.  It's the commit you
>>> are feeding "git show".
>>
>> I think the point (or at least as I understand it) is that the diff is
>> not "git show" for a particular commit. It could be part of a much
>> larger diff that covers many commits.
>>
>> As a non-hypothetical instance, I have a fork of git.git that has
>> various enhancements. I want to feed those enhancements upstream. I need
>> to know which commits should be cherry-picked to get those various
>> enhancements.
>>
>> Looking at "log origin..fork" tells me too many commits, because it
>> includes ones which aren't useful anymore. Either because they already
>> went upstream, or because they were cherry-picked to the fork and their
>> upstream counterparts merged (or even equivalent commits made upstream
>> that obsoleted the features).
>>
>> Looking at "git diff origin fork" tells me what the actual differences
>> are, but it doesn't show me which commits are responsible for them.
>>
>> I can "git blame" each individual line of the diff (starting with "fork"
>> as the tip), but that doesn't work for lines that no longer exist (i.e.,
>> when the interesting change is a deletion).
>>
>>> In order to run blame to find where 'c' came from, you need to start
>>> at the _parent_ of the commit the above output came from, and the
>>> hunk header shows which line range to find the final 'c'.
>>
>> So perhaps that explains my comment more. "blame" is not good for
>> finding which commit took away a line. I've tried using "blame
>> --reverse", but it shows you the parent of the commit you are looking
>> for, which is slightly annoying. :)
>>
>> "git log -S" is probably a better tool for finding that.
>>
>> -Peff

Re: difflame

[Edmundo Carmona Antoranz]

I have been "scripting around git blame --reverse" for some days now.
Mind taking a look? I've been working on blame-deletions for this.

22:41 $ ../difflame/difflame.py HEAD~5 HEAD
diff --git a/file b/file
index b414108..051c298 100644
--- a/file
+++ b/file
@@ -1,6 +1,9 @@
^1513353 (Edmundo 2017-02-02 22:41:45 -0600 1) 1
f20952eb (Edmundo 2017-02-02 22:41:45 -0600 2) 2
bb04dc7c (Edmundo 2017-02-02 22:41:45 -0600 3) 3
-de3c07b (Edmundo 2017-02-02 22:41:47 -060 4) 4
058ea125 (Edmundo 2017-02-02 22:41:45 -0600 4) 5
85fc6b81 (Edmundo 2017-02-02 22:41:45 -0600 5) 6
+2cd990a6 (Edmundo 2017-02-02 22:41:45 -0600 6) 7
+ab0be970 (Edmundo 2017-02-02 22:41:45 -0600 7) 8
+944452c0 (Edmundo 2017-02-02 22:41:45 -0600 8) 9
+6641edb0 (Edmundo 2017-02-02 22:41:45 -0600 9) 10


$ git show de3c07b
commit de3c07bc21a83472d5c5ddf172dcb742665924dd (HEAD -> master)
Author: Edmundo <eantoranz@gmail.com>
Date:   Thu Feb 2 22:41:47 2017 -0600

   drop 4

diff --git a/file b/file
index f00c965..051c298 100644
--- a/file
+++ b/file
@@ -1,7 +1,6 @@
1
2
3
-4
5
6
7


Next step: solve the
find-real-deletion-revision-when-there-are-multiple-child-nodes
problem.... and let me read the discussion around git blame --reverse.

Thanks in advance.

On Mon, Jan 30, 2017 at 8:37 PM, Edmundo Carmona Antoranz
<eantoranz@gmail.com> wrote:
> Maybe a little work on blame to get the actual revision where some
> lines were "deleted"?
>
> Something like git blame --blame-deletion that could be based on --reverse?
>
> On Mon, Jan 30, 2017 at 7:35 PM, Edmundo Carmona Antoranz
> <eantoranz@gmail.com> wrote:
>> I'm thinking of something like this:
>>
>> Say I just discovered a problem in a file.... I want to see who worked
>> on it since some revision that I know is working fine (or even
>> something as generic as HEAD~100..). It could be a number of people
>> with different revisions. I would diff to see what changed.... and
>> blame the added lines (blame reverse to try to get as close as
>> possible with a single command in case I want to see what happened
>> with something that was deleted). If I could get blame information of
>> added/deleted lines in a single run, that would help a lot.
>>
>> Lo and behold: difflame.
>>
>>
>>
>> On Mon, Jan 30, 2017 at 5:26 PM, Jeff King <peff@peff.net> wrote:
>>> On Mon, Jan 30, 2017 at 01:08:41PM -0800, Junio C Hamano wrote:
>>>
>>>> Jeff King <peff@peff.net> writes:
>>>>
>>>> > On Tue, Jan 17, 2017 at 11:24:02PM -0600, Edmundo Carmona Antoranz wrote:
>>>> >
>>>> >> For a very long time I had wanted to get the output of diff to include
>>>> >> blame information as well (to see when something was added/removed).
>>>> >
>>>> > This is something I've wanted, too. The trickiest part, though, is
>>>> > blaming deletions, because git-blame only tracks the origin of content,
>>>> > not the origin of a change.
>>>>
>>>> Hmph, this is a comment without looking at what difflame does
>>>> internally, so you can ignore me if I am misunderstood what problem
>>>> you are pointing out, but I am not sure how "tracks the origin of
>>>> content" could be a problem.
>>>>
>>>> If output from "git show" says this:
>>>>
>>>>       --- a/file
>>>>       +++ b/file
>>>>       @@ -1,5 +1,6 @@
>>>>        a
>>>>        b
>>>>       -c
>>>>       +C
>>>>       +D
>>>>        d
>>>>        e
>>>>
>>>> in order to annotate lines 'a', 'b', 'd', and 'e' for their origin,
>>>> you would run 'blame' on the commit the above output was taken from
>>>> (or its parent---they are in the context so either would be OK).
>>>>
>>>> You know where 'C' and 'D' came from already.  It's the commit you
>>>> are feeding "git show".
>>>
>>> I think the point (or at least as I understand it) is that the diff is
>>> not "git show" for a particular commit. It could be part of a much
>>> larger diff that covers many commits.
>>>
>>> As a non-hypothetical instance, I have a fork of git.git that has
>>> various enhancements. I want to feed those enhancements upstream. I need
>>> to know which commits should be cherry-picked to get those various
>>> enhancements.
>>>
>>> Looking at "log origin..fork" tells me too many commits, because it
>>> includes ones which aren't useful anymore. Either because they already
>>> went upstream, or because they were cherry-picked to the fork and their
>>> upstream counterparts merged (or even equivalent commits made upstream
>>> that obsoleted the features).
>>>
>>> Looking at "git diff origin fork" tells me what the actual differences
>>> are, but it doesn't show me which commits are responsible for them.
>>>
>>> I can "git blame" each individual line of the diff (starting with "fork"
>>> as the tip), but that doesn't work for lines that no longer exist (i.e.,
>>> when the interesting change is a deletion).
>>>
>>>> In order to run blame to find where 'c' came from, you need to start
>>>> at the _parent_ of the commit the above output came from, and the
>>>> hunk header shows which line range to find the final 'c'.
>>>
>>> So perhaps that explains my comment more. "blame" is not good for
>>> finding which commit took away a line. I've tried using "blame
>>> --reverse", but it shows you the parent of the commit you are looking
>>> for, which is slightly annoying. :)
>>>
>>> "git log -S" is probably a better tool for finding that.
>>>
>>> -Peff

Re: difflame

[Edmundo Carmona Antoranz]

On Thu, Feb 2, 2017 at 10:46 PM, Edmundo Carmona Antoranz
<eantoranz@gmail.com> wrote:
> I have been "scripting around git blame --reverse" for some days now.
> Mind taking a look? I've been working on blame-deletions for this.

blame-deletions branch, that is

Sorry for the previous top-posting.