Git log: filtering by date and by lines do not work together

Git log: filtering by date and by lines do not work together

[Мария Долгополова]

[-- Attachment #1: Type: text/plain, Size: 1200 bytes --]

Hi! I would like to report the bug.

I want to get commits that change specific lines in a specific file
and were made before a specific date.

Steps:
1) I apply filtering by date: --before="2016-11-18"
Result is in the picture FilteringByDate.png. Commit
570778797effd69bd4536c51125b7d2e8f654c08 is presented.

2) I apply filtering by lines:
-L33,34:src/main/java/org/toradocu/translator/Subject.java
Result is in the picture FilteringByLines.png . Commit
570778797effd69bd4536c51125b7d2e8f654c08 is presented.

3) Finally, I apply filtering by lines and by date:
--before="2016-11-18"
-L33,34:src/main/java/org/toradocu/translator/Subject.java
Result is in the picture FilteringByDatesAndLines.png . Commit
570778797effd69bd4536c51125b7d2e8f654c08 is NOT presented.

Expected Result: the commit is presented in filter 1 and in filter 2,
so it must be presented in filter “1 AND 2”

Actual Result: the commit is NOT presented in filter “1 AND 2”

Environment: git version 2.27.0.windows.1

 I beg you to provide me with a workaround. I am doing research work
on analyzing commits in open source projects. Therefore, this
functionality is very important to me.

[-- Attachment #2: FilteringByDate.png --]
[-- Type: image/png, Size: 43821 bytes --]

[-- Attachment #3: FilteringByDatesAndLines.png --]
[-- Type: image/png, Size: 13292 bytes --]

[-- Attachment #4: FilteringByLines.png --]
[-- Type: image/png, Size: 63551 bytes --]

Re: Git log: filtering by date and by lines do not work together

[René Scharfe]

Am 03.07.20 um 16:14 schrieb Мария Долгополова:
> Hi! I would like to report the bug.
>
> I want to get commits that change specific lines in a specific file
> and were made before a specific date.
>
> Steps:
> 1) I apply filtering by date: --before="2016-11-18"
> Result is in the picture FilteringByDate.png. Commit
> 570778797effd69bd4536c51125b7d2e8f654c08 is presented.
>
> 2) I apply filtering by lines:
> -L33,34:src/main/java/org/toradocu/translator/Subject.java
> Result is in the picture FilteringByLines.png . Commit
> 570778797effd69bd4536c51125b7d2e8f654c08 is presented.
>
> 3) Finally, I apply filtering by lines and by date:
> --before="2016-11-18"
> -L33,34:src/main/java/org/toradocu/translator/Subject.java
> Result is in the picture FilteringByDatesAndLines.png . Commit
> 570778797effd69bd4536c51125b7d2e8f654c08 is NOT presented.
>
> Expected Result: the commit is presented in filter 1 and in filter 2,
> so it must be presented in filter “1 AND 2”
>
> Actual Result: the commit is NOT presented in filter “1 AND 2”
>
> Environment: git version 2.27.0.windows.1
>
>  I beg you to provide me with a workaround. I am doing research work
> on analyzing commits in open source projects. Therefore, this
> functionality is very important to me.

You could do the date filtering on the output of git log, e.g.:

  git log -L33,34:src/main/java/org/toradocu/translator/Subject.java --date=short --color=always |
  awk -v before=2016-11-18 '
    /^[^ ]*commit/ {state=1; header=""}
    /^[^ ]*Date:/ {if ($2 <= before) {state=2; printf "%s", header} else {state=0}}
    state == 1 {header = header $0 ORS}
    state == 2 {print}
  '

This command is a bit unwieldy, so you might want to put the AWK
script into a file, then you could use it like this:

  git log -L33,34:src/main/java/org/toradocu/translator/Subject.java --date=short --color=always |
  awk -v before=2016-11-18 -f gitlogbefore.awk

René

[PATCH] revision: disable min_age optimization with line-log

[René Scharfe]

If one of the options --before, --min-age or --until is given,
limit_list() filters out younger commits early on.  Line-log needs all
those commits to trace the movement of line ranges, though.  Skip this
optimization if both are used together.

Reported-by: Мария Долгополова <dolgopolovamariia@gmail.com>
Signed-off-by: René Scharfe <l.s.r@web.de>
---
Needs careful review -- I'm not familiar with that the line-log code and
the revision traversal machinery is a bit scary.  AFAIU the min_age
check is done again in get_commit_action(), so this patch shouldn't
cause underage commits to be shown, but I'm not sure.  Test coverage for
the three options is spotty. :-/

 revision.c          | 3 ++-
 t/t4211-line-log.sh | 8 ++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/revision.c b/revision.c
index ebb4d2a0f2..3bdc1bbf2a 100644
--- a/revision.c
+++ b/revision.c
@@ -1410,7 +1410,8 @@ static int limit_list(struct rev_info *revs)
 				continue;
 			break;
 		}
-		if (revs->min_age != -1 && (commit->date > revs->min_age))
+		if (revs->min_age != -1 && (commit->date > revs->min_age) &&
+		    !revs->line_level_traverse)
 			continue;
 		date = commit->date;
 		p = &commit_list_insert(commit, p)->next;
diff --git a/t/t4211-line-log.sh b/t/t4211-line-log.sh
index 1428eae262..e186c83250 100755
--- a/t/t4211-line-log.sh
+++ b/t/t4211-line-log.sh
@@ -240,10 +240,12 @@ test_expect_success 'setup for checking line-log and parent oids' '
 	EOF
 	git add file.c &&
 	test_tick &&
+	first_tick=$test_tick &&
 	git commit -m "Add func1() and func2() in file.c" &&

 	echo 1 >other-file &&
 	git add other-file &&
+	test_tick &&
 	git commit -m "Add other-file" &&

 	sed -e "s/F1/F1 + 1/" file.c >tmp &&
@@ -283,4 +285,10 @@ test_expect_success 'parent oids with parent rewriting' '
 	test_cmp expect actual
 '

+test_expect_success 'line-log with --before' '
+	echo $root_oid >expect &&
+	git log --format=%h --no-patch -L:func2:file.c --before=$first_tick >actual &&
+	test_cmp expect actual
+'
+
 test_done
--
2.27.0

Re: [PATCH] revision: disable min_age optimization with line-log

[Junio C Hamano]

René Scharfe <l.s.r@web.de> writes:

> If one of the options --before, --min-age or --until is given,
> limit_list() filters out younger commits early on.  Line-log needs all
> those commits to trace the movement of line ranges, though.  Skip this
> optimization if both are used together.
>
> Reported-by: Мария Долгополова <dolgopolovamariia@gmail.com>
> Signed-off-by: René Scharfe <l.s.r@web.de>
> ---
> Needs careful review -- I'm not familiar with that the line-log code and
> the revision traversal machinery is a bit scary.  AFAIU the min_age
> check is done again in get_commit_action(), so this patch shouldn't
> cause underage commits to be shown, but I'm not sure.  Test coverage for
> the three options is spotty. :-/

I am not familiar with the line-log code, either, but anyway.  This
starts queuing commits that are too young in the list of commits to
be processed, but later stages in the output callchain would decide
that they are too young to be shown in get_commit_action() that is
called from simplify_commit(), so the overall effect of this change
is to show these commits to the line-log machinery but filter them
out of the final output phase (with the parent rewriting, the
line-log machinery sees these commits in prepare_revision_walk(),
and without it, each of these commits is shown to the line-log
machinery immediately before get_commit_action() decides that it is
too young to be shown).  And the effect you want to gain by showing
these commits to the line-log machinery is to ensure that the range
of lines in the starting point is adjusted for the changes each of
these commits makes.

Which makes sense to me.

Thanks, will queue.

>  revision.c          | 3 ++-
>  t/t4211-line-log.sh | 8 ++++++++
>  2 files changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/revision.c b/revision.c
> index ebb4d2a0f2..3bdc1bbf2a 100644
> --- a/revision.c
> +++ b/revision.c
> @@ -1410,7 +1410,8 @@ static int limit_list(struct rev_info *revs)
>  				continue;
>  			break;
>  		}
> -		if (revs->min_age != -1 && (commit->date > revs->min_age))
> +		if (revs->min_age != -1 && (commit->date > revs->min_age) &&
> +		    !revs->line_level_traverse)
>  			continue;
>  		date = commit->date;
>  		p = &commit_list_insert(commit, p)->next;
> diff --git a/t/t4211-line-log.sh b/t/t4211-line-log.sh
> index 1428eae262..e186c83250 100755
> --- a/t/t4211-line-log.sh
> +++ b/t/t4211-line-log.sh
> @@ -240,10 +240,12 @@ test_expect_success 'setup for checking line-log and parent oids' '
>  	EOF
>  	git add file.c &&
>  	test_tick &&
> +	first_tick=$test_tick &&
>  	git commit -m "Add func1() and func2() in file.c" &&
>
>  	echo 1 >other-file &&
>  	git add other-file &&
> +	test_tick &&
>  	git commit -m "Add other-file" &&
>
>  	sed -e "s/F1/F1 + 1/" file.c >tmp &&
> @@ -283,4 +285,10 @@ test_expect_success 'parent oids with parent rewriting' '
>  	test_cmp expect actual
>  '
>
> +test_expect_success 'line-log with --before' '
> +	echo $root_oid >expect &&
> +	git log --format=%h --no-patch -L:func2:file.c --before=$first_tick >actual &&
> +	test_cmp expect actual
> +'
> +
>  test_done
> --
> 2.27.0