[PATCH] Import.pm: When purging replace a purged file with a zero length file

[PATCH] Import.pm: When purging replace a purged file with a zero length file

[Eric W. Biederman]

This ensures that the number of added files remains the same and thus
the article numbers derived from a repository will remain the same.

I think this is the last place in public-inbox that has to be tweaked to
guarantee the generated article number will remain the same in an public
inbox archive.

Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
---
 lib/PublicInbox/Import.pm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm
index bfa7a8053297..3df7d98f298b 100644
--- a/lib/PublicInbox/Import.pm
+++ b/lib/PublicInbox/Import.pm
@@ -519,11 +519,12 @@ sub purge_oids {
 			push @buf, $buf;
 		} elsif (/^M 100644 ([a-f0-9]+) (\w+)/) {
 			my ($oid, $path) = ($1, $2);
+			$tree->{$path} = 1;
 			if ($purge->{$oid}) {
 				push @oids, $oid;
-				delete $tree->{$path};
+				my $cmd = "M 100644 inline $path\ndata 0\n\n";
+				push @buf, $cmd;
 			} else {
-				$tree->{$path} = 1;
 				push @buf, $_;
 			}
 		} elsif (/^D (\w+)/) {
-- 
2.17.1

Re: [PATCH] Import.pm: When purging replace a purged file with a zero length file

[Eric Wong]

"Eric W. Biederman" <ebiederm@xmission.com> wrote:
> 
> This ensures that the number of added files remains the same and thus
> the article numbers derived from a repository will remain the same.
> 
> I think this is the last place in public-inbox that has to be tweaked to
> guarantee the generated article number will remain the same in an public
> inbox archive.

OK, definitely desirable.

> Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
> ---
>  lib/PublicInbox/Import.pm | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm
> index bfa7a8053297..3df7d98f298b 100644
> --- a/lib/PublicInbox/Import.pm
> +++ b/lib/PublicInbox/Import.pm
> @@ -519,11 +519,12 @@ sub purge_oids {
>  			push @buf, $buf;
>  		} elsif (/^M 100644 ([a-f0-9]+) (\w+)/) {
>  			my ($oid, $path) = ($1, $2);
> +			$tree->{$path} = 1;
>  			if ($purge->{$oid}) {
>  				push @oids, $oid;
> -				delete $tree->{$path};
> +				my $cmd = "M 100644 inline $path\ndata 0\n\n";
> +				push @buf, $cmd;
>  			} else {
> -				$tree->{$path} = 1;
>  				push @buf, $_;
>  			}
>  		} elsif (/^D (\w+)/) {
> -- 

OK.  I haven't checked, but is the indexing/re-indexing code
able to deal with zero-byte messages?  Thanks.

Re: [PATCH] Import.pm: When purging replace a purged file with a zero length file

[Eric W. Biederman]

Eric Wong <e@80x24.org> writes:

> "Eric W. Biederman" <ebiederm@xmission.com> wrote:
>> 
>> This ensures that the number of added files remains the same and thus
>> the article numbers derived from a repository will remain the same.
>> 
>> I think this is the last place in public-inbox that has to be tweaked to
>> guarantee the generated article number will remain the same in an public
>> inbox archive.
>
> OK, definitely desirable.
>
>> Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
>> ---
>>  lib/PublicInbox/Import.pm | 5 +++--
>>  1 file changed, 3 insertions(+), 2 deletions(-)
>> 
>> diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm
>> index bfa7a8053297..3df7d98f298b 100644
>> --- a/lib/PublicInbox/Import.pm
>> +++ b/lib/PublicInbox/Import.pm
>> @@ -519,11 +519,12 @@ sub purge_oids {
>>  			push @buf, $buf;
>>  		} elsif (/^M 100644 ([a-f0-9]+) (\w+)/) {
>>  			my ($oid, $path) = ($1, $2);
>> +			$tree->{$path} = 1;
>>  			if ($purge->{$oid}) {
>>  				push @oids, $oid;
>> -				delete $tree->{$path};
>> +				my $cmd = "M 100644 inline $path\ndata 0\n\n";
>> +				push @buf, $cmd;
>>  			} else {
>> -				$tree->{$path} = 1;
>>  				push @buf, $_;
>>  			}
>>  		} elsif (/^D (\w+)/) {
>> -- 
>
> OK.  I haven't checked, but is the indexing/re-indexing code
> able to deal with zero-byte messages?  Thanks.

The v2mirror test covers this case and it doesn't seem to have any
problems.  The v2mirror performs an inex_sync after the purge and looks
for warnings and doesn't get any.  So I think we are ok.  Skimming
through the code I don't see any obvious issues either.

Eric