From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS6315 166.70.0.0/16 X-Spam-Status: No, score=-3.7 required=3.0 tests=AWL,BAYES_00, RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from out01.mta.xmission.com (out01.mta.xmission.com [166.70.13.231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 2478B1F62D; Sat, 7 Jul 2018 18:22:43 +0000 (UTC) Received: from in01.mta.xmission.com ([166.70.13.51]) by out01.mta.xmission.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.87) (envelope-from ) id 1fbrr0-0004eH-Gf; Sat, 07 Jul 2018 12:22:42 -0600 Received: from [97.119.167.31] (helo=x220.xmission.com) by in01.mta.xmission.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.87) (envelope-from ) id 1fbrqz-00037g-Nm; Sat, 07 Jul 2018 12:22:42 -0600 From: ebiederm@xmission.com (Eric W. Biederman) To: Eric Wong Cc: meta@public-inbox.org References: <87a7r6z1cy.fsf@xmission.com> <20180705231346.GA6524@dcvr> <87601turnf.fsf@xmission.com> <87o9flt496.fsf@xmission.com> <87efggt5i8.fsf_-_@xmission.com> <20180706222251.GA14747@dcvr> Date: Sat, 07 Jul 2018 13:22:28 -0500 In-Reply-To: <20180706222251.GA14747@dcvr> (Eric Wong's message of "Fri, 6 Jul 2018 22:22:51 +0000") Message-ID: <87d0vysy6z.fsf_-_@xmission.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-XM-SPF: eid=1fbrqz-00037g-Nm;;;mid=<87d0vysy6z.fsf_-_@xmission.com>;;;hst=in01.mta.xmission.com;;;ip=97.119.167.31;;;frm=ebiederm@xmission.com;;;spf=neutral X-XM-AID: U2FsdGVkX19RizRTcjhiU9rR+BNs+QbmFv+DuFZazB8= X-SA-Exim-Connect-IP: 97.119.167.31 X-SA-Exim-Mail-From: ebiederm@xmission.com Subject: [PATCH] Import: Don't copy nulls from emails into git X-SA-Exim-Version: 4.2.1 (built Thu, 05 May 2016 13:38:54 -0600) X-SA-Exim-Scanned: Yes (on in01.mta.xmission.com) List-Id: Recently I ran git --git-dir=lkml/git/1.git fsck and it reported: > warning in commit 299dbd50b6995c6debe2275f0df984ce697fb4cc: nulInCommit: NULL byte inthe commit object body Which I found quite scary. Nulls in the wrong place have a bad tendency to make programs misbehave. It turns out someone had placed "=?iso-8859-1?q?=00?=" at the end of their subject line. Which is the mime encoding for NULL. Email::Mime had correctly decoded the header, and then public-inbox had simply copied the contents of the header into the subject line of the git commit. To prevent that from causing problems replace nulls in such subject lines with spaces. Signed-off-by: Eric Biederman --- lib/PublicInbox/Import.pm | 2 ++ t/nulsubject.t | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 35 insertions(+) create mode 100644 t/nulsubject.t diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm index 250a2db31e97..8c1819209e58 100644 --- a/lib/PublicInbox/Import.pm +++ b/lib/PublicInbox/Import.pm @@ -405,6 +405,8 @@ sub add { print $w "reset $ref\n" or wfail; } + # Mime decoding can create nulls replace them with spaces to protect git + $subject =~ s/\0/ /; utf8::encode($subject); print $w "commit $ref\nmark :$commit\n", "author $name <$email> $author_time_raw\n", diff --git a/t/nulsubject.t b/t/nulsubject.t new file mode 100644 index 000000000000..bb05be8589e7 --- /dev/null +++ b/t/nulsubject.t @@ -0,0 +1,33 @@ +# Copyright (C) 2016-2018 all contributors +# License: AGPL-3.0+ +use strict; +use warnings; +use Test::More; +use File::Temp qw/tempdir/; + +use_ok 'PublicInbox::Import'; +use_ok 'PublicInbox::Git'; +my $tmpdir = tempdir('pi-nulsubject-XXXXXX', TMPDIR => 1, CLEANUP => 1); +my $git_dir = "$tmpdir/a.git"; + +{ + is(system(qw(git init -q --bare), $git_dir), 0, 'git init ok'); + my $git = PublicInbox::Git->new($git_dir); + my $im = PublicInbox::Import->new($git, 'testbox', 'test@example'); + $im->add(Email::MIME->create( + header => [ + From => 'a@example.com', + To => 'b@example.com', + 'Content-Type' => 'text/plain', + Subject => ' A subject line with a null =?iso-8859-1?q?=00?= see!', + 'Message-ID' => '', + ], + body => "hello world\n", + )); + $im->done; + is(system(qw(git --git-dir), $git_dir, 'fsck', '--strict'), 0, 'git fsck ok'); +} + +done_testing(); + +1; -- 2.17.1