Re: Extra newline when retrieving messages

[Konstantin Ryabitsev <konstantin@linuxfoundation.org>]

On Thu, Dec 10, 2020 at 08:55:40PM +0000, Eric Wong wrote:
> Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> > Hello:
> > 
> > While investigating why some of the messages retrieved via 
> > lore.kernel.org were failing DKIM checks, I realized that 
> > public-inbox-httpd appends an extra newline to message bodies. This 
> > newline isn't present in git backends, just in messages retrieved via 
> > (at least) public-inbox-httpd. 
> 
> It looks like a regression from commit dbdc7a42dd885523 (2016-04-11);
> which now doesn't make sense, though very little is making sense
> to me nowadays :<

Actually, my conclusions were wrong and the problem is not with the 
newline -- it's the extra inserted headers. So you're not the only 
person confused here. :)

The message that tripped me up is this one:

https://lore.kernel.org/alsa-devel/20201210152541.191728-1-amadeuszx.slawinski@linux.intel.com/raw

The DKIM signature in that message is:

DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=alsa-project.org;
	s=default; t=1607605851;
	bh=RloCOZ1mS9qShbBGvPnUerOnMg14SHidcYi1OxvdtE0=;
	h=From:To:Subject:Date:Cc:List-Id:List-Unsubscribe:List-Archive:
	 List-Post:List-Help:List-Subscribe:From;
	b=nBOdKYbMY44uVJMlw++UIMx03JWn334I/F5uyH5hmhU9h9/cFQTfZL0WAwPzzG7nL
	 pT9M4ElwsSqu8isrhJd7QV8q8DNvu+cRmGKbGOVLIEMhlYU87iHvIATKQmchLQv3xR
	 OixHf1955oAoTFU/n5eqjaZdLhyfUtSo5oCdxg7Y=

The problem here is that List-Archive is included in the list of signed 
headers, but when we retrieve the message via lore.kernel.org, it 
inserts an additional couple of headers:

Archived-At: <https://lore.kernel.org/alsa-devel/20201210152541.191728-1-amadeuszx.slawinski@linux.intel.com/>
List-Archive: <https://lore.kernel.org/alsa-devel/>

This is what causes DKIM verification to fail, and NOT the newline:

$ curl -s \
  https://lore.kernel.org/alsa-devel/20201210152541.191728-1-amadeuszx.slawinski@linux.intel.com/raw \
  | dkimverify
signature verification failed

$ curl -s \
  https://lore.kernel.org/alsa-devel/20201210152541.191728-1-amadeuszx.slawinski@linux.intel.com/raw \
  | grep -v '<https://lore' | dkimverify
signature ok

So, I was confused as well and there is no need to fix the newline.

That said, should public-inbox consider this case when generating the 
/raw and /t.mbox.gz messages? If the Archived-At and List-Archive 
headers are listed in the DKIM-Signature header, skip inserting them 
into the generated message?

-K