git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
* git cat-file --batch surprising carriage return behavior
@ 2019-10-08 19:22 Joey Hess
  2019-10-08 20:00 ` Jeff King
  0 siblings, 1 reply; 4+ messages in thread
From: Joey Hess @ 2019-10-08 19:22 UTC (permalink / raw)
  To: git

[-- Attachment #1: Type: text/plain, Size: 668 bytes --]

I'm surprised to find that git cat-file --batch, on a Linux system,
strips the \r from an input like "HEAD:foo\r\n"

It's obvious, of course, that it will remove the newline, and so this
interface cannot be used to query about a filename that, for some
horrible reason[1], contains a newline. But very surprising that it
cannot be used for filename that contains a carriage return, at least
on a non-Windows system.

The docs for cat-file --batch say the list of objects is separated by
linefeeds. I don't know if updating the docs is the best fix.
(I'd be happy to use a -z if it had one.)

-- 
see shy jo

[1] aka "a large enough number of monkeys"

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: git cat-file --batch surprising carriage return behavior
  2019-10-08 19:22 git cat-file --batch surprising carriage return behavior Joey Hess
@ 2019-10-08 20:00 ` Jeff King
  2019-10-09 15:28   ` Joey Hess
  0 siblings, 1 reply; 4+ messages in thread
From: Jeff King @ 2019-10-08 20:00 UTC (permalink / raw)
  To: Joey Hess; +Cc: git

On Tue, Oct 08, 2019 at 03:22:57PM -0400, Joey Hess wrote:

> I'm surprised to find that git cat-file --batch, on a Linux system,
> strips the \r from an input like "HEAD:foo\r\n"
> 
> It's obvious, of course, that it will remove the newline, and so this
> interface cannot be used to query about a filename that, for some
> horrible reason[1], contains a newline. But very surprising that it
> cannot be used for filename that contains a carriage return, at least
> on a non-Windows system.

This is likely due to b42ca3dd0f (cat-file: read batch stream with
strbuf_getline(), 2015-10-28), and the matching c8aa9fdf5d (strbuf: make
strbuf_getline_crlf() global, 2015-10-28).

I agree it's a bit surprising (though OTOH, I imagine the old behavior
surprised some people in the opposite direction).

> The docs for cat-file --batch say the list of objects is separated by
> linefeeds. I don't know if updating the docs is the best fix.
> (I'd be happy to use a -z if it had one.)

Yeah, I agree that a -z option is the best path forward. For non-z
input, I'm tempted to say we could unquote entries that start with a
double-quote (the match to how we handle filenames in non-z diff
output). That would mean breaking compatibility for refnames that start
with a quote, though. If we just add a new "-z", that's less disruptive
_and_ easier to use.

I suspect it's not entirely sufficient for clean input, though. You're
not feeding filenames but rather full "object names". I wouldn't be
surprised if we mis-parse "$rev:$path" when $path has "@{}" or similar
in it.

So what you may actually want is some more robust input format that lets
you specify the filename as an independent NUL-terminated entity.

-Peff

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: git cat-file --batch surprising carriage return behavior
  2019-10-08 20:00 ` Jeff King
@ 2019-10-09 15:28   ` Joey Hess
  2019-10-11  6:21     ` Jeff King
  0 siblings, 1 reply; 4+ messages in thread
From: Joey Hess @ 2019-10-09 15:28 UTC (permalink / raw)
  To: Jeff King; +Cc: git

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]

Jeff King wrote:
> If we just add a new "-z", that's less disruptive_and_ easier to use.

Agreed. 

> I suspect it's not entirely sufficient for clean input, though. You're
> not feeding filenames but rather full "object names". I wouldn't be
> surprised if we mis-parse "$rev:$path" when $path has "@{}" or similar
> in it.

Nothing I've tried along the lines of "HEAD:{yesterday}" has misparsed
the part after the colon as anything but a filename.

The one I can think of where there's a parse ambiguity is that while
:foo gets file foo, :1:foo does not get file "1:foo". Instead it's
treated as a stage number. Using either HEAD:1:foo or :./1:foo
will avoid that ambiguity.

-- 
see shy jo

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: git cat-file --batch surprising carriage return behavior
  2019-10-09 15:28   ` Joey Hess
@ 2019-10-11  6:21     ` Jeff King
  0 siblings, 0 replies; 4+ messages in thread
From: Jeff King @ 2019-10-11  6:21 UTC (permalink / raw)
  To: Joey Hess; +Cc: git

On Wed, Oct 09, 2019 at 11:28:51AM -0400, Joey Hess wrote:

> > I suspect it's not entirely sufficient for clean input, though. You're
> > not feeding filenames but rather full "object names". I wouldn't be
> > surprised if we mis-parse "$rev:$path" when $path has "@{}" or similar
> > in it.
> 
> Nothing I've tried along the lines of "HEAD:{yesterday}" has misparsed
> the part after the colon as anything but a filename.

It's possible we've fixed them all. We definitely don't parse strictly
left-to-right. The first thing we try to do is strip bits like ^{commit}
off the end, before we even find the colon. But after doing so, we
should generally be left with a resolvable name, and I think that uses
the "basic" parser which will not allow colons. I.e., this:

  mkdir subdir
  echo whatever >subdir/file
  git add subdir
  git commit -m 'add directory'
  git show HEAD:subdir^{tree}

really does look for the file "subdir^{tree}" in HEAD, and not
"HEAD:subdir" as a tree.

We have had bugs in the past; I'm thinking specifically of 8cd4249c4c
(interpret_branch_name: always respect "namelen" parameter, 2014-01-15).
But I couldn't find any problematic inputs after poking around for a few
minutes.

> The one I can think of where there's a parse ambiguity is that while
> :foo gets file foo, :1:foo does not get file "1:foo". Instead it's
> treated as a stage number. Using either HEAD:1:foo or :./1:foo
> will avoid that ambiguity.

Yeah, that makes sense.

-Peff

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, back to index

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-08 19:22 git cat-file --batch surprising carriage return behavior Joey Hess
2019-10-08 20:00 ` Jeff King
2019-10-09 15:28   ` Joey Hess
2019-10-11  6:21     ` Jeff King

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox