Re: [PATCH 0/3] [GSOC][RFC] ref-filter: add contents:raw atom

[Junio C Hamano <gitster@pobox.com>]

ZheNing Hu <adlternative@gmail.com> writes:

> $ printf '%b' "name='a\\\0b\\\0c'\necho -e \"\$name\"" | sh | od -c
> 0000000   a  \0   b  \0   c  \n
> 0000006

This is wrong.  In the above, the variable name does not have a NUL
in it.  It insead has 2-byte sequence "\" and "0" in it, and you are
letting "echo -e" to convert it into binary, which is not portable
at all.

I'd suggest instead to declare that some host languages, like shell,
are not binary-clean and either refuse to process atoms whose values
have NUL in them.  Silently truncating strings at NUL or striping
NULs in strings are also acceptable options if clearly documented.
Claiming that we stuff binaries into variables of the host language,
while not doing so and instead assigning a quoted form, is not good.

I have not thought about Python3 very much.  For the purpose of most
%(placeholders), it is vastly more preferrable to use str (i.e. text
sequence type) as opposed to bytes, as you do not have to .decode()
to use the resulting "string", but even for things like %(refname),
it is not technically kosher to assume that the contents are UTF-8
encoded text, as filenames used to represent refnames are merely a
sequence of bytes except NUL, but for consistency with binary gunk,
we might have to emit everything as bytes.  I dunno.

> In shell or python2/3, we can replace'\0' with "\\0".

Not for shell.  We should declare that it is not supported to feed
binary to shell.