Re: [PATCH 0/3] [GSOC][RFC] ref-filter: add contents:raw atom

[ZheNing Hu <adlternative@gmail.com>]

>
> Another thing to keep in mind is that not all host languages may be
> capable of expressing a string with NUL in it.  Most notably, shell.
> The --shell quoting rule used by for-each-ref would produce an
> equivalent of the "script" produced like this:
>
>         $ tr Q '\000' >script <<\EOF
>         #!/bin/sh
>         varname='varQname'
>         echo "$varname"
>         EOF
>
> but I do not think it would say 'var' followed by a NUL followed by
> 'name'.  The NUL is likely lost when assigned to the variable.
>

Yes, in the following example you mentioned earlier, I have also
noticed the loss of '\0'.

> >     git for-each-ref --format='
> >               name=%(refname)
> >               var=%(placeholder)
> >                 mkdir -p "$(dirname "$name")"
> >               printf "%%s" "$var" >"$name"
> >     ' --shell | /bin/sh
> >

> So for some host languages, binaries may be useless with or without
> quoting.  But for ones that can use strings to hold arbitrary byte
> sequence, it should be OK to let for-each-ref to quote the byte
> sequence as a string literal for the language (so that the exact
> byte sequence will end up being in the variable after assignment).
>

I agree, and maybe some'\0' can be escaped appropriately to let host
languages recognize it....

> That reminds me of another thing.  The --python thing was written
> back when Python3 was still a distant dream and strings were the
> appropriate type for a random sequence of bytes (as opposed to
> unicode, which cannot have a random sequence of bytes).  Somebody
> needs to check if it needs any update to work with Python3.

$ printf '%b' "name='a\\\0b\\\0c'\nprint(name)" | python2.7 | od -c
0000000   a  \0   b  \0   c  \n
0000006

$ printf '%b' "name='a\\\0b\\\0c'\necho -e \"\$name\"" | sh | od -c
0000000   a  \0   b  \0   c  \n
0000006

In shell or python2/3, we can replace'\0' with "\\0".

In Tcl and perl, they are ok with '\0'.

$ printf '%b' "set name \"a\0b\0c\"\nputs \$name" | tclsh | od -c
0000000   '   a  \0   b  \0   c   '  \n
0000010

$ printf '%b' "\$name = 'a\0b\0c';\n print \"\$name\""  | perl | od -c
0000000   a  \0   b  \0   c
0000005

So I currently think that a possible strategy is to modify
`python_quote_buf_with_size()` and `sq_quote_buf_with_size()` from
'\0' to "\\0".

Thanks!

--
ZheNing Hu