RFC: Modernizing the contept of plumbing v.s. porcelain

RFC: Modernizing the contept of plumbing v.s. porcelain

[Ævar Arnfjörð Bjarmason]

A lot of external guides and people's mental models of /usr/bin/git as a
scriptable client reference the concept of plumbing & porcelain. Just
one such example [1] prompted me to write this E-Mail.

I've wondered if we shouldn't be updating this concept to reflect the
reality on the ground in the git command ecosystem.

I.e. if you look at "git help git"'s list of plumbing v.s. porcelain it
makes no mention or distinction between those commands & functionalities
that are truly transitory "porcelain". E.g. the specific error message a
command might return, and those that are effectively plumbing. E.g. some
"git config" functionality, "git init", the pretty formats in "git log"
etc.

I'm not quite sure what I'm proposing if anything, just putting out
feelers to see if others think this documentary status quo has drifted
from reality.

One potential change would be to mostly/entirely remove the
"porcelain/ancillary/plumbing" distinction in "git help git" (except
maybe e.g. "hash-object") and instead make a mention of the status of
the command at the top of its own manpage, which could then also
(e.g. in the case of "git log") document the API reliability of its
various sub-features.

1. https://gitlab.com/gitlab-org/gitaly/-/blob/afc90e3c2/doc/serverside_git_usage.md#L11-17

Re: RFC: Modernizing the contept of plumbing v.s. porcelain

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> A lot of external guides and people's mental models of /usr/bin/git as a
> scriptable client reference the concept of plumbing & porcelain. Just
> one such example [1] prompted me to write this E-Mail.
>
> I've wondered if we shouldn't be updating this concept to reflect the
> reality on the ground in the git command ecosystem.

The example tells me that they are trying to be a good ecosystem
citizen by sticking to the plumbing and refraining from using
Porcelain command when writing their script.  The practice gives
them assurance that we won't unilaterally break them, and gives us
the freedom to improve Porcelain for human consumption.

> I.e. if you look at "git help git"'s list of plumbing v.s. porcelain it
> makes no mention or distinction between those commands & functionalities
> that are truly transitory "porcelain". E.g. the specific error message a
> command might return, and those that are effectively plumbing. E.g. some
> "git config" functionality, "git init", the pretty formats in "git log"
> etc.
>
> I'm not quite sure what I'm proposing if anything, just putting out
> feelers to see if others think this documentary status quo has drifted
> from reality.
>
> One potential change would be to mostly/entirely remove the
> "porcelain/ancillary/plumbing" distinction in "git help git" (except
> maybe e.g. "hash-object") and instead make a mention of the status of
> the command at the top of its own manpage, which could then also
> (e.g. in the case of "git log") document the API reliability of its
> various sub-features.
>
> 1. https://gitlab.com/gitlab-org/gitaly/-/blob/afc90e3c2/doc/serverside_git_usage.md#L11-17

I am not sure what you want to propose as a solution, but even
before that, what problem you are perceiving.  Are you wondering if
it may be a better general direction for us to tell "no, no, there
is no value in sticking to the plumbing because we will break you
anyway in the future" to those who wrote [1]?

Re: RFC: Modernizing the contept of plumbing v.s. porcelain

[Felipe Contreras]

On Wed, Dec 9, 2020 at 5:02 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>
> A lot of external guides and people's mental models of /usr/bin/git as a
> scriptable client reference the concept of plumbing & porcelain. Just
> one such example [1] prompted me to write this E-Mail.
>
> I've wondered if we shouldn't be updating this concept to reflect the
> reality on the ground in the git command ecosystem.
>
> I.e. if you look at "git help git"'s list of plumbing v.s. porcelain it
> makes no mention or distinction between those commands & functionalities
> that are truly transitory "porcelain". E.g. the specific error message a
> command might return, and those that are effectively plumbing. E.g. some
> "git config" functionality, "git init", the pretty formats in "git log"
> etc.
>
> I'm not quite sure what I'm proposing if anything, just putting out
> feelers to see if others think this documentary status quo has drifted
> from reality.

One option would be to split git into two binaries: "git" and
"git-tool". Obviously the latter would be plumbing.

We could slowly move the documentation to git-tool and by doing so we
could see that if a porcelain man page has too many links to git-tool
documentation, that's some area of opportunity.

Every time you access a git-tool command inside git, it still would
work, but you will get a warning: "you are using a plumbing command,
use git-tool instead". Scripts could enable GIT_TOOL_MODE=1 if they
are going to access many of these commands and don't want to
s/git/git-tool/.

I would be a ton of work, but it's something I see value in doing.

Cheers.

-- 
Felipe Contreras

Re: RFC: Modernizing the contept of plumbing v.s. porcelain

[Ævar Arnfjörð Bjarmason]

On Wed, Dec 09 2020, Junio C Hamano wrote:

> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> A lot of external guides and people's mental models of /usr/bin/git as a
>> scriptable client reference the concept of plumbing & porcelain. Just
>> one such example [1] prompted me to write this E-Mail.
>>
>> I've wondered if we shouldn't be updating this concept to reflect the
>> reality on the ground in the git command ecosystem.
>
> The example tells me that they are trying to be a good ecosystem
> citizen by sticking to the plumbing and refraining from using
> Porcelain command when writing their script.  The practice gives
> them assurance that we won't unilaterally break them, and gives us
> the freedom to improve Porcelain for human consumption.
>
>> I.e. if you look at "git help git"'s list of plumbing v.s. porcelain it
>> makes no mention or distinction between those commands & functionalities
>> that are truly transitory "porcelain". E.g. the specific error message a
>> command might return, and those that are effectively plumbing. E.g. some
>> "git config" functionality, "git init", the pretty formats in "git log"
>> etc.
>>
>> I'm not quite sure what I'm proposing if anything, just putting out
>> feelers to see if others think this documentary status quo has drifted
>> from reality.
>>
>> One potential change would be to mostly/entirely remove the
>> "porcelain/ancillary/plumbing" distinction in "git help git" (except
>> maybe e.g. "hash-object") and instead make a mention of the status of
>> the command at the top of its own manpage, which could then also
>> (e.g. in the case of "git log") document the API reliability of its
>> various sub-features.
>>
>> 1. https://gitlab.com/gitlab-org/gitaly/-/blob/afc90e3c2/doc/serverside_git_usage.md#L11-17
>
> I am not sure what you want to propose as a solution, but even
> before that, what problem you are perceiving.  Are you wondering if
> it may be a better general direction for us to tell "no, no, there
> is no value in sticking to the plumbing because we will break you
> anyway in the future" to those who wrote [1]?

I was updating that documentation, and ended up going for:

    +#### Plumbing v.s. porcelain
    +
     `git(1)` is the default choice to access repositories for Gitaly. Not all
    -commands that are available should be used in the Gitaly code base. Git makes
    -a distinction between porcelain and plumbing commands. Porcelain commands are
    -intended for the end-user and are the user-interface of the default `git`
    -client, where plumbing commands are intended for scripted use or to build
    -another porcelain. Gitaly should only use plumbing commands. `man 1 git`
    -contains a section on the low level plumbing.
    +commands that are available should be used in the Gitaly code base.
    +
    +Git makes a distinction between porcelain and plumbing
    +commands. Porcelain commands are intended for the end-user and are the
    +user-interface of the default `git` client, where plumbing commands
    +are intended for scripted use or to build another porcelain.
    +
    +Generally speaking, Gitaly should only use plumbing commands. `man 1
    +git` contains a section on the low level plumbing. However, a lot of
    +git's plumbing-like functionality is exposed as commands not marked as
    +plumbing, but whose API reliability can be considered the
    +same. E.g. `git log`'s `--pretty=` formats, `git config -l -z`, the
    +documented exit codes of `git remote` etc..
    +
    +We should use good judgement when choosing what commands and command
    +functionality to use, with the aim of not having gitaly break due to
    +e.g. an error message being rephrased or functionality the upstream
    +`git` maintainers don't consider plumbing-like being removed or
    +altered.
    +
    +#### Executing Git commands

I.e. the existing advice was to say "just use plumbing", now it takes a
more nuanced view of e.g. pointing out that certin porcelain commands
have "-z" options that can be considered as reliable as things
explicitly marked as plumbing.

It's widespread traditional wisdom when discussing git that there's
plumbing and porcelain, but I think ever since all the builtins were
shellscripts way-back-when this distinction has blurred.

We now have some plumbing tools whole entire functionality is considered
a stable API, some porcelain tools you shouldn't rely on at all for
that, but a large set of tools that are in-between somewhere. E.g. maybe
their output format(s) are "plumbing-like", or some options (such as
"-z") but not others.

I think updating our documentation to reflect this would be a good idea,
and I'm willing to write those patches. Just thought I'd weather-balloon
what others thought about it first.

I think a good way forward there would be:

 1. Keep the high-level/low-level list in "man git", but not mention
    plumbing/porcelain so prominently.

    Some of that's ... a bit suspect, e.g. "its low-level commands are
    sufficient to support development of alternative porcelains", but
    then git-config(1) is porcelain. These days you're not getting very
    far in developing your own alternate porcelain with just the
    plumbing commands.

 2. Either inline at the bottom, or probably better in a new
    gitplumbing(5) (or gitapi(5) or something) explain the nuance, that
    some commads are pure-plumbing, some pure-porcelain, and some are
    hybrids.

    That you can follow some general rules (does it have "-z" output,
    can probably be relied on) to determine "plumbing-like", or
    porcelain-like (is it stdout/stderr output in English, does it
    change under i18n?), that not all manpages might explicitly call out
    plumbing / porcelain, and that when in doubt ask the list.

Re: RFC: Modernizing the contept of plumbing v.s. porcelain

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> I.e. the existing advice was to say "just use plumbing", now it takes a
> more nuanced view of e.g. pointing out that certin porcelain commands
> have "-z" options that can be considered as reliable as things
> explicitly marked as plumbing.

Yup, and we have --porcelain option to some commands that are meant
to help Porcelain writers.

> It's widespread traditional wisdom when discussing git that there's
> plumbing and porcelain, but I think ever since all the builtins were
> shellscripts way-back-when this distinction has blurred.

Yeah, it is somewhat unfortunate that it is human nature to be
excited by shiny new toys that are end results; some of the newer
features we added and only available at the Porcelain level may be
better made available to the plumbing, but that is one of the
easiest corners authors can cut.

>  2. Either inline at the bottom, or probably better in a new
>     gitplumbing(5) (or gitapi(5) or something) explain the nuance, that
>     some commads are pure-plumbing, some pure-porcelain, and some are
>     hybrids.
>
>     That you can follow some general rules (does it have "-z" output,
>     can probably be relied on) to determine "plumbing-like", or
>     porcelain-like (is it stdout/stderr output in English, does it
>     change under i18n?), that not all manpages might explicitly call out
>     plumbing / porcelain, and that when in doubt ask the list.

OK.