Re: On shipping more of our technical docs as manpages

[Jeff King <peff@peff.net>]

On Wed, Sep 26, 2018 at 10:44:33PM +0200, Ævar Arnfjörð Bjarmason wrote:

> My bias here is that I've also contributed actively to the perl project
> in the past, and with that project you can get an overview of *all* of
> the docs by typing:
> 
>     man perl
> 
> That includes stuff like perl585delta(1) which we'd stick in
> Documentation/RelNotes, and "Internals and C Language Interface". Most
> of what we'd put in Documentation/technical/api-* & headers is in
> perlapi(1).

I like the perl documentation, too. But I think most of what you're
talking about there is the public API, where you have many readers. But
here we're talking about internal functions (albeit ones we hope to see
reused across the codebase). My concern is mostly: is the work in
maintaining this documentation-formatting system worth the number of
readers it will get?

There are two numbers there, and I'm guessing at both of them. ;)

If you're interested in pulling documentation out of the header files
and generating asciidoc from it, I'm happy to at least try keeping it up
to date. When we started putting this information into header files, we
used "/**" to start the comment, as a special marker to indicate it was
worth pulling out. I don't know how well we've maintained that
convention, but it's a starting point.

> I spent an embarrassingly long time submitting patches to git before
> discovering that Documentation/technical/api-*.txt even existed, I think
> something like 1-2 years ago.

That's another thing that I think is improved by keeping the
documentation and code together. If you find one, you find the other.
You just have to "somehow" find one, which is what you're getting at
below.

> We have very well documented stuff like strbuf.h (mostly thanks to you),
> but how is such documentation supposed to be discovered? Something like:
> 
>     git grep -A20 '^/\*$' -- *.h
>     <hold in page down>
> 
> ?
> 
> In terms of getting an overview it's indistinguishable from
> comments. I.e. there's nothing like an index of:
> 
>     man git-api-strbuf     ==> working with strings
>     man git-api-sha1-array ==> list, iterate and binary lookup SHA1s

I agree that is a problem, especially for contributors less familiar
with the code base. But I think generating an index is a separate (and
much easier) problem than formatting all of the documentation.

We already have the /** convention I mentioned above. Could we have
another micro-format like:

  /** API:strbuf - working with strings */

in each header file? That would make generating such an index pretty
trivial.

> I'm also not in the habit of opening the *.h file for something, I
> usually just start reading the *.c and only later discover there's some
> API docs in the relevant *.h (or not).

Interesting. I'm not totally opposed to putting the documentation
alongside the actual code. It does feel a bit cluttered to me, but I
think you're right that it keeps everything as close together as
possible.

> It means you can't avoid seeing it or updating it when source
> spelunking, and it leaves header files short, which is useful when you'd
> like to get a general API overview without all the docs. Our documented
> headers are quite fat, e.g. strbuf.h is 60% of the size of strbuf.c, but
> less than 20% when you strip the docs.

I don't use folds in my editor, and I guess nobody else in this thread
does, either. But they may be a reasonable tool for "wow, there are
comments, declarations, and definitions all together and I just want to
view one of them". In vim, try "set foldmethod=syntax" and then "zc" to
close the folds.

I kind of hate it myself, but just another option to explore.

> > I'm mildly negative on this, just because it introduces extra work on
> > people writing the documentation. Now it has to follow special
> > formatting rules, and sometimes the source is uglier (e.g., the horrible
> > +-continuation in lists). Are authors now responsible for formatting any
> > changes they make to make sure they look good in asciidoc? Or are people
> > who care about the formatted output going to come along afterwards and
> > submit fixup patches? Either way it seems like make-work.
> 
> This part I'm slightly confused by. If you're just saying "let's
> document stuff in *.h files in free-flowing text", fine. But if we're
> talking about the difference between Documentation/technical/api-*.txt
> and having the same stuff in manpages, then the stuff in api-*.txt *is*
> already asciidoc, and we have a target to format it all up and ship it
> with git, and distros ship that.

Yes, the "free-flowing text" thing is more or less what I'm saying. I
suspect what's in technical/* currently probably has formatting
problems, and people update it (if we're lucky!) without regard to what
the result looks like.

> E.g. on Debian you can "apt install git-doc" and browse stuff like
> file:///usr/share/doc/git-doc/technical/api-argv-array.html which is the
> HTML formatted version, same for all the other api-*.txt docs.

IMHO this is just silly. What am I as an end user going to do with
api-argv-array.html?

> This E-mail is much longer than I intended, sorry about that. I didn't
> just mean to bitch about how we ship docs, but I was wondering if there
> was a desire to move to something like what I've outlined above, or
> whether the status quo was mostly by design and intended.

I think we're actually half-way between status quos. The stuff in
technical/api-* has existed for a long time. We've been slowly moving
stuff over to header files, but there's still a bit of stuff there.

Building those files as asciidoc is one of those things that just
cropped up along the way. I have never thought it was useful myself, but
somebody bothered to write the patches, so OK. The person who did so is
not even somebody who has otherwise worked on the project, AFAICT.

-Peff