git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "SZEDER Gábor" <szeder.dev@gmail.com>
To: "brian m. carlson" <sandals@crustytoothpaste.net>
Cc: git@vger.kernel.org, "Jeff King" <peff@peff.net>,
	"Martin Ågren" <martin.agren@gmail.com>,
	"Junio C Hamano" <gitster@pobox.com>
Subject: Re: [PATCH v3] Documentation: fix build with Asciidoctor 2
Date: Sun, 15 Sep 2019 11:59:52 +0200	[thread overview]
Message-ID: <20190915095952.GA6190@szeder.dev> (raw)
In-Reply-To: <20190914194919.748935-1-sandals@crustytoothpaste.net>

On Sat, Sep 14, 2019 at 07:49:19PM +0000, brian m. carlson wrote:
> Our documentation toolchain has traditionally been built around DocBook
> 4.5.  This version of DocBook is the last DTD-based version of DocBook.
> In 2009, DocBook 5 was introduced using namespaces and its syntax is
> expressed in RELAX NG, which is more expressive and allows a wider
> variety of syntax forms.
> 
> Asciidoctor, one of the alternatives for building our documentation,
> moved support for DocBook 4.5 out of core in its recent 2.0 release and
> now only supports DocBook 5 in the main release.  The DocBoook 4.5
> converter is still available as a separate component, but this is not
> available in most distro packages.  This would not be a problem but for
> the fact that we use xmlto, which is still stuck in the DocBook 4.5 era.
> 
> xmlto performs DTD validation as part of the build process.  This is not
> problematic for DocBook 4.5, which has a valid DTD, but it clearly
> cannot work for DocBook 5, since no DTD can adequately express its full
> syntax.  In addition, even if xmlto did support RELAX NG validation,
> that wouldn't be sufficient because it uses the libxml2-based xmllint to
> do so, which has known problems with validating interleaves in RELAX NG.
> 
> Fortunately, there's an easy way forward: ask Asciidoctor to use its
> DocBook 5 backend and tell xmlto to skip validation.  Asciidoctor has
> supported DocBook 5 since v0.1.4 in 2013 and xmlto has supported
> skipping validation for probably longer than that.
> 
> We also need to teach xmlto how to use the namespaced DocBook XSLT
> stylesheets instead of the non-namespaced ones it usually uses.
> Normally these stylesheets are interchangeable, but the non-namespaced
> ones have a bug that causes them not to strip whitespace automatically
> from certain elements when namespaces are in use.  This results in
> additional whitespace at the beginning of list elements, which is
> jarring and unsightly.
> 
> We can do this by passing a custom stylesheet with the -x option that
> simply imports the namespaced stylesheets via a URL.  Any system with
> support for XML catalogs will automatically look this URL up and
> reference a local copy instead without us having to know where this
> local copy is located.  We know that anyone using xmlto will already
> have catalogs set up properly since the DocBook 4.5 DTD used during
> validation is also looked up via catalogs.  All major Linux
> distributions distribute the necessary stylesheets and have built-in
> catalog support, and Homebrew does as well, albeit with a requirement to
> set an environment variable to enable catalog support.
> 
> On the off chance that someone lacks support for catalogs, it is
> possible for xmlto (via xmllint) to download the stylesheets from the
> URLs in question, although this will likely perform poorly enough to
> attract attention.  People still have the option of using the prebuilt
> documentation that we ship, so happily this should not be an impediment.
> 
> Finally, we need to filter out some messages from other stylesheets that
> occur when invoking dblatex in the CI job.  This tool strips namespaces
> much like the unnamespaced DocBook stylesheets and prints similar
> messages.  If we permit these messages to be printed to standard error,
> our documentation CI job will fail because we check standard error for
> unexpected output.  Due to dblatex's reliance on Python 2, we may need
> to revisit its use in the future, in which case this problem may go
> away, but this can be delayed until a future patch.
> 
> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>

>  Documentation/Makefile     | 4 +++-
>  Documentation/manpage.xsl  | 3 +++
>  azure-pipelines.yml        | 2 +-
>  ci/install-dependencies.sh | 2 +-
>  ci/test-documentation.sh   | 2 ++
>  5 files changed, 10 insertions(+), 3 deletions(-)
>  create mode 100644 Documentation/manpage.xsl
> 
> diff --git a/Documentation/Makefile b/Documentation/Makefile
> index 76f2ecfc1b..d94f47c5c9 100644
> --- a/Documentation/Makefile
> +++ b/Documentation/Makefile
> @@ -197,11 +197,13 @@ ifdef USE_ASCIIDOCTOR
>  ASCIIDOC = asciidoctor
>  ASCIIDOC_CONF =
>  ASCIIDOC_HTML = xhtml5
> -ASCIIDOC_DOCBOOK = docbook45
> +ASCIIDOC_DOCBOOK = docbook5
>  ASCIIDOC_EXTRA += -acompat-mode -atabsize=8
>  ASCIIDOC_EXTRA += -I. -rasciidoctor-extensions
>  ASCIIDOC_EXTRA += -alitdd='&\#x2d;&\#x2d;'
>  DBLATEX_COMMON =
> +XMLTO_EXTRA += --skip-validation
> +XMLTO_EXTRA += -x manpage.xsl
>  endif
>  
>  SHELL_PATH ?= $(SHELL)
> diff --git a/Documentation/manpage.xsl b/Documentation/manpage.xsl
> new file mode 100644
> index 0000000000..ef64bab17a
> --- /dev/null
> +++ b/Documentation/manpage.xsl
> @@ -0,0 +1,3 @@
> +<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
> +	<xsl:import href="http://docbook.sourceforge.net/release/xsl-ns/current/manpages/docbook.xsl" />
> +</xsl:stylesheet>
> diff --git a/azure-pipelines.yml b/azure-pipelines.yml
> index c329b7218b..34031b182a 100644
> --- a/azure-pipelines.yml
> +++ b/azure-pipelines.yml
> @@ -374,7 +374,7 @@ jobs:
>         test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
>  
>         sudo apt-get update &&
> -       sudo apt-get install -y asciidoc xmlto asciidoctor &&
> +       sudo apt-get install -y asciidoc xmlto asciidoctor docbook-xsl-ns &&
>  
>         export ALREADY_HAVE_ASCIIDOCTOR=yes. &&
>         export jobname=Documentation &&
> diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
> index 8cc72503cb..a76f348484 100755
> --- a/ci/install-dependencies.sh
> +++ b/ci/install-dependencies.sh
> @@ -53,7 +53,7 @@ StaticAnalysis)
>  	;;
>  Documentation)
>  	sudo apt-get -q update
> -	sudo apt-get -q -y install asciidoc xmlto
> +	sudo apt-get -q -y install asciidoc xmlto docbook-xsl-ns

Ok, with this package installed the build passed on Travis CI.

>  	test -n "$ALREADY_HAVE_ASCIIDOCTOR" ||
>  	gem install --version 1.5.8 asciidoctor

So, since the documentation can now be built with Asciidoctor v2, is
it already time to remove this '--version 1.5.8'?

> diff --git a/ci/test-documentation.sh b/ci/test-documentation.sh
> index d49089832d..b3e76ef863 100755
> --- a/ci/test-documentation.sh
> +++ b/ci/test-documentation.sh
> @@ -8,6 +8,8 @@
>  filter_log () {
>  	sed -e '/^GIT_VERSION = /d' \
>  	    -e '/^    \* new asciidoc flags$/d' \
> +	    -e '/stripped namespace before processing/d' \
> +	    -e '/Attributed.*IDs for element/d' \

I haven't seen this latter message in the CI builds, neither with
Asciidoctor v1.5.8 nor with v2.  Do we really need this filter, then?
Where does this message come from?

>  	    "$1"
>  }
>  

  reply	other threads:[~2019-09-15 10:00 UTC|newest]

Thread overview: 71+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-03-17 14:47 [PATCH] asciidoctor-extensions: provide `<refmiscinfo/>` Martin Ågren
2019-03-17 19:44 ` Todd Zullinger
2019-03-17 20:03   ` Martin Ågren
2019-03-19  7:02   ` Martin Ågren
2019-03-20 18:17     ` Todd Zullinger
2019-03-22 21:01       ` Martin Ågren
2019-03-23 19:27         ` Todd Zullinger
2019-03-24 12:16           ` Jeff King
2019-03-24 16:21             ` Todd Zullinger
2019-03-25 15:06               ` Jeff King
2019-03-25 19:00                 ` Todd Zullinger
2019-03-27  1:06                   ` Todd Zullinger
2019-03-27 10:05                     ` SZEDER Gábor
2019-03-28  0:06                     ` brian m. carlson
2019-03-30 18:00                       ` Todd Zullinger
2019-03-30 21:04                         ` brian m. carlson
2019-04-05  2:17                           ` Todd Zullinger
2019-04-05 18:46                             ` Jeff King
2019-03-28  2:54                     ` Jeff King
2019-03-28  3:33                       ` Jeff King
2019-03-19  2:46 ` Jeff King
2019-03-19  2:59   ` Jeff King
2019-03-19  3:55     ` Junio C Hamano
2019-03-19  7:33       ` Martin Ågren
2019-03-19  7:36         ` Martin Ågren
2019-09-03 18:51           ` [PATCH v2 0/2] " Martin Ågren
2019-09-03 18:51             ` [PATCH v2 1/2] " Martin Ågren
2019-09-03 18:51             ` [PATCH v2 2/2] doc-diff: replace --cut-header-footer with --cut-footer Martin Ågren
2019-09-03 23:16             ` [PATCH v2 0/2] asciidoctor-extensions: provide `<refmiscinfo/>` brian m. carlson
2019-09-05 19:28               ` Martin Ågren
2019-09-04  3:26             ` Jeff King
2019-09-05 19:35               ` Martin Ågren
2019-09-07  6:45                 ` Jeff King
2019-09-07 14:06                   ` Martin Ågren
2019-09-08 21:30                     ` Jeff King
2019-09-06 23:29               ` brian m. carlson
2019-09-07  4:40                 ` Jeff King
2019-09-07 16:53                   ` brian m. carlson
2019-09-07 17:07                 ` [PATCH] Documentation: fix build with Asciidoctor 2 brian m. carlson
2019-09-08 10:48                   ` Jeff King
2019-09-08 17:18                     ` brian m. carlson
2019-09-08 21:21                       ` Jeff King
2019-09-08 22:24                         ` brian m. carlson
2019-09-09 17:37                           ` Junio C Hamano
2019-09-10 18:44                           ` Jeff King
2019-09-11 23:19                             ` brian m. carlson
2019-09-08 14:13                   ` SZEDER Gábor
2019-09-08 21:32                     ` Jeff King
2019-09-13  1:52                 ` [PATCH v2] " brian m. carlson
2019-09-13  5:06                   ` Jeff King
2019-09-13 17:06                     ` Junio C Hamano
2019-09-16 10:47                     ` Martin Ågren
2019-09-16 17:43                       ` Junio C Hamano
2019-09-14  7:53                   ` SZEDER Gábor
2019-09-14 19:44                     ` brian m. carlson
2019-09-14 19:49                 ` [PATCH v3] " brian m. carlson
2019-09-15  9:59                   ` SZEDER Gábor [this message]
2019-09-15 21:26                     ` brian m. carlson
2019-09-15 22:05                       ` SZEDER Gábor
2019-09-15 22:14                         ` brian m. carlson
2019-09-16 10:51                       ` Martin Ågren
2019-09-15 22:43                 ` [PATCH v4] " brian m. carlson
2019-09-16 19:00             ` [PATCH v3 0/3] asciidoctor-extensions: provide `<refmiscinfo/>` Martin Ågren
2019-09-16 19:00               ` [PATCH v3 1/3] Doc/Makefile: give mansource/-version/-manual attributes Martin Ågren
2019-09-16 19:00               ` [PATCH v3 2/3] asciidoctor-extensions: provide `<refmiscinfo/>` Martin Ågren
2019-09-16 19:00               ` [PATCH v3 3/3] doc-diff: replace --cut-header-footer with --cut-footer Martin Ågren
2019-03-19  3:30   ` [PATCH] asciidoctor-extensions: provide `<refmiscinfo/>` Jeff King
2019-03-19  7:12     ` Martin Ågren
2019-03-19  7:43       ` Jeff King
2019-03-20 18:32         ` Todd Zullinger
2019-03-19  7:10   ` Martin Ågren

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190915095952.GA6190@szeder.dev \
    --to=szeder.dev@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=martin.agren@gmail.com \
    --cc=peff@peff.net \
    --cc=sandals@crustytoothpaste.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).