git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: Johannes Schindelin via GitGitGadget <gitgitgadget@gmail.com>
Cc: git@vger.kernel.org, Johannes Schindelin <johannes.schindelin@gmx.de>
Subject: Re: [PATCH 5/9] tests: refactor --write-junit-xml code
Date: Wed, 26 Jan 2022 01:10:01 +0100	[thread overview]
Message-ID: <220126.86o83zfj5m.gmgdl@evledraar.gmail.com> (raw)
In-Reply-To: <94dcbe1bc437dc9738cfc635e9085d743eb4ec98.1643050574.git.gitgitgadget@gmail.com>


On Mon, Jan 24 2022, Johannes Schindelin via GitGitGadget wrote:

> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>
> The code writing JUnit XML is interspersed directly with all the code in
> `t/test-lib.sh`, and it is therefore not only ill-separated, but
> introducing yet another output format would make the situation even
> worse.
>
> Let's introduce an abstraction layer by hiding the JUnit XML code behind
> four new functions that are supposed to be called before and after each
> test and test case.
>
> This is not just an academic exercise, refactoring for refactoring's
> sake. We _actually_ want to introduce such a new output format, to
> make it substantially easier to diagnose test failures in our GitHub
> workflow, therefore we do need this refactoring.

I'm a bit confused about the need to patch this JUnit code & to
generalize test-lib.sh to emit things in three machine-readable formats
(TAP, JUnit, and now this Markdown format).

In
https://lore.kernel.org/git/nycvar.QRO.7.76.6.2112201834050.347@tvgsbejvaqbjf.bet/
you replied to my patch to remove this dead code with wanting to keep
it, because:
    
    The reason is that there are still some things that Azure Pipelines can do
    that GitHub workflows cannot, for example:
    
    - present the logs of failed tests in an intuitive manner,
    
    - re-run _only_ failed jobs.

Which is fair enough, but in this series we're further patching it, but
it's still not used anywhere in-tree at the end of it, or am I missing
something?

This series is seeking to address 1/2 of the points you mentioned, and
presumably the latter is a question of us juggling around our GitHub CI
job definitions.

Then in 6/9 you note:

    This seems not to faze Azure Pipelines' XML parser, but it still is
    incorrect, so let's fix it.

So this is running on Azure somehow, but not via the previously in-tree
azure-pipeline.yml removed in your 6081d3898fe (ci: retire the Azure
Pipelines definition, 2020-04-11)?

  reply	other threads:[~2022-01-26  0:14 UTC|newest]

Thread overview: 98+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-01-24 18:56 [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 1/9] ci: fix code style Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 2/9] ci/run-build-and-tests: take a more high-level view Johannes Schindelin via GitGitGadget
2022-01-24 23:22   ` Eric Sunshine
2022-01-25 14:34     ` Johannes Schindelin
2022-01-24 18:56 ` [PATCH 3/9] ci: make it easier to find failed tests' logs in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-01-25 23:48   ` Ævar Arnfjörð Bjarmason
2022-01-24 18:56 ` [PATCH 4/9] ci/run-build-and-tests: add some structure to the GitHub workflow output Johannes Schindelin via GitGitGadget
2022-02-23 12:13   ` Phillip Wood
2022-02-25 13:40     ` Johannes Schindelin
2022-01-24 18:56 ` [PATCH 5/9] tests: refactor --write-junit-xml code Johannes Schindelin via GitGitGadget
2022-01-26  0:10   ` Ævar Arnfjörð Bjarmason [this message]
2022-01-24 18:56 ` [PATCH 6/9] test(junit): avoid line feeds in XML attributes Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 7/9] ci: optionally mark up output in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 8/9] ci: use `--github-workflow-markup` " Johannes Schindelin via GitGitGadget
2022-01-24 18:56 ` [PATCH 9/9] ci: call `finalize_test_case_output` a little later Johannes Schindelin via GitGitGadget
2022-01-26  0:25 ` [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful Ævar Arnfjörð Bjarmason
2022-01-27 16:31 ` CI "grouping" within jobs v.s. lighter split-out jobs (was: [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful) Ævar Arnfjörð Bjarmason
2022-02-19 23:46 ` [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful Johannes Schindelin
2022-02-20  2:44   ` Junio C Hamano
2022-02-20 15:25     ` Johannes Schindelin
2022-02-21  8:09       ` Ævar Arnfjörð Bjarmason
2022-02-22 10:26         ` Johannes Schindelin
2022-02-20 12:47   ` Ævar Arnfjörð Bjarmason
2022-02-22 10:30     ` Johannes Schindelin
2022-02-22 13:31       ` Ævar Arnfjörð Bjarmason
2022-02-23 12:07         ` Phillip Wood
2022-02-25 12:39           ` Ævar Arnfjörð Bjarmason
2022-02-25 14:10           ` Johannes Schindelin
2022-02-25 18:16             ` Junio C Hamano
2022-02-26 18:43               ` Junio C Hamano
2022-03-01  2:59                 ` Junio C Hamano
2022-03-01  6:35                   ` Junio C Hamano
2022-03-01 10:18                   ` Johannes Schindelin
2022-03-01 16:52                     ` Junio C Hamano
2022-03-01 10:10                 ` Johannes Schindelin
2022-03-01 16:57                   ` Junio C Hamano
2022-03-01 10:20               ` Johannes Schindelin
2022-03-04  7:38               ` win+VS environment has "cut" but not "paste"? Junio C Hamano
2022-03-04  9:04                 ` Ævar Arnfjörð Bjarmason
2022-03-07 15:51                   ` Johannes Schindelin
2022-03-07 17:05                     ` Junio C Hamano
2022-03-09 13:02                       ` Johannes Schindelin
2022-03-10 15:23                         ` Ævar Arnfjörð Bjarmason
2022-03-07 15:48                 ` Johannes Schindelin
2022-03-07 16:58                   ` Junio C Hamano
2022-03-02 10:58             ` [PATCH 0/9] ci: make Git's GitHub workflow output much more helpful Phillip Wood
2022-03-07 16:07               ` Johannes Schindelin
2022-03-07 17:11                 ` Junio C Hamano
2022-03-09 11:44                   ` Ævar Arnfjörð Bjarmason
2022-03-07 17:12                 ` Phillip Wood
2022-03-01 10:24 ` [PATCH v2 " Johannes Schindelin via GitGitGadget
2022-03-01 10:24   ` [PATCH v2 1/9] ci: fix code style Johannes Schindelin via GitGitGadget
2022-03-01 10:24   ` [PATCH v2 2/9] ci/run-build-and-tests: take a more high-level view Johannes Schindelin via GitGitGadget
2022-03-01 10:24   ` [PATCH v2 3/9] ci: make it easier to find failed tests' logs in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-03-01 10:24   ` [PATCH v2 4/9] ci/run-build-and-tests: add some structure to the GitHub workflow output Johannes Schindelin via GitGitGadget
2022-03-01 10:24   ` [PATCH v2 5/9] tests: refactor --write-junit-xml code Johannes Schindelin via GitGitGadget
2022-03-01 10:24   ` [PATCH v2 6/9] test(junit): avoid line feeds in XML attributes Johannes Schindelin via GitGitGadget
2022-03-01 10:24   ` [PATCH v2 7/9] ci: optionally mark up output in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-03-01 10:24   ` [PATCH v2 8/9] ci: use `--github-workflow-markup` " Johannes Schindelin via GitGitGadget
2022-03-01 10:24   ` [PATCH v2 9/9] ci: call `finalize_test_case_output` a little later Johannes Schindelin via GitGitGadget
2022-03-01 19:07   ` [PATCH v2 0/9] ci: make Git's GitHub workflow output much more helpful Junio C Hamano
2022-03-02 12:22   ` Ævar Arnfjörð Bjarmason
2022-03-07 15:57     ` Johannes Schindelin
2022-03-07 16:05       ` Ævar Arnfjörð Bjarmason
2022-03-07 17:36         ` Junio C Hamano
2022-03-09 10:56           ` Ævar Arnfjörð Bjarmason
2022-03-09 13:20         ` Johannes Schindelin
2022-03-09 19:39           ` Junio C Hamano
2022-03-09 19:47           ` Ævar Arnfjörð Bjarmason
2022-03-25  0:48   ` Victoria Dye
2022-03-25  9:02     ` Ævar Arnfjörð Bjarmason
2022-03-25 18:38       ` Victoria Dye
2022-05-21 21:42     ` Johannes Schindelin
2022-05-21 23:05       ` Junio C Hamano
2022-05-22 18:48         ` Johannes Schindelin
2022-05-22 19:10           ` Junio C Hamano
2022-05-23 12:58             ` Johannes Schindelin
2022-05-22 23:27         ` Junio C Hamano
2022-05-23 18:55           ` Junio C Hamano
2022-05-23 19:21             ` Johannes Schindelin
2022-05-23  9:05         ` Ævar Arnfjörð Bjarmason
2022-05-23 18:41           ` Johannes Schindelin
2022-05-24  8:40             ` Ævar Arnfjörð Bjarmason
2022-05-21 22:18   ` [PATCH v3 00/12] " Johannes Schindelin via GitGitGadget
2022-05-21 22:18     ` [PATCH v3 01/12] ci: fix code style Johannes Schindelin via GitGitGadget
2022-05-21 22:18     ` [PATCH v3 02/12] tests: refactor --write-junit-xml code Johannes Schindelin via GitGitGadget
2022-05-21 22:18     ` [PATCH v3 03/12] test(junit): avoid line feeds in XML attributes Johannes Schindelin via GitGitGadget
2022-05-21 22:18     ` [PATCH v3 04/12] ci/run-build-and-tests: take a more high-level view Johannes Schindelin via GitGitGadget
2022-05-21 22:18     ` [PATCH v3 05/12] ci: make it easier to find failed tests' logs in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-05-21 22:18     ` [PATCH v3 06/12] ci/run-build-and-tests: add some structure to the GitHub workflow output Johannes Schindelin via GitGitGadget
2022-05-21 22:18     ` [PATCH v3 07/12] ci: optionally mark up output in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-05-21 22:18     ` [PATCH v3 08/12] ci(github): skip the logs of the successful test cases Johannes Schindelin via GitGitGadget
2022-05-24 10:47       ` Ævar Arnfjörð Bjarmason
2022-05-21 22:18     ` [PATCH v3 09/12] ci(github): avoid printing test case preamble twice Victoria Dye via GitGitGadget
2022-05-21 22:18     ` [PATCH v3 10/12] ci: use `--github-workflow-markup` in the GitHub workflow Johannes Schindelin via GitGitGadget
2022-05-21 22:18     ` [PATCH v3 11/12] ci(github): mention where the full logs can be found Johannes Schindelin via GitGitGadget
2022-05-21 22:18     ` [PATCH v3 12/12] ci: call `finalize_test_case_output` a little later Johannes Schindelin via GitGitGadget

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=220126.86o83zfj5m.gmgdl@evledraar.gmail.com \
    --to=avarab@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitgitgadget@gmail.com \
    --cc=johannes.schindelin@gmx.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).