git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>,
	"Johannes Schindelin" <Johannes.Schindelin@gmx.de>,
	"Phillip Wood" <phillip.wood123@gmail.com>,
	"Victoria Dye" <vdye@github.com>,
	"Eric Sunshine" <ericsunshine@gmail.com>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH v3 12/12] CI: add a "linux-cmake-test" to run cmake & ctest on linux
Date: Tue,  1 Nov 2022 23:51:36 +0100	[thread overview]
Message-ID: <patch-v3-12.12-c27f620dfa3-20221101T225022Z-avarab@gmail.com> (raw)
In-Reply-To: <cover-v3-00.12-00000000000-20221101T225022Z-avarab@gmail.com>

Since [1] the "cmake" build method should work properly on Linux, but
as seen in preceding commits there were various bugs in it, which are
hopefully now all fixed.

To ensure that it doesn't break again let's add a "linux-cmake-ctest"
target to build and test "cmake" on Linux, in addition to that we'll
also run the tests with "ctest" instead of "make" or "prove", so we
can assert that testing with that method works..

This also stress tests running "cmake" (and "ctest") out of a build
directory that isn't the top-level. The "vs-build" job uses "cmake"
since [2], but clobbers the top-level "Makefile" and builds in the
top-level directory.

That was the reason for why we didn't spot that various tests still
required missing "mergetools/*" etc, which was fixed by using
"$GIT_SOURCE_DIR" in a preceding commit.

Since the "ci/lib.sh" already creates and exports a
GIT_TEST_OPTS="..." we'll pick that up in our CI, see the preceding
commit. Because we pass the "--verbose-log -x
--github-workflow-markup" in the GitHub CI as a result the interaction
with "handle_failed_tests" here works correctly. I.e. on failure we'll
have saved "t/test-results/*.{exit,out,markup}" files relevant to the
failing test(s).

1. f31b6244950 (Merge branch 'yw/cmake-updates', 2022-06-07)
2. 4c2c38e800f (ci: modification of main.yml to use cmake for vs-build
   job, 2020-06-26)

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 .github/workflows/main.yml |  3 +++
 ci/run-build-and-tests.sh  | 13 +++++++++++--
 2 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index bd6f75b8e0f..68c2e134646 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -238,6 +238,9 @@ jobs:
             os: ubuntu
             cc_package: gcc-8
             pool: ubuntu-latest
+          - jobname: linux-cmake-ctest
+            cc: gcc
+            pool: ubuntu-latest
           - jobname: osx-clang
             cc: clang
             pool: macos-latest
diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 8ebff425967..a3ae5ff3972 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -45,10 +45,19 @@ pedantic)
 	;;
 esac
 
-group Build make
+mc=
+if test "$jobname" = "linux-cmake-ctest"
+then
+	cb=contrib/buildsystems
+	group CMake cmake -S "$cb" -B "$cb/out"
+	mc="-C $cb/out"
+fi
+
+group Build make $mc
+
 if test -n "$run_tests"
 then
-	group "Run tests" make test ||
+	group "Run tests" make $mc test ||
 	handle_failed_tests
 fi
 check_unignored_build_artifacts
-- 
2.38.0.1280.g8136eb6fab2


  parent reply	other threads:[~2022-11-01 22:53 UTC|newest]

Thread overview: 113+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-10-21  9:44 [PATCH 0/9] cmake: fix *nix & general issues, no test-lib.sh editing, ctest in CI Ævar Arnfjörð Bjarmason
2022-10-21  9:44 ` [PATCH 1/9] cmake: don't copy chainlint.pl to build directory Ævar Arnfjörð Bjarmason
2022-10-21  9:44 ` [PATCH 2/9] cmake: chmod +x the bin-wrappers/* & SCRIPT_{SH,PERL} & git-p4 Ævar Arnfjörð Bjarmason
2022-10-21 14:18   ` Phillip Wood
2022-10-21 14:43     ` Ævar Arnfjörð Bjarmason
2022-10-21  9:44 ` [PATCH 3/9] cmake & test-lib.sh: add a $GIT_SOURCE_DIR variable Ævar Arnfjörð Bjarmason
2022-10-21  9:44 ` [PATCH 4/9] cmake: set "USE_LIBPCRE2" in "GIT-BUILD-OPTIONS" for test-lib.sh Ævar Arnfjörð Bjarmason
2022-10-21  9:44 ` [PATCH 5/9] test-lib.sh: support a "GIT_TEST_BUILD_DIR" Ævar Arnfjörð Bjarmason
2022-10-21  9:44 ` [PATCH 6/9] cmake: use GIT_TEST_BUILD_DIR instead of editing hack Ævar Arnfjörð Bjarmason
2022-10-21 14:18   ` Phillip Wood
2022-10-21 14:43     ` Ævar Arnfjörð Bjarmason
2022-10-25 13:29       ` Phillip Wood
2022-10-21  9:44 ` [PATCH 7/9] cmake: support using GIT_TEST_OPTS from the environment Ævar Arnfjörð Bjarmason
2022-10-21 14:18   ` Phillip Wood
2022-10-21 15:45     ` Ævar Arnfjörð Bjarmason
2022-10-25 13:38       ` Phillip Wood
2022-10-25 20:18         ` Ævar Arnfjörð Bjarmason
2022-10-21  9:44 ` [PATCH 8/9] cmake: copy over git-p4.py for t983[56] perforce test Ævar Arnfjörð Bjarmason
2022-10-21  9:44 ` [PATCH 9/9] CI: add a "linux-cmake-test" to run cmake & ctest on linux Ævar Arnfjörð Bjarmason
2022-10-21 10:29 ` [PATCH 0/9] cmake: fix *nix & general issues, no test-lib.sh editing, ctest in CI Johannes Schindelin
2022-10-21 14:27 ` Phillip Wood
2022-10-21 15:55   ` Ævar Arnfjörð Bjarmason
2022-10-25 13:22     ` Phillip Wood
2022-10-27  3:26 ` [PATCH v2 00/11] cmake: document, fix on *nix, add CI Ævar Arnfjörð Bjarmason
2022-10-27  3:26   ` [PATCH v2 01/11] cmake: don't "mkdir -p" and "cd" in build instructions Ævar Arnfjörð Bjarmason
2022-10-27  3:26   ` [PATCH v2 02/11] cmake: update instructions for portable CMakeLists.txt Ævar Arnfjörð Bjarmason
2022-10-27  3:35     ` Eric Sunshine
2022-10-27  3:26   ` [PATCH v2 03/11] cmake: don't copy chainlint.pl to build directory Ævar Arnfjörð Bjarmason
2022-10-27  3:26   ` [PATCH v2 04/11] cmake: chmod +x the bin-wrappers/* & SCRIPT_{SH,PERL} & git-p4 Ævar Arnfjörð Bjarmason
2022-10-27  3:26   ` [PATCH v2 05/11] cmake & test-lib.sh: add a $GIT_SOURCE_DIR variable Ævar Arnfjörð Bjarmason
2022-10-27  3:26   ` [PATCH v2 06/11] cmake: set "USE_LIBPCRE2" in "GIT-BUILD-OPTIONS" for test-lib.sh Ævar Arnfjörð Bjarmason
2022-10-27  3:26   ` [PATCH v2 07/11] test-lib.sh: support a "GIT_TEST_BUILD_DIR" Ævar Arnfjörð Bjarmason
2022-10-27  3:26   ` [PATCH v2 08/11] Makefile + cmake: use environment, not GIT-BUILD-DIR Ævar Arnfjörð Bjarmason
2022-10-27  3:26   ` [PATCH v2 09/11] cmake: support GIT_TEST_OPTS, abstract away WIN32 defaults Ævar Arnfjörð Bjarmason
2022-10-27  3:26   ` [PATCH v2 10/11] cmake: copy over git-p4.py for t983[56] perforce test Ævar Arnfjörð Bjarmason
2022-10-27  3:26   ` [PATCH v2 11/11] CI: add a "linux-cmake-test" to run cmake & ctest on linux Ævar Arnfjörð Bjarmason
2022-11-01 22:51   ` [PATCH v3 00/12] cmake: document, fix on *nix, add CI Ævar Arnfjörð Bjarmason
2022-11-01 22:51     ` [PATCH v3 01/12] cmake: don't "mkdir -p" and "cd" in build instructions Ævar Arnfjörð Bjarmason
2022-11-03  9:37       ` Phillip Wood
2022-11-01 22:51     ` [PATCH v3 02/12] cmake: update instructions for portable CMakeLists.txt Ævar Arnfjörð Bjarmason
2022-11-01 22:58       ` Eric Sunshine
2022-11-03  9:42       ` Phillip Wood
2022-11-03 16:00         ` Ævar Arnfjörð Bjarmason
2022-11-01 22:51     ` [PATCH v3 03/12] cmake: don't copy chainlint.pl to build directory Ævar Arnfjörð Bjarmason
2022-11-01 22:51     ` [PATCH v3 04/12] cmake: chmod +x the bin-wrappers/* & SCRIPT_{SH,PERL} & git-p4 Ævar Arnfjörð Bjarmason
2022-11-01 22:51     ` [PATCH v3 05/12] cmake & test-lib.sh: add a $GIT_SOURCE_DIR variable Ævar Arnfjörð Bjarmason
2022-11-01 22:51     ` [PATCH v3 06/12] cmake: set "USE_LIBPCRE2" in "GIT-BUILD-OPTIONS" for test-lib.sh Ævar Arnfjörð Bjarmason
2022-11-01 22:51     ` [PATCH v3 07/12] test-lib.sh: support a "GIT_TEST_BUILD_DIR" Ævar Arnfjörð Bjarmason
2022-11-01 22:51     ` [PATCH v3 08/12] Makefile + cmake: use environment, not GIT-BUILD-DIR Ævar Arnfjörð Bjarmason
2022-11-01 22:51     ` [PATCH v3 09/12] cmake: support GIT_TEST_OPTS, abstract away WIN32 defaults Ævar Arnfjörð Bjarmason
2022-11-03 10:03       ` Phillip Wood
2022-11-03 15:43         ` Ævar Arnfjörð Bjarmason
2022-11-01 22:51     ` [PATCH v3 10/12] cmake: increase test timeout on Windows only Ævar Arnfjörð Bjarmason
2022-11-01 22:51     ` [PATCH v3 11/12] cmake: copy over git-p4.py for t983[56] perforce test Ævar Arnfjörð Bjarmason
2022-11-01 22:51     ` Ævar Arnfjörð Bjarmason [this message]
2022-11-03 16:37     ` [PATCH v4 00/14] cmake: document, fix on *nix, add CI Ævar Arnfjörð Bjarmason
2022-11-03 16:37       ` [PATCH v4 01/14] cmake: don't invoke msgfmt with --statistics Ævar Arnfjörð Bjarmason
2022-11-03 16:37       ` [PATCH v4 02/14] cmake: use "-S" and "-B" to specify source and build directories Ævar Arnfjörð Bjarmason
2022-11-03 16:37       ` [PATCH v4 03/14] cmake: update instructions for portable CMakeLists.txt Ævar Arnfjörð Bjarmason
2022-11-03 16:37       ` [PATCH v4 04/14] cmake: don't copy chainlint.pl to build directory Ævar Arnfjörð Bjarmason
2022-11-03 16:37       ` [PATCH v4 05/14] cmake: chmod +x the bin-wrappers/* & SCRIPT_{SH,PERL} & git-p4 Ævar Arnfjörð Bjarmason
2022-11-03 16:37       ` [PATCH v4 06/14] cmake & test-lib.sh: add a $GIT_SOURCE_DIR variable Ævar Arnfjörð Bjarmason
2022-11-03 16:37       ` [PATCH v4 07/14] cmake: set "USE_LIBPCRE2" in "GIT-BUILD-OPTIONS" for test-lib.sh Ævar Arnfjörð Bjarmason
2022-11-03 16:37       ` [PATCH v4 08/14] test-lib.sh: support a "GIT_TEST_BUILD_DIR" Ævar Arnfjörð Bjarmason
2022-11-03 16:37       ` [PATCH v4 09/14] Makefile + cmake: use environment, not GIT-BUILD-DIR Ævar Arnfjörð Bjarmason
2022-11-03 16:38       ` [PATCH v4 10/14] cmake: support GIT_TEST_OPTS, abstract away WIN32 defaults Ævar Arnfjörð Bjarmason
2022-11-03 16:38       ` [PATCH v4 11/14] cmake: increase test timeout on Windows only Ævar Arnfjörð Bjarmason
2022-11-03 16:38       ` [PATCH v4 12/14] cmake: only look for "sh" in "C:/Program Files" on Windows Ævar Arnfjörð Bjarmason
2022-11-03 16:38       ` [PATCH v4 13/14] cmake: copy over git-p4.py for t983[56] perforce test Ævar Arnfjörð Bjarmason
2022-11-03 16:38       ` [PATCH v4 14/14] CI: add a "linux-cmake-test" to run cmake & ctest on linux Ævar Arnfjörð Bjarmason
2022-11-05  0:27       ` [PATCH v4 00/14] cmake: document, fix on *nix, add CI Taylor Blau
2022-11-08 14:42       ` Phillip Wood
2022-12-02 11:28       ` [PATCH v5 00/15] " Ævar Arnfjörð Bjarmason
2022-12-02 11:28         ` [PATCH v5 01/15] cmake: don't invoke msgfmt with --statistics Ævar Arnfjörð Bjarmason
2022-12-02 11:28         ` [PATCH v5 02/15] cmake: use "-S" and "-B" to specify source and build directories Ævar Arnfjörð Bjarmason
2022-12-03  5:14           ` Eric Sunshine
2022-12-02 11:28         ` [PATCH v5 03/15] cmake: update instructions for portable CMakeLists.txt Ævar Arnfjörð Bjarmason
2022-12-02 11:28         ` [PATCH v5 04/15] cmake: don't copy chainlint.pl to build directory Ævar Arnfjörð Bjarmason
2022-12-02 11:28         ` [PATCH v5 05/15] cmake: chmod +x the bin-wrappers/* & SCRIPT_{SH,PERL} & git-p4 Ævar Arnfjörð Bjarmason
2022-12-02 11:28         ` [PATCH v5 06/15] cmake & test-lib.sh: add a $GIT_SOURCE_DIR variable Ævar Arnfjörð Bjarmason
2022-12-02 11:28         ` [PATCH v5 07/15] cmake: set "USE_LIBPCRE2" in "GIT-BUILD-OPTIONS" for test-lib.sh Ævar Arnfjörð Bjarmason
2022-12-02 11:28         ` [PATCH v5 08/15] Makefile + test-lib.sh: don't prefer cmake-built to make-built git Ævar Arnfjörð Bjarmason
2022-12-02 11:28         ` [PATCH v5 09/15] test-lib.sh: support a "GIT_TEST_BUILD_DIR" Ævar Arnfjörð Bjarmason
2022-12-03  5:22           ` Eric Sunshine
2022-12-02 11:28         ` [PATCH v5 10/15] cmake: optionally be able to run tests before "ctest" Ævar Arnfjörð Bjarmason
2022-12-02 11:28         ` [PATCH v5 11/15] cmake: support GIT_TEST_OPTS, abstract away WIN32 defaults Ævar Arnfjörð Bjarmason
2022-12-03  5:38           ` Eric Sunshine
2022-12-03 13:10             ` Ævar Arnfjörð Bjarmason
2022-12-03 16:58               ` Eric Sunshine
2022-12-02 11:28         ` [PATCH v5 12/15] cmake: increase test timeout on Windows only Ævar Arnfjörð Bjarmason
2022-12-02 11:28         ` [PATCH v5 13/15] cmake: only look for "sh" in "C:/Program Files" on Windows Ævar Arnfjörð Bjarmason
2022-12-02 11:28         ` [PATCH v5 14/15] cmake: copy over git-p4.py for t983[56] perforce test Ævar Arnfjörð Bjarmason
2022-12-02 11:28         ` [PATCH v5 15/15] CI: add a "linux-cmake-test" to run cmake & ctest on linux Ævar Arnfjörð Bjarmason
2022-12-03  5:47           ` Eric Sunshine
2022-12-06  2:08         ` [PATCH v6 00/15] cmake: document, fix on *nix, add CI Ævar Arnfjörð Bjarmason
2022-12-06  2:08           ` [PATCH v6 01/15] cmake: don't invoke msgfmt with --statistics Ævar Arnfjörð Bjarmason
2022-12-06  2:09           ` [PATCH v6 02/15] cmake: use "-S" and "-B" to specify source and build directories Ævar Arnfjörð Bjarmason
2022-12-06  2:09           ` [PATCH v6 03/15] cmake: update instructions for portable CMakeLists.txt Ævar Arnfjörð Bjarmason
2022-12-06  2:09           ` [PATCH v6 04/15] cmake: don't copy chainlint.pl to build directory Ævar Arnfjörð Bjarmason
2022-12-06  2:09           ` [PATCH v6 05/15] cmake: chmod +x the bin-wrappers/* & SCRIPT_{SH,PERL} & git-p4 Ævar Arnfjörð Bjarmason
2022-12-06  2:09           ` [PATCH v6 06/15] cmake & test-lib.sh: add a $GIT_SOURCE_DIR variable Ævar Arnfjörð Bjarmason
2022-12-06  2:09           ` [PATCH v6 07/15] cmake: set "USE_LIBPCRE2" in "GIT-BUILD-OPTIONS" for test-lib.sh Ævar Arnfjörð Bjarmason
2022-12-06  2:09           ` [PATCH v6 08/15] Makefile + test-lib.sh: don't prefer cmake-built to make-built git Ævar Arnfjörð Bjarmason
2022-12-06  2:09           ` [PATCH v6 09/15] test-lib.sh: support a "GIT_TEST_BUILD_DIR" Ævar Arnfjörð Bjarmason
2022-12-06  2:09           ` [PATCH v6 10/15] cmake: optionally be able to run tests before "ctest" Ævar Arnfjörð Bjarmason
2022-12-06  2:09           ` [PATCH v6 11/15] cmake: support GIT_TEST_OPTS, abstract away WIN32 defaults Ævar Arnfjörð Bjarmason
2022-12-06  2:09           ` [PATCH v6 12/15] cmake: increase test timeout on Windows only Ævar Arnfjörð Bjarmason
2022-12-06  2:09           ` [PATCH v6 13/15] cmake: only look for "sh" in "C:/Program Files" on Windows Ævar Arnfjörð Bjarmason
2022-12-06  2:09           ` [PATCH v6 14/15] cmake: copy over git-p4.py for t983[56] perforce test Ævar Arnfjörð Bjarmason
2022-12-06  2:09           ` [PATCH v6 15/15] CI: add a "linux-cmake-test" to run cmake & ctest on linux Ævar Arnfjörð Bjarmason
2022-12-07 15:09           ` [PATCH v6 00/15] cmake: document, fix on *nix, add CI Phillip Wood
2022-12-08  0:06             ` Ævar Arnfjörð Bjarmason
2022-11-03  9:34   ` [PATCH v2 00/11] " Phillip Wood

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=patch-v3-12.12-c27f620dfa3-20221101T225022Z-avarab@gmail.com \
    --to=avarab@gmail.com \
    --cc=Johannes.Schindelin@gmx.de \
    --cc=ericsunshine@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=phillip.wood123@gmail.com \
    --cc=vdye@github.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).