git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Jiang Xin <worldhello.net@gmail.com>
To: "Git List" <git@vger.kernel.org>,
	"Junio C Hamano" <gitster@pobox.com>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
	"Johannes Schindelin" <johannes.schindelin@gmx.de>
Cc: Jiang Xin <zhiyou.jx@alibaba-inc.com>,
	Jiang Xin <worldhello.net@gmail.com>
Subject: [PATCH v4 0/4] Fix broken CI on newer github-actions runner image
Date: Fri, 25 Nov 2022 17:59:50 +0800	[thread overview]
Message-ID: <20221125095954.4826-1-worldhello.net@gmail.com> (raw)
In-Reply-To: <20221124153934.12470-1-worldhello.net@gmail.com>

From: Jiang Xin <zhiyou.jx@alibaba-inc.com>

GitHub CI runner image "ubuntu-latest" will upgrade to "ubuntu-22.04"
soon. Our CI will break on the new runner image because there is not
"gcc-8" package in ubuntu-22.04. See log of CI#10:

 * https://github.com/jiangxin/git-ci-test/actions/runs/3546701459/jobs/5956020103#step:3:106

Another issue in CI#10 is "pd4 -V” failed without an error output. See:

 * https://github.com/jiangxin/git-ci-test/actions/runs/3546701459/jobs/5956019973#step:3:315

This first issue is fixed in patch 1/4 by downgrading the version of
the runner image for jobs which need "gcc-8". The second issue is
fixed in patch 2/4 by removing the pipe from "p4d -V".  Then, we can
see there was a sigfault when running "p4d -V". See log of CI#11:

 * https://github.com/jiangxin/git-ci-test/actions/runs/3546723251/jobs/5956063536#step:3:311

This issue is fixed in patch 3/4 by upgrading version of p4 on Linux
to the same version as on macOS. But all p4 related test cases failed
becasue python was missing on ubuntu-22.04.  See the log of CI#12:

 * https://github.com/jiangxin/git-ci-test/actions/runs/3546734918/jobs/5956087614#step:4:22733

This issue is fixed in patch 4/4 by installing python2/python3 on
Linux.  All test cases passed. See log of CI#13:

 * https://github.com/jiangxin/git-ci-test/actions/runs/3546744257

# range-diff v3..v4

1:  1c0d639487 ! 1:  28588dad56 github-actions: run gcc-8 on ubuntu-20.04 image
    @@ Commit message
         "ubuntu-20.04" to version "ubuntu-22.04". It will fail to find and
         install "gcc-8" package on the new runner image.
     
    -    Change the runner images from "ubuntu-latest" to "ubuntu-20.04" in order
    -    to run with "gcc-8" as a dependency.
    +    Change some of the runner images from "ubuntu-latest" to "ubuntu-20.04"
    +    in order to install "gcc-8" as a dependency.
     
    -    Instead of use the environment "$runs_on_pool" as below:
    +    The first revision of this patch tried to replace "$runs_on_pool" in
    +    "ci/*.sh" with a new "$runs_on_os" environment variable based on the
    +    "os" filed in the matrix strategy. But these "os" fields in matrix
    +    strategies are obsolete legacies from commit [1] and commit [2], and
    +    are no longer useful. So remove these unused "os" fields.
     
    -        case "$runs_on_pool" in
    -        ubuntu-20.04 | ubuntu-latest)
    -            ;;
    -
    -    we can reuse the os field in the matrix, and use a new environment
    -    "$runs_on_os" as below:
    -
    -        case "$runs_on_os" in
    -        ubuntu)
    -            ;;
    -
    -    In this way, we can change the "ubuntu-latest" runner image to any
    -    version such as "ubuntu-22.04" to test CI behavior in advance.
    +    [1]: c08bb26010 (CI: rename the "Linux32" job to lower-case "linux32",
    +                     2021-11-23)
    +    [2]: 25715419bf (CI: don't run "make test" twice in one job, 2021-11-23)
     
         Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    +    Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
         Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
     
      ## .github/workflows/main.yml ##
     @@ .github/workflows/main.yml: jobs:
    -         vector:
    -           - jobname: linux-clang
    -             cc: clang
    -+            os: ubuntu
                  pool: ubuntu-latest
                - jobname: linux-sha256
                  cc: clang
    -@@ .github/workflows/main.yml: jobs:
    +-            os: ubuntu
    +             pool: ubuntu-latest
                - jobname: linux-gcc
                  cc: gcc
                  cc_package: gcc-8
     -            pool: ubuntu-latest
    -+            os: ubuntu
     +            pool: ubuntu-20.04
                - jobname: linux-TEST-vars
                  cc: gcc
     -            os: ubuntu
                  cc_package: gcc-8
     -            pool: ubuntu-latest
    -+            os: ubuntu
     +            pool: ubuntu-20.04
                - jobname: osx-clang
                  cc: clang
    -+            os: macos
                  pool: macos-latest
    -           - jobname: osx-gcc
    -             cc: gcc
    -             cc_package: gcc-9
    -+            os: macos
    -             pool: macos-latest
    -           - jobname: linux-gcc-default
    -             cc: gcc
    -+            os: ubuntu
    -             pool: ubuntu-latest
    -           - jobname: linux-leaks
    -             cc: gcc
    -+            os: ubuntu
    -             pool: ubuntu-latest
    -           - jobname: linux-asan
    -             cc: gcc
    -+            os: ubuntu
    -             pool: ubuntu-latest
    -           - jobname: linux-ubsan
    -             cc: gcc
    -+            os: ubuntu
    -             pool: ubuntu-latest
    -     env:
    -       CC: ${{matrix.vector.cc}}
    -       CC_PACKAGE: ${{matrix.vector.cc_package}}
    -       jobname: ${{matrix.vector.jobname}}
    --      runs_on_pool: ${{matrix.vector.pool}}
    -+      runs_on_os: ${{matrix.vector.os}}
    -     runs-on: ${{matrix.vector.pool}}
    -     steps:
    -     - uses: actions/checkout@v2
    +@@ .github/workflows/main.yml: jobs:
    +         - jobname: linux-musl
    +           image: alpine
    +         - jobname: linux32
    +-          os: ubuntu32
    +           image: daald/ubuntu32:xenial
    +         - jobname: pedantic
    +           image: fedora
     
      ## ci/install-dependencies.sh ##
     @@ ci/install-dependencies.sh: UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
    -  tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
       libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl"
      
    --case "$runs_on_pool" in
    + case "$runs_on_pool" in
     -ubuntu-latest)
    -+case "$runs_on_os" in
    -+ubuntu)
    ++ubuntu-*)
      	sudo apt-get -q update
      	sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
      		$UBUNTU_COMMON_PKGS $CC_PACKAGE
    -@@ ci/install-dependencies.sh: ubuntu-latest)
    - 		cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs .
    - 	popd
    - 	;;
    --macos-latest)
    -+macos)
    - 	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
    - 	# Uncomment this if you want to run perf tests:
    - 	# brew install gnu-time
     
      ## ci/lib.sh ##
    -@@ ci/lib.sh: export DEFAULT_TEST_TARGET=prove
    - export GIT_TEST_CLONE_2GB=true
    +@@ ci/lib.sh: export GIT_TEST_CLONE_2GB=true
      export SKIP_DASHED_BUILT_INS=YesPlease
      
    --case "$runs_on_pool" in
    + case "$runs_on_pool" in
     -ubuntu-latest)
    -+case "$runs_on_os" in
    -+ubuntu)
    ++ubuntu-*)
      	if test "$jobname" = "linux-gcc-default"
      	then
      		break
    -@@ ci/lib.sh: ubuntu-latest)
    - 	GIT_LFS_PATH="$HOME/custom/git-lfs"
    - 	export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
    - 	;;
    --macos-latest)
    -+macos)
    - 	if [ "$jobname" = osx-gcc ]
    - 	then
    - 		MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"
2:  fed880b8bd ! 2:  b914f1d2fc ci: show error message of "p4 -V"
    @@ Metadata
     Author: Jiang Xin <zhiyou.jx@alibaba-inc.com>
     
      ## Commit message ##
    -    ci: show error message of "p4 -V"
    +    ci: remove the pipe after "p4 -V" to cache errors
     
    -    When installing p4 as a dependency, we used to pipe output of "p4 -V" to
    -    validate the installation, but this would hide potential errors of p4.
    -    E.g.: A broken p4 installation fails to run.
    +    When installing p4 as a dependency, we used to pipe output of "p4 -V"
    +    and "p4d -V" to validate the installation and output a condensed version
    +    information. But this would hide potential errors of p4 and would stop
    +    with an empty output. E.g.: p4d version 16.2 running on ubuntu 22.04
    +    causes sigfaults.
     
    -    Add some instructions to show errors of command "p4 -V", so we can see
    -    why the command output doesn't match.
    +    By removing the pipe after "p4 -V" and "p4d -V", we may get a verbose
    +    output, and stop immediately on errors becuase we have "set -e" in
    +    "ci/lib.sh". Since we won't look at these trace logs unless something
    +    fails, so just including the raw output seems most sensible.
     
         Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de>
    +    Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
         Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
     
      ## ci/install-dependencies.sh ##
    @@ ci/install-dependencies.sh: esac
      then
      	echo "$(tput setaf 6)Perforce Server Version$(tput sgr0)"
     -	p4d -V | grep Rev.
    -+	p4d -V | grep Rev. || { echo >&2 "p4d: bad version"; p4d -V; exit 1; }
    ++	p4d -V
      	echo "$(tput setaf 6)Perforce Client Version$(tput sgr0)"
     -	p4 -V | grep Rev.
    -+	p4 -V | grep Rev. || { echo >&2 "p4: bad version"; p4 -V; exit 1; }
    ++	p4 -V
      else
      	echo >&2 "WARNING: perforce wasn't installed, see above for clues why"
      fi
3:  da2f516fc9 = 3:  4aa2b30c36 ci: p4 on Linux has the same version as on macOS
4:  bd1850cc93 = 4:  35909fd459 ci: install python on ubuntu
      
--

Jiang Xin (4):
  github-actions: run gcc-8 on ubuntu-20.04 image
  ci: remove the pipe after "p4 -V" to cache errors
  ci: p4 on Linux has the same version as on macOS
  ci: install python on ubuntu

 .github/workflows/main.yml |  7 ++-----
 ci/install-dependencies.sh | 12 ++++++------
 ci/lib.sh                  | 11 +++++------
 3 files changed, 13 insertions(+), 17 deletions(-)

-- 
2.39.0.rc0


  reply	other threads:[~2022-11-25 10:01 UTC|newest]

Thread overview: 47+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-11-23 15:02 [PATCH 0/2] Use fixed github-actions runner image Jiang Xin
2022-11-23 15:02 ` [PATCH 1/2] github-actions: run gcc-8 on ubuntu-20.04 image Jiang Xin
2022-11-24  8:11   ` Johannes Schindelin
2022-11-23 15:02 ` [PATCH 2/2] ci: upgrade version of p4 Jiang Xin
2022-11-24  8:16   ` Johannes Schindelin
2022-11-24  8:41     ` Johannes Schindelin
2022-11-24  8:54       ` Johannes Schindelin
2022-11-24  9:17         ` Jiang Xin
2022-11-24  9:41           ` Johannes Schindelin
2022-11-24  9:15     ` Jiang Xin
2022-11-24  8:18 ` [PATCH 0/2] Use fixed github-actions runner image Johannes Schindelin
2022-11-24  9:05 ` [PATCH v2 0/3] Fix broken CI on newer " Jiang Xin
2022-11-24  9:44   ` Johannes Schindelin
2022-11-24 10:48     ` Johannes Schindelin
2022-11-24 11:23       ` Jiang Xin
2022-11-24 12:28       ` python 2 EOL (was: [PATCH v2 0/3] Fix broken CI on newer github-actions runner image) Ævar Arnfjörð Bjarmason
2022-11-25  7:11         ` python 2 EOL Junio C Hamano
2022-11-24 15:39   ` [PATCH v3 0/4] Fix broken CI on newer github-actions runner image Jiang Xin
2022-11-25  9:59     ` Jiang Xin [this message]
2022-11-25  9:59     ` [PATCH v4 1/4] github-actions: run gcc-8 on ubuntu-20.04 image Jiang Xin
2022-11-27  0:24       ` Junio C Hamano
2022-11-25  9:59     ` [PATCH v4 2/4] ci: remove the pipe after "p4 -V" to cache errors Jiang Xin
2022-11-27  0:24       ` Junio C Hamano
2022-11-27  9:14         ` Jiang Xin
2022-11-25  9:59     ` [PATCH v4 3/4] ci: p4 on Linux has the same version as on macOS Jiang Xin
2022-11-27  0:28       ` Junio C Hamano
2022-11-25  9:59     ` [PATCH v4 4/4] ci: install python on ubuntu Jiang Xin
2022-11-27  0:30       ` Junio C Hamano
2022-11-27  9:01         ` Jiang Xin
2022-11-27 23:36           ` Junio C Hamano
2022-11-24 15:39   ` [PATCH v3 1/4] github-actions: run gcc-8 on ubuntu-20.04 image Jiang Xin
2022-11-24 16:29     ` Ævar Arnfjörð Bjarmason
2022-11-24 15:39   ` [PATCH v3 2/4] ci: show error message of "p4 -V" Jiang Xin
2022-11-24 16:10     ` Ævar Arnfjörð Bjarmason
2022-11-25  4:48       ` Junio C Hamano
2022-11-24 15:39   ` [PATCH v3 3/4] ci: p4 on Linux has the same version as on macOS Jiang Xin
2022-11-24 15:39   ` [PATCH v3 4/4] ci: install python on ubuntu Jiang Xin
2022-11-24  9:05 ` [PATCH v2 1/3] github-actions: run gcc-8 on ubuntu-20.04 image Jiang Xin
2022-11-24 10:46   ` Ævar Arnfjörð Bjarmason
2022-11-25  7:21     ` Junio C Hamano
2022-11-24  9:05 ` [PATCH v2 2/3] ci: upgrade version of p4 to 21.2 Jiang Xin
2022-11-24 10:55   ` Ævar Arnfjörð Bjarmason
2022-11-24 12:56     ` Jiang Xin
2022-11-24  9:05 ` [PATCH v2 3/3] ci: install python on ubuntu Jiang Xin
2022-11-24 11:02   ` Ævar Arnfjörð Bjarmason
2022-11-24 11:37     ` Jiang Xin
2022-11-24 12:23       ` Ævar Arnfjörð Bjarmason

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20221125095954.4826-1-worldhello.net@gmail.com \
    --to=worldhello.net@gmail.com \
    --cc=avarab@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=johannes.schindelin@gmx.de \
    --cc=zhiyou.jx@alibaba-inc.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).