git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "SZEDER Gábor" <szeder.dev@gmail.com>
To: Junio C Hamano <gitster@pobox.com>
Cc: "Duy Nguyen" <pclouds@gmail.com>, "Jeff King" <peff@peff.net>,
	"Lars Schneider" <larsxschneider@gmail.com>,
	"Johannes Schindelin" <Johannes.Schindelin@gmx.de>,
	"Thomas Gummerer" <t.gummerer@gmail.com>,
	"Brandon Williams" <bmwill@google.com>,
	git@vger.kernel.org, "SZEDER Gábor" <szeder.dev@gmail.com>
Subject: [PATCH 4/5] travis-ci: don't run the test suite as root in the 32 bit Linux build
Date: Mon, 22 Jan 2018 14:32:19 +0100	[thread overview]
Message-ID: <20180122133220.18587-5-szeder.dev@gmail.com> (raw)
In-Reply-To: <20180122133220.18587-1-szeder.dev@gmail.com>

Travis CI runs the 32 bit Linux build job in a Docker container, where
all commands are executed as root by default.  Therefore, ever since
we added this build job in 88dedd5e7 (Travis: also test on 32-bit
Linux, 2017-03-05), we have a bit of code to create a user in the
container matching the ID of the host user and then to run the test
suite as this user.  Matching the host user ID is important, because
otherwise the host user would have no access to any files written by
processes running in the container, notably the logs of failed tests
couldn't be included in the build job's trace log.

Alas, this piece of code never worked, because it sets the variable
holding the user name ($CI_USER) in a subshell, meaning it doesn't
have any effect by the time we get to the point to actually use the
variable to switch users with 'su'.  So all this time we were running
the test suite as root.

Reorganize that piece of code in 'ci/run-linux32-build.sh' a bit to
avoid that problematic subshell and to ensure that we switch to the
right user.  Furthermore, make the script's optional host user ID
option mandatory, so running the build accidentally as root will
become harder when debugging locally.  If someone really wants to run
the test suite as root, whatever the reasons might be, it'll still be
possible to do so by explicitly passing '0' as host user ID.

Finally, one last catch: since commit 7e72cfcee (travis-ci: save prove
state for the 32 bit Linux build, 2017-12-27) the 'prove' test harness
has been writing its state to the Travis CI cache directory from
within the Docker container while running as root.  After this patch
'prove' will run as a regular user, so in future build jobs it won't
be able overwrite a previously written, still root-owned state file,
resulting in build job failures.  To resolve this we should manually
delete caches containing such root-owned files, but that would be a
hassle.  Instead, work this around by changing the owner of the whole
contents of the cache directory to the host user ID.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
---
 ci/run-linux32-build.sh  | 30 +++++++++++++++++++++++++-----
 ci/run-linux32-docker.sh |  2 +-
 2 files changed, 26 insertions(+), 6 deletions(-)

diff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh
index c9476d6598..e37e1d2d5f 100755
--- a/ci/run-linux32-build.sh
+++ b/ci/run-linux32-build.sh
@@ -3,11 +3,17 @@
 # Build and test Git in a 32-bit environment
 #
 # Usage:
-#   run-linux32-build.sh [host-user-id]
+#   run-linux32-build.sh <host-user-id>
 #
 
 set -ex
 
+if test $# -ne 1 || test -z "$1"
+then
+	echo >&2 "usage: run-linux32-build.sh <host-user-id>"
+	exit 1
+fi
+
 # Update packages to the latest available versions
 linux32 --32bit i386 sh -c '
     apt update >/dev/null &&
@@ -18,11 +24,25 @@ linux32 --32bit i386 sh -c '
 # If this script runs inside a docker container, then all commands are
 # usually executed as root. Consequently, the host user might not be
 # able to access the test output files.
-# If a host user id is given, then create a user "ci" with the host user
-# id to make everything accessible to the host user.
+# If a non 0 host user id is given, then create a user "ci" with that
+# user id to make everything accessible to the host user.
 HOST_UID=$1
-CI_USER=$USER
-test -z $HOST_UID || (CI_USER="ci" && useradd -u $HOST_UID $CI_USER)
+if test $HOST_UID -eq 0
+then
+	# Just in case someone does want to run the test suite as root.
+	CI_USER=root
+else
+	CI_USER=ci
+	useradd -u $HOST_UID $CI_USER
+	# Due to a bug the test suite was run as root in the past, so
+	# a prove state file created back then is only accessible by
+	# root.  Now that bug is fixed, the test suite is run as a
+	# regular user, but the prove state file coming from Travis
+	# CI's cache might still be owned by root.
+	# Make sure that this user has rights to any cached files,
+	# including an existing prove state file.
+	test -n "$cache_dir" && chown -R $HOST_UID:$HOST_UID "$cache_dir"
+fi
 
 # Build and test
 linux32 --32bit i386 su -m -l $CI_USER -c "
diff --git a/ci/run-linux32-docker.sh b/ci/run-linux32-docker.sh
index 15288ea2cf..21637903ce 100755
--- a/ci/run-linux32-docker.sh
+++ b/ci/run-linux32-docker.sh
@@ -9,7 +9,7 @@ docker pull daald/ubuntu32:xenial
 
 # Use the following command to debug the docker build locally:
 # $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/bash daald/ubuntu32:xenial
-# root@container:/# /usr/src/git/ci/run-linux32-build.sh
+# root@container:/# /usr/src/git/ci/run-linux32-build.sh <host-user-id>
 
 container_cache_dir=/tmp/travis-cache
 
-- 
2.16.1.80.gc0eec9753d


  parent reply	other threads:[~2018-01-22 13:32 UTC|newest]

Thread overview: 95+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-12-10 21:21 [PATCH 0/3] fixes for split index mode Thomas Gummerer
2017-12-10 21:22 ` [PATCH 1/3] repository: fix repo_read_index with submodules Thomas Gummerer
2017-12-11 18:54   ` Brandon Williams
2017-12-11 20:37     ` Thomas Gummerer
2017-12-10 21:22 ` [PATCH 2/3] prune: fix pruning with multiple worktrees and split index Thomas Gummerer
2017-12-11 19:09   ` Brandon Williams
2017-12-11 21:39     ` Thomas Gummerer
2017-12-10 21:22 ` [PATCH 3/3] travis: run tests with GIT_TEST_SPLIT_INDEX Thomas Gummerer
2017-12-10 23:37   ` Eric Sunshine
2017-12-11 21:09   ` SZEDER Gábor
2017-12-11 21:42     ` Thomas Gummerer
2017-12-12 15:54       ` Lars Schneider
2017-12-12 19:15         ` Junio C Hamano
2017-12-12 20:15           ` Thomas Gummerer
2017-12-12 20:51             ` Junio C Hamano
2017-12-13 23:21               ` Thomas Gummerer
2017-12-13 17:21           ` Lars Schneider
2017-12-13 17:38             ` Junio C Hamano
2017-12-13 17:46               ` Lars Schneider
2017-12-13 23:28                 ` Thomas Gummerer
2017-12-17 22:51 ` [PATCH v2 0/3] fixes for split index mode Thomas Gummerer
2017-12-17 22:51   ` [PATCH v2 1/3] repository: fix repo_read_index with submodules Thomas Gummerer
2017-12-18 18:01     ` Brandon Williams
2017-12-18 23:05       ` Thomas Gummerer
2017-12-18 23:05         ` Brandon Williams
2017-12-17 22:51   ` [PATCH v2 2/3] prune: fix pruning with multiple worktrees and split index Thomas Gummerer
2017-12-18 18:19     ` Brandon Williams
2018-01-03 22:18       ` Thomas Gummerer
2018-01-04 19:12         ` Junio C Hamano
2017-12-17 22:51   ` [PATCH v2 3/3] travis: run tests with GIT_TEST_SPLIT_INDEX Thomas Gummerer
2017-12-18 18:16     ` Lars Schneider
2018-01-04 20:13       ` Thomas Gummerer
2018-01-05 11:03         ` Lars Schneider
2018-01-07 20:02         ` Thomas Gummerer
2018-01-07 22:30   ` [PATCH v3 0/3] fixes for split index mode Thomas Gummerer
2018-01-07 22:30     ` [PATCH v3 1/3] read-cache: fix reading the shared index for other repos Thomas Gummerer
2018-01-08 10:41       ` Duy Nguyen
2018-01-08 22:41         ` Thomas Gummerer
2018-01-13 22:33           ` Thomas Gummerer
2018-01-08 23:38         ` Brandon Williams
2018-01-09  1:24           ` Duy Nguyen
2018-01-16 21:42       ` Brandon Williams
2018-01-17  0:16         ` Duy Nguyen
2018-01-17  0:32           ` Brandon Williams
2018-01-17 18:16           ` Jonathan Nieder
2018-01-18 10:19             ` Duy Nguyen
2018-01-19 21:57       ` Junio C Hamano
2018-01-20 11:58         ` Thomas Gummerer
2018-01-22  6:14           ` Junio C Hamano
2018-01-27 12:18             ` Thomas Gummerer
2018-02-07 22:41             ` Junio C Hamano
2018-01-07 22:30     ` [PATCH v3 2/3] split-index: don't write cache tree with null oid entries Thomas Gummerer
2018-01-07 22:30     ` [PATCH v3 3/3] travis: run tests with GIT_TEST_SPLIT_INDEX Thomas Gummerer
2018-01-13 22:37     ` [PATCH v3 4/3] read-cache: don't try to write index if we can't write shared index Thomas Gummerer
2018-01-14  9:36       ` Duy Nguyen
2018-01-14 10:18         ` [PATCH 1/3] read-cache.c: change type of "temp" in write_shared_index() Nguyễn Thái Ngọc Duy
2018-01-14 10:18           ` [PATCH 2/3] read-cache.c: move tempfile creation/cleanup out of write_shared_index Nguyễn Thái Ngọc Duy
2018-01-14 10:18           ` [PATCH 3/3] read-cache: don't write index twice if we can't write shared index Nguyễn Thái Ngọc Duy
2018-01-18 11:36             ` SZEDER Gábor
2018-01-18 12:47               ` Duy Nguyen
2018-01-18 13:29                 ` Jeff King
2018-01-18 13:36                   ` Duy Nguyen
2018-01-18 15:00                     ` Duy Nguyen
2018-01-18 21:37                       ` Jeff King
2018-01-18 22:32                         ` SZEDER Gábor
2018-01-19  0:30                           ` Duy Nguyen
2018-01-22 13:32                           ` [PATCH 0/5] Travis CI: don't run the test suite as root in the 32 bit Linux build SZEDER Gábor
2018-01-22 13:32                             ` [PATCH 1/5] travis-ci: use 'set -x' for the commands under 'su' " SZEDER Gábor
2018-01-22 13:32                             ` [PATCH 2/5] travis-ci: use 'set -e' in the 32 bit Linux build job SZEDER Gábor
2018-01-23 16:26                               ` Jeff King
2018-01-23 16:32                                 ` Jeff King
2018-01-24 12:12                                 ` SZEDER Gábor
2018-01-24 15:49                                   ` Jeff King
2018-01-22 13:32                             ` [PATCH 3/5] travis-ci: don't repeat the path of the cache directory SZEDER Gábor
2018-01-23 16:30                               ` Jeff King
2018-01-24 13:14                                 ` SZEDER Gábor
2018-01-22 13:32                             ` SZEDER Gábor [this message]
2018-01-23 16:43                               ` [PATCH 4/5] travis-ci: don't run the test suite as root in the 32 bit Linux build Jeff King
2018-01-24 13:45                                 ` SZEDER Gábor
2018-01-24 15:56                                   ` Jeff King
2018-01-24 18:01                                     ` Jeff King
2018-01-24 19:51                                       ` Jeff King
2018-01-22 13:32                             ` [PATCH 5/5] travis-ci: don't fail if user already exists on 32 bit Linux build job SZEDER Gábor
2018-01-23 16:46                               ` Jeff King
2018-01-24  0:32                                 ` Duy Nguyen
2018-01-24 19:39                                 ` SZEDER Gábor
2018-01-22 18:27                 ` [PATCH 3/3] read-cache: don't write index twice if we can't write shared index SZEDER Gábor
2018-01-22 19:46                   ` Eric Sunshine
2018-01-22 22:10                     ` SZEDER Gábor
2018-01-24  9:11                   ` Duy Nguyen
2018-01-26 22:44                   ` Lars Schneider
2018-01-14 14:29         ` [PATCH v3 4/3] read-cache: don't try to write index " Thomas Gummerer
2018-01-18 21:53     ` [PATCH v3 0/3] fixes for split index mode Thomas Gummerer
2018-01-19 18:34       ` Junio C Hamano
2018-01-19 21:11         ` Thomas Gummerer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180122133220.18587-5-szeder.dev@gmail.com \
    --to=szeder.dev@gmail.com \
    --cc=Johannes.Schindelin@gmx.de \
    --cc=bmwill@google.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=larsxschneider@gmail.com \
    --cc=pclouds@gmail.com \
    --cc=peff@peff.net \
    --cc=t.gummerer@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).