From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>,
"Matthieu Moy" <git@matthieu-moy.fr>,
"Johannes Schindelin" <Johannes.Schindelin@gmx.de>,
"Antoine Beaupré" <anarcat@debian.org>,
"Đoàn Trần Công Danh" <congdanhqx@gmail.com>,
"imon Legner" <Simon.Legner@gmail.com>,
"Eric Sunshine" <sunshine@sunshineco.com>,
"Jeff King" <peff@peff.net>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH v2 00/18] remote-mediawiki: fix RCE issue, and the tests
Date: Mon, 21 Sep 2020 12:39:42 +0200 [thread overview]
Message-ID: <20200921104000.2304-1-avarab@gmail.com> (raw)
In-Reply-To: <20200916102918.29805-1-avarab@gmail.com>
This series now has a fix for a remote code execution which previously
was only being discussed on the closed git-security list. Per
discussion there the issue is being made public.
Basically, we expect that almost nobody is using this code in the
first place so there wasn't any interest in a point release, and there
wasn't any downstream interest in an embargo either.
This v2 addresses (hopefully) all the public & git-security commends
on the v1s of this series.
Simon Legner (1):
remote-mediawiki: fix duplicate revisions being imported
Ævar Arnfjörð Bjarmason (17):
remote-mediawiki doc: correct link to GitHub project
remote-mediawiki doc: link to MediaWiki's current version
remote-mediawiki doc: don't hardcode Debian PHP versions
remote-mediawiki tests: use the login/password variables
remote-mediawiki tests: use a 10 character password
remote-mediawiki tests: use test_cmp in tests
remote-mediawiki tests: change `[]` to `test`
remote-mediawiki tests: use "$dir/" instead of "$dir."
remote-mediawiki tests: use a more idiomatic dispatch table
remote-mediawiki tests: replace deprecated Perl construct
remote-mediawiki tests: use inline PerlIO for readability
remote-mediawiki tests: use CLI installer
remote-mediawiki tests: annotate failing tests
remote-mediawiki: provide a list form of run_git()
remote-mediawiki: convert to quoted run_git() invocation
remote-mediawiki: annotate unquoted uses of run_git()
remote-mediawiki: use "sh" to eliminate unquoted commands
contrib/mw-to-git/git-mw.perl | 2 +-
contrib/mw-to-git/git-remote-mediawiki.perl | 80 +++++----
contrib/mw-to-git/git-remote-mediawiki.txt | 2 +-
contrib/mw-to-git/t/.gitignore | 2 +-
contrib/mw-to-git/t/README | 10 +-
contrib/mw-to-git/t/install-wiki/.gitignore | 1 -
.../t/install-wiki/LocalSettings.php | 129 --------------
.../mw-to-git/t/install-wiki/db_install.php | 120 -------------
contrib/mw-to-git/t/t9360-mw-to-git-clone.sh | 8 +-
.../t/t9363-mw-to-git-export-import.sh | 9 +-
contrib/mw-to-git/t/test-gitmw-lib.sh | 162 +++++++++---------
contrib/mw-to-git/t/test-gitmw.pl | 22 ++-
contrib/mw-to-git/t/test.config | 23 +--
13 files changed, 169 insertions(+), 401 deletions(-)
delete mode 100644 contrib/mw-to-git/t/install-wiki/.gitignore
delete mode 100644 contrib/mw-to-git/t/install-wiki/LocalSettings.php
delete mode 100644 contrib/mw-to-git/t/install-wiki/db_install.php
Range-diff:
1: 846fcf6e6a ! 1: 9279eed8ea remote-mediawiki doc: bump recommended PHP version to 7.3
@@ Metadata
Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
## Commit message ##
- remote-mediawiki doc: bump recommended PHP version to 7.3
+ remote-mediawiki doc: don't hardcode Debian PHP versions
- Change the version in the documentation to what's currently in Debian
- stable. Ideally we wouldn't have to keep changing this version, but if
- it's going to be hardcoded let's use something that works on a modern
- installation.
+ Change the hardcoded version 5 PHP versions to the version-agnostic
+ packages. Currently Debian stable's version is 7.3, and there's a
+ php7.3, php7.3-cli etc. package available (but no php5-*).
+
+ The corresponding version-less package is a dependency package which
+ depends on whatever the current stable version is. By not hardcoding
+ the version these instructions won't be out of date when the next
+ Debian/Ubuntu release happens.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
@@ contrib/mw-to-git/t/README: install the following packages (Debian/Ubuntu names,
-* php5-cli
-* php5-curl
-* php5-sqlite
-+* php7.3
-+* php7.3-cgi
-+* php7.3-cli
-+* php7.3-curl
-+* php7.3-sqlite
++* php
++* php-cgi
++* php-cli
++* php-curl
++* php-sqlite
Principles and Technical Choices
--------------------------------
2: 83910fbfde = 2: 5aca7b2fb4 remote-mediawiki tests: use the login/password variables
3: 6e93ab0e28 ! 3: 66cdbc967e remote-mediawiki tests: use a 10 character password
@@ Metadata
## Commit message ##
remote-mediawiki tests: use a 10 character password
- In more recent versions of MediaWiki this is a requirement, e.g. the current stable version of 1.32.2.
+ In more recent versions of MediaWiki this is a requirement, e.g. the
+ current stable version of 1.32.2.
The web installer now refuses our old 9 character password, the
command-line one (will be used in a subsequent change) will accept it,
4: 8f89eb334c ! 4: 10f7542bc3 remote-mediawiki tests: use test_cmp in tests
@@ Metadata
## Commit message ##
remote-mediawiki tests: use test_cmp in tests
- Change code that used an ad-hoc diff invocation to use our test_cmp
- helper instead. I'm also changing the order of arguments to be the
- standard "test_cmp <expected> <actual>".
+ Change code that used an ad-hoc "diff -b" invocation to use our
+ test_cmp helper instead. I'm also changing the order of arguments to
+ be the standard "test_cmp <expected> <actual>".
+
+ Using test_cmp has different semantics since the "-b" option to diff
+ causes it to ignore whitespace, but in these cases the use of "-b" was
+ just meaningless boilerplate. The desired semantics here are to
+ compare "git log" lines with know-good data, so we don't want to
+ ignore whitespace.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
5: b748cab648 < -: ---------- remote-mediawiki tests: guard test_cmp with test_path_is_file
6: 5fdfdf02bb = 5: 4e2fb4b445 remote-mediawiki tests: change `[]` to `test`
7: 706ca0e23d ! 6: 5a1362d003 remote-mediawiki tests: use "$dir/" instead of "$dir."
@@ Commit message
remote-mediawiki tests: use "$dir/" instead of "$dir."
Change UI messages to use "$dir/" instead of "$dir.". I think this is
- less confusing.
+ less confusing when referring to an absolute directory path.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
@@ contrib/mw-to-git/t/test-gitmw-lib.sh: wiki_install () {
"$MW_FILENAME. "\
"Please fix your connection and launch the script again."
- echo "$MW_FILENAME downloaded in $(pwd). "\
-+ echo "$MW_FILENAME downloaded in $(pwd)/ "\
- "You can delete it later if you want."
+- "You can delete it later if you want."
++ echo "$MW_FILENAME downloaded in $(pwd)/;" \
++ "you can delete it later if you want."
else
- echo "Reusing existing $MW_FILENAME downloaded in $(pwd)."
+ echo "Reusing existing $MW_FILENAME downloaded in $(pwd)/"
8: 34dde50515 = 7: b79b0053ae remote-mediawiki tests: use a more idiomatic dispatch table
9: d45c8f3412 ! 8: 05a9701841 remote-mediawiki tests: replace deprecated Perl construct
@@ Metadata
## Commit message ##
remote-mediawiki tests: replace deprecated Perl construct
- The use of the encoding pragma has been a hard error since Perl 5.18,
- which was released in 2013. What this script really wanted to do was
- to decode @ARGV and write out some files with the UTF-8 PerlIO
- layer. Let's just do that explicitly instead.
+ The use of the encoding pragma has been a hard error since Perl
+ 5.18 (released in 2013).
+
+ What this script really wanted to do was to decode @ARGV and write out
+ some files with the UTF-8 PerlIO layer. Let's just do that explicitly
+ instead.
+
+ This explicitly does not retain the previous UTF-8 semantics of the
+ script. The "encoding" pragma had all sorts of global effects (program
+ text being UTF-8, stdin/stdout etc.). But the only thing that was
+ required was decoding @ARGV and writing out UTF-8 data, which is
+ currently facilitated with the "open" pragma.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
10: 75dbb1f772 = 9: ceecab2bf0 remote-mediawiki tests: use inline PerlIO for readability
11: 6d3b1e9b60 ! 10: cc00c528cb remote-mediawiki tests: use CLI installer
@@ contrib/mw-to-git/t/test-gitmw-lib.sh: wiki_install () {
fi
- # Fetch MediaWiki's archive if not already present in the TMP directory
-+ # Fetch MediaWiki's archive if not already present in download directory
++ # Fetch MediaWiki's archive if not already present in the
++ # download directory
+ mkdir -p "$FILES_FOLDER_DOWNLOAD"
MW_FILENAME="mediawiki-$MW_VERSION_MAJOR.$MW_VERSION_MINOR.tar.gz"
- cd "$TMP"
12: 3c29add4d2 = 11: d7fb81d8a2 remote-mediawiki: fix duplicate revisions being imported
13: 2c3580c8db ! 12: b9b10aed72 remote-mediawiki tests: annotate failing tests
@@ Commit message
intermittent test failures. Let's mark these as failing so we can have
an otherwise passing test suite.
+ We need to add an extra test_path_is_file() here because since
+ d572f52a64 ("test_cmp: diagnose incorrect arguments", 2020-08-09)
+ test_cmp has errored out with a BUG if one of the test arguments
+ doesn't exist, without that the test would still fail even without
+ test_expect_failure().
+
1. https://github.com/Git-Mediawiki/Git-Mediawiki/issues/56
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
@@ contrib/mw-to-git/t/t9363-mw-to-git-export-import.sh: test_expect_success 'git p
test_when_finished "rm -rf mw_dir mw_dir_clone" &&
git clone -c remote.origin.mediaimport=true \
mediawiki::'"$WIKI_URL"' mw_dir_clone &&
+ test_cmp mw_dir_clone/Foo.txt mw_dir/Foo.txt &&
+ (cd mw_dir_clone && git checkout HEAD^) &&
+ (cd mw_dir && git checkout HEAD^) &&
++ test_path_is_file mw_dir_clone/Foo.txt &&
+ test_cmp mw_dir_clone/Foo.txt mw_dir/Foo.txt
+ '
+
-: ---------- > 13: 7bea20a373 remote-mediawiki: provide a list form of run_git()
-: ---------- > 14: 46189e2e58 remote-mediawiki: convert to quoted run_git() invocation
-: ---------- > 15: 2ad06f7334 remote-mediawiki: annotate unquoted uses of run_git()
-: ---------- > 16: 41cfcab3af remote-mediawiki: use "sh" to eliminate unquoted commands
--
2.28.0.297.g1956fa8f8d
next prev parent reply other threads:[~2020-09-21 10:40 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-16 10:29 [PATCH 00/15] remote-mediawiki: various fixes to make tests pass Ævar Arnfjörð Bjarmason
2020-09-16 10:29 ` [PATCH 01/15] remote-mediawiki doc: correct link to GitHub project Ævar Arnfjörð Bjarmason
2020-09-16 10:29 ` [PATCH 02/15] remote-mediawiki doc: link to MediaWiki's current version Ævar Arnfjörð Bjarmason
2020-09-16 10:29 ` [PATCH 03/15] remote-mediawiki doc: bump recommended PHP version to 7.3 Ævar Arnfjörð Bjarmason
2020-09-16 13:47 ` Đoàn Trần Công Danh
2020-09-16 20:41 ` Junio C Hamano
2020-09-16 10:29 ` [PATCH 04/15] remote-mediawiki tests: use the login/password variables Ævar Arnfjörð Bjarmason
2020-09-16 10:29 ` [PATCH 05/15] remote-mediawiki tests: use a 10 character password Ævar Arnfjörð Bjarmason
2020-09-16 10:29 ` [PATCH 06/15] remote-mediawiki tests: use test_cmp in tests Ævar Arnfjörð Bjarmason
2020-09-16 18:38 ` Jeff King
2020-09-16 10:29 ` [PATCH 07/15] remote-mediawiki tests: guard test_cmp with test_path_is_file Ævar Arnfjörð Bjarmason
2020-09-16 14:04 ` Đoàn Trần Công Danh
2020-09-16 16:53 ` Eric Sunshine
2020-09-16 21:13 ` Junio C Hamano
2020-10-03 7:04 ` [PATCH] test_cmp: diagnose incorrect arguments more precisely Eric Sunshine
2020-10-03 17:22 ` Junio C Hamano
2020-09-21 8:54 ` [PATCH 07/15] remote-mediawiki tests: guard test_cmp with test_path_is_file Ævar Arnfjörð Bjarmason
2020-09-21 10:42 ` Ævar Arnfjörð Bjarmason
2020-09-16 18:41 ` Jeff King
2020-09-16 10:29 ` [PATCH 08/15] remote-mediawiki tests: change `[]` to `test` Ævar Arnfjörð Bjarmason
2020-09-16 10:29 ` [PATCH 09/15] remote-mediawiki tests: use "$dir/" instead of "$dir." Ævar Arnfjörð Bjarmason
2020-09-16 18:43 ` Jeff King
2020-09-16 21:15 ` Junio C Hamano
2020-09-16 10:29 ` [PATCH 10/15] remote-mediawiki tests: use a more idiomatic dispatch table Ævar Arnfjörð Bjarmason
2020-09-16 10:29 ` [PATCH 11/15] remote-mediawiki tests: replace deprecated Perl construct Ævar Arnfjörð Bjarmason
2020-09-16 18:49 ` Jeff King
2020-09-16 10:29 ` [PATCH 12/15] remote-mediawiki tests: use inline PerlIO for readability Ævar Arnfjörð Bjarmason
2020-09-16 10:29 ` [PATCH 13/15] remote-mediawiki tests: use CLI installer Ævar Arnfjörð Bjarmason
2020-09-16 10:29 ` [PATCH 14/15] remote-mediawiki: fix duplicate revisions being imported Ævar Arnfjörð Bjarmason
2020-09-16 10:29 ` [PATCH 15/15] remote-mediawiki tests: annotate failing tests Ævar Arnfjörð Bjarmason
2020-09-16 18:57 ` [PATCH 00/15] remote-mediawiki: various fixes to make tests pass Jeff King
2020-09-17 22:28 ` Junio C Hamano
2020-09-16 19:46 ` Johannes Schindelin
2020-09-21 10:15 ` Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` Ævar Arnfjörð Bjarmason [this message]
2020-09-25 6:50 ` [PATCH v2 00/18] remote-mediawiki: fix RCE issue, and the tests Jeff King
2020-09-21 10:39 ` [PATCH v2 01/18] remote-mediawiki doc: correct link to GitHub project Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` [PATCH v2 02/18] remote-mediawiki doc: link to MediaWiki's current version Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` [PATCH v2 03/18] remote-mediawiki doc: don't hardcode Debian PHP versions Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` [PATCH v2 04/18] remote-mediawiki tests: use the login/password variables Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` [PATCH v2 05/18] remote-mediawiki tests: use a 10 character password Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` [PATCH v2 06/18] remote-mediawiki tests: use test_cmp in tests Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` [PATCH v2 07/18] remote-mediawiki tests: change `[]` to `test` Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` [PATCH v2 08/18] remote-mediawiki tests: use "$dir/" instead of "$dir." Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` [PATCH v2 09/18] remote-mediawiki tests: use a more idiomatic dispatch table Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` [PATCH v2 10/18] remote-mediawiki tests: replace deprecated Perl construct Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` [PATCH v2 11/18] remote-mediawiki tests: use inline PerlIO for readability Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` [PATCH v2 12/18] remote-mediawiki tests: use CLI installer Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` [PATCH v2 13/18] remote-mediawiki: fix duplicate revisions being imported Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` [PATCH v2 14/18] remote-mediawiki tests: annotate failing tests Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` [PATCH v2 15/18] remote-mediawiki: provide a list form of run_git() Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` [PATCH v2 16/18] remote-mediawiki: convert to quoted run_git() invocation Ævar Arnfjörð Bjarmason
2020-09-21 10:39 ` [PATCH v2 17/18] remote-mediawiki: annotate unquoted uses of run_git() Ævar Arnfjörð Bjarmason
2020-09-21 10:40 ` [PATCH v2 18/18] remote-mediawiki: use "sh" to eliminate unquoted commands Ævar Arnfjörð Bjarmason
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200921104000.2304-1-avarab@gmail.com \
--to=avarab@gmail.com \
--cc=Johannes.Schindelin@gmx.de \
--cc=Simon.Legner@gmail.com \
--cc=anarcat@debian.org \
--cc=congdanhqx@gmail.com \
--cc=git@matthieu-moy.fr \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=peff@peff.net \
--cc=sunshine@sunshineco.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).