From: Christian Couder <christian.couder@gmail.com>
To: git@vger.kernel.org
Cc: Junio C Hamano <gitster@pobox.com>, Jeff King <peff@peff.net>,
Ben Peart <Ben.Peart@microsoft.com>,
Nguyen Thai Ngoc Duy <pclouds@gmail.com>,
Mike Hommey <mh@glandium.org>,
Lars Schneider <larsxschneider@gmail.com>,
Eric Wong <e@80x24.org>,
Christian Couder <chriscool@tuxfamily.org>
Subject: [RFC/PATCH v4 23/49] t0420: add test with HTTP external odb
Date: Tue, 20 Jun 2017 09:54:57 +0200 [thread overview]
Message-ID: <20170620075523.26961-24-chriscool@tuxfamily.org> (raw)
In-Reply-To: <20170620075523.26961-1-chriscool@tuxfamily.org>
This tests that an apache web server can be used as an
external object database and store files in their native
format instead of converting them to a Git object.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
t/t0420-transfer-http-e-odb.sh | 150 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 150 insertions(+)
create mode 100755 t/t0420-transfer-http-e-odb.sh
diff --git a/t/t0420-transfer-http-e-odb.sh b/t/t0420-transfer-http-e-odb.sh
new file mode 100755
index 0000000000..716d722e97
--- /dev/null
+++ b/t/t0420-transfer-http-e-odb.sh
@@ -0,0 +1,150 @@
+#!/bin/sh
+
+test_description='tests for transfering external objects to an HTTPD server'
+
+. ./test-lib.sh
+
+# If we don't specify a port, the current test number will be used
+# which will not work as it is less than 1024, so it can only be used by root.
+LIB_HTTPD_PORT=$(expr ${this_test#t} + 12000)
+
+. "$TEST_DIRECTORY"/lib-httpd.sh
+
+start_httpd apache-e-odb.conf
+
+# odb helper script must see this
+export HTTPD_URL
+
+write_script odb-http-helper <<\EOF
+die() {
+ printf >&2 "%s\n" "$@"
+ exit 1
+}
+echo >&2 "odb-http-helper args:" "$@"
+case "$1" in
+have)
+ list_url="$HTTPD_URL/list/"
+ curl "$list_url" ||
+ die "curl '$list_url' failed"
+ ;;
+get)
+ get_url="$HTTPD_URL/list/?sha1=$2"
+ curl "$get_url" ||
+ die "curl '$get_url' failed"
+ ;;
+put)
+ sha1="$2"
+ size="$3"
+ kind="$4"
+ upload_url="$HTTPD_URL/upload/?sha1=$sha1&size=$size&type=$kind"
+ curl --data-binary @- --include "$upload_url" >out ||
+ die "curl '$upload_url' failed"
+ ref_hash=$(echo "$sha1 $size $kind" | GIT_NO_EXTERNAL_ODB=1 git hash-object -w -t blob --stdin) || exit
+ git update-ref refs/odbs/magic/"$sha1" "$ref_hash"
+ ;;
+*)
+ die "unknown command '$1'"
+ ;;
+esac
+EOF
+HELPER="\"$PWD\"/odb-http-helper"
+
+
+test_expect_success 'setup repo with a root commit and the helper' '
+ test_commit zero &&
+ git config odb.magic.command "$HELPER" &&
+ git config odb.magic.plainObjects "true"
+'
+
+test_expect_success 'setup another repo from the first one' '
+ git init other-repo &&
+ (cd other-repo &&
+ git remote add origin .. &&
+ git pull origin master &&
+ git checkout master &&
+ git log)
+'
+
+UPLOADFILENAME="hello_apache_upload.txt"
+
+UPLOAD_URL="$HTTPD_URL/upload/?sha1=$UPLOADFILENAME&size=123&type=blob"
+
+test_expect_success 'can upload a file' '
+ echo "Hello Apache World!" >hello_to_send.txt &&
+ echo "How are you?" >>hello_to_send.txt &&
+ curl --data-binary @hello_to_send.txt --include "$UPLOAD_URL" >out_upload
+'
+
+LIST_URL="$HTTPD_URL/list/"
+
+test_expect_success 'can list uploaded files' '
+ curl --include "$LIST_URL" >out_list &&
+ grep "$UPLOADFILENAME" out_list
+'
+
+test_expect_success 'can delete uploaded files' '
+ curl --data "delete" --include "$UPLOAD_URL&delete=1" >out_delete &&
+ curl --include "$LIST_URL" >out_list2 &&
+ ! grep "$UPLOADFILENAME" out_list2
+'
+
+FILES_DIR="httpd/www/files"
+
+test_expect_success 'new blobs are transfered to the http server' '
+ test_commit one &&
+ hash1=$(git ls-tree HEAD | grep one.t | cut -f1 | cut -d\ -f3) &&
+ echo "$hash1-4-blob" >expected &&
+ ls "$FILES_DIR" >actual &&
+ test_cmp expected actual
+'
+
+test_expect_success 'blobs can be retrieved from the http server' '
+ git cat-file blob "$hash1" &&
+ git log -p >expected
+'
+
+test_expect_success 'update other repo from the first one' '
+ (cd other-repo &&
+ git fetch origin "refs/odbs/magic/*:refs/odbs/magic/*" &&
+ test_must_fail git cat-file blob "$hash1" &&
+ git config odb.magic.command "$HELPER" &&
+ git config odb.magic.plainObjects "true" &&
+ git cat-file blob "$hash1" &&
+ git pull origin master)
+'
+
+test_expect_success 'local clone from the first repo' '
+ mkdir my-clone &&
+ (cd my-clone &&
+ git clone .. . &&
+ git cat-file blob "$hash1")
+'
+
+test_expect_success 'no-local clone from the first repo fails' '
+ mkdir my-other-clone &&
+ (cd my-other-clone &&
+ test_must_fail git clone --no-local .. .) &&
+ rm -rf my-other-clone
+'
+
+test_expect_success 'no-local clone from the first repo with helper succeeds' '
+ mkdir my-other-clone &&
+ (cd my-other-clone &&
+ git clone -c odb.magic.command="$HELPER" \
+ -c odb.magic.plainObjects="true" \
+ --no-local .. .) &&
+ rm -rf my-other-clone
+'
+
+test_expect_success 'no-local initial-refspec clone succeeds' '
+ mkdir my-other-clone &&
+ (cd my-other-clone &&
+ git config odb.magic.command "$HELPER" &&
+ git config odb.magic.plainObjects "true" &&
+ git -c odb.magic.command="$HELPER" -c odb.magic.plainObjects="true" \
+ clone --no-local --initial-refspec "refs/odbs/magic/*:refs/odbs/magic/*" .. .)
+'
+
+stop_httpd
+
+test_done
--
2.13.1.565.gbfcd7a9048
next prev parent reply other threads:[~2017-06-20 7:57 UTC|newest]
Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-06-20 7:54 [RFC/PATCH v4 00/49] Add initial experimental external ODB support Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 01/49] builtin/clone: get rid of 'value' strbuf Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 02/49] t0021/rot13-filter: refactor packet reading functions Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 03/49] t0021/rot13-filter: improve 'if .. elsif .. else' style Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 04/49] Add Git/Packet.pm from parts of t0021/rot13-filter.pl Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 05/49] t0021/rot13-filter: use Git/Packet.pm Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 06/49] Git/Packet.pm: improve error message Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 07/49] Git/Packet.pm: add packet_initialize() Christian Couder
2017-06-23 18:55 ` Ben Peart
2017-06-20 7:54 ` [RFC/PATCH v4 08/49] Git/Packet: add capability functions Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 09/49] Add initial external odb support Christian Couder
2017-06-23 19:49 ` Ben Peart
2017-06-20 7:54 ` [RFC/PATCH v4 10/49] external odb foreach Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 11/49] t0400: add 'put' command to odb-helper script Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 12/49] external odb: add write support Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 13/49] external-odb: accept only blobs for now Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 14/49] t0400: add test for external odb write support Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 15/49] Add GIT_NO_EXTERNAL_ODB env variable Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 16/49] Add t0410 to test external ODB transfer Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 17/49] lib-httpd: pass config file to start_httpd() Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 18/49] lib-httpd: add upload.sh Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 19/49] lib-httpd: add list.sh Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 20/49] lib-httpd: add apache-e-odb.conf Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 21/49] odb-helper: add 'store_plain_objects' to 'struct odb_helper' Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 22/49] pack-objects: don't pack objects in external odbs Christian Couder
2017-06-20 7:54 ` Christian Couder [this message]
2017-06-20 7:54 ` [RFC/PATCH v4 24/49] odb-helper: start fault in implementation Christian Couder
2017-06-20 7:54 ` [RFC/PATCH v4 25/49] external-odb: add external_odb_fault_in_object() Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 26/49] odb-helper: add script_mode Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 27/49] Documentation: add read-object-protocol.txt Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 28/49] contrib: add long-running-read-object/example.pl Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 29/49] Add t0410 to test read object mechanism Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 30/49] odb-helper: add read_object_process() Christian Couder
2017-07-10 15:57 ` Ben Peart
2017-06-20 7:55 ` [RFC/PATCH v4 31/49] external-odb: add external_odb_get_capabilities() Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 32/49] t04*: add 'get_cap' support to helpers Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 33/49] odb-helper: call odb_helper_lookup() with 'have' capability Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 34/49] odb-helper: fix odb_helper_fetch_object() for read_object Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 35/49] Add t0460 to test passing git objects Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 36/49] odb-helper: add read_packetized_git_object_to_fd() Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 37/49] odb-helper: add read_packetized_plain_object_to_fd() Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 38/49] Add t0470 to test passing plain objects Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 39/49] odb-helper: add write_object_process() Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 40/49] Add t0480 to test "have" capability and plain objects Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 41/49] external-odb: add external_odb_do_fetch_object() Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 42/49] odb-helper: advertise 'have' capability Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 43/49] odb-helper: advertise 'put' capability Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 44/49] odb-helper: add have_object_process() Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 45/49] clone: add initial param to write_remote_refs() Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 46/49] clone: add --initial-refspec option Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 47/49] clone: disable external odb before initial clone Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 48/49] Add test for 'clone --initial-refspec' Christian Couder
2017-06-20 7:55 ` [RFC/PATCH v4 49/49] t: add t0430 to test cloning using bundles Christian Couder
2017-06-20 13:48 ` [RFC/PATCH v4 00/49] Add initial experimental external ODB support Christian Couder
2017-06-23 18:24 ` Ben Peart
2017-07-01 19:41 ` Christian Couder
2017-07-01 20:12 ` Christian Couder
2017-07-01 20:33 ` Junio C Hamano
2017-07-02 4:25 ` Christian Couder
2017-07-03 16:56 ` Junio C Hamano
2017-07-06 17:36 ` Ben Peart
2017-09-15 12:56 ` Christian Couder
2017-07-12 19:06 ` Jonathan Tan
2017-09-15 13:16 ` Christian Couder
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170620075523.26961-24-chriscool@tuxfamily.org \
--to=christian.couder@gmail.com \
--cc=Ben.Peart@microsoft.com \
--cc=chriscool@tuxfamily.org \
--cc=e@80x24.org \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=larsxschneider@gmail.com \
--cc=mh@glandium.org \
--cc=pclouds@gmail.com \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).