From: Jonathan Nieder <jrnieder@gmail.com>
To: Sverre Rabbelier <srabbelier@gmail.com>
Cc: Git List <git@vger.kernel.org>,
Daniel Barkalow <barkalow@iabervon.org>,
Ramkumar Ramachandra <artagnon@gmail.com>,
vcs-fast-import-devs@lists.launchpad.net
Subject: Re: [PATCH 01/13] fast-import: add the 'done' command
Date: Sun, 13 Feb 2011 03:42:12 -0600 [thread overview]
Message-ID: <20110213094212.GA25435@elie> (raw)
In-Reply-To: <1283053540-27042-2-git-send-email-srabbelier@gmail.com>
Hi Sverre et al,
Sverre Rabbelier wrote:
> Currently the only way to end an import stream is to close it, which
> is not desirable when the stream that's being used is shared.
Here's a variation on the same theme, with notes indicating
what remains to be fixed. Maybe it can save someone some time.
-- 8< --
From: Sverre Rabbelier <srabbelier@gmail.com>
Date: Sat, 28 Aug 2010 22:45:28 -0500
Subject: fast-import: introduce 'done' command
Add a 'done' command that causes fast-import to stop reading from the
stream and exit.
If the new --done command line flag was passed on the command line
(or a "feature done" declaration included at the start of the stream),
make the 'done' command mandatory. So "git fast-import --done"'s
input format will be prefix-free, making errors easier to detect when
they show up as early termination at some convenient time of the
upstream of a pipe writing to fast-import.
Another possible application of the 'done' command would to be allow a
fast-import stream that is only a small part of a larger encapsulating
stream to be easily parsed, leaving the file offset after the "done\n"
so the other application can pick up from there. This patch does not
teach fast-import to do that --- fast-import still uses buffered input
(stdio).
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Documentation/git-fast-import.txt | 25 ++++++++++++++++++++++
fast-import.c | 14 ++++++++++++
t/t9300-fast-import.sh | 42 +++++++++++++++++++++++++++++++++++++
3 files changed, 81 insertions(+), 0 deletions(-)
diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt
index c3a2766..d0efdf8 100644
--- a/Documentation/git-fast-import.txt
+++ b/Documentation/git-fast-import.txt
@@ -101,6 +101,12 @@ OPTIONS
when the `cat-blob` command is encountered in the stream.
The default behaviour is to write to `stdout`.
+--done::
+ Require a `done` command at the end of the stream.
+ This option might be useful for detecting errors that
+ cause the frontend to terminate before it has started to
+ write a stream.
+
--export-pack-edges=<file>::
After creating a packfile, print a line of data to
<file> listing the filename of the packfile and the last
@@ -329,6 +335,11 @@ and control the current import process. More detailed discussion
standard output. This command is optional and is not needed
to perform an import.
+`done`::
+ Marks the end of the stream. This command is optional
+ unless the `done` feature was requested using the
+ `--done` command line option or `feature done` command.
+
`cat-blob`::
Causes fast-import to print a blob in 'cat-file --batch'
format to the file descriptor set with `--cat-blob-fd` or
@@ -958,6 +969,11 @@ notes::
Versions of fast-import not supporting notes will exit
with a message indicating so.
+done::
+ Error out if the stream ends without a 'done' command.
+ Without this feature, errors causing the frontend to end
+ abruptly at a convenient point in the stream can go
+ undetected.
`option`
~~~~~~~~
@@ -987,6 +1003,15 @@ not be passed as option:
* cat-blob-fd
* force
+`done`
+~~~~~~
+If the `done` feature is not in use, treated as if EOF was read.
+This can be used to tell fast-import to finish early.
+
+If the `--done` command line option or `feature done` command is
+in use, the `done` command is mandatory and marks the end of the
+stream.
+
Crash Reports
-------------
If fast-import is supplied invalid input it will terminate with a
diff --git a/fast-import.c b/fast-import.c
index 3886a1b..cbcf61f 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -365,6 +365,7 @@ static unsigned int cmd_save = 100;
static uintmax_t next_mark;
static struct strbuf new_data = STRBUF_INIT;
static int seen_data_command;
+static int require_explicit_termination;
/* Signal handling */
static volatile sig_atomic_t checkpoint_requested;
@@ -2999,6 +3000,8 @@ static int parse_one_feature(const char *feature, int from_stream)
relative_marks_paths = 1;
} else if (!prefixcmp(feature, "no-relative-marks")) {
relative_marks_paths = 0;
+ } else if (!strcmp(feature, "done")) {
+ require_explicit_termination = 1;
} else if (!prefixcmp(feature, "force")) {
force_update = 1;
} else if (!strcmp(feature, "notes")) {
@@ -3150,6 +3153,8 @@ int main(int argc, const char **argv)
parse_reset_branch();
else if (!strcmp("checkpoint", command_buf.buf))
parse_checkpoint();
+ else if (!strcmp("done", command_buf.buf))
+ break;
else if (!prefixcmp(command_buf.buf, "progress "))
parse_progress();
else if (!prefixcmp(command_buf.buf, "feature "))
@@ -3169,6 +3174,15 @@ int main(int argc, const char **argv)
if (!seen_data_command)
parse_argv();
+ /*
+ * NEEDSWORK: we should report input errors before
+ * errno has a chance to be clobbered.
+ */
+ if (ferror(stdin))
+ die("error reading input");
+ if (require_explicit_termination && feof(stdin))
+ die("stream ends early");
+
end_packfile();
dump_branches();
diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index 52ac0e5..a366ee2 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -2121,6 +2121,48 @@ test_expect_success 'R: quiet option results in no stats being output' '
test_cmp empty output
'
+test_expect_success 'R: feature done means terminating "done" is mandatory' '
+ echo feature done | test_must_fail git fast-import &&
+ test_must_fail git fast-import --done </dev/null
+'
+
+test_expect_success 'R: terminating "done" with trailing gibberish is ok' '
+ git fast-import <<-\EOF &&
+ feature done
+ done
+ trailing gibberish
+ EOF
+ git fast-import <<-\EOF
+ done
+ more trailing gibberish
+ EOF
+'
+
+test_expect_success 'R: terminating "done" within commit' '
+ cat >expect <<-\EOF &&
+ OBJID
+ :000000 100644 OBJID OBJID A hello.c
+ :000000 100644 OBJID OBJID A hello2.c
+ EOF
+ git fast-import <<-EOF &&
+ commit refs/heads/done-ends
+ committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+ data <<EOT
+ Commit terminated by "done" command
+ EOT
+ M 100644 inline hello.c
+ data <<EOT
+ Hello, world.
+ EOT
+ C hello.c hello2.c
+ done
+ EOF
+ git rev-list done-ends |
+ git diff-tree -r --stdin --root --always |
+ sed -e "s/$_x40/OBJID/g" >actual &&
+ test_cmp expect actual
+'
+
cat >input <<EOF
option git non-existing-option
EOF
--
1.7.4.1
next prev parent reply other threads:[~2011-02-13 9:42 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-08-29 3:45 [PATCH 00/13] remote helper improvements Sverre Rabbelier
2010-08-29 3:45 ` [PATCH 01/13] fast-import: add the 'done' command Sverre Rabbelier
2010-08-29 18:59 ` Daniel Barkalow
2010-08-29 20:23 ` Sverre Rabbelier
2010-08-29 21:24 ` Jonathan Nieder
2010-08-29 21:28 ` Sverre Rabbelier
2010-08-29 22:32 ` Jonathan Nieder
2010-08-30 0:30 ` Sverre Rabbelier
2010-08-30 2:02 ` Jonathan Nieder
2010-08-30 2:08 ` Sverre Rabbelier
2010-08-30 2:12 ` Jonathan Nieder
2011-02-13 9:42 ` Jonathan Nieder [this message]
2010-08-29 3:45 ` [PATCH 02/13] fast-export: support done feature Sverre Rabbelier
2010-08-29 19:15 ` Daniel Barkalow
2010-08-29 20:25 ` Sverre Rabbelier
2010-08-29 23:42 ` Tay Ray Chuan
2010-08-30 0:32 ` Sverre Rabbelier
2010-08-29 3:45 ` [PATCH 03/13] transport-helper: factor out push_update_refs_status Sverre Rabbelier
2010-08-29 21:36 ` Jonathan Nieder
2010-08-29 21:45 ` Sverre Rabbelier
2010-08-29 3:45 ` [PATCH 04/13] transport-helper: check status code of finish_command Sverre Rabbelier
2010-08-29 21:52 ` Jonathan Nieder
2010-08-29 3:45 ` [PATCH 05/13] transport-helper: use the new done feature to properly do imports Sverre Rabbelier
2010-08-29 22:02 ` Jonathan Nieder
2010-08-30 0:28 ` Sverre Rabbelier
2010-08-29 3:45 ` [RFC PATCH 06/13] transport-helper: update ref status after push with export Sverre Rabbelier
2010-08-29 22:25 ` Jonathan Nieder
2010-08-30 0:29 ` Sverre Rabbelier
2010-08-29 3:45 ` [PATCH 07/13] transport-helper: change import semantics Sverre Rabbelier
2010-08-29 19:29 ` Daniel Barkalow
2010-08-29 20:26 ` Sverre Rabbelier
2010-08-29 3:45 ` [PATCH 08/13] transport-helper: export should disconnect too Sverre Rabbelier
2010-08-29 19:32 ` Daniel Barkalow
2010-08-29 20:28 ` Sverre Rabbelier
2010-08-29 3:45 ` [PATCH 09/13] transport-helper: Use capname for gitdir capability too Sverre Rabbelier
2010-08-30 1:05 ` Jonathan Nieder
2010-08-29 3:45 ` [PATCH 10/13] transport-helper: implement marks location as capability Sverre Rabbelier
2010-08-29 19:52 ` Daniel Barkalow
2010-08-29 20:17 ` Sverre Rabbelier
2010-08-30 1:31 ` Jonathan Nieder
2010-08-30 1:35 ` Sverre Rabbelier
2010-08-29 3:45 ` [PATCH 11/13] remote-curl: accept empty line as terminator Sverre Rabbelier
2010-08-30 1:39 ` Jonathan Nieder
2010-08-30 2:02 ` Sverre Rabbelier
2010-08-29 3:45 ` [PATCH 12/13] git-remote-testgit: only push for non-local repositories Sverre Rabbelier
2010-08-30 1:48 ` Jonathan Nieder
2010-08-30 1:59 ` Sverre Rabbelier
2010-08-30 2:09 ` Jonathan Nieder
2010-08-29 3:45 ` [PATCH 13/13] git-remote-testgit: fix error handling Sverre Rabbelier
2010-08-30 1:53 ` [PATCH 00/13] remote helper improvements Jonathan Nieder
2010-08-30 2:01 ` Sverre Rabbelier
[not found] ` <1283137728899-5476616.post@n2.nabble.com>
2010-08-30 5:54 ` Sverre Rabbelier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110213094212.GA25435@elie \
--to=jrnieder@gmail.com \
--cc=artagnon@gmail.com \
--cc=barkalow@iabervon.org \
--cc=git@vger.kernel.org \
--cc=srabbelier@gmail.com \
--cc=vcs-fast-import-devs@lists.launchpad.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).