From: Josh Steadmon <steadmon@google.com>
To: git@vger.kernel.org
Subject: Proposed approaches to supporting HTTP remotes in "git archive"
Date: Fri, 27 Jul 2018 14:47:00 -0700 [thread overview]
Message-ID: <CANq=j3tK7QeBJOC7VNWkh4+WBNibMJJp5YUkd9te5NaYwukAow@mail.gmail.com> (raw)
[-- Attachment #1: Type: text/plain, Size: 3672 bytes --]
# Supporting HTTP remotes in "git archive"
We would like to allow remote archiving from HTTP servers. There are a
few possible implementations to be discussed:
## Shallow clone to temporary repo
This approach builds on existing endpoints. Clients will connect to the
remote server's git-upload-pack service and fetch a shallow clone of the
requested commit into a temporary local repo. The write_archive()
function is then called on the local clone to write out the requested
archive.
### Benefits
* This can be implemented entirely in builtin/archive.c. No new service
endpoints or server code are required.
* The archive is generated and compressed on the client side. This
reduces CPU load on the server (for compressed archives) which would
otherwise be a potential DoS vector.
* This provides a git-native way to archive any HTTP servers that
support the git-upload-pack service; some providers (including GitHub)
do not currently allow the git-upload-archive service.
### Drawbacks
* Archives generated remotely may not be bit-for-bit identical compared
to those generated locally, if the versions of git used on the client
and on the server differ.
* This requires higher bandwidth compared to transferring a compressed
archive generated on the server.
## Use git-upload-archive
This approach requires adding support for the git-upload-archive
endpoint to the HTTP backend. Clients will connect to the remote
server's git-upload-archive service and the server will generate the
archive which is then delivered to the client.
### Benefits
* Matches existing "git archive" behavior for other remotes.
* Requires less bandwidth to send a compressed archive than a shallow
clone.
* Resulting archive does not depend in any way on the client
implementation.
### Drawbacks
* Implementation is more complicated; it will require changes to (at
least) builtin/archive.c, http-backend.c, and
builtin/upload-archive.c.
* Generates more CPU load on the server when compressing archives. This
is potentially a DoS vector.
* Does not allow archiving from servers that don't support the
git-upload-archive service.
## Add a new protocol v2 "archive" command
I am still a bit hazy on the exact details of this approach, please
forgive any inaccuracies (I'm a new contributor and haven't examined
custom v2 commands in much detail yet).
This approach builds off the existing v2 upload-pack endpoint. The
client will issue an archive command (with options to select particular
paths or a tree-ish). The server will generate the archive and deliver
it to the client.
### Benefits
* Requires less bandwidth to send a compressed archive than a shallow
clone.
* Resulting archive does not depend in any way on the client
implementation.
### Drawbacks
* Generates more CPU load on the server when compressing archives. This
is potentially a DoS vector.
* Servers must support the v2 protocol (although the client could
potentially fallback to some other supported remote archive
functionality).
### Unknowns
* I am not clear on the relative complexity of this approach compared to
the others, and would appreciate any guidance offered.
## Summary
Personally, I lean towards the first approach. It could give us an
opportunity to remove server-side complexity; there is no reason that
the shallow-clone approach must be restricted to the HTTP transport, and
we could re-implement other transports using this method. Additionally,
it would allow clients to pull archives from remotes that would not
otherwise support it.
That said, I am happy to work on whichever approach the community deems
most worthwhile.
[-- Attachment #2: Type: text/html, Size: 4097 bytes --]
next reply other threads:[~2018-07-27 21:47 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-27 21:47 Josh Steadmon [this message]
2018-07-27 21:56 ` Proposed approaches to supporting HTTP remotes in "git archive" Jonathan Nieder
2018-07-27 22:00 ` Jonathan Nieder
2018-07-27 22:32 ` Junio C Hamano
2018-07-29 11:54 ` René Scharfe
2018-07-28 18:52 ` brian m. carlson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CANq=j3tK7QeBJOC7VNWkh4+WBNibMJJp5YUkd9te5NaYwukAow@mail.gmail.com' \
--to=steadmon@google.com \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).