git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
From: Yaroslav Halchenko <yoh@onerussian.com>
To: git <git@vger.kernel.org>
Subject: [wishlist] support of cloning recursively from non-bare submodule hierarchies?
Date: Thu, 13 Dec 2018 12:19:17 -0500
Message-ID: <20181213171917.GC4633@hopa.kiewit.dartmouth.edu> (raw)

Example - on http://datasets.datalad.org we have a few hundred datasets
organized into a hierarchy as git submodules.  Each  git submodules carries its
own .git/ directory so they are "self sufficient" and we could readily assess
their sizes, and "cut the tree" at any level without looking for the
supermodule somewhere high up in the tree.

.gitmodules typically has relative paths for the url and path for the
submodules there, the form which I think we chose because it used to work (I
could be utterly wrong! but I think it was done in an informed fashion)
for git clone --recursive:

	$> curl http://datasets.datalad.org/labs/gobbini/famface/.gitmodules
	[submodule "data"]
		path = data
		url = ./data

and possibly outside:

	$> curl http://datasets.datalad.org/labs/gobbini/famface/data/.gitmodules 
	[submodule "scripts/mridefacer"]
		path = scripts/mridefacer
		url = https://github.com/yarikoptic/mridefacer

But unfortunately git doesn't even consider such (valid AFAIK) situation
while cloning where url has to have .git suffix but repository is not bare and
a relative "data" path (or "./data" url) is referring to the worktree.

	$> git clone --recursive http://datasets.datalad.org/labs/gobbini/famface/.git 
	Cloning into 'famface'...
	remote: Counting objects: 61, done.
	remote: Compressing objects: 100% (54/54), done.
	remote: Total 61 (delta 14), reused 0 (delta 0)
	Unpacking objects: 100% (61/61), done.
	Submodule 'data' (http://datasets.datalad.org/labs/gobbini/famface/.git/data) registered for path 'data'
	Cloning into '/tmp/famface/data'...
	fatal: repository 'http://datasets.datalad.org/labs/gobbini/famface/.git/data/' not found
	fatal: clone of 'http://datasets.datalad.org/labs/gobbini/famface/.git/data' into submodule path '/tmp/famface/data' failed
	Failed to clone 'data'. Retry scheduled
	Cloning into '/tmp/famface/data'...
	fatal: repository 'http://datasets.datalad.org/labs/gobbini/famface/.git/data/' not found
	fatal: clone of 'http://datasets.datalad.org/labs/gobbini/famface/.git/data' into submodule path '/tmp/famface/data' failed
	Failed to clone 'data' a second time, aborting

on the server I use the "smart HTTP" git backend, but not sure if that is the one to blame, since
I do not see in the logs any attempt to get the /data from not under .git/:

	10.31.188.88 - - [13/Dec/2018:12:18:38 -0500] "GET /labs/gobbini/famface/.git/info/refs?service=git-upload-pack HTTP/1.1" 200 681 "-" "git/2.20.0.rc2.403.gdbc3b29805"
	10.31.188.88 - - [13/Dec/2018:12:18:38 -0500] "POST /labs/gobbini/famface/.git/git-upload-pack HTTP/1.1" 200 69276 "-" "git/2.20.0.rc2.403.gdbc3b29805"

	==> datasets.datalad.org-error.log <==
	[Thu Dec 13 12:18:38.673447 2018] [core:info] [pid 7570:tid 140683541153536] [client 10.31.188.88:32794] AH00128: File does not exist: /srv/datasets.datalad.org/www/labs/gobbini/famface/.git/data/info/refs

	==> datasets.datalad.org-access-comb.log <==
	10.31.188.88 - - [13/Dec/2018:12:18:38 -0500] "GET /labs/gobbini/famface/.git/data/info/refs?service=git-upload-pack HTTP/1.1" 404 485 "-" "git/2.20.0.rc2.403.gdbc3b29805"

	==> datasets.datalad.org-error.log <==
	[Thu Dec 13 12:18:38.689277 2018] [core:info] [pid 7572:tid 140683574724352] [client 10.31.188.88:32796] AH00128: File does not exist: /srv/datasets.datalad.org/www/labs/gobbini/famface/.git/data/info/refs

	==> datasets.datalad.org-access-comb.log <==
	10.31.188.88 - - [13/Dec/2018:12:18:38 -0500] "GET /labs/gobbini/famface/.git/data/info/refs?service=git-upload-pack HTTP/1.1" 404 485 "-" "git/2.20.0.rc2.403.gdbc3b29805"


-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        

             reply index

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-12-13 17:19 Yaroslav Halchenko [this message]
2018-12-13 21:59 ` Stefan Beller

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181213171917.GC4633@hopa.kiewit.dartmouth.edu \
    --to=yoh@onerussian.com \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox