From: David Aguilar <firstname.lastname@example.org>
To: Tom Cook <email@example.com>
Cc: "brian m. carlson" <firstname.lastname@example.org>,
Git Mailing List <email@example.com>
Subject: Re: Bug: All git operations fail when .git contains a non-existent gitdir
Date: Sun, 29 Aug 2021 17:38:21 -0700 [thread overview]
Message-ID: <CAJDDKr7vXRxhwuTN=xFxqQkt_zt_Hpi3bYfc7rotcUAFjau9Sg@mail.gmail.com> (raw)
On Thu, Jul 22, 2021 at 6:16 AM Tom Cook <firstname.lastname@example.org> wrote:
> The easiest way to reproduce it is this:
> $ mkdir test
> $ cd test
> $ echo "gitdir: /foo/bar" > .git
> $ git ls-remote https://github.com/torvalds/linux
> We happen to use overlay mounts in our build system in a way that maps
> a git submodule from one place to another so that its "gitdir" is
> invalid and then attempt a `git ls-remote` from that location which
> unexpectedly fails. But the above reproduces the problem well enough.
I'm making a guess about your setup. To me it sounds like tweaking the
overlayfs or bind mounts to also map the .git directory of the parent
repository (at just the right relative location inside the build
environment) is the right approach towards making the .git-file links
Submodules point to the parent repository's .git/modules/ directory
these days so poking those paths through into the mapped file system
is the solution that's going to lead to the least amount of
hard-to-diagnose issues in the future.
The .git-file uses relative paths so it is possible to remap and
relocate them as you're doing, but it's going to require another mount
in the right relative location to do it.
It seems doable, even for most edge cases. For example if you end up
mapping some/deeply/nested/submod into /shallow inside your
container(?) and the .git-file contains
"../../../../.git/modules/submod", that might seem impossible to
resolve from a "/shallow" directory, but actually it's fine.
The solution in that situation is to map the parent repo's .git/ dir
to the "/.git" filesystem root inside the build environment. Unix
treats "/../../../../../../" as equivalent to "/" so if the .git-file
has a bunch of "../../../" parent-dir traversing going on then it'll
still work from a one-level deep "/shallow" directory. The parent git
repo directory would be expected to be found at "/.git" (with a
"modules" directory inside it) because the parent directory of "/" is
This should mesh right in with the setup you described because you're
already using overlay mounts to synthesize a filesystem.
There are edge cases where this approach breaks down, though. One of
these cases is submodules inside submodules. Luckily it doesn't sound
like you have that problem.
There are patches now that special-case being able to run ls-remote
<url> in a broken setup, but the real fix is to extend the build
environment's filesystem view so that the submodules have access to
the underlying .git/modules/ storage.
Hopefully it's just a --mount or --volume docker(?) command-line flag
that has to be extended to poke the parent .git/ repository contents
into the environment. That's the avenue I would explore if I were
already down a path of using mount tricks.
I hope that helps and maybe sparks an idea that you can run with. I
certainly don't mean it to be a dismissive, "go change your setup"
kind of advice, but it would be interesting to know more details since
it seems like achieving correctness is going to require doing that
based on the limited information in this thread about the specifics of
Also, I wouldn't be surprised if, even after the "git ls-remote"
robustness fixes are applied, you'll just stumble onto the next git
command that can only be resolved by mapping in the parent repo's .git
next prev parent reply other threads:[~2021-08-30 0:39 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-21 9:17 Bug: All git operations fail when .git contains a non-existent gitdir Tom Cook
2021-07-21 22:58 ` brian m. carlson
2021-07-22 13:13 ` Tom Cook
2021-07-22 14:07 ` [PATCH] setup: only die on invalid .git under RUN_SETUP Ævar Arnfjörð Bjarmason
2021-07-22 19:06 ` Junio C Hamano
2021-07-22 21:08 ` Ævar Arnfjörð Bjarmason
2021-07-23 1:59 ` Junio C Hamano
2021-07-23 8:42 ` Tom Cook
2021-07-22 20:50 ` Andrei Rybak
2021-07-23 9:33 ` Ævar Arnfjörð Bjarmason
2021-07-23 15:21 ` Junio C Hamano
2021-07-23 8:23 ` Bug: All git operations fail when .git contains a non-existent gitdir Atharva Raykar
2021-07-23 8:39 ` Tom Cook
2021-07-23 15:47 ` Ævar Arnfjörð Bjarmason
2021-07-23 17:02 ` Atharva Raykar
2021-08-30 0:38 ` David Aguilar [this message]
2021-08-31 14:16 ` Tom Cook
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).