Re: [PATCH 1/4] docs: add a question on syncing repositories to the FAQ

["brian m. carlson" <sandals@crustytoothpaste.net>]

[-- Attachment #1: Type: text/plain, Size: 4171 bytes --]

On 2021-02-28 at 13:01:04, Ævar Arnfjörð Bjarmason wrote:
> make this more understandable by going to the heart of the matter, i.e.:
> 
>  * We prefer push/pull/bundle to copy/replicate .git content
> 
>  * Regardless, a .git directory can be copied across systems just fine
>    if you recursively guarantee snapshot integrity, e.g. it doesn't
>    depend on the endian-ness of the OS, or has anything like symlinks in
>    there made by git itself.

I'll revise to make this clearer.

>  * Anything which copies .git data on a concurrently updated repo can
>    lead to corruption, whether that's cp -R, rsync with any combination
>    of flags, some cloud syncing service that expects to present that
>    tree to two computers without guaranteeing POSIX fs semantics between
>    the two etc.
> 
>  * A common pitfall with such copying of a .git directory is that file
>    deletions are also critical, e.g. rsync without --delete is almost
>    guaranteed to produce a corrupt .git if repeated enough times
>    (e.g. git might prefer stale loose refs over now-packed ones).

I'll include that as well.

>  * It's OK to copy .git between system that differ in their support of
>    symbolic links, but the work tree may be in an inconsistent state and
>    need some manner of "git reset" to repair it.

And this.

> And, not sure if this is correct:
> 
>  * It may be OK to edit a .git directory on a non-POSIX conforming fs
>    (but perhaps validate the result with "git fsck"). But it's not OK to
>    have two writing git processes work on such a repository at the same
>    time. Keep in mind that certain operations and default settings (such
>    as background gc, see `gc.autoDetach` in linkgit:git-config[1]) might
>    result in two processes working on the directory even if you're
>    changing it only in one terminal window at a time.

I'll reflect the fact that only one process may modify the repository at
a time.

> I.e. to go a bit beyond the docs you have of basically saying "there be
> dragons in non-POSIX" and describe the particular scenarios where it can
> go wrong. Something like the above still leaves the door open to users
> using cloud syncing services, which they can then judge for themselves
> as being OK or not. I'm sure there's some that are far from POSIX
> compliance that are OK in practice if the above warnings are observed.

Unfortunately, it's a bit hard to make a concise entry that explains all
the different ways that using a non-POSIX compliant file system can
break things.  We've seen NFS where open(2) is broken, both with
permissions and O_EXCL; cloud syncing services that restore files that
have been deleted; DAV mounts that don't support the necessary
semantics; systems where O_APPEND doesn't have POSIX semantics; and a
whole host of other sadness.  I don't therefore think that we want to
tell people that we think that using a file system that doesn't support
POSIX semantics is okay, because in most cases, they are not.

Users frequently try to judge for themselves what works and then they
try one of the above things which clearly does not, so saying, "Try it
and see if it breaks," just tends to result in users complaining about
repository corruption later on.  I'm really tired of answering these
same questions again and again and telling users that their repositories
are hosed and that they've lost data, so I want to be definitive that we
don't recommend or support these various broken environments and that
users should not use them.  It may be in rare cases that users have
extensive knowledge about the behavior of their file systems and Git's
requirements and can make a sound judgment to use a non-POSIX file
system, but the people on the planet who can do that effectively are
almost all Git developers.  I will state that "File systems that don't
provide POSIX semantics, such as DAV mounts, shouldn't be used without
fully understanding the situation and requirements," which I think is
the most generous recommendation we can safely give.
-- 
brian m. carlson (he/him or they/them)
Houston, Texas, US

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]