Re: Git for games working group

["Ævar Arnfjörð Bjarmason" <avarab@gmail.com>]

On Mon, Sep 17 2018, Taylor Blau wrote:

> On Sun, Sep 16, 2018 at 04:55:13PM +0200, Ævar Arnfjörð Bjarmason wrote:
>> In the hypothetical git-annex-like case (simplifying a bit for the
>> purposes this explanation), for every FILE in your tree you have a
>> corresponding FILE.lock file, but it's not a boolean, but a log of who's
>> asked for locks, i.e. lines of:
>>
>>     <repository UUID> <ts> <state> <who (email?)> <explanation?>
>>
>> E.g.:
>>
>>     $ cat Makefile.lock
>>     my-random-per-repo-id 2018-09-15 1 avarab@gmail.com "refactoring all Makefiles"
>>     my-random-per-repo-id 2018-09-16 0 avarab@gmail.com "done!"
>>
>> This log is append-only, when clients encounter conflicts there's a
>> merge driver to ensure that all updates are kept.
>
> Certainly. I think that there are two things that aren't well expressed
> under this mechanism:
>
>   1. Having a log of locks held against that (a) file doesn't prevent us
>      from introducing merge conflicts at the <file>.lock level, so we're
>      reliant upon the caller first running 'git pull' and hoping that no
>      one beats them out to locking and pushing their lock.

I was eliding a lot of details about how git-annex works under the
hood.

In reality under git-annex it's not a Makefile.lock file, but there's a
dedicated branch (called "git-annex") that stores this sort of metadata,
i.e. who has copies of the the "Makefile" file. That branch has
dedicated merge drivers for the files it manages, so you never get into
these sorts of conflicts.

But yeah, the ad-hoc example I mentioned of:

    echo We created a lock for this >Makefile.lock

*Would* conflict if two users picked a different string, so in practice
you'd need something standard there, i.e. everyone would just echo
"magic git-annex lock" to the file & track it, so even if they did that
same action in parallel it wouldn't conflict.

There's surely other aspects of that square peg of large file tracking
not fitting the round hole of file locking, the point of my write-up was
not that *that* solution is perfect, but there's prior art here that's
very easily adopted to distributed locking if someone wanted to scratch
that itch, since the notion of keeping a log of who has/hasn't gotten a
file is very similar to a log of who has/hasn't locked some file(s) in
the tree.

>   2. Multi-file locks, e.g., "I need to lock file(s) X, Y, and Z
>      together." This isn't possible in Git LFS today with the existing "git
>      lfs lock" command (I had to check, but it takes only _one_ filename as
>      its argument).
>
>      Perhaps it would be nice to support something like this someday in
>      Git LFS, but I think we would have to reimagine how this would look
>      in your file.lock scheme.

If you can do it for 1 file you can do it for N with a for-loop, no? So
is this just a genreal UI issue in git-annex where some commands don't
take lists of filenames (or git pathspecs) to operate on, or a more
general issue with locking?