Re: Git for games working group

[David Aguilar <davvid@gmail.com>]

On Fri, Sep 14, 2018 at 02:13:28PM -0700, John Austin wrote:
> Hey Taylor,
> 
> Great to have your support! I think LFS has done a great job so far
> solving the large file issue. I've been working myself on strategies
> for handling binary conflicts, and particularly how to do it in a
> git-friendly way (ie. avoiding as much centralization as possible and
> playing into the commit/branching model of git). I've got to a loose
> design that I like, but it'd be good to get some feedback, as well as
> hearing what other game devs would want in a binary conflict system.
> 
> - John

Hey John, thanks for LFS, and thanks to Taylor for bringing up this topic.

Regarding file locking, the gitolite docs are insightful:
http://gitolite.com/gitolite/locking/index.html

File locking is how P4 handles binary conflicts.  It's actually
conflict prevention -- the locks prevent users from stepping
on each other without needing to actually talk to each other.

(I've always believed that this is actually a social problem
 (not a technical one) that is best served by better communication,
 but there's no doubt that having a technical guard in place is useful
 in many scenarios.)

From the POV of using Git as a P4 replacement, the locking support in
git-lfs seems like a fine solution to prevent binary conflicts.

https://github.com/git-lfs/git-lfs/wiki/File-Locking

Are there any missing features that would help improve LFS solution?


Locking is just one aspect of binary conflicts.

In a lock-free world, another aspect is tooling around dealing
with actual conflicts.  It seems like the main challenges there are
related to introspection of changes and mechanisms for combining
changes.

Combining changes is inherently file-format specific, and I suspect
that native authoring tools are best used in those scenarios.
Maybe LFS can help deal with binary conflicts by having short and sweet
ways to grab the "base", "their" and "our" versions of the conflict
files.

Example:

	git lfs checkout --theirs --to theirs.wav conflict.wav
	git lfs checkout --ours --to ours.wav conflict.wav
	git lfs checkout --base --to base.wav conflict.wav

Then the user can use {ours,theirs,base}.wav to produce the
resolved result using their usual authoring tools.

From the plumbing perspective, we already have the tools to
do this today, but they're not really user-friendly because
they require the user to use "git cat-file --filters --path=..."
and redirect the output to get at their changes.

Not sure if git-lfs is the right place for that kind of helper
wrapper command, but it's not a bad place for it either.
That said, none of these are user-friendly for non-Gits that
might be intimidated by a command-line.

Is there anything we could add to git-cola to help?

Being able to save the different conflicted index stages to
separately named files seems like an obvious feature that
would help users when confronted with a binary conflict.

With LFS and the ongoing work related to MVFS, shallow clone,
and partial checkout, the reasons to use P4 over Git are becoming
less and less compelling.  It'd be great to polish the game asset
workflows further so that we can have a cohesive approach to
doing game asset development using Git that is easy enough for
non-technical users to use and understand.

I mention git-cola because it's a Git porcelain that already has
git-lfs support and I'm very much in favor of improving workflows
related to interacting with LFS, large files, repos, and binary content.

Are there other rough edges around (large) binary files that can be improved?

One thought that comes to mind is diffing -- I imagine that we
might want to use different diff tools depending on the file format.
Currently git-difftool uses a single tool for all files, but it seems
like being able to use different tools, based on the file type, could
be helpful.  Not sure if difftool is the right place for that, but
being able to specify different tools per-file seems be useful in
that scenario.

Another avenue that could use help is documentation about suggested
workflows.  Git's core documentation talks about various
large-file-centric features in isolation, but it'd be good to have a
single user-centric document (not unlike gitworkflows) to document best
practices for dealing with large files, repos, game assets, etc.

That alone would help dispel the myth that Git is unsuitable for
large repos, large files, and binary content.
-- 
David