From: Michael J Gruber <git@drmicha.warpmail.net>
To: Yann Droneaud <yann@droneaud.fr>
Cc: git@vger.kernel.org
Subject: Re: inode problem when using git on a sshfs filesystem
Date: Thu, 17 Feb 2011 09:37:44 +0100 [thread overview]
Message-ID: <4D5CDE58.5000905@drmicha.warpmail.net> (raw)
In-Reply-To: <1297893854.4097.43.camel@dworkin.quest-ce.net>
Yann Droneaud venit, vidit, dixit 16.02.2011 23:04:
> Hi,
>
> For some days, my usage of git is not as seamless as before.
>
> I'm using git along sshfs/fuse (don't blame me for that), and
> each time I try to rebase one of my branch, I have a conflict when applying
> the third commit. Doing the same operation on a local filesystem works without any problem.
>
> ===== Part one: git =====
>
> When I try to rebase one specific branch, git rebase failed when applying the third commit,
> telling me about uncommited
>
> I've tried to do it from scratch, using git format-patch / git am
> but git am also abort on the third patch with the error message:
>
> error: <path>/<filename>: does not match index
>
> So I've tried to diagnose the problem using :
>
> - git diff / git status : doesn't return anything.
>
> - git ls-tree HEAD -l <path>/<filename> : returns the correct mode and file size.
>
> - git log --raw --all --full-history -- <path>/<filename> :
> returns valid information matching the one retrieved above.
>
> - git hash-object <path>/<filename> :
> gives the correct sha1 for the file, as recorded in the patch extracted using git format-patch
> and reported by git ls-tree or git log (see before)
>
> - git diff-files : it shows a lot a file, all of them in same directory
>
> :100644 100644 <sha1> 0000000000000000000000000000000000000000 M <path>/<filename0>
> :100644 100644 <sha1> 0000000000000000000000000000000000000000 M <path>/<filename1>
> :100644 100644 <sha1> 0000000000000000000000000000000000000000 M <path>/<filename2>
> :100644 100644 <sha1> 0000000000000000000000000000000000000000 M <path>/<filename3>
> :100644 100644 <sha1> 0000000000000000000000000000000000000000 M <path>/<filename>
>
> BTW, there's no conflict when applying the patch manually with patch: the patch itself is fine
> Using git apply --index also work, but only if it's applied alone:
> apply each patches in series and git apply fails in the same third patch.
>
> After diving into git source code and some debugging session, I've found
> that the inode number recorded in the active_cache doesn't match the one
> on the filesystem for <pach>/<filename> : that's why git apply --index refuse to apply the patch.
>
> Then I tried to monitor stat() information for the file in <path> during
> git operations.
> 1) After applying the first patch, files in <path> were affected different inode number
> 2) Using strace, I checked that git apply didn't make anything specials to thoses files.
> The only thing strange git did, was trying to unlink(<path>), but this failed since the <path>
> directory wasn't empty.
>
> Note: the first patch remove, change and add some files in <path> directory, while
> the third patch changes another file in <path> directory
>
> As a workaround: running git diff / git diff --cached / git status between each
> git apply --index command seems to update the cache and allows me to apply all the patches
> without problem. But it's not an easy path to follow when rebasing branches.
>
> Surprisingly, when looking at strace output, it seems that git apply, once work done, is calling lstat()
> for all the files under <path>, and it sees the new inodes allocated to those files, but I don't know what
> it is doing with those information, if it's not stored in the index.
>
> To conclude, it was a bit hard to diagnose from git point of view.
>
> ====== Part two: sshfs / fuse ======
>
> At this time sshfs seems to be guilty of bad behavior, breaking somes POSIX rules.
>
> So I tried the following testcase on another computer to reproduce the
> problem outside of git.
>
> Here the results:
>
> $ mkdir dir
> $ touch dir/a dir/b
> $ stat -t dir/*
> dir/a 0 0 81b4 500 500 15 3 1 0 0 1297882724 1297882724 1297882724 4096
> dir/b 0 0 81b4 500 500 15 4 1 0 0 1297882726 1297882726 1297882726 4096
> $ rmdir dir
> rmdir: failed to remove `dir1': Operation not permitted
> $ stat -t dir/*
> dir/a 0 0 81b4 500 500 15 6 1 0 0 1297882724 1297882724 1297882724 4096
> dir/b 0 0 81b4 500 500 15 7 1 0 0 1297882726 1297882726 1297882726 4096
>
> One can see that inode 3 became inode 6 and inode 4 became inode 7 after the failed
> unlink operation on dir. Which seems to be a bit uncommon for me.
>
> Note: on a local filesystem, rmdir failed with message rmdir: failed to remove `dir1': Directory not empty
>
> I try to add some debug support to fuse / sshfs in order to locate more precisely the problem:
> (lines beginning by -/+ where added by me in libfuse, line beginning with --/++ in sshfs)
>
> $ sshfs localhost:<export> <mount> -o sshfs_debug,debug,cache=no -d -f -s
>
> unique: 22, opcode: FORGET (2), nodeid: 4, insize: 48, pid: 0
> - forget 4
> FORGET 4/1
> DELETE: 4
> + forget 4
> unique: 23, opcode: FORGET (2), nodeid: 3, insize: 48, pid: 0
> - forget 3
> FORGET 3/1
> DELETE: 3
> + forget 3
> unique: 24, opcode: RMDIR (11), nodeid: 1, insize: 44, pid: 9044
> - rmdir 1 dir
> rmdir /dir
> -- rmdir(/dir)
> [00020] RMDIR
> [00020] STATUS 28bytes (0ms)
> ++ rmdir(/dir) = -1
> unique: 24, error: -1 (Operation not permitted), outsize: 16
> + rmdir 1 dir
> unique: 25, opcode: FORGET (2), nodeid: 2, insize: 48, pid: 0
> - forget 2
> FORGET 2/1
> DELETE: 2
> + forget 2
>
> One can see that the reference to files under the directory are asked by
> the kernel to be forgotten, even if the directory is not yet removed.
>
> This seems a bit illogical since a directory with files under it can't
> be removed (but FORGET could apply to file deleted but still referenced
> by a process).
>
> Note: if the file is opened, the inode associated to the file name
> didn't change. Hopefully.
>
> I've tried to reproduce the problem with other virtual filesystem like
> shm / tmpfs / devtmpfs / ramfs : no problem.
>
> I've also tried with NFS (local), and there's no problem too (the inode
> numbers reported from NFS client side are the same than the server
> side).
>
> So it seems this a FUSE only problem, and I haven't found exactly why.
>
> Regards.
>
fuse-sshfs is a bit of a pita whenever you try to treat it like a "real"
file-system... OTOH, git makes assumptions on stability of device ids,
e.g. On such a (non)-fs, I'd suggest to try in this order of preference:
- Work from a local clone :) Pushing over ssh is fine.
- Have your workdir on a local file system, gitdir on sshfs.
- Try whether core.fsyncobjectfiles helps (shot in the dark).
- Try whether core.ignoreStat works for you. (This impacts your workflow!)
Michael
next prev parent reply other threads:[~2011-02-17 8:41 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-02-16 22:04 inode problem when using git on a sshfs filesystem Yann Droneaud
2011-02-17 8:37 ` Michael J Gruber [this message]
[not found] ` <1297893854.4097.43.camel-vNW8ozRvgWupuGC+iAP0z+TW4wlIGRCZ@public.gmane.org>
2011-02-17 10:44 ` [sshfs] " Miklos Szeredi
2011-02-17 11:54 ` Yann Droneaud
2011-02-17 13:08 ` Miklos Szeredi
2011-02-17 18:11 ` Yann Droneaud
2011-02-17 19:59 ` Miklos Szeredi
2011-02-18 7:41 ` [fuse-devel] " Goswin von Brederlow
2011-02-17 12:05 ` Yann Droneaud
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4D5CDE58.5000905@drmicha.warpmail.net \
--to=git@drmicha.warpmail.net \
--cc=git@vger.kernel.org \
--cc=yann@droneaud.fr \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).