From: Jeff King <peff@peff.net>
To: Brandon Williams <bmwill@google.com>
Cc: Junio C Hamano <gitster@pobox.com>, git@vger.kernel.org
Subject: Re: What's cooking in git.git (Nov 2016, #06; Mon, 28)
Date: Wed, 30 Nov 2016 19:19:02 -0500 [thread overview]
Message-ID: <20161201001901.mjof24j2qsklnf6f@sigill.intra.peff.net> (raw)
In-Reply-To: <20161201000824.GE192901@google.com> <20161201000605.GD192901@google.com>
On Wed, Nov 30, 2016 at 04:06:05PM -0800, Brandon Williams wrote:
> On 11/30, Jeff King wrote:
> > So I think there is some other chdir(). I'm not sure if there is an easy
> > way to get a backtrace on every call to chdir() in every thread. I'm
> > sure somebody more clever than me could figure out how to make gdb do it
> > automatically, but it might be workable manually. I think the chdir was
> > in the main thread.
> >
> > -Peff
>
> Yeah maybe we're missing something else...
>
> How did you run strace with your stress script?
It's hidden in the patch I sent a moment ago, but basically just "strace
-o foo.out" will dump the trace in the trash directory. After the stress
script runs, you can "cat fail/trash*/foo.out".
> > (gdb) bt
> > #0 chdir () at ../sysdeps/unix/syscall-template.S:84
> > #1 0x00005555555fe259 in real_path_internal (path=0x5555559f6b30 "su:b/../.git/modules/su:b", die_on_error=1)
> > at abspath.c:84
> > #2 0x00005555555fe48a in real_path (path=0x5555559f6b30 "su:b/../.git/modules/su:b") at abspath.c:135
> > #3 0x00005555556d09e6 in read_gitfile_gently (path=0x5555559f6ac0 "su:b/.git", return_error_code=0x0)
> > at setup.c:555
> > #4 0x00005555556d19cf in resolve_gitdir (suspect=0x5555559f6ac0 "su:b/.git") at setup.c:1021
> > #5 0x00005555556e7e34 in is_submodule_populated (path=0x5555559f5ec8 "su:b") at submodule.c:244
> > #6 0x00005555555a0f05 in grep_submodule (opt=0x7fffffffd8b0, sha1=0x0, filename=0x5555559f5ec8 "su:b",
> > path=0x5555559f5ec8 "su:b") at builtin/grep.c:619
> > #7 0x00005555555a12ac in grep_cache (opt=0x7fffffffd8b0, pathspec=0x7fffffffd880, cached=0) at builtin/grep.c:700
> > #8 0x00005555555a36cb in cmd_grep (argc=0, argv=0x7fffffffdf40, prefix=0x0) at builtin/grep.c:1257
> > #9 0x000055555556603b in run_builtin (p=0x5555559b3ad8 <commands+984>, argc=4, argv=0x7fffffffdf40) at git.c:373
> > #10 0x00005555555662bc in handle_builtin (argc=4, argv=0x7fffffffdf40) at git.c:572
> > #11 0x000055555556641a in run_argv (argcp=0x7fffffffddfc, argv=0x7fffffffddf0) at git.c:630
> > #12 0x00005555555665a8 in cmd_main (argc=4, argv=0x7fffffffdf40) at git.c:702
> > #13 0x00005555555fde47 in main (argc=7, argv=0x7fffffffdf28) at common-main.c:40
> >
> > So is_submodule_populated() needs to take a lock. But what's really
> > gross is that the _other_ threads need to lock just to call lstat().
> > Presumably it could be done as a reader/writer type of lock where many
> > "reader" threads can take the "I need to lstat()" lock simultaneously,
> > but block when an "I'm going to chdir()" writer holds it.
>
> Oh interesting, I wonder if there is a way to not have to perform a
> chdir since taking a lock to lstat wouldn't be ideal.
I don't think so. It comes from real_path(), which needs to either
chdir(), or start interpreting symbolic links itself (and madness that
way lies).
I think with a reader/writer lock as I described it wouldn't be too bad.
The common case would pay only the locking cost and not ever block,
since submodules are rare (and they're super-heavyweight to descend into
anyway).
I think putting it at the individual lstat() would be way too low, but
probably you could do it right before calling grep_source(). It may even
be possible to do some of the submodule work ahead of time while holding
grep_lock().
> Thanks for helping out with this!
I wasn't planning on it, but this turned into an intriguing puzzle. ;)
-Peff
next prev parent reply other threads:[~2016-12-01 0:19 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-11-29 0:15 What's cooking in git.git (Nov 2016, #06; Mon, 28) Junio C Hamano
2016-11-29 1:05 ` Brandon Williams
2016-11-29 6:37 ` Jeff King
2016-11-29 6:51 ` Jeff King
2016-11-30 19:54 ` Brandon Williams
2016-11-30 23:28 ` Brandon Williams
2016-11-30 23:32 ` Jeff King
2016-11-30 23:40 ` Jeff King
2016-11-30 23:42 ` Brandon Williams
2016-11-30 23:46 ` Jeff King
2016-11-30 23:57 ` Brandon Williams
2016-11-30 23:59 ` Jeff King
2016-12-01 0:04 ` Jeff King
2016-12-01 0:08 ` Brandon Williams
2016-12-01 0:14 ` Stefan Beller
2016-12-01 1:14 ` Brandon Williams
2016-12-01 0:06 ` Brandon Williams
2016-12-01 0:19 ` Jeff King [this message]
2016-11-30 23:43 ` Stefan Beller
2016-12-01 7:09 ` Johannes Sixt
2016-12-01 7:19 ` Jeff King
2016-11-29 6:59 ` Jeff King
2016-11-29 18:31 ` Junio C Hamano
2016-11-29 18:37 ` Jeff King
2016-11-29 19:21 ` Stefan Beller
2016-11-29 19:26 ` Junio C Hamano
2016-11-29 19:29 ` Stefan Beller
2016-11-30 0:25 ` Stefan Beller
2016-12-01 8:30 ` bw/transport-protocol-policy Jeff King
2016-12-01 18:14 ` bw/transport-protocol-policy Brandon Williams
2016-12-01 19:20 ` bw/transport-protocol-policy Jeff King
2016-12-01 19:35 ` bw/transport-protocol-policy Brandon Williams
2016-12-01 19:46 ` bw/transport-protocol-policy Jeff King
2016-12-01 19:53 ` bw/transport-protocol-policy Brandon Williams
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20161201001901.mjof24j2qsklnf6f@sigill.intra.peff.net \
--to=peff@peff.net \
--cc=bmwill@google.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).