From: Johannes Sixt <j6t@kdbg.org>
To: Jeff King <peff@peff.net>
Cc: git@vger.kernel.org
Subject: Re: [PATCH 4/4] send-pack: abort sideband demuxer on pack-objects error
Date: Thu, 14 Apr 2011 22:43:33 +0200 [thread overview]
Message-ID: <201104142243.33522.j6t@kdbg.org> (raw)
In-Reply-To: <20110414202110.GA6525@sigill.intra.peff.net>
On Donnerstag, 14. April 2011, Jeff King wrote:
> On Thu, Apr 14, 2011 at 09:36:25PM +0200, Johannes Sixt wrote:
> > On Donnerstag, 14. April 2011, Jeff King wrote:
> > > Obviously it totally breaks the start_async abstraction if the called
> > > code needs to care whether it forked or not. But we can use that to our
> > > advantage, since it means start_async callers must assume the interface
> > > is very limited. So I think we can do something like:
> > >
> > > 1. Async code declares which file descriptors it cares about. This
> > > would automatically include the pipe we give to it, of course.
> > > So the declared ones for a sideband demuxer would be stderr, and
> > > some network fd for reading.
> > >
> > > 2. In the pthreads case, we do nothing. In the forked case, the child
> > > closes every descriptor except the "interesting" ones.
> > >
> > > And that solves this problem, and the general case that async-callers
> > > have no idea if they have just leaked pipe descriptors in the forked
> > > case.
> >
> > Sounds like a plan. How do you close all file descriptors? Just iterate
> > up to getrlimit(RLIMIT_NOFILE)?
>
> Sadly, yes, I think that is what we would have to do. It does feel like
> an awful hack. And it will interact badly with things like valgrind,
> which open descriptors behind the scenes (but can properly handle
> the forking).
>
> I just don't see another way around it for the general case. The
> "usual" fix for this sort of thing is that the descriptors should have
> close-on-exec set, but that doesn't work for us here, because we are
> only forking.
>
> It's sufficiently ugly (and still possible to break in the pthreads
> case!) that it may be worth not worrying about the general case at all,
> and just fixing this one with the explicit close.
>
> > > I'm still slightly confused, though, because I never see that
> > > descriptor get closed in the threaded case. So I still don't understand
> > > why it _doesn't_ deadlock with pthreads.
> >
> > In the threaded case, this fd is closed by start_command(), where it is
> > passed as po.out in pack_objects(). In the fork case this is too late
> > because a duplicate was already inherited to the sideband demuxer.
>
> Hrm, I see the code now. That seems like an odd thing to do to me.
Why so? It's a matter of resource ownership: If you pass a positive value, you
give away ownership; if you pass -1, you gain ownership; if you pass 0,
ownership remains unchanged.
> Doesn't it disallow:
>
> /* set up a command */
> const char **argv = { "some", "command" };
> struct child_process c;
> c.argv = argv;
> c.out = fd;
>
> /* run it */
> run_command(&c);
>
> /* now tack our own output to the end */
> write(fd, "foo", 3);
You would have to dup() the fd before run_command().
> And even weirder, we only do the close for high file descriptors. So you
> _can_ do that above if "fd" is stdout, but not with an arbitrary fd.
Ah, right, that's a bit dubious. The reason is that if you want to tell the
child process to use the parent's stdout for its own stdout, you specify 0
aka "no special treatement", i.e. just inherit from the parent, not 1. IOW, 1
is never a sane candidate to be assigned to c.out.
-- Hannes
next prev parent reply other threads:[~2011-04-14 20:43 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-03-31 18:42 [PATCH 0/4] fix hang in git push when pack-objects fails Jeff King
2011-03-31 18:43 ` [PATCH 1/4] teach wait_or_whine a "quiet" mode Jeff King
2011-03-31 20:56 ` Johannes Sixt
2011-04-01 1:35 ` Jeff King
2011-03-31 18:44 ` [PATCH 2/4] finish_async: be quiet when waiting for async process Jeff King
2011-03-31 18:44 ` [PATCH 3/4] run-command: allow aborting async code prematurely Jeff King
2011-04-01 9:36 ` Erik Faye-Lund
2011-04-01 13:59 ` Jeff King
2011-03-31 18:44 ` [PATCH 4/4] send-pack: abort sideband demuxer on pack-objects error Jeff King
2011-04-13 19:53 ` Johannes Sixt
2011-04-14 13:54 ` Jeff King
2011-04-14 19:36 ` Johannes Sixt
2011-04-14 20:21 ` Jeff King
2011-04-14 20:43 ` Johannes Sixt [this message]
2011-04-14 20:51 ` Jeff King
2011-04-14 21:05 ` Johannes Sixt
2011-04-14 21:21 ` Junio C Hamano
2011-04-24 20:42 ` [PATCH/RFC 1/2] send-pack --stateless-rpc: properly close the outgoing channel Johannes Sixt
2011-04-24 20:49 ` [PATCH 2/2] send-pack: avoid deadlock when pack-object dies early Johannes Sixt
2011-04-25 16:50 ` Jeff King
2011-04-25 17:41 ` Johannes Sixt
2011-04-25 17:51 ` Junio C Hamano
2011-04-25 21:04 ` [PATCH v2] " Johannes Sixt
2011-04-26 8:23 ` Jeff King
2011-04-25 16:40 ` [PATCH/RFC 1/2] send-pack --stateless-rpc: properly close the outgoing channel Jeff King
2011-03-31 18:45 ` [PATCH 5/4] run-command: implement abort_async for pthreads Jeff King
2011-04-01 9:41 ` Erik Faye-Lund
2011-04-01 10:15 ` Erik Faye-Lund
2011-04-01 17:27 ` Johannes Sixt
2011-04-01 17:38 ` Jeff King
2011-04-01 19:26 ` Erik Faye-Lund
2011-04-01 19:33 ` Erik Faye-Lund
2011-04-01 19:42 ` Johannes Sixt
2011-04-01 19:57 ` Erik Faye-Lund
2011-04-01 20:05 ` Jeff King
2011-04-01 20:13 ` Erik Faye-Lund
2011-04-01 20:17 ` Jeff King
2011-04-01 20:18 ` Jeff King
2011-04-01 20:34 ` Erik Faye-Lund
2011-04-01 20:36 ` Johannes Sixt
2011-04-01 20:41 ` Erik Faye-Lund
2011-04-01 20:18 ` Johannes Sixt
2011-04-01 20:31 ` Erik Faye-Lund
2011-04-01 21:16 ` Jeff King
2011-04-02 12:27 ` Erik Faye-Lund
2011-04-01 14:00 ` Jeff King
2011-03-31 20:45 ` [PATCH 0/4] fix hang in git push when pack-objects fails Johannes Sixt
2011-04-01 1:34 ` Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=201104142243.33522.j6t@kdbg.org \
--to=j6t@kdbg.org \
--cc=git@vger.kernel.org \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).