mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Johannes Sixt <>
To: "Ævar Arnfjörð Bjarmason" <>,
	"Mike Hommey" <>
Subject: Re: Closing fds twice when using remote helpers
Date: Wed, 15 May 2019 19:59:49 +0200	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

Am 15.05.19 um 13:43 schrieb Ævar Arnfjörð Bjarmason:
> On Wed, May 15 2019, Mike Hommey wrote:
>> Hi,
>> I started getting a weird error message during some test case involving
>> git-cinnabar, which is a remote-helper to access mercurial
>> repositories.
>> The error says:
>> fatal: mmap failed: Bad file descriptor
>> ... which was not making much sense. Some debugging later, and it turns
>> out this is what happens:
>> - start_command is called for fast-import

I guess, you request fast_import->out = -1.

>> - start_command is called again for git-remote-hg, passing the
>>   fast_import->out as cmd->in.


>> - in start_command, we end up on the line of code that does
>>   close(cmd->in), so fast_import->out/cmd->in is now closed

Yes. That's how the interface is specified.

>> - much later, in disconnect_helper, we call close(data->helper->out),
>>   where data->helper is the cmd for fast-import, and that fd was already
>> closed above.

That must is wrong. Passing a fd to start_command() relinquishes

>> - Except, well, fds being what they are, we in fact just closed a fd
>>   from a packed_git->pack_fd. So, when use_pack is later called, and
>>   tries to mmap data from that pack, it fails because the file
>>   descriptor was closed.

Either dup() the file descriptor, or mmap() before you call the
consuming start_command().

>> I'm not entirely sure how to address this... Any ideas?
>> Relatedly, use_pack calls xmmap, which does its own error handling and
>> die()s in case of error, but then goes on to do its own check with a
>> different error message (which, in fact, could be more useful in other
>> cases). It seems like it should call xmmap_gently instead.
> The "obvious" hacky fix is to pass in some "I own it, don't close it"
> new flag in the child_process struct.
> In fact we used to have such a thing in the code, see e72ae28895
> ("start_command(), .in/.out/.err = -1: Callers must close the file
> descriptor", 2008-02-16).

That's a different thing. -1 tells that a pipe end should be passed back
to the caller. Of course, it must not be closed by start_command.

But if the caller passes their own fd *into* start_command/run_command,
then the caller must not close it.

> So we could bring it back, but I wonder if a better long-term solution
> is to refactor the API to have explicit start_command() ->
> free_command() steps, even if the free() is something that happens
> implicitly unless some "gutsy" function is called.

*Shrug* I'd use C++ to make the interface a no-brainer.

-- Hannes

  reply	other threads:[~2019-05-15 17:59 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-05-15 10:56 Closing fds twice when using remote helpers Mike Hommey
2019-05-15 11:43 ` Ævar Arnfjörð Bjarmason
2019-05-15 17:59   ` Johannes Sixt [this message]
2019-05-15 22:08     ` Mike Hommey
2019-05-15 23:53       ` Jeff King
2019-05-16  0:48         ` Mike Hommey
2019-05-16  3:28           ` Jeff King
2019-05-16  8:35             ` Mike Hommey
2019-05-16 21:47               ` Jeff King
2019-05-16 22:02                 ` Mike Hommey
2019-05-16  0:31 ` Mike Hommey
2019-05-16  0:37   ` [PATCH 1/2] dup() the input fd for fast-import used for " Mike Hommey
2019-05-16  0:37     ` [PATCH 2/2] Use xmmap_gently instead of xmmap in use_pack Mike Hommey
2019-05-16  3:34       ` Jeff King
2019-05-16  3:28     ` [PATCH 1/2] dup() the input fd for fast-import used for remote helpers Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).