git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Bryan Turner <bturner@atlassian.com>
To: Jeff King <peff@peff.net>
Cc: Derrick Stolee <stolee@gmail.com>, Andreas Krey <a.krey@gmx.de>,
	Git Users <git@vger.kernel.org>
Subject: Re: Avoid race condition between fetch and repack/gc?
Date: Mon, 16 Mar 2020 16:40:13 -0700	[thread overview]
Message-ID: <CAGyf7-HQH7hjuYSmOYeOEvZBWkUzjU4M8Wr50FNPYgG4NZ=5UA@mail.gmail.com> (raw)
In-Reply-To: <20200316172729.GA1072073@coredump.intra.peff.net>

On Mon, Mar 16, 2020 at 10:27 AM Jeff King <peff@peff.net> wrote:
>
>   - you don't use config like core.packedGitLimit that encourages Git to
>     drop mmap'd windows. On 64-bit systems, this doesn't help at all (we
>     have plenty of address space, and since the maps are read-only, the
>     OS is free to drop the clean pages if there's memory pressure). On
>     32-bit systems, it's a necessity (and I'd expect a 32-bit server to
>     run into this issue a lot for large repositories).

I could be wrong, but I'm going to _guess_ from the :7999 in the
example output that the repository is hosted with Bitbucket Server.
Assuming my guess is correct, Bitbucket Server _does_ set
"core.packedgitlimit=256m" (and "core.packedgitwindowsize=32m", for
what it's worth). Those settings are applied specifically because
we've found they _do_ impact Git's overall memory usage when serving
clones in particular, which is important for cases where the system is
serving dozens (or hundreds) of concurrent clones.

Of course, we don't always do a great job of re-testing
once-beneficial settings later, which means sometimes they end up
being based on outdated observations. Perhaps we should prioritize
some additional testing here, especially on 64-bit systems. (We've
been setting "core.packedgitlimit" since back when Bitbucket Server
was called Stash and supported Git 1.7.6 on the server.)

That said, though, you note "core.packedgitlimit" is necessary on
32-bit servers and, unfortunately, we do still support Bitbucket
Server on 32-bit OSes. Maybe we should investigate applying (or not)
the flags depending on the platform. Sadly, that's not necessarily
simple to do since just because the _OS_ is 64-bit doesn't mean _Git_
is; it's pretty trivial to run 32-bit Git on 64-bit Windows, for
example.

Bryan

  reply	other threads:[~2020-03-16 23:40 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-16  8:23 Avoid race condition between fetch and repack/gc? Andreas Krey
2020-03-16 12:10 ` Derrick Stolee
2020-03-16 17:17   ` Nasser Grainawi
2020-03-16 17:27   ` Jeff King
2020-03-16 23:40     ` Bryan Turner [this message]
2020-03-17 18:41       ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAGyf7-HQH7hjuYSmOYeOEvZBWkUzjU4M8Wr50FNPYgG4NZ=5UA@mail.gmail.com' \
    --to=bturner@atlassian.com \
    --cc=a.krey@gmx.de \
    --cc=git@vger.kernel.org \
    --cc=peff@peff.net \
    --cc=stolee@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).