From: Junio C Hamano <gitster@pobox.com>
To: Steffen Prohaska <prohaska@zib.de>
Cc: "Linus Torvalds" <torvalds@linux-foundation.org>,
git@vger.kernel.org, "Johannes Sixt" <j6t@kdbg.org>,
"John Keeping" <john@keeping.me.uk>,
"Jonathan Nieder" <jrnieder@gmail.com>,
"Kyle J. McKay" <mackyle@gmail.com>,
"Torsten Bögershausen" <tboegi@web.de>,
"Eric Sunshine" <sunshine@sunshineco.com>
Subject: Re: [PATCH v5 1/2] xread, xwrite: Limit size of IO, fixing IO of 2GB and more on Mac OS X
Date: Tue, 20 Aug 2013 12:37:49 -0700 [thread overview]
Message-ID: <xmqqli3w5f1u.fsf@gitster.dls.corp.google.com> (raw)
In-Reply-To: <1376981035-23284-2-git-send-email-prohaska@zib.de> (Steffen Prohaska's message of "Tue, 20 Aug 2013 08:43:54 +0200")
Steffen Prohaska <prohaska@zib.de> writes:
> diff --git a/wrapper.c b/wrapper.c
> index 6a015de..97e3cf7 100644
> --- a/wrapper.c
> +++ b/wrapper.c
> @@ -131,6 +131,14 @@ void *xcalloc(size_t nmemb, size_t size)
> }
>
> /*
> + * Limit size of IO chunks, because huge chunks only cause pain. OS X 64-bit
> + * buggy, returning EINVAL if len >= INT_MAX; and even in the absense of bugs,
s/buggy/is &/ perhaps?
> + * large chunks can result in bad latencies when you decide to kill the
> + * process.
> + */
> +#define MAX_IO_SIZE (8*1024*1024)
> +
> +/*
> * xread() is the same a read(), but it automatically restarts read()
> * operations with a recoverable error (EAGAIN and EINTR). xread()
> * DOES NOT GUARANTEE that "len" bytes is read even if the data is available.
> @@ -138,6 +146,8 @@ void *xcalloc(size_t nmemb, size_t size)
> ssize_t xread(int fd, void *buf, size_t len)
> {
> ssize_t nr;
> + if (len > MAX_IO_SIZE)
> + len = MAX_IO_SIZE;
> while (1) {
> nr = read(fd, buf, len);
> if ((nr < 0) && (errno == EAGAIN || errno == EINTR))
> @@ -154,6 +164,8 @@ ssize_t xread(int fd, void *buf, size_t len)
> ssize_t xwrite(int fd, const void *buf, size_t len)
> {
> ssize_t nr;
> + if (len > MAX_IO_SIZE)
> + len = MAX_IO_SIZE;
> while (1) {
> nr = write(fd, buf, len);
> if ((nr < 0) && (errno == EAGAIN || errno == EINTR))
next prev parent reply other threads:[~2013-08-20 19:37 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-08-17 12:40 [PATCH] xread(): Fix read error when filtering >= 2GB on Mac OS X Steffen Prohaska
2013-08-17 15:27 ` John Keeping
2013-08-17 15:56 ` Torsten Bögershausen
2013-08-17 17:16 ` Johannes Sixt
2013-08-17 18:57 ` Jonathan Nieder
2013-08-17 20:25 ` Kyle J. McKay
2013-08-17 21:23 ` Jonathan Nieder
2013-08-19 6:38 ` [PATCH v2] compat: Fix read() of 2GB and more " Steffen Prohaska
2013-08-19 7:54 ` John Keeping
2013-08-19 8:20 ` Steffen Prohaska
2013-08-19 8:20 ` Johannes Sixt
2013-08-19 8:25 ` Stefan Beller
2013-08-19 8:40 ` Johannes Sixt
2013-08-19 8:28 ` Steffen Prohaska
2013-08-19 8:21 ` [PATCH v3] " Steffen Prohaska
2013-08-19 13:59 ` Eric Sunshine
2013-08-19 16:33 ` Junio C Hamano
2013-08-19 15:41 ` [PATCH v4] " Steffen Prohaska
2013-08-19 16:04 ` Linus Torvalds
2013-08-19 16:37 ` Steffen Prohaska
2013-08-19 17:24 ` Junio C Hamano
2013-08-19 17:16 ` Junio C Hamano
2013-08-19 17:28 ` Linus Torvalds
2013-08-19 21:56 ` Kyle J. McKay
2013-08-19 22:51 ` Linus Torvalds
2013-08-27 4:59 ` Junio C Hamano
2013-08-20 6:43 ` [PATCH v5 0/2] Fix IO of >=2GB on Mac OS X by limiting IO chunks Steffen Prohaska
2013-08-20 6:43 ` [PATCH v5 1/2] xread, xwrite: Limit size of IO, fixing IO of 2GB and more on Mac OS X Steffen Prohaska
2013-08-20 19:37 ` Junio C Hamano [this message]
2013-08-21 19:50 ` Torsten Bögershausen
2013-08-20 6:43 ` [PATCH v5 2/2] Revert "compate/clipped-write.c: large write(2) fails on Mac OS X/XNU" Steffen Prohaska
2013-08-21 13:46 ` [PATCH v6 0/2] Fix IO >= 2GB on Mac, fixed typo Steffen Prohaska
2013-08-21 13:46 ` [PATCH v5 1/2] xread, xwrite: Limit size of IO, fixing IO of 2GB and more on Mac OS X Steffen Prohaska
2013-08-21 13:46 ` [PATCH v5 2/2] Revert "compate/clipped-write.c: large write(2) fails on Mac OS X/XNU" Steffen Prohaska
2013-08-21 15:58 ` [PATCH v6 0/2] Fix IO >= 2GB on Mac, fixed typo Junio C Hamano
2013-08-19 8:27 ` [PATCH v2] compat: Fix read() of 2GB and more on Mac OS X Johannes Sixt
2013-08-19 14:41 ` Torsten Bögershausen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqqli3w5f1u.fsf@gitster.dls.corp.google.com \
--to=gitster@pobox.com \
--cc=git@vger.kernel.org \
--cc=j6t@kdbg.org \
--cc=john@keeping.me.uk \
--cc=jrnieder@gmail.com \
--cc=mackyle@gmail.com \
--cc=prohaska@zib.de \
--cc=sunshine@sunshineco.com \
--cc=tboegi@web.de \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).