From: Ramkumar Ramachandra <artagnon@gmail.com>
To: Jonathan Nieder <jrnieder@gmail.com>
Cc: Git Mailing List <git@vger.kernel.org>,
David Michael Barr <david.barr@cordelta.com>,
Sverre Rabbelier <srabbelier@gmail.com>,
Junio C Hamano <gitster@pobox.com>
Subject: Re: [PATCH 3/8] Add memory pool library
Date: Thu, 15 Jul 2010 21:12:13 +0200 [thread overview]
Message-ID: <20100715191213.GJ22574@debian> (raw)
In-Reply-To: <20100715185706.GA2774@burratino>
Hi Jonathan,
Jonathan Nieder writes:
> Ramkumar Ramachandra wrote:
>
> > void pre_commit(void);
> >
> > Write the pool to file.
>
> Except as a proof of concept, this is the wrong API to have. The problem
> is that the caller cannot choose the filename, so it ends up being a .bin
> file in the current directory, wherever that is.
>
> The log message leaves out a subtlety: this also increases the
> ‘committed’ value, and bookkeeping for that might be useful to some
> callers.
>
> In other words:
>
> > +static MAYBE_UNUSED void pre##_init(void) \
> > +{ \
> > + struct stat st; \
> > + pre##_pool.file = fopen(#pre ".bin", "a+"); \
> > + rewind(pre##_pool.file); \
> > + fstat(fileno(pre##_pool.file), &st); \
> > + pre##_pool.size = st.st_size / sizeof(obj_t); \
> > + pre##_pool.committed = pre##_pool.size; \
> > + pre##_pool.capacity = pre##_pool.size * 2; \
> > + if (pre##_pool.capacity < initial_capacity) \
> > + pre##_pool.capacity = initial_capacity; \
> > + pre##_pool.base = malloc(pre##_pool.capacity * sizeof(obj_t)); \
> > + fread(pre##_pool.base, sizeof(obj_t), pre##_pool.size, pre##_pool.file); \
> > +} \
>
> If you just want something working, I’d suggest stubbing this out:
>
> static MAYBE_UNUSED void pre##_init(void) \
> { \
> } \
>
> It even almost makes sense as API: the _init function does all
> initialization tasks required, which is to say, none. (The {0, ...}
> initializer already has taken care of setting all fields to 0).
>
> > +static MAYBE_UNUSED void pre##_commit(void) \
> > +{ \
> > + pre##_pool.committed += fwrite(pre##_pool.base + pre##_pool.committed, \
> > + sizeof(obj_t), pre##_pool.size - pre##_pool.committed, \
> > + pre##_pool.file); \
> > +} \
>
> This can be simplified
>
> static MAYBE_UNUSED void pre##_commit(void) \
> { \
> pre##_pool.committed = pre##_pool.size; \
> } \
>
> In other words, maybe something like this on top? This includes the
> vestigal _init() function which really should not be there (it is
> confusing that some callers use it and others don’t). I did not
> spend much time on it because in the end I suspect we might throw
> obj_pool away anyway.
Oh, right. I remember that you asked to turn off persistence for this
merge. We can include persistence it in a later series.
Junio: Could you squash this diff into the commit?
diff --git a/vcs-svn/obj_pool.h b/vcs-svn/obj_pool.h
index f60c872..7a256d4 100644
--- a/vcs-svn/obj_pool.h
+++ b/vcs-svn/obj_pool.h
@@ -20,17 +20,6 @@ static struct { \
} pre##_pool = { 0, 0, 0, NULL, NULL}; \
static MAYBE_UNUSED void pre##_init(void) \
{ \
- struct stat st; \
- pre##_pool.file = fopen(#pre ".bin", "a+"); \
- rewind(pre##_pool.file); \
- fstat(fileno(pre##_pool.file), &st); \
- pre##_pool.size = st.st_size / sizeof(obj_t); \
- pre##_pool.committed = pre##_pool.size; \
- pre##_pool.capacity = pre##_pool.size * 2; \
- if (pre##_pool.capacity < initial_capacity) \
- pre##_pool.capacity = initial_capacity; \
- pre##_pool.base = malloc(pre##_pool.capacity * sizeof(obj_t)); \
- fread(pre##_pool.base, sizeof(obj_t), pre##_pool.size, pre##_pool.file); \
} \
static MAYBE_UNUSED uint32_t pre##_alloc(uint32_t count) \
{ \
@@ -62,9 +51,7 @@ static MAYBE_UNUSED obj_t *pre##_pointer(uint32_t offset) \
} \
static MAYBE_UNUSED void pre##_commit(void) \
{ \
- pre##_pool.committed += fwrite(pre##_pool.base + pre##_pool.committed, \
- sizeof(obj_t), pre##_pool.size - pre##_pool.committed, \
- pre##_pool.file); \
+ pre##_pool.committed = pre##_pool.size; \
} \
static MAYBE_UNUSED void pre##_reset(void) \
{ \
next prev parent reply other threads:[~2010-07-15 19:10 UTC|newest]
Thread overview: 79+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-07-15 16:22 [PATCH 0/8] Resurrect rr/svn-export Ramkumar Ramachandra
2010-07-15 16:22 ` [PATCH 1/8] Export parse_date_basic() to convert a date string to timestamp Ramkumar Ramachandra
2010-07-15 17:25 ` Jonathan Nieder
2010-07-15 22:54 ` Junio C Hamano
2010-07-15 16:22 ` [PATCH 2/8] Introduce vcs-svn lib Ramkumar Ramachandra
2010-07-15 17:46 ` Jonathan Nieder
2010-07-15 19:15 ` Ramkumar Ramachandra
2010-07-15 16:22 ` [PATCH 3/8] Add memory pool library Ramkumar Ramachandra
2010-07-15 18:57 ` Jonathan Nieder
2010-07-15 19:12 ` Ramkumar Ramachandra [this message]
2010-07-15 16:23 ` [PATCH 4/8] Add treap implementation Ramkumar Ramachandra
2010-07-15 19:09 ` Jonathan Nieder
2010-07-15 19:18 ` Ramkumar Ramachandra
2010-07-15 16:23 ` [PATCH 5/8] Add string-specific memory pool Ramkumar Ramachandra
2010-07-15 16:23 ` [PATCH 6/8] Add stream helper library Ramkumar Ramachandra
2010-07-15 19:19 ` Jonathan Nieder
2010-07-15 16:23 ` [PATCH 7/8] Add infrastructure to write revisions in fast-export format Ramkumar Ramachandra
2010-07-15 19:28 ` Jonathan Nieder
2010-07-15 16:23 ` [PATCH 8/8] Add SVN dump parser Ramkumar Ramachandra
2010-07-15 19:52 ` Jonathan Nieder
2010-07-15 20:04 ` Jonathan Nieder
2010-07-16 10:13 ` [PATCH 0/8] Resurrect rr/svn-export Jonathan Nieder
2010-07-16 10:16 ` [PATCH 3/9] Add memory pool library Jonathan Nieder
2010-07-16 10:23 ` [PATCH 4/9] Add treap implementation Jonathan Nieder
2010-07-16 18:26 ` Jonathan Nieder
2010-08-09 21:57 ` [PATCH 0/10] rr/svn-export reroll Jonathan Nieder
2010-08-09 22:01 ` [PATCH 01/10] Export parse_date_basic() to convert a date string to timestamp Jonathan Nieder
2010-08-09 22:04 ` [PATCH 02/10] Introduce vcs-svn lib Jonathan Nieder
2010-08-09 22:11 ` [PATCH 03/10] Add memory pool library Jonathan Nieder
2010-08-09 22:17 ` [PATCH 04/10] Add treap implementation Jonathan Nieder
2010-08-12 17:22 ` Junio C Hamano
2010-08-12 22:02 ` Jonathan Nieder
2010-08-12 22:11 ` Jonathan Nieder
2010-08-12 22:44 ` Junio C Hamano
2010-08-09 22:34 ` [PATCH 05/10] Add string-specific memory pool Jonathan Nieder
2010-08-12 17:22 ` Junio C Hamano
2010-08-12 21:30 ` Jonathan Nieder
2010-08-09 22:39 ` [PATCH 06/10] Add stream helper library Jonathan Nieder
2010-08-09 22:48 ` [PATCH 07/10] Infrastructure to write revisions in fast-export format Jonathan Nieder
2010-08-09 22:55 ` [PATCH 08/10] SVN dump parser Jonathan Nieder
2010-08-12 17:22 ` Junio C Hamano
2010-08-09 22:55 ` PATCH 09/10] Update svn-fe manual Jonathan Nieder
2010-08-09 22:58 ` [PATCH 10/10] svn-fe manual: Clarify warning about deltas in dump files Jonathan Nieder
2010-08-10 12:53 ` [PATCH 0/10] rr/svn-export reroll Ramkumar Ramachandra
2010-08-11 1:53 ` Jonathan Nieder
2010-10-11 2:34 ` [PATCH/WIP 00/16] svn delta applier Jonathan Nieder
2010-10-11 2:37 ` [PATCH 01/16] vcs-svn: Eliminate global byte_buffer[] array Jonathan Nieder
2010-10-11 2:39 ` [PATCH 03/16] vcs-svn: Collect line_buffer data in a struct Jonathan Nieder
2010-10-11 2:41 ` [PATCH 04/16] vcs-svn: Teach line_buffer to handle multiple input files Jonathan Nieder
2010-10-11 2:44 ` [PATCH 05/16] vcs-svn: Make buffer_skip_bytes() report partial reads Jonathan Nieder
2010-10-11 2:46 ` [PATCH 06/16] vcs-svn: Improve support for reading large files Jonathan Nieder
2010-10-11 2:47 ` [PATCH 07/16] vcs-svn: Add binary-safe read() function Jonathan Nieder
2010-10-11 2:47 ` [PATCH 08/16] vcs-svn: Let callers peek ahead to find stream end Jonathan Nieder
2010-10-11 2:51 ` [PATCH 09/16] vcs-svn: Allow input errors to be detected early Jonathan Nieder
2010-10-11 2:52 ` [PATCH 10/16] vcs-svn: Allow character-oriented input Jonathan Nieder
2010-10-11 2:53 ` [PATCH 11/16] vcs-svn: Add code to maintain a sliding view of a file Jonathan Nieder
2010-10-11 2:55 ` [PATCH 12/16] vcs-svn: Learn to parse variable-length integers Jonathan Nieder
2010-10-11 2:58 ` [PATCH 13/16] vcs-svn: Learn to check for SVN\0 magic Jonathan Nieder
2010-10-11 2:59 ` [PATCH 14/16] compat: helper for detecting unsigned overflow Jonathan Nieder
2010-10-11 3:00 ` [PATCH 15/16] t9010 (svn-fe): Eliminate dependency on svn perl bindings Jonathan Nieder
2010-10-11 3:11 ` [PATCH 02/16] vcs-svn: Replace buffer_read_string() memory pool with a strbuf Jonathan Nieder
2010-10-11 4:01 ` [PATCH/RFC 16'/16] vcs-svn: Add svn delta parser Jonathan Nieder
2010-10-13 9:17 ` [PATCH/RFC 0/11] Building up the " Jonathan Nieder
2010-10-13 9:19 ` [PATCH 01/11] fixup! vcs-svn: Learn to parse variable-length integers Jonathan Nieder
2010-10-13 9:21 ` [PATCH 02/11] vcs-svn: Skeleton of an svn delta parser Jonathan Nieder
2010-10-13 9:30 ` [PATCH 03/11] vcs-svn: Read the preimage while applying deltas Jonathan Nieder
2010-10-14 21:45 ` Sam Vilain
2010-10-14 23:40 ` Jonathan Nieder
2010-10-13 9:35 ` [PATCH 04/11] vcs-svn: Read inline data from deltas Jonathan Nieder
2010-10-13 9:38 ` [PATCH 05/11] vcs-svn: Read instructions " Jonathan Nieder
2010-10-13 9:39 ` [PATCH 06/11] vcs-svn: Implement copyfrom_data delta instruction Jonathan Nieder
2010-10-13 9:41 ` [PATCH 07/11] vcs-svn: Check declared number of output bytes Jonathan Nieder
2010-10-13 9:48 ` [PATCH 08/11] vcs-svn: Reject deltas that do not consume all inline data Jonathan Nieder
2010-10-13 9:50 ` [PATCH 09/11] vcs-svn: Let deltas use data from postimage Jonathan Nieder
2010-10-13 9:53 ` [PATCH 10/11] vcs-svn: Reject deltas that read past end of preimage Jonathan Nieder
2010-10-13 9:58 ` [PATCH 11/11] vcs-svn: Allow deltas to copy from preimage Jonathan Nieder
2010-10-13 10:00 ` Jonathan Nieder
2010-10-18 17:00 ` [PATCH/RFC 0/11] Building up the delta parser Ramkumar Ramachandra
2010-10-18 17:03 ` Jonathan Nieder
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20100715191213.GJ22574@debian \
--to=artagnon@gmail.com \
--cc=david.barr@cordelta.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jrnieder@gmail.com \
--cc=srabbelier@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).