From: Thomas Gummerer <t.gummerer@gmail.com>
To: "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>, git@vger.kernel.org
Cc: tr@thomasrast.ch, "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
Subject: Re: [PATCH/WIP v2 02/14] read-cache: new extension to mark what file is watched
Date: Fri, 17 Jan 2014 12:19:05 +0100 [thread overview]
Message-ID: <87ha92c0uu.fsf@gmail.com> (raw)
In-Reply-To: <1389952060-12297-3-git-send-email-pclouds@gmail.com>
Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:
> If an entry is "watched", git lets an external program decide if the
> entry is modified or not. It's more like --assume-unchanged, but
> designed to be controlled by machine.
>
> We are running out of on-disk ce_flags, so instead of extending
> on-disk entry format again, "watched" flags are in-core only and
> stored as extension instead.
As you said yourself in http://thread.gmane.org/gmane.comp.version-control.git/240339/focus=240385
this is not quite true. As for your explanation there,
> Anyway using extended flags means 2 extra bytes per entry for
> almost every entry in this case (and for index v5 it means redoing
> crc32 for almost every entry too when the bit is updated) so it may
> still be a good idea to keep the new flag separate.
I don't think adding 2 extra bytes would be too bad, since we are
already using 62 bytes plus the bytes for the filename for each index
entry, so it would be a less than 3% increase in the index file size.
(And the extended flags may be used anyway in some cases)
As for index-v5 (if that's ever going to happen), it depends mostly on
how often the CE_WATCHED is going to be updated, to decide whether it
makes sense to store this as extension.
That said, I don't care too deeply if it's stored one way or another,
but I think it would be good to update the commit message with a better
rationale for the choice.
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
> cache.h | 2 ++
> read-cache.c | 41 ++++++++++++++++++++++++++++++++++++++++-
> 2 files changed, 42 insertions(+), 1 deletion(-)
>
> diff --git a/cache.h b/cache.h
> index a09d622..069dce7 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -168,6 +168,8 @@ struct cache_entry {
> /* used to temporarily mark paths matched by pathspecs */
> #define CE_MATCHED (1 << 26)
>
> +#define CE_WATCHED (1 << 27)
> +
> /*
> * Extended on-disk flags
> */
> diff --git a/read-cache.c b/read-cache.c
> index fe1d153..6f21e3f 100644
> --- a/read-cache.c
> +++ b/read-cache.c
> @@ -33,6 +33,7 @@ static struct cache_entry *refresh_cache_entry(struct cache_entry *ce, int reall
> #define CACHE_EXT(s) ( (s[0]<<24)|(s[1]<<16)|(s[2]<<8)|(s[3]) )
> #define CACHE_EXT_TREE 0x54524545 /* "TREE" */
> #define CACHE_EXT_RESOLVE_UNDO 0x52455543 /* "REUC" */
> +#define CACHE_EXT_WATCH 0x57415443 /* "WATC" */
>
> struct index_state the_index;
>
> @@ -1293,6 +1294,19 @@ static int verify_hdr(struct cache_header *hdr,
> return 0;
> }
>
> +static void read_watch_extension(struct index_state *istate, uint8_t *data,
> + unsigned long sz)
> +{
> + int i;
> + if ((istate->cache_nr + 7) / 8 != sz) {
> + error("invalid 'WATC' extension");
> + return;
> + }
> + for (i = 0; i < istate->cache_nr; i++)
> + if (data[i / 8] & (1 << (i % 8)))
> + istate->cache[i]->ce_flags |= CE_WATCHED;
> +}
> +
> static int read_index_extension(struct index_state *istate,
> const char *ext, void *data, unsigned long sz)
> {
> @@ -1303,6 +1317,9 @@ static int read_index_extension(struct index_state *istate,
> case CACHE_EXT_RESOLVE_UNDO:
> istate->resolve_undo = resolve_undo_read(data, sz);
> break;
> + case CACHE_EXT_WATCH:
> + read_watch_extension(istate, data, sz);
> + break;
> default:
> if (*ext < 'A' || 'Z' < *ext)
> return error("index uses %.4s extension, which we do not understand",
> @@ -1781,7 +1798,7 @@ int write_index(struct index_state *istate, int newfd)
> {
> git_SHA_CTX c;
> struct cache_header hdr;
> - int i, err, removed, extended, hdr_version;
> + int i, err, removed, extended, hdr_version, has_watches = 0;
> struct cache_entry **cache = istate->cache;
> int entries = istate->cache_nr;
> struct stat st;
> @@ -1790,6 +1807,8 @@ int write_index(struct index_state *istate, int newfd)
> for (i = removed = extended = 0; i < entries; i++) {
> if (cache[i]->ce_flags & CE_REMOVE)
> removed++;
> + else if (cache[i]->ce_flags & CE_WATCHED)
> + has_watches++;
>
> /* reduce extended entries if possible */
> cache[i]->ce_flags &= ~CE_EXTENDED;
> @@ -1861,6 +1880,26 @@ int write_index(struct index_state *istate, int newfd)
> if (err)
> return -1;
> }
> + if (has_watches) {
> + int id, sz = (entries - removed + 7) / 8;
> + uint8_t *data = xmalloc(sz);
> + memset(data, 0, sz);
> + for (i = 0, id = 0; i < entries && has_watches; i++) {
> + struct cache_entry *ce = cache[i];
> + if (ce->ce_flags & CE_REMOVE)
> + continue;
> + if (ce->ce_flags & CE_WATCHED) {
> + data[id / 8] |= 1 << (id % 8);
> + has_watches--;
> + }
> + id++;
> + }
> + err = write_index_ext_header(&c, newfd, CACHE_EXT_WATCH, sz) < 0
> + || ce_write(&c, newfd, data, sz) < 0;
> + free(data);
> + if (err)
> + return -1;
> + }
>
> if (ce_flush(&c, newfd) || fstat(newfd, &st))
> return -1;
> --
> 1.8.5.1.208.g05b12ea
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Thomas
next prev parent reply other threads:[~2014-01-17 11:18 UTC|newest]
Thread overview: 72+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-01-12 11:03 [PATCH 0/6] inotify support Nguyễn Thái Ngọc Duy
2014-01-12 11:03 ` [PATCH 1/6] read-cache: save trailing sha-1 Nguyễn Thái Ngọc Duy
2014-01-12 11:03 ` [PATCH 2/6] read-cache: new extension to mark what file is watched Nguyễn Thái Ngọc Duy
2014-01-13 17:02 ` Jonathan Nieder
2014-01-14 1:25 ` Duy Nguyen
2014-01-14 1:39 ` Duy Nguyen
2014-01-12 11:03 ` [PATCH 3/6] read-cache: connect to file watcher Nguyễn Thái Ngọc Duy
2014-01-15 10:58 ` Jeff King
2014-01-12 11:03 ` [PATCH 4/6] read-cache: get "updated" path list from " Nguyễn Thái Ngọc Duy
2014-01-12 11:03 ` [PATCH 5/6] read-cache: ask file watcher to watch files Nguyễn Thái Ngọc Duy
2014-01-12 11:03 ` [PATCH 6/6] file-watcher: support inotify Nguyễn Thái Ngọc Duy
2014-01-17 9:47 ` [PATCH/WIP v2 00/14] inotify support Nguyễn Thái Ngọc Duy
2014-01-17 9:47 ` [PATCH/WIP v2 01/14] read-cache: save trailing sha-1 Nguyễn Thái Ngọc Duy
2014-01-17 9:47 ` [PATCH/WIP v2 02/14] read-cache: new extension to mark what file is watched Nguyễn Thái Ngọc Duy
2014-01-17 11:19 ` Thomas Gummerer [this message]
2014-01-19 17:06 ` Thomas Rast
2014-01-20 1:38 ` Duy Nguyen
2014-01-17 9:47 ` [PATCH/WIP v2 03/14] read-cache: connect to file watcher Nguyễn Thái Ngọc Duy
2014-01-17 15:24 ` Torsten Bögershausen
2014-01-17 16:21 ` Duy Nguyen
2014-01-17 9:47 ` [PATCH/WIP v2 04/14] read-cache: ask file watcher to watch files Nguyễn Thái Ngọc Duy
2014-01-17 9:47 ` [PATCH/WIP v2 05/14] read-cache: put some limits on file watching Nguyễn Thái Ngọc Duy
2014-01-19 17:06 ` Thomas Rast
2014-01-20 1:36 ` Duy Nguyen
2014-01-17 9:47 ` [PATCH/WIP v2 06/14] read-cache: get modified file list from file watcher Nguyễn Thái Ngọc Duy
2014-01-17 9:47 ` [PATCH/WIP v2 07/14] read-cache: add config to start file watcher automatically Nguyễn Thái Ngọc Duy
2014-01-17 9:47 ` [PATCH/WIP v2 08/14] read-cache: add GIT_TEST_FORCE_WATCHER for testing Nguyễn Thái Ngọc Duy
2014-01-19 17:04 ` Thomas Rast
2014-01-20 1:32 ` Duy Nguyen
2014-01-17 9:47 ` [PATCH/WIP v2 09/14] file-watcher: add --shutdown and --log options Nguyễn Thái Ngọc Duy
2014-01-17 9:47 ` [PATCH/WIP v2 10/14] file-watcher: automatically quit Nguyễn Thái Ngọc Duy
2014-01-17 9:47 ` [PATCH/WIP v2 11/14] file-watcher: support inotify Nguyễn Thái Ngọc Duy
2014-01-19 17:04 ` [PATCH/WIP v2 00/14] inotify support Thomas Rast
2014-01-20 1:28 ` Duy Nguyen
2014-01-20 21:51 ` Thomas Rast
2014-01-28 10:46 ` Duy Nguyen
2014-02-03 4:28 ` [PATCH v3 00/26] " Nguyễn Thái Ngọc Duy
2014-02-03 4:28 ` [PATCH v3 01/26] pkt-line.c: rename global variable buffer[] to something less generic Nguyễn Thái Ngọc Duy
2014-02-03 4:28 ` [PATCH v3 02/26] pkt-line.c: add packet_write_timeout() Nguyễn Thái Ngọc Duy
2014-02-03 4:28 ` [PATCH v3 03/26] pkt-line.c: add packet_read_line_timeout() Nguyễn Thái Ngọc Duy
2014-02-03 4:28 ` [PATCH v3 04/26] unix-socket: make unlink() optional in unix_stream_listen() Nguyễn Thái Ngọc Duy
2014-02-03 4:28 ` [PATCH v3 05/26] Add git-file-watcher and basic connection handling logic Nguyễn Thái Ngọc Duy
2014-02-03 4:28 ` [PATCH v3 06/26] file-watcher: check socket directory permission Nguyễn Thái Ngọc Duy
2014-02-03 4:28 ` [PATCH v3 07/26] file-watcher: remove socket on exit Nguyễn Thái Ngọc Duy
2014-02-03 4:28 ` [PATCH v3 08/26] file-watcher: add --detach Nguyễn Thái Ngọc Duy
2014-02-03 4:28 ` [PATCH v3 09/26] read-cache: save trailing sha-1 Nguyễn Thái Ngọc Duy
2014-02-03 4:28 ` [PATCH v3 10/26] read-cache: new flag CE_WATCHED to mark what file is watched Nguyễn Thái Ngọc Duy
2014-02-03 4:28 ` [PATCH v3 11/26] Clear CE_WATCHED when set CE_VALID alone Nguyễn Thái Ngọc Duy
2014-02-03 4:29 ` [PATCH v3 12/26] read-cache: basic hand shaking to the file watcher Nguyễn Thái Ngọc Duy
2014-02-03 4:29 ` [PATCH v3 13/26] read-cache: ask file watcher to watch files Nguyễn Thái Ngọc Duy
2014-02-03 4:29 ` [PATCH v3 14/26] read-cache: put some limits on file watching Nguyễn Thái Ngọc Duy
2014-02-03 4:29 ` [PATCH v3 15/26] read-cache: get changed file list from file watcher Nguyễn Thái Ngọc Duy
2014-02-03 4:29 ` [PATCH v3 16/26] git-compat-util.h: add inotify stubs on non-Linux platforms Nguyễn Thái Ngọc Duy
2014-02-03 4:29 ` [PATCH v3 17/26] file-watcher: inotify support, watching part Nguyễn Thái Ngọc Duy
2014-02-03 4:29 ` [PATCH v3 18/26] file-watcher: inotify support, notification part Nguyễn Thái Ngọc Duy
2014-02-03 4:29 ` [PATCH v3 19/26] Wrap CE_VALID test with ce_valid() Nguyễn Thái Ngọc Duy
2014-02-03 4:29 ` [PATCH v3 20/26] read-cache: new variable to verify file-watcher results Nguyễn Thái Ngọc Duy
2014-02-03 4:29 ` [PATCH v3 21/26] Support running file watcher with the test suite Nguyễn Thái Ngọc Duy
2014-02-03 4:29 ` [PATCH v3 22/26] file-watcher: quit if $WATCHER/socket is gone Nguyễn Thái Ngọc Duy
2014-02-03 4:29 ` [PATCH v3 23/26] file-watcher: tests for the daemon Nguyễn Thái Ngọc Duy
2014-02-03 4:29 ` [PATCH v3 24/26] ls-files: print CE_WATCHED as W (or "w" with CE_VALID) Nguyễn Thái Ngọc Duy
2014-02-03 4:29 ` [PATCH v3 25/26] file-watcher: tests for the client side Nguyễn Thái Ngọc Duy
2014-02-03 4:29 ` [PATCH v3 26/26] Disable file-watcher with system inotify on some tests Nguyễn Thái Ngọc Duy
2014-02-08 8:04 ` [PATCH v3 00/26] inotify support Torsten Bögershausen
2014-02-08 8:53 ` Duy Nguyen
2014-02-09 20:19 ` Torsten Bögershausen
2014-02-10 10:37 ` Duy Nguyen
2014-02-10 16:55 ` Torsten Bögershausen
2014-02-10 23:34 ` Duy Nguyen
2014-02-17 12:36 ` Duy Nguyen
2014-02-19 20:35 ` [PATCH 0/6] " Shawn Pearce
2014-02-19 23:45 ` Duy Nguyen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87ha92c0uu.fsf@gmail.com \
--to=t.gummerer@gmail.com \
--cc=git@vger.kernel.org \
--cc=pclouds@gmail.com \
--cc=tr@thomasrast.ch \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).