From: Phillip Wood <phillip.wood123@gmail.com>
To: Victoria Dye via GitGitGadget <gitgitgadget@gmail.com>,
git@vger.kernel.org
Cc: stolee@gmail.com, gitster@pobox.com, newren@gmail.com,
"Taylor Blau" <me@ttaylorr.com>,
"Bagas Sanjaya" <bagasdotme@gmail.com>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
"Victoria Dye" <vdye@github.com>
Subject: Re: [PATCH v3 6/8] reset: make sparse-aware (except --mixed)
Date: Fri, 8 Oct 2021 12:09:30 +0100 [thread overview]
Message-ID: <c1dd1fef-2d48-cc18-5786-10e174b487a7@gmail.com> (raw)
In-Reply-To: <330e0c0977480d0506801854fcaa6c9f2b014569.1633641339.git.gitgitgadget@gmail.com>
Hi Victoria
On 07/10/2021 22:15, Victoria Dye via GitGitGadget wrote:
> From: Victoria Dye <vdye@github.com>
>
> Remove `ensure_full_index` guard on `prime_cache_tree` and update
> `prime_cache_tree_rec` to correctly reconstruct sparse directory entries in
> the cache tree. While processing a tree's entries, `prime_cache_tree_rec`
> must determine whether a directory entry is sparse or not by searching for
> it in the index (*without* expanding the index). If a matching sparse
> directory index entry is found, no subtrees are added to the cache tree
> entry and the entry count is set to 1 (representing the sparse directory
> itself). Otherwise, the tree is assumed to not be sparse and its subtrees
> are recursively added to the cache tree.
I was looking at the callers to prime_cache_tree() this morning and
would like to suggest an alternative approach - just delete
prime_cache_tree() and all of its callers! As far as I can see it is
only ever called after a successful call to unpack_trees() and since
52fca2184d ("unpack-trees: populate cache-tree on successful merge",
2015-07-28) unpack_trees() updates the cache tree for the caller. All
the call sites are pretty obvious apart from the one in
t/help/test-fast-rebase.c where unpack_trees() is called by
merge_switch_to_result()
Best Wishes
Phillip
> Helped-by: Elijah Newren <newren@gmail.com>
> Signed-off-by: Victoria Dye <vdye@github.com>
> ---
> cache-tree.c | 47 ++++++++++++++++++++++--
> cache.h | 10 +++++
> read-cache.c | 27 ++++++++++----
> t/t1092-sparse-checkout-compatibility.sh | 15 +++++++-
> 4 files changed, 86 insertions(+), 13 deletions(-)
>
> diff --git a/cache-tree.c b/cache-tree.c
> index 9be19c85b66..2866101052c 100644
> --- a/cache-tree.c
> +++ b/cache-tree.c
> @@ -740,15 +740,26 @@ out:
> return ret;
> }
>
> +static void prime_cache_tree_sparse_dir(struct cache_tree *it,
> + struct tree *tree)
> +{
> +
> + oidcpy(&it->oid, &tree->object.oid);
> + it->entry_count = 1;
> +}
> +
> static void prime_cache_tree_rec(struct repository *r,
> struct cache_tree *it,
> - struct tree *tree)
> + struct tree *tree,
> + struct strbuf *tree_path)
> {
> struct tree_desc desc;
> struct name_entry entry;
> int cnt;
> + int base_path_len = tree_path->len;
>
> oidcpy(&it->oid, &tree->object.oid);
> +
> init_tree_desc(&desc, tree->buffer, tree->size);
> cnt = 0;
> while (tree_entry(&desc, &entry)) {
> @@ -757,14 +768,40 @@ static void prime_cache_tree_rec(struct repository *r,
> else {
> struct cache_tree_sub *sub;
> struct tree *subtree = lookup_tree(r, &entry.oid);
> +
> if (!subtree->object.parsed)
> parse_tree(subtree);
> sub = cache_tree_sub(it, entry.path);
> sub->cache_tree = cache_tree();
> - prime_cache_tree_rec(r, sub->cache_tree, subtree);
> +
> + /*
> + * Recursively-constructed subtree path is only needed when working
> + * in a sparse index (where it's used to determine whether the
> + * subtree is a sparse directory in the index).
> + */
> + if (r->index->sparse_index) {
> + strbuf_setlen(tree_path, base_path_len);
> + strbuf_grow(tree_path, base_path_len + entry.pathlen + 1);
> + strbuf_add(tree_path, entry.path, entry.pathlen);
> + strbuf_addch(tree_path, '/');
> + }
> +
> + /*
> + * If a sparse index is in use, the directory being processed may be
> + * sparse. To confirm that, we can check whether an entry with that
> + * exact name exists in the index. If it does, the created subtree
> + * should be sparse. Otherwise, cache tree expansion should continue
> + * as normal.
> + */
> + if (r->index->sparse_index &&
> + index_entry_exists(r->index, tree_path->buf, tree_path->len))
> + prime_cache_tree_sparse_dir(sub->cache_tree, subtree);
> + else
> + prime_cache_tree_rec(r, sub->cache_tree, subtree, tree_path);
> cnt += sub->cache_tree->entry_count;
> }
> }
> +
> it->entry_count = cnt;
> }
>
> @@ -772,12 +809,14 @@ void prime_cache_tree(struct repository *r,
> struct index_state *istate,
> struct tree *tree)
> {
> + struct strbuf tree_path = STRBUF_INIT;
> +
> trace2_region_enter("cache-tree", "prime_cache_tree", the_repository);
> cache_tree_free(&istate->cache_tree);
> istate->cache_tree = cache_tree();
>
> - ensure_full_index(istate);
> - prime_cache_tree_rec(r, istate->cache_tree, tree);
> + prime_cache_tree_rec(r, istate->cache_tree, tree, &tree_path);
> + strbuf_release(&tree_path);
> istate->cache_changed |= CACHE_TREE_CHANGED;
> trace2_region_leave("cache-tree", "prime_cache_tree", the_repository);
> }
> diff --git a/cache.h b/cache.h
> index f6295f3b048..1d3e4665562 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -816,6 +816,16 @@ struct cache_entry *index_file_exists(struct index_state *istate, const char *na
> */
> int index_name_pos(struct index_state *, const char *name, int namelen);
>
> +/*
> + * Determines whether an entry with the given name exists within the
> + * given index. The return value is 1 if an exact match is found, otherwise
> + * it is 0. Note that, unlike index_name_pos, this function does not expand
> + * the index if it is sparse. If an item exists within the full index but it
> + * is contained within a sparse directory (and not in the sparse index), 0 is
> + * returned.
> + */
> +int index_entry_exists(struct index_state *, const char *name, int namelen);
> +
> /*
> * Some functions return the negative complement of an insert position when a
> * precise match was not found but a position was found where the entry would
> diff --git a/read-cache.c b/read-cache.c
> index f5d4385c408..c079ece981a 100644
> --- a/read-cache.c
> +++ b/read-cache.c
> @@ -68,6 +68,11 @@
> */
> #define CACHE_ENTRY_PATH_LENGTH 80
>
> +enum index_search_mode {
> + NO_EXPAND_SPARSE = 0,
> + EXPAND_SPARSE = 1
> +};
> +
> static inline struct cache_entry *mem_pool__ce_alloc(struct mem_pool *mem_pool, size_t len)
> {
> struct cache_entry *ce;
> @@ -551,7 +556,10 @@ int cache_name_stage_compare(const char *name1, int len1, int stage1, const char
> return 0;
> }
>
> -static int index_name_stage_pos(struct index_state *istate, const char *name, int namelen, int stage)
> +static int index_name_stage_pos(struct index_state *istate,
> + const char *name, int namelen,
> + int stage,
> + enum index_search_mode search_mode)
> {
> int first, last;
>
> @@ -570,7 +578,7 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in
> first = next+1;
> }
>
> - if (istate->sparse_index &&
> + if (search_mode == EXPAND_SPARSE && istate->sparse_index &&
> first > 0) {
> /* Note: first <= istate->cache_nr */
> struct cache_entry *ce = istate->cache[first - 1];
> @@ -586,7 +594,7 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in
> ce_namelen(ce) < namelen &&
> !strncmp(name, ce->name, ce_namelen(ce))) {
> ensure_full_index(istate);
> - return index_name_stage_pos(istate, name, namelen, stage);
> + return index_name_stage_pos(istate, name, namelen, stage, search_mode);
> }
> }
>
> @@ -595,7 +603,12 @@ static int index_name_stage_pos(struct index_state *istate, const char *name, in
>
> int index_name_pos(struct index_state *istate, const char *name, int namelen)
> {
> - return index_name_stage_pos(istate, name, namelen, 0);
> + return index_name_stage_pos(istate, name, namelen, 0, EXPAND_SPARSE);
> +}
> +
> +int index_entry_exists(struct index_state *istate, const char *name, int namelen)
> +{
> + return index_name_stage_pos(istate, name, namelen, 0, NO_EXPAND_SPARSE) >= 0;
> }
>
> int remove_index_entry_at(struct index_state *istate, int pos)
> @@ -1222,7 +1235,7 @@ static int has_dir_name(struct index_state *istate,
> */
> }
>
> - pos = index_name_stage_pos(istate, name, len, stage);
> + pos = index_name_stage_pos(istate, name, len, stage, EXPAND_SPARSE);
> if (pos >= 0) {
> /*
> * Found one, but not so fast. This could
> @@ -1322,7 +1335,7 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e
> strcmp(ce->name, istate->cache[istate->cache_nr - 1]->name) > 0)
> pos = index_pos_to_insert_pos(istate->cache_nr);
> else
> - pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce));
> + pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce), EXPAND_SPARSE);
>
> /* existing match? Just replace it. */
> if (pos >= 0) {
> @@ -1357,7 +1370,7 @@ static int add_index_entry_with_check(struct index_state *istate, struct cache_e
> if (!ok_to_replace)
> return error(_("'%s' appears as both a file and as a directory"),
> ce->name);
> - pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce));
> + pos = index_name_stage_pos(istate, ce->name, ce_namelen(ce), ce_stage(ce), EXPAND_SPARSE);
> pos = -pos-1;
> }
> return pos + 1;
> diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh
> index 875cdcb0495..4ac93874cb2 100755
> --- a/t/t1092-sparse-checkout-compatibility.sh
> +++ b/t/t1092-sparse-checkout-compatibility.sh
> @@ -756,9 +756,9 @@ test_expect_success 'sparse-index is not expanded' '
> ensure_not_expanded checkout - &&
> ensure_not_expanded switch rename-out-to-out &&
> ensure_not_expanded switch - &&
> - git -C sparse-index reset --hard &&
> + ensure_not_expanded reset --hard &&
> ensure_not_expanded checkout rename-out-to-out -- deep/deeper1 &&
> - git -C sparse-index reset --hard &&
> + ensure_not_expanded reset --hard &&
> ensure_not_expanded restore -s rename-out-to-out -- deep/deeper1 &&
>
> echo >>sparse-index/README.md &&
> @@ -768,6 +768,17 @@ test_expect_success 'sparse-index is not expanded' '
> echo >>sparse-index/untracked.txt &&
> ensure_not_expanded add . &&
>
> + for ref in update-deep update-folder1 update-folder2 update-deep
> + do
> + echo >>sparse-index/README.md &&
> + ensure_not_expanded reset --hard $ref || return 1
> + done &&
> +
> + ensure_not_expanded reset --hard update-deep &&
> + ensure_not_expanded reset --keep base &&
> + ensure_not_expanded reset --merge update-deep &&
> + ensure_not_expanded reset --hard &&
> +
> ensure_not_expanded checkout -f update-deep &&
> test_config -C sparse-index pull.twohead ort &&
> (
>
next prev parent reply other threads:[~2021-10-08 11:09 UTC|newest]
Thread overview: 114+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-30 14:50 [PATCH 0/7] Sparse Index: integrate with reset Victoria Dye via GitGitGadget
2021-09-30 14:50 ` [PATCH 1/7] reset: behave correctly with sparse-checkout Kevin Willford via GitGitGadget
2021-09-30 18:34 ` Junio C Hamano
2021-10-01 14:55 ` Victoria Dye
2021-09-30 14:50 ` [PATCH 2/7] sparse-index: update command for expand/collapse test Victoria Dye via GitGitGadget
2021-09-30 19:17 ` Taylor Blau
2021-09-30 20:11 ` Victoria Dye
2021-09-30 21:32 ` Junio C Hamano
2021-09-30 22:59 ` Victoria Dye
2021-10-01 0:04 ` Junio C Hamano
2021-10-04 13:47 ` Victoria Dye
2021-10-01 9:14 ` Bagas Sanjaya
2021-09-30 14:50 ` [PATCH 3/7] reset: expand test coverage for sparse checkouts Victoria Dye via GitGitGadget
2021-09-30 14:50 ` [PATCH 4/7] reset: integrate with sparse index Victoria Dye via GitGitGadget
2021-09-30 14:50 ` [PATCH 5/7] reset: make sparse-aware (except --mixed) Victoria Dye via GitGitGadget
2021-09-30 14:51 ` [PATCH 6/7] reset: make --mixed sparse-aware Victoria Dye via GitGitGadget
2021-10-01 15:03 ` Victoria Dye
2021-09-30 14:51 ` [PATCH 7/7] unpack-trees: improve performance of next_cache_entry Victoria Dye via GitGitGadget
2021-10-05 13:20 ` [PATCH v2 0/7] Sparse Index: integrate with reset Victoria Dye via GitGitGadget
2021-10-05 13:20 ` [PATCH v2 1/7] reset: behave correctly with sparse-checkout Kevin Willford via GitGitGadget
2021-10-05 19:30 ` Junio C Hamano
2021-10-05 21:59 ` Victoria Dye
2021-10-06 12:44 ` Junio C Hamano
2021-10-06 1:46 ` Elijah Newren
2021-10-06 20:09 ` Victoria Dye
2021-10-06 10:31 ` Bagas Sanjaya
2021-10-05 13:20 ` [PATCH v2 2/7] update-index: add --force-full-index option for expand/collapse test Victoria Dye via GitGitGadget
2021-10-06 2:00 ` Elijah Newren
2021-10-06 20:40 ` Victoria Dye
2021-10-08 3:42 ` Elijah Newren
2021-10-08 17:11 ` Junio C Hamano
2021-10-06 10:33 ` Bagas Sanjaya
2021-10-05 13:20 ` [PATCH v2 3/7] reset: expand test coverage for sparse checkouts Victoria Dye via GitGitGadget
2021-10-06 2:04 ` Elijah Newren
2021-10-05 13:20 ` [PATCH v2 4/7] reset: integrate with sparse index Victoria Dye via GitGitGadget
2021-10-06 2:15 ` Elijah Newren
2021-10-06 17:48 ` Junio C Hamano
2021-10-05 13:20 ` [PATCH v2 5/7] reset: make sparse-aware (except --mixed) Victoria Dye via GitGitGadget
2021-10-06 3:43 ` Elijah Newren
2021-10-06 20:56 ` Victoria Dye
2021-10-06 10:34 ` Bagas Sanjaya
2021-10-05 13:20 ` [PATCH v2 6/7] reset: make --mixed sparse-aware Victoria Dye via GitGitGadget
2021-10-06 4:43 ` Elijah Newren
2021-10-07 14:34 ` Victoria Dye
2021-10-05 13:20 ` [PATCH v2 7/7] unpack-trees: improve performance of next_cache_entry Victoria Dye via GitGitGadget
2021-10-06 10:37 ` Bagas Sanjaya
2021-10-05 15:34 ` [PATCH v2 0/7] Sparse Index: integrate with reset Ævar Arnfjörð Bjarmason
2021-10-05 20:44 ` Victoria Dye
2021-10-06 5:46 ` Elijah Newren
2021-10-07 21:15 ` [PATCH v3 0/8] " Victoria Dye via GitGitGadget
2021-10-07 21:15 ` [PATCH v3 1/8] reset: rename is_missing to !is_in_reset_tree Victoria Dye via GitGitGadget
2021-10-07 21:15 ` [PATCH v3 2/8] reset: preserve skip-worktree bit in mixed reset Kevin Willford via GitGitGadget
2021-10-08 9:04 ` Junio C Hamano
2021-10-07 21:15 ` [PATCH v3 3/8] update-index: add --force-full-index option for expand/collapse test Victoria Dye via GitGitGadget
2021-10-08 2:50 ` Bagas Sanjaya
2021-10-08 5:24 ` Junio C Hamano
2021-10-08 15:47 ` Victoria Dye
2021-10-08 17:19 ` Junio C Hamano
2021-10-11 14:12 ` Derrick Stolee
2021-10-11 15:05 ` Victoria Dye
2021-10-11 15:24 ` Junio C Hamano
2021-10-07 21:15 ` [PATCH v3 4/8] reset: expand test coverage for sparse checkouts Victoria Dye via GitGitGadget
2021-10-07 21:15 ` [PATCH v3 5/8] reset: integrate with sparse index Victoria Dye via GitGitGadget
2021-10-07 21:15 ` [PATCH v3 6/8] reset: make sparse-aware (except --mixed) Victoria Dye via GitGitGadget
2021-10-08 11:09 ` Phillip Wood [this message]
2021-10-08 17:14 ` Victoria Dye
2021-10-08 18:31 ` Junio C Hamano
2021-10-09 11:18 ` Phillip Wood
2021-10-10 22:03 ` Junio C Hamano
2021-10-11 15:55 ` Victoria Dye
2021-10-11 16:16 ` Junio C Hamano
2021-10-12 10:16 ` Phillip Wood
2021-10-12 19:15 ` Junio C Hamano
2021-10-12 10:17 ` Phillip Wood
2021-10-07 21:15 ` [PATCH v3 7/8] reset: make --mixed sparse-aware Victoria Dye via GitGitGadget
2021-11-20 22:02 ` Elijah Newren
2021-11-22 16:46 ` Victoria Dye
2021-10-07 21:15 ` [PATCH v3 8/8] unpack-trees: improve performance of next_cache_entry Victoria Dye via GitGitGadget
2021-10-11 20:30 ` [PATCH v4 0/8] Sparse Index: integrate with reset Victoria Dye via GitGitGadget
2021-10-11 20:30 ` [PATCH v4 1/8] reset: rename is_missing to !is_in_reset_tree Victoria Dye via GitGitGadget
2021-10-11 20:30 ` [PATCH v4 2/8] reset: preserve skip-worktree bit in mixed reset Victoria Dye via GitGitGadget
2021-10-22 4:19 ` Emily Shaffer
2021-10-25 15:59 ` Victoria Dye
2021-10-11 20:30 ` [PATCH v4 3/8] sparse-index: update command for expand/collapse test Victoria Dye via GitGitGadget
2021-10-11 20:30 ` [PATCH v4 4/8] reset: expand test coverage for sparse checkouts Victoria Dye via GitGitGadget
2021-10-11 20:30 ` [PATCH v4 5/8] reset: integrate with sparse index Victoria Dye via GitGitGadget
2021-10-11 20:30 ` [PATCH v4 6/8] reset: make sparse-aware (except --mixed) Victoria Dye via GitGitGadget
2021-10-11 20:30 ` [PATCH v4 7/8] reset: make --mixed sparse-aware Victoria Dye via GitGitGadget
2021-10-11 20:30 ` [PATCH v4 8/8] unpack-trees: improve performance of next_cache_entry Victoria Dye via GitGitGadget
2021-10-27 14:39 ` [PATCH v5 0/8] Sparse Index: integrate with reset Victoria Dye via GitGitGadget
2021-10-27 14:39 ` [PATCH v5 1/8] reset: rename is_missing to !is_in_reset_tree Victoria Dye via GitGitGadget
2021-10-27 14:39 ` [PATCH v5 2/8] reset: preserve skip-worktree bit in mixed reset Victoria Dye via GitGitGadget
2021-10-27 14:39 ` [PATCH v5 3/8] sparse-index: update command for expand/collapse test Victoria Dye via GitGitGadget
2021-10-27 14:39 ` [PATCH v5 4/8] reset: expand test coverage for sparse checkouts Victoria Dye via GitGitGadget
2021-10-27 14:39 ` [PATCH v5 5/8] reset: integrate with sparse index Victoria Dye via GitGitGadget
2021-10-27 14:39 ` [PATCH v5 6/8] reset: make sparse-aware (except --mixed) Victoria Dye via GitGitGadget
2021-10-27 14:39 ` [PATCH v5 7/8] reset: make --mixed sparse-aware Victoria Dye via GitGitGadget
2021-11-20 22:05 ` Elijah Newren
2021-11-22 16:54 ` Victoria Dye
2021-10-27 14:39 ` [PATCH v5 8/8] unpack-trees: improve performance of next_cache_entry Victoria Dye via GitGitGadget
2021-11-20 22:13 ` [PATCH v5 0/8] Sparse Index: integrate with reset Elijah Newren
2021-11-29 15:52 ` [PATCH v6 " Victoria Dye via GitGitGadget
2021-11-29 15:52 ` [PATCH v6 1/8] reset: rename is_missing to !is_in_reset_tree Victoria Dye via GitGitGadget
2021-11-29 15:52 ` [PATCH v6 2/8] reset: preserve skip-worktree bit in mixed reset Victoria Dye via GitGitGadget
2021-11-29 15:52 ` [PATCH v6 3/8] sparse-index: update command for expand/collapse test Victoria Dye via GitGitGadget
2021-11-29 15:52 ` [PATCH v6 4/8] reset: expand test coverage for sparse checkouts Victoria Dye via GitGitGadget
2021-11-29 15:52 ` [PATCH v6 5/8] reset: integrate with sparse index Victoria Dye via GitGitGadget
2021-11-29 15:52 ` [PATCH v6 6/8] reset: make sparse-aware (except --mixed) Victoria Dye via GitGitGadget
2021-11-29 15:52 ` [PATCH v6 7/8] reset: make --mixed sparse-aware Victoria Dye via GitGitGadget
2021-11-29 15:52 ` [PATCH v6 8/8] unpack-trees: improve performance of next_cache_entry Victoria Dye via GitGitGadget
2021-11-29 18:37 ` [PATCH v6 0/8] Sparse Index: integrate with reset Elijah Newren
2021-11-29 19:44 ` Victoria Dye
2021-11-30 3:59 ` Elijah Newren
2021-12-01 20:26 ` Ævar Arnfjörð Bjarmason
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c1dd1fef-2d48-cc18-5786-10e174b487a7@gmail.com \
--to=phillip.wood123@gmail.com \
--cc=avarab@gmail.com \
--cc=bagasdotme@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitgitgadget@gmail.com \
--cc=gitster@pobox.com \
--cc=me@ttaylorr.com \
--cc=newren@gmail.com \
--cc=phillip.wood@dunelm.org.uk \
--cc=stolee@gmail.com \
--cc=vdye@github.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).