git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
From: Elijah Newren <newren@gmail.com>
To: Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Cc: Git Mailing List <git@vger.kernel.org>,
	Junio C Hamano <gitster@pobox.com>,
	Derrick Stolee <dstolee@microsoft.com>
Subject: Re: [PATCH v3 15/17] sparse-checkout: update working directory in-process
Date: Sat, 12 Oct 2019 15:57:14 -0700
Message-ID: <CABPp-BF+JWYZfDqp2Tn4AEKVp4b0YMA=Mbz4Nz62D-gGgiduYQ@mail.gmail.com> (raw)
In-Reply-To: <a6f17e9a77d86f8ec856ea08617d1c1af2853d54.1570478905.git.gitgitgadget@gmail.com>

On Mon, Oct 7, 2019 at 1:08 PM Derrick Stolee via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> From: Derrick Stolee <dstolee@microsoft.com>
>
> The sparse-checkout builtin used 'git read-tree -mu HEAD' to update the
> skip-worktree bits in the index and to update the working directory.
> This extra process is overly complex, and prone to failure. It also
> requires that we write our changes to the sparse-checkout file before
> trying to update the index.
>
> Remove this extra process call by creating a direct call to
> unpack_trees() in the same way 'git read-tree -mu HEAD' does. In
> adition, provide an in-memory list of patterns so we can avoid

s/adition/addition/

> reading from the sparse-checkout file. This allows us to test a
> proposed change to the file before writing to it.
>
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
>  builtin/read-tree.c                |  2 +-
>  builtin/sparse-checkout.c          | 85 +++++++++++++++++++++++++-----
>  t/t1091-sparse-checkout-builtin.sh | 17 ++++++
>  unpack-trees.c                     |  5 +-
>  unpack-trees.h                     |  3 +-
>  5 files changed, 95 insertions(+), 17 deletions(-)
>
> diff --git a/builtin/read-tree.c b/builtin/read-tree.c
> index 69963d83dc..d7eeaa26ec 100644
> --- a/builtin/read-tree.c
> +++ b/builtin/read-tree.c
> @@ -186,7 +186,7 @@ int cmd_read_tree(int argc, const char **argv, const char *cmd_prefix)
>
>         if (opts.reset || opts.merge || opts.prefix) {
>                 if (read_cache_unmerged() && (opts.prefix || opts.merge))
> -                       die("You need to resolve your current index first");
> +                       die(_("You need to resolve your current index first"));

A good change, but isn't this unrelated to the current commit?

>                 stage = opts.merge = 1;
>         }
>         resolve_undo_clear();
> diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c
> index 25786f8bb0..542d57fac6 100644
> --- a/builtin/sparse-checkout.c
> +++ b/builtin/sparse-checkout.c
> @@ -7,6 +7,11 @@
>  #include "run-command.h"
>  #include "strbuf.h"
>  #include "string-list.h"
> +#include "cache.h"
> +#include "cache-tree.h"
> +#include "lockfile.h"
> +#include "resolve-undo.h"
> +#include "unpack-trees.h"
>
>  static char const * const builtin_sparse_checkout_usage[] = {
>         N_("git sparse-checkout [init|list|set|disable] <options>"),
> @@ -60,18 +65,53 @@ static int sparse_checkout_list(int argc, const char **argv)
>         return 0;
>  }
>
> -static int update_working_directory(void)
> +static int update_working_directory(struct pattern_list *pl)
>  {
> -       struct argv_array argv = ARGV_ARRAY_INIT;
>         int result = 0;
> -       argv_array_pushl(&argv, "read-tree", "-m", "-u", "HEAD", NULL);
> +       struct unpack_trees_options o;
> +       struct lock_file lock_file = LOCK_INIT;
> +       struct object_id oid;
> +       struct tree *tree;
> +       struct tree_desc t;
> +       struct repository *r = the_repository;
>
> -       if (run_command_v_opt(argv.argv, RUN_GIT_CMD)) {
> -               error(_("failed to update index with new sparse-checkout paths"));
> -               result = 1;
> +       if (repo_read_index_unmerged(r))
> +               die(_("You need to resolve your current index first"));

Well, at least that ensures that the user gets a good error message.
I'm not sure I like the error, because e.g. if a user hits a conflict
while merging in a sparse checkout and wants to return to a non-sparse
checkout because they think other files might help them resolve the
conflicts, then they ought to be able to do it.  Basically, unless
they are trying use sparsification to remove entries from the working
directory that differ from the index (and conflicted entries always
differ), then it seems like we should be able to support
sparsification despite the presence of conflicts.

Your series is long enough, doesn't make this problem any worse (and
appears to make it slightly better), and so you really don't need to
tackle that problem in this series. I'm just stating a gripe with
sparse checkouts again.  :-)

[...]

>  static void insert_recursive_pattern(struct pattern_list *pl, struct strbuf *path)
>  {
> -       struct pattern_entry *e = xmalloc(sizeof(struct pattern_entry));
> +       struct pattern_entry *e = xmalloc(sizeof(*e));

This is a good fix, but shouldn't it be squashed into the
"sparse-checkout: init and set in cone mode" commit from earlier in
your series?

> @@ -262,12 +308,21 @@ static int write_patterns_and_update(struct pattern_list *pl)
>  {
>         char *sparse_filename;
>         FILE *fp;
> -
> +       int result;
> +

Trailing whitespace that should be cleaned up.

>         if (!core_apply_sparse_checkout) {
>                 warning(_("core.sparseCheckout is disabled, so changes to the sparse-checkout file will have no effect"));
>                 warning(_("run 'git sparse-checkout init' to enable the sparse-checkout feature"));
>         }
>
> +       result = update_working_directory(pl);
> +
> +       if (result) {
> +               clear_pattern_list(pl);
> +               update_working_directory(NULL);
> +               return result;
> +       }
> +
>         sparse_filename = get_sparse_checkout_filename();
>         fp = fopen(sparse_filename, "w");
>
> @@ -277,9 +332,11 @@ static int write_patterns_and_update(struct pattern_list *pl)
>                 write_patterns_to_file(fp, pl);
>
>         fclose(fp);
> +
>         free(sparse_filename);
> +       clear_pattern_list(pl);
>
> -       return update_working_directory();
> +       return 0;
>  }
>
>  static void strbuf_to_cone_pattern(struct strbuf *line, struct pattern_list *pl)
> @@ -330,6 +387,7 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix)
>                 struct strbuf line = STRBUF_INIT;
>                 hashmap_init(&pl.recursive_hashmap, pl_hashmap_cmp, NULL, 0);
>                 hashmap_init(&pl.parent_hashmap, pl_hashmap_cmp, NULL, 0);
> +               pl.use_cone_patterns = 1;
>
>                 if (set_opts.use_stdin) {
>                         while (!strbuf_getline(&line, stdin))
> @@ -375,7 +433,8 @@ static int sparse_checkout_disable(int argc, const char **argv)
>         fprintf(fp, "/*\n");
>         fclose(fp);
>
> -       if (update_working_directory())
> +       core_apply_sparse_checkout = 1;
> +       if (update_working_directory(NULL))
>                 die(_("error while refreshing working directory"));
>
>         unlink(sparse_filename);
> diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh
> index ee4d361787..82eb5fb2f8 100755
> --- a/t/t1091-sparse-checkout-builtin.sh
> +++ b/t/t1091-sparse-checkout-builtin.sh
> @@ -199,11 +199,13 @@ test_expect_success 'cone mode: init and set' '
>                 a
>                 deep
>         EOF
> +       test_cmp dir expect &&
>         ls repo/deep >dir  &&
>         cat >expect <<-EOF &&
>                 a
>                 deeper1
>         EOF
> +       test_cmp dir expect &&
>         ls repo/deep/deeper1 >dir  &&
>         cat >expect <<-EOF &&
>                 a
> @@ -245,4 +247,19 @@ test_expect_success 'cone mode: set with nested folders' '
>         test_cmp repo/.git/info/sparse-checkout expect
>  '
>
> +test_expect_success 'revert to old sparse-checkout on bad update' '
> +       echo update >repo/deep/deeper2/a &&
> +       cp repo/.git/info/sparse-checkout expect &&
> +       test_must_fail git -C repo sparse-checkout set deep/deeper1 2>err &&
> +       test_i18ngrep "Cannot update sparse checkout" err &&
> +       test_cmp repo/.git/info/sparse-checkout expect &&
> +       ls repo/deep >dir &&
> +       cat >expect <<-EOF &&
> +               a
> +               deeper1
> +               deeper2
> +       EOF
> +       test_cmp dir expect
> +'
> +
>  test_done
> diff --git a/unpack-trees.c b/unpack-trees.c
> index edf0fb4673..f0fee5adf2 100644
> --- a/unpack-trees.c
> +++ b/unpack-trees.c
> @@ -1508,7 +1508,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options
>         memset(&pl, 0, sizeof(pl));
>         if (!core_apply_sparse_checkout || !o->update)
>                 o->skip_sparse_checkout = 1;
> -       if (!o->skip_sparse_checkout) {
> +       if (!o->skip_sparse_checkout && !o->pl) {
>                 char *sparse = git_pathdup("info/sparse-checkout");
>                 pl.use_cone_patterns = core_sparse_checkout_cone;
>                 if (add_patterns_from_file_to_list(sparse, "", 0, &pl, NULL) < 0)
> @@ -1681,7 +1681,8 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options
>
>  done:
>         trace_performance_leave("unpack_trees");
> -       clear_pattern_list(&pl);
> +       if (!o->keep_pattern_list)
> +               clear_pattern_list(&pl);
>         return ret;
>
>  return_failed:
> diff --git a/unpack-trees.h b/unpack-trees.h
> index f2eee0c7c5..ca94a421a5 100644
> --- a/unpack-trees.h
> +++ b/unpack-trees.h
> @@ -59,7 +59,8 @@ struct unpack_trees_options {
>                      quiet,
>                      exiting_early,
>                      show_all_errors,
> -                    dry_run;
> +                    dry_run,
> +                    keep_pattern_list;
>         const char *prefix;
>         int cache_bottom;
>         struct dir_struct *dir;
> --

The rest looks reasonable.

  reply index

Thread overview: 93+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-20 15:11 [PATCH 0/9] [RFC] New sparse-checkout builtin and "cone" mode Derrick Stolee via GitGitGadget
2019-08-20 15:11 ` [PATCH 1/9] sparse-checkout: create builtin with 'list' subcommand Derrick Stolee via GitGitGadget
2019-08-23 22:30   ` Elijah Newren
2019-08-20 15:11 ` [PATCH 2/9] sparse-checkout: create 'init' subcommand Derrick Stolee via GitGitGadget
2019-08-23 23:02   ` Elijah Newren
2019-09-11 14:27     ` Derrick Stolee
2019-09-11 20:28     ` Derrick Stolee
2019-08-20 15:11 ` [PATCH 3/9] clone: add --sparse mode Derrick Stolee via GitGitGadget
2019-08-23 23:17   ` Elijah Newren
2019-09-18 13:51     ` Derrick Stolee
2019-08-20 15:11 ` [PATCH 4/9] sparse-checkout: 'add' subcommand Derrick Stolee via GitGitGadget
2019-08-23 23:30   ` Elijah Newren
2019-09-18 13:55     ` Derrick Stolee
2019-09-18 14:56       ` Elijah Newren
2019-09-18 17:23         ` Derrick Stolee
2019-08-20 15:11 ` [PATCH 5/9] sparse-checkout: create 'disable' subcommand Derrick Stolee via GitGitGadget
2019-08-23 23:50   ` Elijah Newren
2019-08-20 15:11 ` [PATCH 6/9] trace2:experiment: clear_ce_flags_1 Jeff Hostetler via GitGitGadget
2019-08-24  0:08   ` Elijah Newren
2019-08-20 15:11 ` [PATCH 7/9] sparse-checkout: add 'cone' mode Derrick Stolee via GitGitGadget
2019-08-24  0:31   ` Elijah Newren
2019-08-20 15:11 ` [PATCH 8/9] sparse-checkout: use hashmaps for cone patterns Derrick Stolee via GitGitGadget
2019-08-24  4:56   ` Elijah Newren
2019-08-20 15:11 ` [PATCH 9/9] sparse-checkout: init and add in cone mode Derrick Stolee via GitGitGadget
2019-08-24  5:07   ` Elijah Newren
2019-08-21 21:52 ` [PATCH 0/9] [RFC] New sparse-checkout builtin and "cone" mode Elijah Newren
2019-08-22 13:10   ` Derrick Stolee
2019-08-22 14:25     ` Derrick Stolee
2019-08-24  5:40     ` Elijah Newren
2019-08-26 13:29       ` Derrick Stolee
2019-08-26 18:16         ` Elijah Newren
2019-08-26 19:16           ` Derrick Stolee
2019-09-02 17:55       ` Eric Sunshine
2019-09-19 14:43 ` [PATCH v2 00/11] " Derrick Stolee via GitGitGadget
2019-09-19 14:43   ` [PATCH v2 01/11] sparse-checkout: create builtin with 'list' subcommand Derrick Stolee via GitGitGadget
2019-10-05 19:22     ` Elijah Newren
2019-09-19 14:43   ` [PATCH v2 02/11] sparse-checkout: create 'init' subcommand Derrick Stolee via GitGitGadget
2019-10-05 19:34     ` Elijah Newren
2019-09-19 14:43   ` [PATCH v2 03/11] clone: add --sparse mode Derrick Stolee via GitGitGadget
2019-10-05 19:40     ` Elijah Newren
2019-10-07 13:56       ` Derrick Stolee
2019-09-19 14:43   ` [PATCH v2 04/11] sparse-checkout: 'set' subcommand Derrick Stolee via GitGitGadget
2019-10-05 22:44     ` Elijah Newren
2019-10-06  0:30       ` Elijah Newren
2019-10-07 18:26         ` Derrick Stolee
2019-10-11 22:24           ` Elijah Newren
2019-09-19 14:43   ` [PATCH v2 05/11] sparse-checkout: add '--stdin' option to set subcommand Derrick Stolee via GitGitGadget
2019-09-19 14:43   ` [PATCH v2 06/11] sparse-checkout: create 'disable' subcommand Derrick Stolee via GitGitGadget
2019-10-06  4:10     ` Elijah Newren
2019-10-07 19:12       ` Derrick Stolee
2019-09-19 14:43   ` [PATCH v2 07/11] trace2: add region in clear_ce_flags Jeff Hostetler via GitGitGadget
2019-10-06  4:13     ` Elijah Newren
2019-09-19 14:43   ` [PATCH v2 08/11] sparse-checkout: add 'cone' mode Derrick Stolee via GitGitGadget
2019-10-06  4:22     ` Elijah Newren
2019-10-07 19:15       ` Derrick Stolee
2019-09-19 14:43   ` [PATCH v2 09/11] sparse-checkout: use hashmaps for cone patterns Derrick Stolee via GitGitGadget
2019-09-19 20:59     ` Derrick Stolee
2019-09-20 14:37       ` Derrick Stolee
2019-09-19 14:43   ` [PATCH v2 10/11] sparse-checkout: init and set in cone mode Derrick Stolee via GitGitGadget
2019-09-19 14:43   ` [PATCH v2 11/11] unpack-trees: hash less " Derrick Stolee via GitGitGadget
2019-10-01 13:40   ` [PATCH v2 00/11] New sparse-checkout builtin and "cone" mode Derrick Stolee
2019-10-01 16:54     ` Elijah Newren
2019-10-01 18:15       ` Derrick Stolee
2019-10-03 22:28     ` Junio C Hamano
2019-10-07 20:08   ` [PATCH v3 00/17] " Derrick Stolee via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 01/17] sparse-checkout: create builtin with 'list' subcommand Derrick Stolee via GitGitGadget
2019-10-11 22:01       ` Elijah Newren
2019-10-07 20:08     ` [PATCH v3 02/17] sparse-checkout: create 'init' subcommand Derrick Stolee via GitGitGadget
2019-10-11 22:14       ` Elijah Newren
2019-10-07 20:08     ` [PATCH v3 03/17] clone: add --sparse mode Derrick Stolee via GitGitGadget
2019-10-11 22:20       ` Elijah Newren
2019-10-07 20:08     ` [PATCH v3 04/17] sparse-checkout: 'set' subcommand Derrick Stolee via GitGitGadget
2019-10-11 22:26       ` Elijah Newren
2019-10-11 22:30         ` Elijah Newren
2019-10-07 20:08     ` [PATCH v3 05/17] sparse-checkout: add '--stdin' option to set subcommand Derrick Stolee via GitGitGadget
2019-10-11 22:27       ` Elijah Newren
2019-10-07 20:08     ` [PATCH v3 06/17] sparse-checkout: create 'disable' subcommand Derrick Stolee via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 07/17] trace2: add region in clear_ce_flags Jeff Hostetler via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 08/17] sparse-checkout: add 'cone' mode Derrick Stolee via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 09/17] sparse-checkout: use hashmaps for cone patterns Derrick Stolee via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 10/17] sparse-checkout: init and set in cone mode Derrick Stolee via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 11/17] unpack-trees: hash less " Derrick Stolee via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 13/17] read-tree: show progress by default Derrick Stolee via GitGitGadget
2019-10-12 22:16       ` Elijah Newren
2019-10-07 20:08     ` [PATCH v3 12/17] unpack-trees: add progress to clear_ce_flags() Derrick Stolee via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 14/17] sparse-checkout: sanitize for nested folders Derrick Stolee via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 15/17] sparse-checkout: update working directory in-process Derrick Stolee via GitGitGadget
2019-10-12 22:57       ` Elijah Newren [this message]
2019-10-07 20:08     ` [PATCH v3 16/17] sparse-checkout: write using lockfile Derrick Stolee via GitGitGadget
2019-10-12 22:59       ` Elijah Newren
2019-10-07 20:08     ` [PATCH v3 17/17] sparse-checkout: cone mode should not interact with .gitignore Derrick Stolee via GitGitGadget
2019-10-12 23:00       ` Elijah Newren
2019-10-12 23:22     ` [PATCH v3 00/17] New sparse-checkout builtin and "cone" mode Elijah Newren

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CABPp-BF+JWYZfDqp2Tn4AEKVp4b0YMA=Mbz4Nz62D-gGgiduYQ@mail.gmail.com' \
    --to=newren@gmail.com \
    --cc=dstolee@microsoft.com \
    --cc=git@vger.kernel.org \
    --cc=gitgitgadget@gmail.com \
    --cc=gitster@pobox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror http://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox