git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
From: Derrick Stolee <stolee@gmail.com>
To: Elijah Newren <newren@gmail.com>,
	Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Cc: Git Mailing List <git@vger.kernel.org>,
	Junio C Hamano <gitster@pobox.com>,
	Derrick Stolee <dstolee@microsoft.com>
Subject: Re: [PATCH 2/9] sparse-checkout: create 'init' subcommand
Date: Wed, 11 Sep 2019 16:28:45 -0400
Message-ID: <ab98c292-a1b1-ff47-2e8e-f74b7ade2b94@gmail.com> (raw)
In-Reply-To: <CABPp-BEmwSwg4tgJg6nVG8a3Hpn_g-=ZjApZF4EiJO+qVgu4uw@mail.gmail.com>

On 8/23/2019 7:02 PM, Elijah Newren wrote:
> On Tue, Aug 20, 2019 at 8:13 AM Derrick Stolee via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>> +static int sc_read_tree(void)
>> +{
>> +       struct argv_array argv = ARGV_ARRAY_INIT;
>> +       int result = 0;
>> +       argv_array_pushl(&argv, "read-tree", "-m", "-u", "HEAD", NULL);
>> +
>> +       if (run_command_v_opt(argv.argv, RUN_GIT_CMD)) {
>> +               error(_("failed to update index with new sparse-checkout paths"));
>> +               result = 1;
>> +       }
> 
> `git read-tree -m -u HEAD` will fail if the index has any higher stage
> entries in it, even if those higher stage entries correspond to files
> which are included in the sparseness patterns and thus would not need
> an update.  It might be nice if we can find a way to provide a better
> error message, and/or implement the read-tree -m -u HEAD internally in
> a way that will allow us to not fail if the conflicted files are
> included in the sparse set.

I agree that this is not the _best_ thing to do, but it does mimic the
current recommendation for a user interacting with sparse-checkout.

I'll rename this helper to something like "update_working_directory()"
so it can be swapped with a different implementation later, after we
work out those usability kinks.

The other thing that is needed here: allow reverting the sparse-checkout
settings if this fails. I'll isolate that to a new commit so we can
examine that behavior carefully.

> 
>> +
>> +       argv_array_clear(&argv);
>> +       return result;
>> +}
>> +
>> +static int sc_enable_config(void)
>> +{
>> +       struct argv_array argv = ARGV_ARRAY_INIT;
>> +       int result = 0;
>> +       argv_array_pushl(&argv, "config", "--add", "core.sparseCheckout", "true", NULL);
> > Why --add?  That seems really odd to me.

Yeah, that's a mistake. Good find.

> 
> This should also have "--worktree".  And this function should either
> set extensions.worktreeConfig to true or die if it isn't already set;
> not sure which.  There's some UI and documentation stuff to figure out
> here...

I was planning to switch my `git config` subcommand to use in-process
methods, but I'm struggling to find a way to ensure we follow the
`--worktree` option. It likely would work if extensions.worktreeConfig
was enabled when the process starts, but adding it in-process likely
causes a problem.

> 
>> +
>> +       if (run_command_v_opt(argv.argv, RUN_GIT_CMD)) {
>> +               error(_("failed to enable core.sparseCheckout"));
>> +               result = 1;
>> +       }
>> +
>> +       argv_array_clear(&argv);
>> +       return result;
>> +}
>> +
>> +static int delete_directory(const struct object_id *oid, struct strbuf *base,
>> +               const char *pathname, unsigned mode, int stage, void *context)
>> +{
>> +       struct strbuf dirname = STRBUF_INIT;
>> +       struct stat sb;
>> +
>> +       strbuf_addstr(&dirname, the_repository->worktree);
>> +       strbuf_addch(&dirname, '/');
>> +       strbuf_addstr(&dirname, pathname);
>> +
>> +       if (stat(dirname.buf, &sb) || !(sb.st_mode & S_IFDIR))
>> +               return 0;
>> +
>> +       if (remove_dir_recursively(&dirname, 0))
> 
> flags = 0 implies not REMOVE_DIR_EMPTY_ONLY.  I'm not familiar with
> remove_dir_recursively(), but won't this delete everything...including
> untracked files?  If so, that sounds like a bug.
This whole thing isn't needed any more, since read-tree does the right
thing.

> 
>> +               warning(_("failed to remove directory '%s'"),
>> +                       dirname.buf);
>> +
>> +       strbuf_release(&dirname);
>> +       return 0;
>> +}
>> +
>> +static int sparse_checkout_init(int argc, const char **argv)
>> +{
>> +       struct tree *t;
>> +       struct object_id oid;
>> +       struct exclude_list el;
>> +       static struct pathspec pathspec;
>> +       char *sparse_filename;
>> +       FILE *fp;
>> +       int res;
>> +
>> +       if (sc_enable_config())
>> +               return 1;
>> +
>> +       memset(&el, 0, sizeof(el));
>> +
>> +       sparse_filename = get_sparse_checkout_filename();
>> +       res = add_excludes_from_file_to_list(sparse_filename, "", 0, &el, NULL);
> 
> But 'el' isn't used again?  Why are we getting the list of files from
> sparse_filename then?

This is the only way I could think to check that the sparse-checkout file parses well without just doing the file open myself. Maybe we only need to check if the file exists (and is not empty).

>> +
>> +       /* If we already have a sparse-checkout file, use it. */
>> +       if (res >= 0) {
>> +               free(sparse_filename);
>> +               goto reset_dir;
>> +       }
>> +
>> +       /* initial mode: all blobs at root */
>> +       fp = fopen(sparse_filename, "w");
>> +       free(sparse_filename);
>> +       fprintf(fp, "/*\n!/*/*\n");
>> +       fclose(fp);
> 
> Makes sense.
> 
>> +
>> +       /* remove all directories in the root, if tracked by Git */
>> +       if (get_oid("HEAD", &oid)) {
>> +               /* assume we are in a fresh repo */
>> +               return 0;
>> +       }
>> +
>> +       t = parse_tree_indirect(&oid);
>> +
>> +       parse_pathspec(&pathspec, PATHSPEC_ALL_MAGIC &
>> +                                 ~(PATHSPEC_FROMTOP | PATHSPEC_LITERAL),
>> +                      PATHSPEC_PREFER_CWD,
>> +                      "", NULL);
>> +
>> +       if (read_tree_recursive(the_repository, t, "", 0, 0, &pathspec,
>> +                               delete_directory, NULL))
>> +               return 1;
> 
> Since this is only needed on Windows, as per your commit message,
> should it be #ifdef'd?  Or is this actually a bug that should be fixed
> in "git read-tree -mu HEAD"?

(this will not be needed, but thanks!)
 
>> +
>> +reset_dir:
>> +       return sc_read_tree();
>> +}
>> +
> 
> The rest looks fine.

Thanks,
-Stolee


  parent reply index

Thread overview: 154+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-20 15:11 [PATCH 0/9] [RFC] New sparse-checkout builtin and "cone" mode Derrick Stolee via GitGitGadget
2019-08-20 15:11 ` [PATCH 1/9] sparse-checkout: create builtin with 'list' subcommand Derrick Stolee via GitGitGadget
2019-08-23 22:30   ` Elijah Newren
2019-08-20 15:11 ` [PATCH 2/9] sparse-checkout: create 'init' subcommand Derrick Stolee via GitGitGadget
2019-08-23 23:02   ` Elijah Newren
2019-09-11 14:27     ` Derrick Stolee
2019-09-11 20:28     ` Derrick Stolee [this message]
2019-08-20 15:11 ` [PATCH 3/9] clone: add --sparse mode Derrick Stolee via GitGitGadget
2019-08-23 23:17   ` Elijah Newren
2019-09-18 13:51     ` Derrick Stolee
2019-08-20 15:11 ` [PATCH 4/9] sparse-checkout: 'add' subcommand Derrick Stolee via GitGitGadget
2019-08-23 23:30   ` Elijah Newren
2019-09-18 13:55     ` Derrick Stolee
2019-09-18 14:56       ` Elijah Newren
2019-09-18 17:23         ` Derrick Stolee
2019-08-20 15:11 ` [PATCH 6/9] trace2:experiment: clear_ce_flags_1 Jeff Hostetler via GitGitGadget
2019-08-24  0:08   ` Elijah Newren
2019-08-20 15:11 ` [PATCH 5/9] sparse-checkout: create 'disable' subcommand Derrick Stolee via GitGitGadget
2019-08-23 23:50   ` Elijah Newren
2019-08-20 15:11 ` [PATCH 7/9] sparse-checkout: add 'cone' mode Derrick Stolee via GitGitGadget
2019-08-24  0:31   ` Elijah Newren
2019-08-20 15:11 ` [PATCH 8/9] sparse-checkout: use hashmaps for cone patterns Derrick Stolee via GitGitGadget
2019-08-24  4:56   ` Elijah Newren
2019-08-20 15:11 ` [PATCH 9/9] sparse-checkout: init and add in cone mode Derrick Stolee via GitGitGadget
2019-08-24  5:07   ` Elijah Newren
2019-08-21 21:52 ` [PATCH 0/9] [RFC] New sparse-checkout builtin and "cone" mode Elijah Newren
2019-08-22 13:10   ` Derrick Stolee
2019-08-22 14:25     ` Derrick Stolee
2019-08-24  5:40     ` Elijah Newren
2019-08-26 13:29       ` Derrick Stolee
2019-08-26 18:16         ` Elijah Newren
2019-08-26 19:16           ` Derrick Stolee
2019-09-02 17:55       ` Eric Sunshine
2019-09-19 14:43 ` [PATCH v2 00/11] " Derrick Stolee via GitGitGadget
2019-09-19 14:43   ` [PATCH v2 01/11] sparse-checkout: create builtin with 'list' subcommand Derrick Stolee via GitGitGadget
2019-10-05 19:22     ` Elijah Newren
2019-09-19 14:43   ` [PATCH v2 02/11] sparse-checkout: create 'init' subcommand Derrick Stolee via GitGitGadget
2019-10-05 19:34     ` Elijah Newren
2019-09-19 14:43   ` [PATCH v2 03/11] clone: add --sparse mode Derrick Stolee via GitGitGadget
2019-10-05 19:40     ` Elijah Newren
2019-10-07 13:56       ` Derrick Stolee
2019-09-19 14:43   ` [PATCH v2 05/11] sparse-checkout: add '--stdin' option to set subcommand Derrick Stolee via GitGitGadget
2019-09-19 14:43   ` [PATCH v2 04/11] sparse-checkout: 'set' subcommand Derrick Stolee via GitGitGadget
2019-10-05 22:44     ` Elijah Newren
2019-10-06  0:30       ` Elijah Newren
2019-10-07 18:26         ` Derrick Stolee
2019-10-11 22:24           ` Elijah Newren
2019-09-19 14:43   ` [PATCH v2 06/11] sparse-checkout: create 'disable' subcommand Derrick Stolee via GitGitGadget
2019-10-06  4:10     ` Elijah Newren
2019-10-07 19:12       ` Derrick Stolee
2019-09-19 14:43   ` [PATCH v2 07/11] trace2: add region in clear_ce_flags Jeff Hostetler via GitGitGadget
2019-10-06  4:13     ` Elijah Newren
2019-09-19 14:43   ` [PATCH v2 09/11] sparse-checkout: use hashmaps for cone patterns Derrick Stolee via GitGitGadget
2019-09-19 20:59     ` Derrick Stolee
2019-09-20 14:37       ` Derrick Stolee
2019-09-19 14:43   ` [PATCH v2 08/11] sparse-checkout: add 'cone' mode Derrick Stolee via GitGitGadget
2019-10-06  4:22     ` Elijah Newren
2019-10-07 19:15       ` Derrick Stolee
2019-09-19 14:43   ` [PATCH v2 10/11] sparse-checkout: init and set in cone mode Derrick Stolee via GitGitGadget
2019-09-19 14:43   ` [PATCH v2 11/11] unpack-trees: hash less " Derrick Stolee via GitGitGadget
2019-10-01 13:40   ` [PATCH v2 00/11] New sparse-checkout builtin and "cone" mode Derrick Stolee
2019-10-01 16:54     ` Elijah Newren
2019-10-01 18:15       ` Derrick Stolee
2019-10-03 22:28     ` Junio C Hamano
2019-10-07 20:08   ` [PATCH v3 00/17] " Derrick Stolee via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 01/17] sparse-checkout: create builtin with 'list' subcommand Derrick Stolee via GitGitGadget
2019-10-11 22:01       ` Elijah Newren
2019-10-07 20:08     ` [PATCH v3 02/17] sparse-checkout: create 'init' subcommand Derrick Stolee via GitGitGadget
2019-10-11 22:14       ` Elijah Newren
2019-10-14 20:22         ` Derrick Stolee
2019-10-07 20:08     ` [PATCH v3 03/17] clone: add --sparse mode Derrick Stolee via GitGitGadget
2019-10-11 22:20       ` Elijah Newren
2019-10-07 20:08     ` [PATCH v3 04/17] sparse-checkout: 'set' subcommand Derrick Stolee via GitGitGadget
2019-10-11 22:26       ` Elijah Newren
2019-10-11 22:30         ` Elijah Newren
2019-10-07 20:08     ` [PATCH v3 05/17] sparse-checkout: add '--stdin' option to set subcommand Derrick Stolee via GitGitGadget
2019-10-11 22:27       ` Elijah Newren
2019-10-14 20:28         ` Derrick Stolee
2019-10-07 20:08     ` [PATCH v3 06/17] sparse-checkout: create 'disable' subcommand Derrick Stolee via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 07/17] trace2: add region in clear_ce_flags Jeff Hostetler via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 08/17] sparse-checkout: add 'cone' mode Derrick Stolee via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 09/17] sparse-checkout: use hashmaps for cone patterns Derrick Stolee via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 10/17] sparse-checkout: init and set in cone mode Derrick Stolee via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 11/17] unpack-trees: hash less " Derrick Stolee via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 12/17] unpack-trees: add progress to clear_ce_flags() Derrick Stolee via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 13/17] read-tree: show progress by default Derrick Stolee via GitGitGadget
2019-10-12 22:16       ` Elijah Newren
2019-10-14 20:31         ` Derrick Stolee
2019-10-07 20:08     ` [PATCH v3 14/17] sparse-checkout: sanitize for nested folders Derrick Stolee via GitGitGadget
2019-10-07 20:08     ` [PATCH v3 15/17] sparse-checkout: update working directory in-process Derrick Stolee via GitGitGadget
2019-10-12 22:57       ` Elijah Newren
2019-10-14 20:39         ` Derrick Stolee
2019-10-07 20:08     ` [PATCH v3 16/17] sparse-checkout: write using lockfile Derrick Stolee via GitGitGadget
2019-10-12 22:59       ` Elijah Newren
2019-10-14 20:41         ` Derrick Stolee
2019-10-07 20:08     ` [PATCH v3 17/17] sparse-checkout: cone mode should not interact with .gitignore Derrick Stolee via GitGitGadget
2019-10-12 23:00       ` Elijah Newren
2019-10-12 23:22     ` [PATCH v3 00/17] New sparse-checkout builtin and "cone" mode Elijah Newren
2019-10-15 13:55     ` [PATCH v4 " Derrick Stolee via GitGitGadget
2019-10-15 13:55       ` [PATCH v4 01/17] sparse-checkout: create builtin with 'list' subcommand Derrick Stolee via GitGitGadget
2019-10-16 19:00         ` Elijah Newren
2019-10-21 12:10           ` Derrick Stolee
2019-10-18 16:07         ` SZEDER Gábor
2019-10-21 11:42           ` Derrick Stolee
2019-10-15 13:55       ` [PATCH v4 02/17] sparse-checkout: create 'init' subcommand Derrick Stolee via GitGitGadget
2019-10-15 13:55       ` [PATCH v4 03/17] clone: add --sparse mode Derrick Stolee via GitGitGadget
2019-10-15 13:55       ` [PATCH v4 04/17] sparse-checkout: 'set' subcommand Derrick Stolee via GitGitGadget
2019-10-15 13:55       ` [PATCH v4 05/17] sparse-checkout: add '--stdin' option to set subcommand Derrick Stolee via GitGitGadget
2019-10-15 13:55       ` [PATCH v4 06/17] sparse-checkout: create 'disable' subcommand Derrick Stolee via GitGitGadget
2019-10-18 16:31         ` SZEDER Gábor
2019-10-15 13:55       ` [PATCH v4 07/17] trace2: add region in clear_ce_flags Jeff Hostetler via GitGitGadget
2019-10-15 13:55       ` [PATCH v4 08/17] sparse-checkout: add 'cone' mode Derrick Stolee via GitGitGadget
2019-10-15 13:55       ` [PATCH v4 09/17] sparse-checkout: use hashmaps for cone patterns Derrick Stolee via GitGitGadget
2019-10-18 15:31         ` SZEDER Gábor
2019-10-21 11:44           ` Derrick Stolee
2019-10-15 13:55       ` [PATCH v4 10/17] sparse-checkout: init and set in cone mode Derrick Stolee via GitGitGadget
2019-10-15 13:55       ` [PATCH v4 11/17] unpack-trees: hash less " Derrick Stolee via GitGitGadget
2019-10-15 13:55       ` [PATCH v4 12/17] unpack-trees: add progress to clear_ce_flags() Derrick Stolee via GitGitGadget
2019-10-15 13:56       ` [PATCH v4 13/17] read-tree: show progress by default Derrick Stolee via GitGitGadget
2019-10-15 13:56       ` [PATCH v4 14/17] sparse-checkout: sanitize for nested folders Derrick Stolee via GitGitGadget
2019-10-15 13:56       ` [PATCH v4 15/17] sparse-checkout: update working directory in-process Derrick Stolee via GitGitGadget
2019-10-18 20:24         ` SZEDER Gábor
2019-10-18 20:40           ` SZEDER Gábor
2019-10-21 11:59             ` Derrick Stolee
2019-10-15 13:56       ` [PATCH v4 16/17] sparse-checkout: write using lockfile Derrick Stolee via GitGitGadget
2019-10-15 13:56       ` [PATCH v4 17/17] sparse-checkout: cone mode should not interact with .gitignore Derrick Stolee via GitGitGadget
2019-10-16 20:07       ` [PATCH v4 00/17] New sparse-checkout builtin and "cone" mode Elijah Newren
2019-10-17 23:53       ` Jon Simons
2019-10-21 12:08         ` Derrick Stolee
2019-10-21 13:56       ` [PATCH v5 " Derrick Stolee via GitGitGadget
2019-10-21 13:56         ` [PATCH v5 01/17] sparse-checkout: create builtin with 'list' subcommand Derrick Stolee via GitGitGadget
2019-10-21 13:56         ` [PATCH v5 02/17] sparse-checkout: create 'init' subcommand Derrick Stolee via GitGitGadget
2019-10-21 13:56         ` [PATCH v5 03/17] clone: add --sparse mode Derrick Stolee via GitGitGadget
2019-10-21 13:56         ` [PATCH v5 04/17] sparse-checkout: 'set' subcommand Derrick Stolee via GitGitGadget
2019-10-21 13:56         ` [PATCH v5 05/17] sparse-checkout: add '--stdin' option to set subcommand Derrick Stolee via GitGitGadget
2019-10-21 13:56         ` [PATCH v5 06/17] sparse-checkout: create 'disable' subcommand Derrick Stolee via GitGitGadget
2019-10-21 13:56         ` [PATCH v5 07/17] trace2: add region in clear_ce_flags Jeff Hostetler via GitGitGadget
2019-10-21 13:56         ` [PATCH v5 08/17] sparse-checkout: add 'cone' mode Derrick Stolee via GitGitGadget
2019-10-21 13:56         ` [PATCH v5 09/17] sparse-checkout: use hashmaps for cone patterns Derrick Stolee via GitGitGadget
2019-10-21 13:56         ` [PATCH v5 10/17] sparse-checkout: init and set in cone mode Derrick Stolee via GitGitGadget
2019-10-21 13:56         ` [PATCH v5 11/17] unpack-trees: hash less " Derrick Stolee via GitGitGadget
2019-10-21 13:56         ` [PATCH v5 12/17] unpack-trees: add progress to clear_ce_flags() Derrick Stolee via GitGitGadget
2019-10-21 13:56         ` [PATCH v5 13/17] read-tree: show progress by default Derrick Stolee via GitGitGadget
2019-10-21 15:04           ` Phillip Wood
2019-10-21 15:14             ` Derrick Stolee
2019-10-23  3:48               ` Junio C Hamano
2019-10-23 12:50                 ` Derrick Stolee
2019-10-24 10:18                   ` Phillip Wood
2019-10-24 10:18                 ` Phillip Wood
2019-10-21 13:56         ` [PATCH v5 14/17] sparse-checkout: sanitize for nested folders Derrick Stolee via GitGitGadget
2019-10-21 13:56         ` [PATCH v5 15/17] sparse-checkout: update working directory in-process Derrick Stolee via GitGitGadget
2019-10-21 13:56         ` [PATCH v5 16/17] sparse-checkout: write using lockfile Derrick Stolee via GitGitGadget
2019-10-21 13:56         ` [PATCH v5 17/17] sparse-checkout: cone mode should not interact with .gitignore Derrick Stolee via GitGitGadget
2019-10-23  3:52         ` [PATCH v5 00/17] New sparse-checkout builtin and "cone" mode Junio C Hamano

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ab98c292-a1b1-ff47-2e8e-f74b7ade2b94@gmail.com \
    --to=stolee@gmail.com \
    --cc=dstolee@microsoft.com \
    --cc=git@vger.kernel.org \
    --cc=gitgitgadget@gmail.com \
    --cc=gitster@pobox.com \
    --cc=newren@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git