git@vger.kernel.org mailing list mirror (one of many)
 help / Atom feed
From: Brandon Williams <bmwill@google.com>
To: Stefan Beller <sbeller@google.com>
Cc: "git@vger.kernel.org" <git@vger.kernel.org>
Subject: Re: [PATCH] pull: optionally rebase submodules
Date: Mon, 15 May 2017 16:09:19 -0700
Message-ID: <20170515230919.GF79147@google.com> (raw)
In-Reply-To: <CAGZ79kYYX_AMZm7Di8cUA_eiDS+SSAGnJyrDLcC5U7POk7WdSw@mail.gmail.com>

On 05/11, Stefan Beller wrote:
> On Thu, May 11, 2017 at 10:24 AM, Brandon Williams <bmwill@google.com> wrote:
> > Teach pull to optionally update submodules when '--recurse-submodules'
> > is provided.  This will teach pull to run 'submodule update --rebase'
> > when the '--recurse-submodules' and '--rebase' flags are given.
> >
> > Signed-off-by: Brandon Williams <bmwill@google.com>
> > ---
> >
> > Pull is already a shortcut for running fetch followed by a merge/rebase, so why
> > not have it be a shortcut for running 'submodule update --rebase' when the
> > recurse flag is given!
> 
> I have not thought about the implications of this shortcut, as opposed to
> actually implementing it in C (which presumably would contain more checks).
> Will do.

Well this would be a short up until we actually implement recursion in
merge and rebase.  For rebase we may want to wait until its completely
ported to C since that effort is still underway.  Alternatively we can avoid
this shortcut and wait until rebase is finished being ported.

> 
> >
> >  builtin/pull.c            | 30 ++++++++++++++-
> >  t/t5572-pull-submodule.sh | 97 +++++++++++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 125 insertions(+), 2 deletions(-)
> >
> > diff --git a/builtin/pull.c b/builtin/pull.c
> > index dd1a4a94e..d73d654e6 100644
> > --- a/builtin/pull.c
> > +++ b/builtin/pull.c
> > @@ -77,6 +77,7 @@ static const char * const pull_usage[] = {
> >  /* Shared options */
> >  static int opt_verbosity;
> >  static char *opt_progress;
> > +static int recurse_submodules;
> >
> >  /* Options passed to git-merge or git-rebase */
> >  static enum rebase_type opt_rebase = -1;
> > @@ -532,6 +533,17 @@ static int pull_into_void(const struct object_id *merge_head,
> >         return 0;
> >  }
> >
> > +static int  update_submodules(void)
> 
> Maybe s/update_submodules/rebase_submodules/ ?
> 
> > +{
> > +       struct child_process cp = CHILD_PROCESS_INIT;
> > +       cp.git_cmd = 1;
> > +
> > +       argv_array_pushl(&cp.args, "submodule", "update", "--recursive", NULL);
> > +       argv_array_push(&cp.args, "--rebase");
> 
> The --rebase could be part of the _pushl ?
> Also we could set
>     no_stdin = 1
> we do need stdout/err though.

can do.

> 
> 
> > +
> > +       return run_command(&cp);
> > +}
> > +
> >  /**
> >   * Runs git-merge, returning its exit status.
> >   */
> > @@ -816,6 +828,14 @@ int cmd_pull(int argc, const char **argv, const char *prefix)
> >                         oidclr(&rebase_fork_point);
> >         }
> >
> > +       if (opt_recurse_submodules &&
> > +           !strcmp(opt_recurse_submodules, "--recurse-submodules")) {
> 
> So this checks if we pass --recurse-submodules to fetch and if it is not
> a on-demand/no.
> Maybe we'd want to use the same infrastructure as fetch does, such that
> parse_fetch_recurse makes the decision. (Then "--recurse-submodules=TrUe"
> would work as well, IIUC)

Agreed, it may be better to actually parse the switch properly.

> 
> 
> > +               recurse_submodules = 1;
> > +
> > +               if (!opt_rebase)
> > +                       die(_("--recurse-submodules is only valid with --rebase"));
> 
> I wonder if there are existing users of "git pull --recurse --merge";
> as of now this would fetch the submodules (on-demand) and merge
> in the local commits of the superprojects. It sounds like a useful workflow
> which we'd be blocking here? Maybe just do nothing in case of !opt_rebase,
> i.e. make it part of the first condition added in this hunk?
> 
> > +               ret = run_rebase(&curr_head, merge_heads.oid, &rebase_fork_point);
> > +
> > +               if (!ret && recurse_submodules)
> > +                       ret = update_submodules();
> 
> Instead of doing the rebase of submodules here, we may just want to pass
> 'recurse_submodules' into run_rebase, which can do it, too. (It already has
> a 'ret' value, so the main cmd is not as cluttered.
> 
> ---
> Before reviewing the tests, let's step a bit back and talk about the design
> and what is useful to the user. From reading the code, we
>   1) perform a fetch in the superproject
>   2) rebase the superproject (not rewriting any submodule pointers,
>     but that may be ok for now)
>   3) sequentially:
>   3a) fetch each submodule on demand
>   3b) run rebase inside of it.
> 
> 
> (A) On the sequence:
> If in a normal pull --rebase we have a failure, we can fixup the failure
> and then continue via "git rebase --continue"; now if we have a failure
> in 3), we would need to fixup the submodule ourselves and then as
> we lost all progress in the superproject, rerun "pull --rebase --recurse"?

Yeah this may not have the best workflow...

> 
> (B)
> As discussed offline this produces bad results if we have a non-ff
> operation in the superproject, that also has submodule pointer updates.
> So additionally we would need to walk the superprojects local commits
> and check if any submodule is touched.
> 
> 
> >
> > +test_expect_success 'pull --recurse-submodule setup' '
> > +       git init child &&
> 
> test_create_repo child
> 
> > +       (
> > +               cd child &&
> > +               echo "bar" >file &&
> > +               git add file &&
> > +               git commit -m "initial commit"
> 
> test_create_commit -C child
> 
> > +       ) &&
> > +       git init parent &&
> > +       (
> > +               cd parent &&
> > +               echo "foo" >file &&
> > +               git add file &&
> > +               git commit -m "Initial commit" &&
> > +               git submodule add ../child sub &&
> > +               git commit -m "add submodule"
> > +       ) &&
> 
> Same setup comment as for the child
> 
> 
> > +       git clone --recurse-submodule parent super &&
> > +       git -C super/sub checkout master
> 
> I wonder if we want to keep these two commands in each test
> as I noticed some test scripts are horribly messy others have
> a pattern to cleanup after themselves:
> 
> test_expect_....
>     test_when_finished "rm -rf super-clone" &&
>     git clone ... into super-clone
> 
> 
> 
> > +'
> > +
> > +test_expect_success 'pull recursive fails without --rebase' '
> > +       test_must_fail git -C super pull --recurse-submodules 2>actual &&
> > +       test_i18ngrep "recurse-submodules is only valid with --rebase" actual
> > +'
> 
> Side note: another place to add tests could be t5520 or t740*.
> 
> > +test_expect_success 'pull rebase recursing fails with conflicts' '
> > +       git -C super/sub reset --hard HEAD^^ &&
> > +       git -C super reset --hard HEAD^ &&
> > +       (
> > +               cd super/sub &&
> > +               echo "b" >file &&
> > +               git add file &&
> > +               git commit -m "update file"
> > +       ) &&
> > +       test_must_fail git -C super pull --rebase --recurse-submodules
> 
> As discussed above: We'd also want to have a reasonable state here,
> or some advice to the user telling them how to recover. Maybe in a
> first approach we can tell them to re-run "submodule update --rebase"
> after fixing the conflict?
> 
> Thanks,
> Stefan

-- 
Brandon Williams

  reply index

Thread overview: 5+ messages in thread (expand / mbox.gz / Atom feed / [top])
2017-05-11 17:24 Brandon Williams
2017-05-11 18:12 ` Stefan Beller
2017-05-15 23:09   ` Brandon Williams [this message]
2017-05-11 20:00 ` Philip Oakley
2017-05-12 17:30   ` Brandon Williams

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply to all the recipients using the --to, --cc,
  and --in-reply-to switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170515230919.GF79147@google.com \
    --to=bmwill@google.com \
    --cc=git@vger.kernel.org \
    --cc=sbeller@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org mailing list mirror (one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/
       or Tor2web: https://www.tor2web.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox