* git push recurse.submodules behavior changed in 2.13 @ 2017-05-27 13:31 John Shahid 2017-05-29 2:44 ` Junio C Hamano 2017-05-30 18:10 ` Jonathan Nieder 0 siblings, 2 replies; 10+ messages in thread From: John Shahid @ 2017-05-27 13:31 UTC (permalink / raw) To: git Hi all, It looks like the git push recurse-submodules behavior has changed. Currently with 2.13 you cannot run "git push --recurse-submodules=on-demand" if the parent repo is on a different branch than the sub repos, e.g. parent repo is on "develop" and sub-repo on "master". I created a test that can be found here [1]. A bisect shows that the change to propagate refspec [2] to the submodules is the culprit. imho this is an undesired change in behavior. I looked at the code but couldn't see an easy way to fix this issue without breaking the feature mentioned above. The only option I can think of is to control the refspec propagation behavior using a flag, e.g. "--propagate-refspecs" or add another recurse-submodules option, e.g. "--recurse-submodules=propagate" What do you all think ? [1] https://gist.github.com/jvshahid/b778702cc3d825c6887d2707e866a9c8 [2] https://github.com/git/git/commit/06bf4ad1db92c32af38e16d9b7f928edbd647780 ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: git push recurse.submodules behavior changed in 2.13 2017-05-27 13:31 git push recurse.submodules behavior changed in 2.13 John Shahid @ 2017-05-29 2:44 ` Junio C Hamano 2017-05-29 4:20 ` Stefan Beller 2017-05-30 18:10 ` Jonathan Nieder 1 sibling, 1 reply; 10+ messages in thread From: Junio C Hamano @ 2017-05-29 2:44 UTC (permalink / raw) To: Brandon Williams; +Cc: git, John Shahid, Stefan Beller John Shahid <jvshahid@gmail.com> writes: > It looks like the git push recurse-submodules behavior has changed. > Currently with 2.13 you cannot run "git push > --recurse-submodules=on-demand" if the parent repo is on a different > branch than the sub repos, e.g. parent repo is on "develop" and > sub-repo on "master". I created a test that can be found here [1]. > > A bisect shows that the change to propagate refspec [2] to the > submodules is the culprit. imho this is an undesired change in > behavior. I looked at the code but couldn't see an easy way to fix > this issue without breaking the feature mentioned above. The only > option I can think of is to control the refspec propagation behavior > using a flag, e.g. "--propagate-refspecs" or add another > recurse-submodules option, e.g. "--recurse-submodules=propagate" > > What do you all think ? > > [1] https://gist.github.com/jvshahid/b778702cc3d825c6887d2707e866a9c8 > [2] https://github.com/git/git/commit/06bf4ad1db92c32af38e16d9b7f928edbd647780 Brandon? I cannot quite tell from the report what "has changed" refers to, what failures "you cannot run" gets, and if that is a desirable thing to do (i.e. if letting the user run it in such a configuration would somehow break things, actively erroring out may be a deliberate change) or not (i.e. an unintended regression). ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: git push recurse.submodules behavior changed in 2.13 2017-05-29 2:44 ` Junio C Hamano @ 2017-05-29 4:20 ` Stefan Beller 2017-05-30 12:01 ` John Shahid 0 siblings, 1 reply; 10+ messages in thread From: Stefan Beller @ 2017-05-29 4:20 UTC (permalink / raw) To: Junio C Hamano; +Cc: Brandon Williams, git@vger.kernel.org, John Shahid On Sun, May 28, 2017 at 7:44 PM, Junio C Hamano <gitster@pobox.com> wrote: > John Shahid <jvshahid@gmail.com> writes: > >> It looks like the git push recurse-submodules behavior has changed. >> Currently with 2.13 you cannot run "git push >> --recurse-submodules=on-demand" if the parent repo is on a different >> branch than the sub repos, e.g. parent repo is on "develop" and >> sub-repo on "master". I created a test that can be found here [1]. >> >> A bisect shows that the change to propagate refspec [2] to the >> submodules is the culprit. imho this is an undesired change in >> behavior. I looked at the code but couldn't see an easy way to fix >> this issue without breaking the feature mentioned above. The only >> option I can think of is to control the refspec propagation behavior >> using a flag, e.g. "--propagate-refspecs" or add another >> recurse-submodules option, e.g. "--recurse-submodules=propagate" >> >> What do you all think ? >> >> [1] https://gist.github.com/jvshahid/b778702cc3d825c6887d2707e866a9c8 >> [2] https://github.com/git/git/commit/06bf4ad1db92c32af38e16d9b7f928edbd647780 > > Brandon? I cannot quite tell from the report what "has changed" > refers to, what failures "you cannot run" gets, and if that is a > desirable thing to do (i.e. if letting the user run it in such a > configuration would somehow break things, actively erroring out may > be a deliberate change) or not (i.e. an unintended regression). > Before the refspec was passed down into the submodules, we'd just invoke "git push" in the submodule assuming the user setup a remote tracking branch and a push strategy such that "git push" would do the right thing. And because the submodule is configured independently, it doesn't matter which branch you're on in the superproject. Looking at the test[1], you run "git push --recurse-submodules" without any remote/branch that was called out in the commit message[2] to not have changed. Is that understanding correct? Looking at the test cases of [2] we did not test for explicit "still works with no args given", though one could have expected we'd have a test for that already. :/ Thanks, Stefan ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: git push recurse.submodules behavior changed in 2.13 2017-05-29 4:20 ` Stefan Beller @ 2017-05-30 12:01 ` John Shahid 2017-05-30 17:05 ` Brandon Williams 0 siblings, 1 reply; 10+ messages in thread From: John Shahid @ 2017-05-30 12:01 UTC (permalink / raw) To: Stefan Beller; +Cc: Junio C Hamano, Brandon Williams, git@vger.kernel.org Junio, sorry for the poor report. I totally forgot to describe the behavior that i'm currently getting vs what i expect. Expected behavior: We have a parent repo on a branch called "develop" and a submodule on a branch called "master". Prior to git version 2.13 if we had an unpushed commit in the submodule and ran "git push origin develop --recurse-submodules=on-demand" git would happily push the develop branch of the parent repo and the master branch of the submodule, e.g.: > Pushing submodule 'sub' > Counting objects: 2, done. > Delta compression using up to 4 threads. > Compressing objects: 100% (2/2), done. > Writing objects: 100% (2/2), 242 bytes | 0 bytes/s, done. > Total 2 (delta 0), reused 0 (delta 0) > To /home/jvshahid/codez/git/t/trash directory.t9904-diff-branch-submodule-push/sub.git > 3cd2129..69cbc06 master -> master > Counting objects: 2, done. > Delta compression using up to 4 threads. > Compressing objects: 100% (2/2), done. > Writing objects: 100% (2/2), 283 bytes | 0 bytes/s, done. > Total 2 (delta 0), reused 0 (delta 0) > To ../pub.git > 7ff6fca..945bc12 develop -> develop > ok 2 - push if submodule has is on a different branch Actual behavior: After the change mentioned in my previous email, git would propagate the refspec from the parent repo to the submodule, i.e. it would try to push a branch called "develop" in the submodule which would error since no branch with that name exist in the submodule. Here is a sample output with git v2.13.0: > pushing to ref refs/heads/develop:refs/heads/develop > pushging to remote origin > fatal: src refspec 'refs/heads/develop' must name a ref > fatal: process for submodule 'sub' failed > fatal: The remote end hung up unexpectedly I hope this clarifies my bug report. Stefan, one little correction. I don't think the commit called out this behavior. The commit message was talking about unconfigured remotes (i.e. pushing to a url or local path) to not propagate the refspec and preserve the current behavior. Judging from the code and a test case that I wrote, this behavior is working as expected. That is, git won't propagate the refspec. Cheers, JS On Mon, May 29, 2017 at 12:20 AM, Stefan Beller <sbeller@google.com> wrote: > On Sun, May 28, 2017 at 7:44 PM, Junio C Hamano <gitster@pobox.com> wrote: >> John Shahid <jvshahid@gmail.com> writes: >> >>> It looks like the git push recurse-submodules behavior has changed. >>> Currently with 2.13 you cannot run "git push >>> --recurse-submodules=on-demand" if the parent repo is on a different >>> branch than the sub repos, e.g. parent repo is on "develop" and >>> sub-repo on "master". I created a test that can be found here [1]. >>> >>> A bisect shows that the change to propagate refspec [2] to the >>> submodules is the culprit. imho this is an undesired change in >>> behavior. I looked at the code but couldn't see an easy way to fix >>> this issue without breaking the feature mentioned above. The only >>> option I can think of is to control the refspec propagation behavior >>> using a flag, e.g. "--propagate-refspecs" or add another >>> recurse-submodules option, e.g. "--recurse-submodules=propagate" >>> >>> What do you all think ? >>> >>> [1] https://gist.github.com/jvshahid/b778702cc3d825c6887d2707e866a9c8 >>> [2] https://github.com/git/git/commit/06bf4ad1db92c32af38e16d9b7f928edbd647780 >> >> Brandon? I cannot quite tell from the report what "has changed" >> refers to, what failures "you cannot run" gets, and if that is a >> desirable thing to do (i.e. if letting the user run it in such a >> configuration would somehow break things, actively erroring out may >> be a deliberate change) or not (i.e. an unintended regression). >> > > Before the refspec was passed down into the submodules, > we'd just invoke "git push" in the submodule assuming the user > setup a remote tracking branch and a push strategy such that > "git push" would do the right thing. > And because the submodule is configured independently, it > doesn't matter which branch you're on in the superproject. > > Looking at the test[1], you run "git push --recurse-submodules" > without any remote/branch that was called out in the commit > message[2] to not have changed. Is that understanding correct? > > Looking at the test cases of [2] we did not test for explicit > "still works with no args given", though one could have expected > we'd have a test for that already. :/ > > Thanks, > Stefan ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: git push recurse.submodules behavior changed in 2.13 2017-05-30 12:01 ` John Shahid @ 2017-05-30 17:05 ` Brandon Williams 0 siblings, 0 replies; 10+ messages in thread From: Brandon Williams @ 2017-05-30 17:05 UTC (permalink / raw) To: John Shahid; +Cc: Stefan Beller, Junio C Hamano, git@vger.kernel.org On 05/30, John Shahid wrote: > Junio, sorry for the poor report. I totally forgot to describe the > behavior that i'm currently getting vs what i expect. > > Expected behavior: > > We have a parent repo on a branch called "develop" and a submodule on > a branch called "master". Prior to git version 2.13 if we had an > unpushed commit in the submodule and ran "git push origin develop > --recurse-submodules=on-demand" git would happily push the develop > branch of the parent repo and the master branch of the submodule, > e.g.: Yeah my patches would definitely break that kind of workflow because they assumed that if you actually provided a refspec + --recurse that you would want it propagated down. When developing those patches I was trying to avoid needing to add an additional flag to do the propagation but given people were already relying on this behavior it looks like that may be the only course of action. > > > Pushing submodule 'sub' > > Counting objects: 2, done. > > Delta compression using up to 4 threads. > > Compressing objects: 100% (2/2), done. > > Writing objects: 100% (2/2), 242 bytes | 0 bytes/s, done. > > Total 2 (delta 0), reused 0 (delta 0) > > To /home/jvshahid/codez/git/t/trash directory.t9904-diff-branch-submodule-push/sub.git > > 3cd2129..69cbc06 master -> master > > Counting objects: 2, done. > > Delta compression using up to 4 threads. > > Compressing objects: 100% (2/2), done. > > Writing objects: 100% (2/2), 283 bytes | 0 bytes/s, done. > > Total 2 (delta 0), reused 0 (delta 0) > > To ../pub.git > > 7ff6fca..945bc12 develop -> develop > > ok 2 - push if submodule has is on a different branch > > Actual behavior: > > After the change mentioned in my previous email, git would propagate > the refspec from the parent repo to the submodule, i.e. it would try > to push a branch called "develop" in the submodule which would error > since no branch with that name exist in the submodule. Here is a > sample output with git v2.13.0: > > > pushing to ref refs/heads/develop:refs/heads/develop > > pushging to remote origin > > fatal: src refspec 'refs/heads/develop' must name a ref > > fatal: process for submodule 'sub' failed > > fatal: The remote end hung up unexpectedly > > I hope this clarifies my bug report. > > Stefan, one little correction. I don't think the commit called out > this behavior. The commit message was talking about unconfigured > remotes (i.e. pushing to a url or local path) to not propagate the > refspec and preserve the current behavior. Judging from the code and a > test case that I wrote, this behavior is working as expected. That is, > git won't propagate the refspec. > > Cheers, > > JS > > On Mon, May 29, 2017 at 12:20 AM, Stefan Beller <sbeller@google.com> wrote: > > On Sun, May 28, 2017 at 7:44 PM, Junio C Hamano <gitster@pobox.com> wrote: > >> John Shahid <jvshahid@gmail.com> writes: > >> > >>> It looks like the git push recurse-submodules behavior has changed. > >>> Currently with 2.13 you cannot run "git push > >>> --recurse-submodules=on-demand" if the parent repo is on a different > >>> branch than the sub repos, e.g. parent repo is on "develop" and > >>> sub-repo on "master". I created a test that can be found here [1]. > >>> > >>> A bisect shows that the change to propagate refspec [2] to the > >>> submodules is the culprit. imho this is an undesired change in > >>> behavior. I looked at the code but couldn't see an easy way to fix > >>> this issue without breaking the feature mentioned above. The only > >>> option I can think of is to control the refspec propagation behavior > >>> using a flag, e.g. "--propagate-refspecs" or add another > >>> recurse-submodules option, e.g. "--recurse-submodules=propagate" > >>> > >>> What do you all think ? > >>> > >>> [1] https://gist.github.com/jvshahid/b778702cc3d825c6887d2707e866a9c8 > >>> [2] https://github.com/git/git/commit/06bf4ad1db92c32af38e16d9b7f928edbd647780 > >> > >> Brandon? I cannot quite tell from the report what "has changed" > >> refers to, what failures "you cannot run" gets, and if that is a > >> desirable thing to do (i.e. if letting the user run it in such a > >> configuration would somehow break things, actively erroring out may > >> be a deliberate change) or not (i.e. an unintended regression). > >> > > > > Before the refspec was passed down into the submodules, > > we'd just invoke "git push" in the submodule assuming the user > > setup a remote tracking branch and a push strategy such that > > "git push" would do the right thing. > > And because the submodule is configured independently, it > > doesn't matter which branch you're on in the superproject. > > > > Looking at the test[1], you run "git push --recurse-submodules" > > without any remote/branch that was called out in the commit > > message[2] to not have changed. Is that understanding correct? > > > > Looking at the test cases of [2] we did not test for explicit > > "still works with no args given", though one could have expected > > we'd have a test for that already. :/ > > > > Thanks, > > Stefan -- Brandon Williams ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: git push recurse.submodules behavior changed in 2.13 2017-05-27 13:31 git push recurse.submodules behavior changed in 2.13 John Shahid 2017-05-29 2:44 ` Junio C Hamano @ 2017-05-30 18:10 ` Jonathan Nieder 2017-05-31 14:50 ` John Shahid 1 sibling, 1 reply; 10+ messages in thread From: Jonathan Nieder @ 2017-05-30 18:10 UTC (permalink / raw) To: John Shahid; +Cc: git, Brandon Williams, Stefan Beller Hi John, John Shahid wrote: > It looks like the git push recurse-submodules behavior has changed. > Currently with 2.13 you cannot run "git push > --recurse-submodules=on-demand" if the parent repo is on a different > branch than the sub repos, e.g. parent repo is on "develop" and > sub-repo on "master". I created a test that can be found here [1]. > > A bisect shows that the change to propagate refspec [2] to the > submodules is the culprit. imho this is an undesired change in > behavior. I looked at the code but couldn't see an easy way to fix > this issue without breaking the feature mentioned above. The only > option I can think of is to control the refspec propagation behavior > using a flag, e.g. "--propagate-refspecs" or add another > recurse-submodules option, e.g. "--recurse-submodules=propagate" Thanks for reporting. The old behavior of "git push --recurse-submodules" was a little loose (it would run "git push" in the submodule and rely on defaults to determine the behavior) and when propagating refspecs we considered taking the opportunity to tighten that by propagating the default refspec when you run "git push --recurse-submodules" without a refspec. We didn't make that change. If starting over, it might be a natural thing to do, but we didn't want to break existing users. So git push --recurse-submodules=on-demand should continue to work as before. *puzzled* [...] > We have a parent repo on a branch called "develop" and a submodule on > a branch called "master". Prior to git version 2.13 if we had an > unpushed commit in the submodule and ran "git push origin develop > --recurse-submodules=on-demand" git would happily push the develop > branch of the parent repo and the master branch of the submodule, [...] > pushing to ref refs/heads/develop:refs/heads/develop > pushging to remote origin > fatal: src refspec 'refs/heads/develop' must name a ref > fatal: process for submodule 'sub' failed > fatal: The remote end hung up unexpectedly Aha, you are passing a refspec! Can you say more about how this change affects you? Would you be able to push without a refspec, or do you e.g. have scripting that was relying on the existing loose behavior? To put this in more context: part of the ideal that --recurse-submodules options are trying to achieve is to allow thinking of a repository as a single unit, including submodules, most of the time. To this end, "git clone --recurse-submodules" clones a repository including its submodules, "git checkout --recurse-submodules" checks out submodules as part of a checkout operation, avoiding a separate "git submodule update --init" operation, "git fetch --recurse-submodules" fetches upstream changes for both the parent repository and submodules, and so on. Unfortunately "git checkout --recurse-submodules" was missing a piece, impacting this workflow. If I run git checkout --recurse-submodules -b develop then today this creates a "develop" branch in the superproject but not within submodules. This hurts the illusion of the project being a single unit and makes the meaning of operations like git push --recurse-submodules develop:master unclear. Ideas for next steps: 1. If "git checkout --recurse-submodules -b develop" does the right thing, then this situation is harder to get in in the first place. The series at [3] is meant to do that. 2. The error message shown above is very unclear. It leaks implementation details and doesn't give the user a clear hint about what to do to resolve it. Would some message like the following have helped? $ git push --recurse-submodules origin develop fatal: branch "develop" does not exist in submodule "sub" hint: use "git -C sub branch develop <sha1>" to create such a branch hint: or push without a branch name to push the current branch 3. Outside of the output from any particular command, we are missing an equivalent of "man gitworkflows" to describe how the --recurse-submodules options are meant to work. The series at [4] aims to start addressing that. Given all the above, this looks like a real and serious bug but I think to find a different way from --recurse-submodules=propagate to fix it. The idea, especially with [5], is for submodules to work intuitively without requiring special options. This can require more work in the short term getting the default behavior to work well with a variety of use cases but I think it is time well spent. Thoughts? Thanks and hope that helps, Jonathan > [1] https://gist.github.com/jvshahid/b778702cc3d825c6887d2707e866a9c8 > [2] https://github.com/git/git/commit/06bf4ad1db92c32af38e16d9b7f928edbd647780 [3] http://public-inbox.org/git/20170501180058.8063-1-sbeller@google.com/ [4] http://public-inbox.org/git/20170209020855.23486-1-sbeller@google.com/ [5] http://public-inbox.org/git/20170526191017.19155-1-sbeller@google.com/ ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: git push recurse.submodules behavior changed in 2.13 2017-05-30 18:10 ` Jonathan Nieder @ 2017-05-31 14:50 ` John Shahid 2017-06-10 13:28 ` John Shahid 0 siblings, 1 reply; 10+ messages in thread From: John Shahid @ 2017-05-31 14:50 UTC (permalink / raw) To: Jonathan Nieder; +Cc: git, Brandon Williams, Stefan Beller Hi Jonathan. Thanks a lot for the detailed and interesting response. I included my comments/replies inline. tl;dr; 1. I love the idea of having tools treat the repo as a unit. 2. I think propagating the refspec when no refspec is passed on the CLI is very surprising action that should be corrected 3. I propose more changes to the git tools to support the unity view of the repo On Tue, May 30, 2017 at 2:10 PM, Jonathan Nieder <jrnieder@gmail.com> wrote: > Hi John, > > John Shahid wrote: > >> It looks like the git push recurse-submodules behavior has changed. >> Currently with 2.13 you cannot run "git push >> --recurse-submodules=on-demand" if the parent repo is on a different >> branch than the sub repos, e.g. parent repo is on "develop" and >> sub-repo on "master". I created a test that can be found here [1]. >> >> A bisect shows that the change to propagate refspec [2] to the >> submodules is the culprit. imho this is an undesired change in >> behavior. I looked at the code but couldn't see an easy way to fix >> this issue without breaking the feature mentioned above. The only >> option I can think of is to control the refspec propagation behavior >> using a flag, e.g. "--propagate-refspecs" or add another >> recurse-submodules option, e.g. "--recurse-submodules=propagate" > > Thanks for reporting. The old behavior of "git push > --recurse-submodules" was a little loose (it would run "git push" in > the submodule and rely on defaults to determine the behavior) and when > propagating refspecs we considered taking the opportunity to tighten > that by propagating the default refspec when you run "git push > --recurse-submodules" without a refspec. > > We didn't make that change. If starting over, it might be a natural > thing to do, but we didn't want to break existing users. So > > git push --recurse-submodules=on-demand > > should continue to work as before. *puzzled* > > [...] >> We have a parent repo on a branch called "develop" and a submodule on >> a branch called "master". Prior to git version 2.13 if we had an >> unpushed commit in the submodule and ran "git push origin develop >> --recurse-submodules=on-demand" git would happily push the develop >> branch of the parent repo and the master branch of the submodule, > [...] >> pushing to ref refs/heads/develop:refs/heads/develop >> pushging to remote origin >> fatal: src refspec 'refs/heads/develop' must name a ref >> fatal: process for submodule 'sub' failed >> fatal: The remote end hung up unexpectedly > > Aha, you are passing a refspec! Yes, we are used to passing a refspec but we will be happy not to if that preserves the old behavior. In fact, the test case I linked to earlier does not pass a refspec. I wrote the test that way because I initially misunderstood the commit message and thought that not passing a refspec preserves the old behavior. Long story short, I'm happy with whatever option we have to maintain our current workflow. > > Can you say more about how this change affects you? Would you be able > to push without a refspec, or do you e.g. have scripting that was > relying on the existing loose behavior? as i said before, pushing without a refspec would work for us. I'm also talking to the team to see if we can switch the branch names to be consistent, i.e. use either "master" or "develop" both in the parent repo and sub repos. > > To put this in more context: part of the ideal that > --recurse-submodules options are trying to achieve is to allow > thinking of a repository as a single unit, including submodules, most > of the time. To this end, "git clone --recurse-submodules" clones a > repository including its submodules, "git checkout > --recurse-submodules" checks out submodules as part of a checkout > operation, avoiding a separate "git submodule update --init" operation, > "git fetch --recurse-submodules" fetches upstream changes for both the > parent repository and submodules, and so on. > > Unfortunately "git checkout --recurse-submodules" was missing a piece, > impacting this workflow. If I run > > git checkout --recurse-submodules -b develop > > then today this creates a "develop" branch in the superproject but not > within submodules. This hurts the illusion of the project being a single > unit and makes the meaning of operations like > > git push --recurse-submodules develop:master > > unclear. > > Ideas for next steps: > > 1. If "git checkout --recurse-submodules -b develop" does the right > thing, then this situation is harder to get in in the first place. > The series at [3] is meant to do that. > > 2. The error message shown above is very unclear. It leaks > implementation details and doesn't give the user a clear hint > about what to do to resolve it. Would some message like the > following have helped? > > $ git push --recurse-submodules origin develop > fatal: branch "develop" does not exist in submodule "sub" > hint: use "git -C sub branch develop <sha1>" to create such a branch > hint: or push without a branch name to push the current branch > > 3. Outside of the output from any particular command, we are missing > an equivalent of "man gitworkflows" to describe how the > --recurse-submodules options are meant to work. The series at [4] > aims to start addressing that. I would also add to that list, "git pull --rebase -recurse-submodules", "git merge --recurse-submodules", etc. Otherwise "git checkout --recurse-submodules develop" won't make much sense, since you'll have to go to each submodules and make sure the local branch is up to date. I like the idea of treating the entire repo as a single unit. In fact the new feature (to propagate refspec) is very useful to me. For example, when working on feature branches across the entire repo. That said, i think "git push --recurse-submodules=on-demand" should do the least surprising action and push the current branch, where "current" is defined wrt the repo being pushed (and "push.default" of course). Taking the refspec from the parent repo and forcing it on the submodules feels weird to me. > > Given all the above, this looks like a real and serious bug but I > think to find a different way from --recurse-submodules=propagate to > fix it. The idea, especially with [5], is for submodules to work > intuitively without requiring special options. This can require more > work in the short term getting the default behavior to work well with > a variety of use cases but I think it is time well spent. > > Thoughts? > > Thanks and hope that helps, > Jonathan Yes, that definitely helped. This track of work makes more sense now. Thanks again for the detailed response. JS > >> [1] https://gist.github.com/jvshahid/b778702cc3d825c6887d2707e866a9c8 >> [2] https://github.com/git/git/commit/06bf4ad1db92c32af38e16d9b7f928edbd647780 > [3] http://public-inbox.org/git/20170501180058.8063-1-sbeller@google.com/ > [4] http://public-inbox.org/git/20170209020855.23486-1-sbeller@google.com/ > [5] http://public-inbox.org/git/20170526191017.19155-1-sbeller@google.com/ ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: git push recurse.submodules behavior changed in 2.13 2017-05-31 14:50 ` John Shahid @ 2017-06-10 13:28 ` John Shahid 2017-06-12 17:27 ` Stefan Beller 0 siblings, 1 reply; 10+ messages in thread From: John Shahid @ 2017-06-10 13:28 UTC (permalink / raw) To: Jonathan Nieder; +Cc: git, Brandon Williams, Stefan Beller bump. it's been a while and I'm still not clear what the next steps are. I'm happy to send a patch but I would like to get a consensus first. On Wed, May 31, 2017 at 10:50 AM, John Shahid <jvshahid@gmail.com> wrote: > Hi Jonathan. Thanks a lot for the detailed and interesting response. I > included my comments/replies inline. tl;dr; > > 1. I love the idea of having tools treat the repo as a unit. > 2. I think propagating the refspec when no refspec is passed on the > CLI is very surprising action that should be corrected > 3. I propose more changes to the git tools to support the unity view of the repo > > On Tue, May 30, 2017 at 2:10 PM, Jonathan Nieder <jrnieder@gmail.com> wrote: >> Hi John, >> >> John Shahid wrote: >> >>> It looks like the git push recurse-submodules behavior has changed. >>> Currently with 2.13 you cannot run "git push >>> --recurse-submodules=on-demand" if the parent repo is on a different >>> branch than the sub repos, e.g. parent repo is on "develop" and >>> sub-repo on "master". I created a test that can be found here [1]. >>> >>> A bisect shows that the change to propagate refspec [2] to the >>> submodules is the culprit. imho this is an undesired change in >>> behavior. I looked at the code but couldn't see an easy way to fix >>> this issue without breaking the feature mentioned above. The only >>> option I can think of is to control the refspec propagation behavior >>> using a flag, e.g. "--propagate-refspecs" or add another >>> recurse-submodules option, e.g. "--recurse-submodules=propagate" >> >> Thanks for reporting. The old behavior of "git push >> --recurse-submodules" was a little loose (it would run "git push" in >> the submodule and rely on defaults to determine the behavior) and when >> propagating refspecs we considered taking the opportunity to tighten >> that by propagating the default refspec when you run "git push >> --recurse-submodules" without a refspec. >> >> We didn't make that change. If starting over, it might be a natural >> thing to do, but we didn't want to break existing users. So >> >> git push --recurse-submodules=on-demand >> >> should continue to work as before. *puzzled* >> >> [...] >>> We have a parent repo on a branch called "develop" and a submodule on >>> a branch called "master". Prior to git version 2.13 if we had an >>> unpushed commit in the submodule and ran "git push origin develop >>> --recurse-submodules=on-demand" git would happily push the develop >>> branch of the parent repo and the master branch of the submodule, >> [...] >>> pushing to ref refs/heads/develop:refs/heads/develop >>> pushging to remote origin >>> fatal: src refspec 'refs/heads/develop' must name a ref >>> fatal: process for submodule 'sub' failed >>> fatal: The remote end hung up unexpectedly >> >> Aha, you are passing a refspec! > > Yes, we are used to passing a refspec but we will be happy not to if > that preserves the old behavior. In fact, the test case I linked to > earlier does not pass a refspec. I wrote the test that way because I > initially misunderstood the commit message and thought that not > passing a refspec preserves the old behavior. Long story short, I'm > happy with whatever option we have to maintain our current workflow. > >> >> Can you say more about how this change affects you? Would you be able >> to push without a refspec, or do you e.g. have scripting that was >> relying on the existing loose behavior? > > as i said before, pushing without a refspec would work for us. I'm > also talking to the team to see if we can switch the branch names to > be consistent, i.e. use either "master" or "develop" both in the > parent repo and sub repos. > >> >> To put this in more context: part of the ideal that >> --recurse-submodules options are trying to achieve is to allow >> thinking of a repository as a single unit, including submodules, most >> of the time. To this end, "git clone --recurse-submodules" clones a >> repository including its submodules, "git checkout >> --recurse-submodules" checks out submodules as part of a checkout >> operation, avoiding a separate "git submodule update --init" operation, >> "git fetch --recurse-submodules" fetches upstream changes for both the >> parent repository and submodules, and so on. >> >> Unfortunately "git checkout --recurse-submodules" was missing a piece, >> impacting this workflow. If I run >> >> git checkout --recurse-submodules -b develop >> >> then today this creates a "develop" branch in the superproject but not >> within submodules. This hurts the illusion of the project being a single >> unit and makes the meaning of operations like >> >> git push --recurse-submodules develop:master >> >> unclear. >> >> Ideas for next steps: >> >> 1. If "git checkout --recurse-submodules -b develop" does the right >> thing, then this situation is harder to get in in the first place. >> The series at [3] is meant to do that. >> >> 2. The error message shown above is very unclear. It leaks >> implementation details and doesn't give the user a clear hint >> about what to do to resolve it. Would some message like the >> following have helped? >> >> $ git push --recurse-submodules origin develop >> fatal: branch "develop" does not exist in submodule "sub" >> hint: use "git -C sub branch develop <sha1>" to create such a branch >> hint: or push without a branch name to push the current branch >> >> 3. Outside of the output from any particular command, we are missing >> an equivalent of "man gitworkflows" to describe how the >> --recurse-submodules options are meant to work. The series at [4] >> aims to start addressing that. > > I would also add to that list, "git pull --rebase > -recurse-submodules", "git merge --recurse-submodules", etc. Otherwise > "git checkout --recurse-submodules develop" won't make much sense, > since you'll have to go to each submodules and make sure the local > branch is up to date. > > I like the idea of treating the entire repo as a single unit. In fact > the new feature (to propagate refspec) is very useful to me. For > example, when working on feature branches across the entire repo. That > said, i think "git push --recurse-submodules=on-demand" should do the > least surprising action and push the current branch, where "current" > is defined wrt the repo being pushed (and "push.default" of course). > Taking the refspec from the parent repo and forcing it on the > submodules feels weird to me. > >> >> Given all the above, this looks like a real and serious bug but I >> think to find a different way from --recurse-submodules=propagate to >> fix it. The idea, especially with [5], is for submodules to work >> intuitively without requiring special options. This can require more >> work in the short term getting the default behavior to work well with >> a variety of use cases but I think it is time well spent. > >> >> Thoughts? >> >> Thanks and hope that helps, >> Jonathan > > Yes, that definitely helped. This track of work makes more sense now. > Thanks again for the detailed response. > > JS > >> >>> [1] https://gist.github.com/jvshahid/b778702cc3d825c6887d2707e866a9c8 >>> [2] https://github.com/git/git/commit/06bf4ad1db92c32af38e16d9b7f928edbd647780 >> [3] http://public-inbox.org/git/20170501180058.8063-1-sbeller@google.com/ >> [4] http://public-inbox.org/git/20170209020855.23486-1-sbeller@google.com/ >> [5] http://public-inbox.org/git/20170526191017.19155-1-sbeller@google.com/ ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: git push recurse.submodules behavior changed in 2.13 2017-06-10 13:28 ` John Shahid @ 2017-06-12 17:27 ` Stefan Beller 2017-06-16 14:11 ` John Shahid 0 siblings, 1 reply; 10+ messages in thread From: Stefan Beller @ 2017-06-12 17:27 UTC (permalink / raw) To: John Shahid; +Cc: Jonathan Nieder, git@vger.kernel.org, Brandon Williams On Sat, Jun 10, 2017 at 6:28 AM, John Shahid <jvshahid@gmail.com> wrote: > bump. it's been a while and I'm still not clear what the next steps > are. I'm happy to send a patch but I would like to get a consensus > first. What do you want a consensus on? (Is the change in 2.13 a bug or feature? I considered it enough of a feature to not pursue an urgent bug fix. Maybe I misunderstood the discussion) This thread has diverged into lots of things that could be done. Jonathan pointed out 3 possible ways forward. (1) and (3) are being worked on, (2) is a new thing that nobody (not Brandon nor me) have on our radar for what to work on next. So maybe that is a good idea to get started when you want to get into sending a patch? Also I have the impression that there may be one corner case, in which the handling of refspecs is missed, i.e. maybe we'd need a list (in the man page eventually) on what currently happens when recursing in combination with remote/refspecs or other options. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: git push recurse.submodules behavior changed in 2.13 2017-06-12 17:27 ` Stefan Beller @ 2017-06-16 14:11 ` John Shahid 0 siblings, 0 replies; 10+ messages in thread From: John Shahid @ 2017-06-16 14:11 UTC (permalink / raw) To: Stefan Beller; +Cc: Jonathan Nieder, git@vger.kernel.org, Brandon Williams On Mon, Jun 12, 2017 at 1:27 PM, Stefan Beller <sbeller@google.com> wrote: > On Sat, Jun 10, 2017 at 6:28 AM, John Shahid <jvshahid@gmail.com> wrote: >> bump. it's been a while and I'm still not clear what the next steps >> are. I'm happy to send a patch but I would like to get a consensus >> first. > > What do you want a consensus on? > (Is the change in 2.13 a bug or feature? I considered it enough > of a feature to not pursue an urgent bug fix. Maybe I misunderstood > the discussion) I was under the impression that Jonathan and may be others considered the fact that `git push --recurse-submodules=on-demand` doesn't work as before an unintentional change. He asked me previously if pushing without a refspec will work for us and I responded with a yes. The question remains if everyone is on board with change push without refspec to use `push.default` in the parent repo as well as submodules. Cheers, JS ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2017-06-16 14:12 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-05-27 13:31 git push recurse.submodules behavior changed in 2.13 John Shahid 2017-05-29 2:44 ` Junio C Hamano 2017-05-29 4:20 ` Stefan Beller 2017-05-30 12:01 ` John Shahid 2017-05-30 17:05 ` Brandon Williams 2017-05-30 18:10 ` Jonathan Nieder 2017-05-31 14:50 ` John Shahid 2017-06-10 13:28 ` John Shahid 2017-06-12 17:27 ` Stefan Beller 2017-06-16 14:11 ` John Shahid
Code repositories for project(s) associated with this public inbox https://80x24.org/mirrors/git.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).