* git push failing when push.recurseSubmodules on-demand and git commit --amend was used in submodule.
@ 2017-01-29 19:33 Carlo Wood
2017-01-30 1:00 ` Junio C Hamano
0 siblings, 1 reply; 6+ messages in thread
From: Carlo Wood @ 2017-01-29 19:33 UTC (permalink / raw)
To: git
Hi,
there seems to be a problem with using 'git commit --amend' in
git submodules when using 'git push --recurse-submodules=on-demand'
in the parent.
The latter fails, saying "The following submodule paths contain changes
that can not be found on any remote:" for such submodule, even though
the submodule is clean, pushed and reports 'Everything up-to-date'
when trying to push it.
I believe that the reason has to be that the parent repository thinks
that the comment that was amended, but not pushed, must be on the remote
too, while the whole point of amend is that this commit is not pushed.
I wrote a little script that demonstrates the problem.
Please run in an empty directory.
START-OF-SCRIPT
#! /bin/bash
# This script demonstrates a bug in git where it reports
#
# The following submodule paths contain changes that can
# not be found on any remote:
#
# for a submodule that is clean and pushed.
#
# Create an empty directory, put this script in it
# and run the script.
#
# Carlo Wood, 2017/01/29
# Clean a possible previous run:
rm -rf parent remote.parent remote.subm
REMOTE_BASE="$(pwd)"
# Create a 'remote' for the submodule 'subm'.
mkdir remote.subm
pushd remote.subm
git init --bare
popd
# Create a 'remote' for the 'parent' repository.
mkdir remote.parent
pushd remote.parent
git init --bare
popd
# Create initial parent/subm directory structore.
mkdir -p parent/subm
# Create an initial subm git repository.
pushd parent/subm
git init
git remote add local "$REMOTE_BASE/remote.subm"
touch s ; git add s
git commit -m 'Initial commit.'
git push --set-upstream local master
popd
# Create an initial parent git repository with subm as submodule
# and push.recurseSubmodules = on-demand.
pushd parent
git init
git config push.recurseSubmodules on-demand
git remote add local "$REMOTE_BASE/remote.parent"
touch p ; git add p
git submodule add "$REMOTE_BASE/remote.subm" subm
git add .gitmodules subm
git commit -m 'Initial commit.'
git push --set-upstream local master
popd
# Do some commit in subm, but do not push it to the remote.
pushd parent/subm
echo "My frist commit." > s
git commit -a -m 'Change s'
popd
# Add the subm hash to the parent.
pushd parent
git add subm
git commit -m 'Updated subm.'
popd
# Amend the commit in subm (and optionally push it).
pushd parent/subm
echo "My first commit." > s
git commit -a --amend -m 'Change s'
popd
# Correct that in the parent too:
pushd parent
git add subm
git commit -m 'Updated subm.'
popd
# At this point nothing was published yes, so the
# amend shouldn't have caused a problem. But it did.
pushd parent
git push
popd
echo "THE ABOVE ERROR CAN NOW BE REPRODUCED INDEFINITELY,"
echo "FOR EXAMPLE, DO:"
echo
echo "cd parent/subm"
echo "git push"
echo "cd .."
echo "git push"
END-OF-SCRIPT
Tested with current master 4e59582ff70d299f5a88449891e78d15b4b3fabe
Regards,
Carlo
--
Carlo Wood <carlo@alinoe.com>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: git push failing when push.recurseSubmodules on-demand and git commit --amend was used in submodule.
2017-01-29 19:33 git push failing when push.recurseSubmodules on-demand and git commit --amend was used in submodule Carlo Wood
@ 2017-01-30 1:00 ` Junio C Hamano
2017-01-31 22:08 ` Stefan Beller
2017-02-01 1:22 ` Carlo Wood
0 siblings, 2 replies; 6+ messages in thread
From: Junio C Hamano @ 2017-01-30 1:00 UTC (permalink / raw)
To: Carlo Wood, Heiko Voigt, Stefan Beller; +Cc: git
Carlo Wood <carlo@alinoe.com> writes:
> there seems to be a problem with using 'git commit --amend' in
> git submodules when using 'git push --recurse-submodules=on-demand'
> in the parent.
>
> The latter fails, saying "The following submodule paths contain changes
> that can not be found on any remote:" for such submodule, even though
> the submodule is clean, pushed and reports 'Everything up-to-date'
> when trying to push it.
>
> I believe that the reason has to be that the parent repository thinks
> that the comment that was amended, but not pushed, must be on the remote
> too, while the whole point of amend is that this commit is not pushed.
I am not super familiar with the actualy implementation of the
codepaths involved in this, so CC'ed the folks who can help you
better.
I suspect the submodule folks would say it is working as intended,
if \
- you made a commit in the submodule;
- recorded the resulting commit in the superproject;
- you amended the commit in the submodule; and then
- you did "push, while pushing out in the submodule as needed" from
the superproject.
There are two commits in the submodule that are involved in the
above scenario, and the first one before amending is needed by the
other participants of the project in order for them to check out
what you are trying to push in the superproject, because that is
what the superproject's tree records. You somehow need to make that
commit available to them, but after you amended, the original commit
may no longer be reachable from any branch in your submodule, so
even if you (or the "on-demand" mechanism) pushed any and all
branches out, that would not make the needed commit available to
others. If you push your top-level superproject out in such a
situation, you would break others.
I think you have two options.
1. If the amend was done to improve things in submodule but is not
quite ready, then get rid of that amended commit and restore the
branch in the submodule to the state before you amended, i.e.
the tip of the branch will become the same commit as the one
that is recorded in the superproject. Then push the submodule
and the superproject out. After that, move the submodule branch
to point at the amended commit (or record the amended commit as
a child of the commit you pushed out).
2. If the amend is good and ready to go, "git add" to update the
superproject to make that amended result the one that is needed
in the submodule.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: git push failing when push.recurseSubmodules on-demand and git commit --amend was used in submodule.
2017-01-30 1:00 ` Junio C Hamano
@ 2017-01-31 22:08 ` Stefan Beller
2017-02-01 1:10 ` Carlo Wood
2017-02-01 1:22 ` Carlo Wood
1 sibling, 1 reply; 6+ messages in thread
From: Stefan Beller @ 2017-01-31 22:08 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Carlo Wood, Heiko Voigt, git@vger.kernel.org
On Sun, Jan 29, 2017 at 5:00 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Carlo Wood <carlo@alinoe.com> writes:
>
>> there seems to be a problem with using 'git commit --amend' in
>> git submodules when using 'git push --recurse-submodules=on-demand'
>> in the parent.
>>
>> The latter fails, saying "The following submodule paths contain changes
>> that can not be found on any remote:" for such submodule, even though
>> the submodule is clean, pushed and reports 'Everything up-to-date'
>> when trying to push it.
>>
>> I believe that the reason has to be that the parent repository thinks
>> that the comment that was amended, but not pushed, must be on the remote
>> too, while the whole point of amend is that this commit is not pushed.
>
> I am not super familiar with the actualy implementation of the
> codepaths involved in this, so CC'ed the folks who can help you
> better.
>
> I suspect the submodule folks would say it is working as intended,
> if \
>
> - you made a commit in the submodule;
> - recorded the resulting commit in the superproject;
> - you amended the commit in the submodule; and then
> - you did "push, while pushing out in the submodule as needed" from
> the superproject.
Yes, for the current state of affairs, this is it.
>
> There are two commits in the submodule that are involved in the
> above scenario, and the first one before amending is needed by the
> other participants of the project in order for them to check out
> what you are trying to push in the superproject, because that is
> what the superproject's tree records. You somehow need to make that
> commit available to them, but after you amended, the original commit
> may no longer be reachable from any branch in your submodule, so
> even if you (or the "on-demand" mechanism) pushed any and all
> branches out, that would not make the needed commit available to
> others. If you push your top-level superproject out in such a
> situation, you would break others.
In the long term future, we may want to reject non-fastforward
submodule updates. (Not sure if that is feasible)
>
> I think you have two options.
>
> 1. If the amend was done to improve things in submodule but is not
> quite ready, then get rid of that amended commit and restore the
> branch in the submodule to the state before you amended, i.e.
> the tip of the branch will become the same commit as the one
> that is recorded in the superproject. Then push the submodule
> and the superproject out. After that, move the submodule branch
> to point at the amended commit (or record the amended commit as
> a child of the commit you pushed out).
>
> 2. If the amend is good and ready to go, "git add" to update the
> superproject to make that amended result the one that is needed
> in the submodule.
yup.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: git push failing when push.recurseSubmodules on-demand and git commit --amend was used in submodule.
2017-01-31 22:08 ` Stefan Beller
@ 2017-02-01 1:10 ` Carlo Wood
2017-02-04 18:43 ` Stefan Beller
0 siblings, 1 reply; 6+ messages in thread
From: Carlo Wood @ 2017-02-01 1:10 UTC (permalink / raw)
To: Stefan Beller; +Cc: Junio C Hamano, Heiko Voigt, git@vger.kernel.org
On Tue, 31 Jan 2017 14:08:41 -0800
Stefan Beller <sbeller@google.com> wrote:
> On Sun, Jan 29, 2017 at 5:00 PM, Junio C Hamano <gitster@pobox.com>
> wrote:
> > 2. If the amend is good and ready to go, "git add" to update the
> > superproject to make that amended result the one that is needed
> > in the submodule.
>
> yup.
But that is what I am doing. The amended commit IS already
added to the superproject (and pushed to the remote).
Please have a look at my script, this happens here:
# Correct that in the parent too:
pushd parent
git add subm
git commit -m 'Updated subm.'
popd
The commit from before the amend was added to the super
project (but never pushed) but has now been completely
replaced. I still think this is a flaw in git. It shouldn't
not complain and simply push.
--
Carlo Wood <carlo@alinoe.com>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: git push failing when push.recurseSubmodules on-demand and git commit --amend was used in submodule.
2017-01-30 1:00 ` Junio C Hamano
2017-01-31 22:08 ` Stefan Beller
@ 2017-02-01 1:22 ` Carlo Wood
1 sibling, 0 replies; 6+ messages in thread
From: Carlo Wood @ 2017-02-01 1:22 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Heiko Voigt, Stefan Beller, git
On Sun, 29 Jan 2017 17:00:22 -0800
Junio C Hamano <gitster@pobox.com> wrote:
> I suspect the submodule folks would say it is working as intended,
> if \
>
> - you made a commit in the submodule;
> - recorded the resulting commit in the superproject;
> - you amended the commit in the submodule; and then
> - you did "push, while pushing out in the submodule as needed" from
> the superproject.
This is not what I'm doing.
This is what I'm doing (see my script):
- you made a commit in the submodule;
- recorded the resulting commit in the superproject;
- you amended the commit in the submodule;
- you record the amended commit in the superproject; <=== !
- you push the submodule out (or not, the on-demand does that
anyway)
- you try to push the superproject, but that fails,
as long as you use recurseSubmodules=on-demand.
>
> There are two commits in the submodule that are involved in the
> above scenario, and the first one before amending is needed by the
> other participants of the project in order for them to check out
> what you are trying to push in the superproject, because that is
> what the superproject's tree records.
I never pushed anything of that, so the other participants don't
know, nor have, the pre-amended commit.
It is true that the superproject THINKS that the pre-amended commit
is a normal commit though: the last recorded (amended) commit is
internally listed as being on top of the amended commit (which is
incorrect). This is why the superproject assumes that the current
add commit of the submodule needs the pre-amended commit to be
available too. This is not correct however, it is not needed to
be available to others and does not need to be pushed to a remote.
> I think you have two options.
>
> 1. If the amend was done to improve things in submodule but is not
> quite ready, then get rid of that amended commit and restore the
> branch in the submodule to the state before you amended, i.e.
> the tip of the branch will become the same commit as the one
> that is recorded in the superproject. Then push the submodule
> and the superproject out. After that, move the submodule branch
> to point at the amended commit (or record the amended commit as
> a child of the commit you pushed out).
That would work, but would be a horrible workaround for an existing
bug :p
> 2. If the amend is good and ready to go, "git add" to update the
> superproject to make that amended result the one that is needed
> in the submodule.
This was already done, also in the script that I provided.
Yet, the push in the superproject is still rejected.
--
Carlo Wood <carlo@alinoe.com>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: git push failing when push.recurseSubmodules on-demand and git commit --amend was used in submodule.
2017-02-01 1:10 ` Carlo Wood
@ 2017-02-04 18:43 ` Stefan Beller
0 siblings, 0 replies; 6+ messages in thread
From: Stefan Beller @ 2017-02-04 18:43 UTC (permalink / raw)
To: Carlo Wood; +Cc: Junio C Hamano, Heiko Voigt, git@vger.kernel.org
On Tue, Jan 31, 2017 at 5:10 PM, Carlo Wood <carlo@alinoe.com> wrote:
> On Tue, 31 Jan 2017 14:08:41 -0800
> Stefan Beller <sbeller@google.com> wrote:
>
>> On Sun, Jan 29, 2017 at 5:00 PM, Junio C Hamano <gitster@pobox.com>
>> wrote:
>> > 2. If the amend is good and ready to go, "git add" to update the
>> > superproject to make that amended result the one that is needed
>> > in the submodule.
>>
>> yup.
>
> But that is what I am doing. The amended commit IS already
> added to the superproject (and pushed to the remote).
>
> Please have a look at my script, this happens here:
>
> # Correct that in the parent too:
> pushd parent
> git add subm
> git commit -m 'Updated subm.'
> popd
And if you were to use ammend here, too; there would be no problem;
In the parent there are now two commits, the first one pointing at the
first (unammended) commit in the submodule, the second pointing to
the corrected commit.
>
> The commit from before the amend was added to the super
> project (but never pushed) but has now been completely
> replaced. I still think this is a flaw in git. It shouldn't
> not complain and simply push.
The problem here is in the design.
When "on-demand" is set, the parent repo determines which
sumbodules needs pushing, then runs a plain "git push" in them
and then checks again.
The push operation in the submodule did not push out the un-amended
commit (which is correct IMO).
The parent in the second check determines that there is a commit, that
is not pushed, though. Maybe we need an option there "on-demand-but-no-recheck"
as a weaker promise what Git can deliver there.
>
> --
> Carlo Wood <carlo@alinoe.com>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-02-04 18:44 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-01-29 19:33 git push failing when push.recurseSubmodules on-demand and git commit --amend was used in submodule Carlo Wood
2017-01-30 1:00 ` Junio C Hamano
2017-01-31 22:08 ` Stefan Beller
2017-02-01 1:10 ` Carlo Wood
2017-02-04 18:43 ` Stefan Beller
2017-02-01 1:22 ` Carlo Wood
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).