Bug Report: Subtrees and GPG Signed Commits

Bug Report: Subtrees and GPG Signed Commits

[Stephen R Guglielmo]

Hi all,

I've noticed an issue regarding the use of `git subtree add` and `git
subtree pull` when the subtree repository's commit (either HEAD or
whatever commit specified by the subtree command) is signed with GPG.
It seems to work properly if the commit is not signed but previous
commits are.

The gist of the issue is that `git subtree add` does not add the
subree properly and a "fatal: Not a valid object name" error is
thrown. Running `git subtree pull` does not pull any upstream changes
after that ("'subtree' was never added").

I have not done extensive testing, however, below are instructions to
reproduce the issue. This was tested using git version 2.15.1
installed via Homebrew on MacOS. I did not test with the built-in
version of git on MacOS.

Thanks,
Steve

# Create a new repository
mkdir repoA && cd repoA
git init
echo "Test File in Repo A" > FileA
git add -A && git commit -m 'Initial commit in repo A'

# Create a second repository
cd .. && mkdir repoB && cd repoB
git init
echo "Test File in Repo B" > FileB
git add -A && git commit -m 'Initial commit in repo B'

# Create a signed commit in repo B
echo "Signed Commit" >> FileB
git commit -a -S  -m 'Signed commit in repo B'

# Now, add repoB as a subtree of RepoA
cd ../repoA
git subtree add --prefix repoB_subtree/ ../repoB/ master --squash
# Output:
git fetch ../repoB/ master
warning: no common commits
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 6 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (6/6), done.
From ../repoB
 * branch            master     -> FETCH_HEAD
fatal: Not a valid object name gpg: Signature made Sat Jan  6 17:38:31 2018 EST
gpg:                using RSA key 6900E9CFDD39B6A741D601F50999759F2DCF3E7C
gpg: Good signature from "Stephen Robert Guglielmo (Temple University
Computer Services) <srg@temple.edu>" [ultimate]
Primary key fingerprint: 6900 E9CF DD39 B6A7 41D6  01F5 0999 759F 2DCF 3E7C
4b700b1a4ebb9e2c1011aafd6b0f720b38f059a4
# Note, git exits with status 128 at this point.

# FileB was in fact added and staged to repoA, despite the "fatal"
above. Commit it:
git commit -m 'Add repoB subtree'

# Ok, let's make another commit in repoB and try a `subtree pull`
instead of `subtree add`
cd ../repoB
echo "Another Line" >> FileB
git commit -a -S -m 'Another signed commit'
cd ../repoA
git subtree pull --prefix repoB_subtree/ ../repoB master --squash
# Output:
warning: no common commits
remote: Counting objects: 9, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 9 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (9/9), done.
From ../repoB
 * branch            master     -> FETCH_HEAD
Can't squash-merge: 'repoB_subtree' was never added.
# Note, git exits with status 1 at this point.

# RepoB's third commit ('Another signed commit') is not pulled into
the subree in repo A.
# This can be verified by running a diff:
diff -qr --exclude ".git" repoB_subtree ../repoB
# Output:
Files repoB_subtree/FileB and ../repoB/FileB differ

Re: Bug Report: Subtrees and GPG Signed Commits

[Stephen R Guglielmo]

Hi, just following up on this bug report. I have not heard back. Is
there additional information that's needed? Is there a better place to
file bug reports?

Thanks

On Sat, Jan 6, 2018 at 5:45 PM, Stephen R Guglielmo
<srguglielmo@gmail.com> wrote:
> Hi all,
>
> I've noticed an issue regarding the use of `git subtree add` and `git
> subtree pull` when the subtree repository's commit (either HEAD or
> whatever commit specified by the subtree command) is signed with GPG.
> It seems to work properly if the commit is not signed but previous
> commits are.
>
> The gist of the issue is that `git subtree add` does not add the
> subree properly and a "fatal: Not a valid object name" error is
> thrown. Running `git subtree pull` does not pull any upstream changes
> after that ("'subtree' was never added").
>
> I have not done extensive testing, however, below are instructions to
> reproduce the issue. This was tested using git version 2.15.1
> installed via Homebrew on MacOS. I did not test with the built-in
> version of git on MacOS.
>
> Thanks,
> Steve
>
> # Create a new repository
> mkdir repoA && cd repoA
> git init
> echo "Test File in Repo A" > FileA
> git add -A && git commit -m 'Initial commit in repo A'
>
> # Create a second repository
> cd .. && mkdir repoB && cd repoB
> git init
> echo "Test File in Repo B" > FileB
> git add -A && git commit -m 'Initial commit in repo B'
>
> # Create a signed commit in repo B
> echo "Signed Commit" >> FileB
> git commit -a -S  -m 'Signed commit in repo B'
>
> # Now, add repoB as a subtree of RepoA
> cd ../repoA
> git subtree add --prefix repoB_subtree/ ../repoB/ master --squash
> # Output:
> git fetch ../repoB/ master
> warning: no common commits
> remote: Counting objects: 6, done.
> remote: Compressing objects: 100% (2/2), done.
> remote: Total 6 (delta 0), reused 0 (delta 0)
> Unpacking objects: 100% (6/6), done.
> From ../repoB
>  * branch            master     -> FETCH_HEAD
> fatal: Not a valid object name gpg: Signature made Sat Jan  6 17:38:31 2018 EST
> gpg:                using RSA key 6900E9CFDD39B6A741D601F50999759F2DCF3E7C
> gpg: Good signature from "Stephen Robert Guglielmo (Temple University
> Computer Services) <srg@temple.edu>" [ultimate]
> Primary key fingerprint: 6900 E9CF DD39 B6A7 41D6  01F5 0999 759F 2DCF 3E7C
> 4b700b1a4ebb9e2c1011aafd6b0f720b38f059a4
> # Note, git exits with status 128 at this point.
>
> # FileB was in fact added and staged to repoA, despite the "fatal"
> above. Commit it:
> git commit -m 'Add repoB subtree'
>
> # Ok, let's make another commit in repoB and try a `subtree pull`
> instead of `subtree add`
> cd ../repoB
> echo "Another Line" >> FileB
> git commit -a -S -m 'Another signed commit'
> cd ../repoA
> git subtree pull --prefix repoB_subtree/ ../repoB master --squash
> # Output:
> warning: no common commits
> remote: Counting objects: 9, done.
> remote: Compressing objects: 100% (3/3), done.
> remote: Total 9 (delta 0), reused 0 (delta 0)
> Unpacking objects: 100% (9/9), done.
> From ../repoB
>  * branch            master     -> FETCH_HEAD
> Can't squash-merge: 'repoB_subtree' was never added.
> # Note, git exits with status 1 at this point.
>
> # RepoB's third commit ('Another signed commit') is not pulled into
> the subree in repo A.
> # This can be verified by running a diff:
> diff -qr --exclude ".git" repoB_subtree ../repoB
> # Output:
> Files repoB_subtree/FileB and ../repoB/FileB differ

Re: Bug Report: Subtrees and GPG Signed Commits

[Stephen R Guglielmo]

Hi, just following up on this bug report. I have not heard back. Is
there additional information that's needed? Is there a better place to
file bug reports?

Additionally, I have confirmed that this bug still exists with git
version 2.16.1.

Thanks

On Thu, Jan 18, 2018 at 11:19 AM, Stephen R Guglielmo
<srguglielmo@gmail.com> wrote:
> Hi, just following up on this bug report. I have not heard back. Is
> there additional information that's needed? Is there a better place to
> file bug reports?
>
> Thanks
>
> On Sat, Jan 6, 2018 at 5:45 PM, Stephen R Guglielmo
> <srguglielmo@gmail.com> wrote:
>> Hi all,
>>
>> I've noticed an issue regarding the use of `git subtree add` and `git
>> subtree pull` when the subtree repository's commit (either HEAD or
>> whatever commit specified by the subtree command) is signed with GPG.
>> It seems to work properly if the commit is not signed but previous
>> commits are.
>>
>> The gist of the issue is that `git subtree add` does not add the
>> subree properly and a "fatal: Not a valid object name" error is
>> thrown. Running `git subtree pull` does not pull any upstream changes
>> after that ("'subtree' was never added").
>>
>> I have not done extensive testing, however, below are instructions to
>> reproduce the issue. This was tested using git version 2.15.1
>> installed via Homebrew on MacOS. I did not test with the built-in
>> version of git on MacOS.
>>
>> Thanks,
>> Steve
>>
>> # Create a new repository
>> mkdir repoA && cd repoA
>> git init
>> echo "Test File in Repo A" > FileA
>> git add -A && git commit -m 'Initial commit in repo A'
>>
>> # Create a second repository
>> cd .. && mkdir repoB && cd repoB
>> git init
>> echo "Test File in Repo B" > FileB
>> git add -A && git commit -m 'Initial commit in repo B'
>>
>> # Create a signed commit in repo B
>> echo "Signed Commit" >> FileB
>> git commit -a -S  -m 'Signed commit in repo B'
>>
>> # Now, add repoB as a subtree of RepoA
>> cd ../repoA
>> git subtree add --prefix repoB_subtree/ ../repoB/ master --squash
>> # Output:
>> git fetch ../repoB/ master
>> warning: no common commits
>> remote: Counting objects: 6, done.
>> remote: Compressing objects: 100% (2/2), done.
>> remote: Total 6 (delta 0), reused 0 (delta 0)
>> Unpacking objects: 100% (6/6), done.
>> From ../repoB
>>  * branch            master     -> FETCH_HEAD
>> fatal: Not a valid object name gpg: Signature made Sat Jan  6 17:38:31 2018 EST
>> gpg:                using RSA key 6900E9CFDD39B6A741D601F50999759F2DCF3E7C
>> gpg: Good signature from "Stephen Robert Guglielmo (Temple University
>> Computer Services) <srg@temple.edu>" [ultimate]
>> Primary key fingerprint: 6900 E9CF DD39 B6A7 41D6  01F5 0999 759F 2DCF 3E7C
>> 4b700b1a4ebb9e2c1011aafd6b0f720b38f059a4
>> # Note, git exits with status 128 at this point.
>>
>> # FileB was in fact added and staged to repoA, despite the "fatal"
>> above. Commit it:
>> git commit -m 'Add repoB subtree'
>>
>> # Ok, let's make another commit in repoB and try a `subtree pull`
>> instead of `subtree add`
>> cd ../repoB
>> echo "Another Line" >> FileB
>> git commit -a -S -m 'Another signed commit'
>> cd ../repoA
>> git subtree pull --prefix repoB_subtree/ ../repoB master --squash
>> # Output:
>> warning: no common commits
>> remote: Counting objects: 9, done.
>> remote: Compressing objects: 100% (3/3), done.
>> remote: Total 9 (delta 0), reused 0 (delta 0)
>> Unpacking objects: 100% (9/9), done.
>> From ../repoB
>>  * branch            master     -> FETCH_HEAD
>> Can't squash-merge: 'repoB_subtree' was never added.
>> # Note, git exits with status 1 at this point.
>>
>> # RepoB's third commit ('Another signed commit') is not pulled into
>> the subree in repo A.
>> # This can be verified by running a diff:
>> diff -qr --exclude ".git" repoB_subtree ../repoB
>> # Output:
>> Files repoB_subtree/FileB and ../repoB/FileB differ

Re: Bug Report: Subtrees and GPG Signed Commits

[Stefan Beller]

On Tue, Jan 30, 2018 at 11:15 AM, Stephen R Guglielmo
<srguglielmo@gmail.com> wrote:
> Hi, just following up on this bug report. I have not heard back. Is
> there additional information that's needed? Is there a better place to
> file bug reports?
>
> Additionally, I have confirmed that this bug still exists with git
> version 2.16.1.
>
> Thanks
>
> On Thu, Jan 18, 2018 at 11:19 AM, Stephen R Guglielmo
> <srguglielmo@gmail.com> wrote:
>> Hi, just following up on this bug report. I have not heard back. Is
>> there additional information that's needed? Is there a better place to
>> file bug reports?
>>
>> Thanks
>>
>> On Sat, Jan 6, 2018 at 5:45 PM, Stephen R Guglielmo
>> <srguglielmo@gmail.com> wrote:
>>> Hi all,
>>>
>>> I've noticed an issue regarding the use of `git subtree add` and `git
>>> subtree pull` when the subtree repository's commit (either HEAD or
>>> whatever commit specified by the subtree command) is signed with GPG.
>>> It seems to work properly if the commit is not signed but previous
>>> commits are.
>>>
>>> The gist of the issue is that `git subtree add` does not add the
>>> subree properly and a "fatal: Not a valid object name" error is
>>> thrown. Running `git subtree pull` does not pull any upstream changes
>>> after that ("'subtree' was never added").
>>>
>>> I have not done extensive testing, however, below are instructions to
>>> reproduce the issue. This was tested using git version 2.15.1
>>> installed via Homebrew on MacOS. I did not test with the built-in
>>> version of git on MacOS.
>>>
>>> Thanks,
>>> Steve
>>>
>>> # Create a new repository
>>> mkdir repoA && cd repoA
>>> git init
>>> echo "Test File in Repo A" > FileA
>>> git add -A && git commit -m 'Initial commit in repo A'
>>>
>>> # Create a second repository
>>> cd .. && mkdir repoB && cd repoB
>>> git init
>>> echo "Test File in Repo B" > FileB
>>> git add -A && git commit -m 'Initial commit in repo B'
>>>
>>> # Create a signed commit in repo B
>>> echo "Signed Commit" >> FileB
>>> git commit -a -S  -m 'Signed commit in repo B'
>>>
>>> # Now, add repoB as a subtree of RepoA
>>> cd ../repoA
>>> git subtree add --prefix repoB_subtree/ ../repoB/ master --squash
>>> # Output:
>>> git fetch ../repoB/ master
>>> warning: no common commits
>>> remote: Counting objects: 6, done.
>>> remote: Compressing objects: 100% (2/2), done.
>>> remote: Total 6 (delta 0), reused 0 (delta 0)
>>> Unpacking objects: 100% (6/6), done.
>>> From ../repoB
>>>  * branch            master     -> FETCH_HEAD
>>> fatal: Not a valid object name gpg: Signature made Sat Jan  6 17:38:31 2018 EST
>>> gpg:                using RSA key 6900E9CFDD39B6A741D601F50999759F2DCF3E7C
>>> gpg: Good signature from "Stephen Robert Guglielmo (Temple University
>>> Computer Services) <srg@temple.edu>" [ultimate]
>>> Primary key fingerprint: 6900 E9CF DD39 B6A7 41D6  01F5 0999 759F 2DCF 3E7C
>>> 4b700b1a4ebb9e2c1011aafd6b0f720b38f059a4
>>> # Note, git exits with status 128 at this point.
>>>
>>> # FileB was in fact added and staged to repoA, despite the "fatal"
>>> above. Commit it:
>>> git commit -m 'Add repoB subtree'
>>>
>>> # Ok, let's make another commit in repoB and try a `subtree pull`
>>> instead of `subtree add`
>>> cd ../repoB
>>> echo "Another Line" >> FileB
>>> git commit -a -S -m 'Another signed commit'
>>> cd ../repoA
>>> git subtree pull --prefix repoB_subtree/ ../repoB master --squash
>>> # Output:
>>> warning: no common commits
>>> remote: Counting objects: 9, done.
>>> remote: Compressing objects: 100% (3/3), done.
>>> remote: Total 9 (delta 0), reused 0 (delta 0)
>>> Unpacking objects: 100% (9/9), done.
>>> From ../repoB
>>>  * branch            master     -> FETCH_HEAD
>>> Can't squash-merge: 'repoB_subtree' was never added.
>>> # Note, git exits with status 1 at this point.
>>>
>>> # RepoB's third commit ('Another signed commit') is not pulled into
>>> the subree in repo A.
>>> # This can be verified by running a diff:
>>> diff -qr --exclude ".git" repoB_subtree ../repoB
>>> # Output:
>>> Files repoB_subtree/FileB and ../repoB/FileB differ

There has not been feedback for a while on this thread.
I think that is because subtrees are not in anyone's hot
interest area currently.

This is definitely the right place to submit&discuss bugs.
Looking through "git log --format="%ae %s" -S subtree",
it seems as if Avery (apenwarr@gmail.com) was mostly
interested in developing subtrees, though I think he has
moved on. Originally it was invented by Junio, who is
the active maintainer of the project in 68faf68938
(A new merge stragety 'subtree'., 2007-02-15)

Thanks,
Stefan

Re: Bug Report: Subtrees and GPG Signed Commits

[Junio C Hamano]

Stefan Beller <sbeller@google.com> writes:

> There has not been feedback for a while on this thread.
> I think that is because subtrees are not in anyone's hot
> interest area currently.
>
> This is definitely the right place to submit&discuss bugs.
> Looking through "git log --format="%ae %s" -S subtree",
> it seems as if Avery (apenwarr@gmail.com) was mostly
> interested in developing subtrees, though I think he has
> moved on. Originally it was invented by Junio, who is
> the active maintainer of the project in 68faf68938
> (A new merge stragety 'subtree'., 2007-02-15)

Thanks for trying to help, but I have *NOTHING* to do with the "git
subtree" subcommand (and I personally have no interest in it).  What
I did was a subtree merge strategy (i.e. "git merge -s subtree"),
which is totally a different thing.

David Greene offered to take it over in 2015, and then we saw some
activity by David Aguilar in 2016, but otherwise the subcommand from
contrib/ has pretty much been dormant these days.

Re: Bug Report: Subtrees and GPG Signed Commits

[Avery Pennarun]

On Tue, Jan 30, 2018 at 6:24 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Stefan Beller <sbeller@google.com> writes:
>> There has not been feedback for a while on this thread.
>> I think that is because subtrees are not in anyone's hot
>> interest area currently.
>>
>> This is definitely the right place to submit&discuss bugs.
>> Looking through "git log --format="%ae %s" -S subtree",
>> it seems as if Avery (apenwarr@gmail.com) was mostly
>> interested in developing subtrees, though I think he has
>> moved on. Originally it was invented by Junio, who is
>> the active maintainer of the project in 68faf68938
>> (A new merge stragety 'subtree'., 2007-02-15)
>
> Thanks for trying to help, but I have *NOTHING* to do with the "git
> subtree" subcommand (and I personally have no interest in it).  What
> I did was a subtree merge strategy (i.e. "git merge -s subtree"),
> which is totally a different thing.
>
> David Greene offered to take it over in 2015, and then we saw some
> activity by David Aguilar in 2016, but otherwise the subcommand from
> contrib/ has pretty much been dormant these days.

Strictly speaking, the 'git subtree' command does in fact use 'git
merge -s subtree' under the covers, so Junio is at least partly
responsible for giving me the idea :)

I actually have never looked into how signed commits work and although
I still use git-subtree occasionally (it hasn't needed any
maintenance, for my simple use cases), I have never used it with
signed commits.

git-subtree maintains a cache that maps commit ids in the "original
project" with their equivalents in the "merged project."  If there's
something magic about how commit ids work with signed commits, I could
imagine that causing the "no a valid object name" problems.  Or,
git-subtree in --squash mode actually generates new commit objects
using some magic of its own.  If it were to accidentally copy a
signature into a commit that no longer matches the original, I imagine
that new object might get rejected.

Unfortunately I don't have time to look into it.  The git-subtree code
is pretty straightforward, though, so if Stephen has an hour or two to
look deeper it's probably possible to fix it up.  The tool is not
actually as magical and difficult as it might seem at first glance :)

Sorry I can't help more.

Good luck,

Avery

Re: Bug Report: Subtrees and GPG Signed Commits

[Stephen R Guglielmo]

On Tue, Jan 30, 2018 at 6:37 PM, Avery Pennarun <apenwarr@gmail.com> wrote:
> On Tue, Jan 30, 2018 at 6:24 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Stefan Beller <sbeller@google.com> writes:
>>> There has not been feedback for a while on this thread.
>>> I think that is because subtrees are not in anyone's hot
>>> interest area currently.
>>>
>>> This is definitely the right place to submit&discuss bugs.
>>> Looking through "git log --format="%ae %s" -S subtree",
>>> it seems as if Avery (apenwarr@gmail.com) was mostly
>>> interested in developing subtrees, though I think he has
>>> moved on. Originally it was invented by Junio, who is
>>> the active maintainer of the project in 68faf68938
>>> (A new merge stragety 'subtree'., 2007-02-15)
>>
>> Thanks for trying to help, but I have *NOTHING* to do with the "git
>> subtree" subcommand (and I personally have no interest in it).  What
>> I did was a subtree merge strategy (i.e. "git merge -s subtree"),
>> which is totally a different thing.
>>
>> David Greene offered to take it over in 2015, and then we saw some
>> activity by David Aguilar in 2016, but otherwise the subcommand from
>> contrib/ has pretty much been dormant these days.
>
> Strictly speaking, the 'git subtree' command does in fact use 'git
> merge -s subtree' under the covers, so Junio is at least partly
> responsible for giving me the idea :)
>
> I actually have never looked into how signed commits work and although
> I still use git-subtree occasionally (it hasn't needed any
> maintenance, for my simple use cases), I have never used it with
> signed commits.
>
> git-subtree maintains a cache that maps commit ids in the "original
> project" with their equivalents in the "merged project."  If there's
> something magic about how commit ids work with signed commits, I could
> imagine that causing the "no a valid object name" problems.  Or,
> git-subtree in --squash mode actually generates new commit objects
> using some magic of its own.  If it were to accidentally copy a
> signature into a commit that no longer matches the original, I imagine
> that new object might get rejected.
>
> Unfortunately I don't have time to look into it.  The git-subtree code
> is pretty straightforward, though, so if Stephen has an hour or two to
> look deeper it's probably possible to fix it up.  The tool is not
> actually as magical and difficult as it might seem at first glance :)
>
> Sorry I can't help more.
>
> Good luck,
>
> Avery

Thanks all for the discussion/replies.

We use subtrees extensively in our environment right now. The "sub"
repos (90+) are located on GitHub, while the "main/parent" repo is
provided by a vendor on website hosting infrastructure.

I will take a look at:
git/Documentation/CodingGuidelines
git/Documentation/SubmittingPatches
git/contrib/subtree/

Should I follow up in this thread with a patch (it might be a while)?

Thanks!
Steve

Re: Bug Report: Subtrees and GPG Signed Commits

[Junio C Hamano]

Stephen R Guglielmo <srguglielmo@gmail.com> writes:

> On Tue, Jan 30, 2018 at 6:37 PM, Avery Pennarun <apenwarr@gmail.com> wrote:
>>
>> Sorry I can't help more.
>>
>> Good luck,
>>
>> Avery
>
> Thanks all for the discussion/replies.
>
> We use subtrees extensively in our environment right now. The "sub"
> repos (90+) are located on GitHub, while the "main/parent" repo is
> provided by a vendor on website hosting infrastructure.
>
> I will take a look at:
> git/Documentation/CodingGuidelines
> git/Documentation/SubmittingPatches
> git/contrib/subtree/
>
> Should I follow up in this thread with a patch (it might be a while)?

These three are good place to start at.  You may find the output of
"git shortlog --no-merges --since=N.months contrib/subtree" and "git
blame contrib/subtree" also a good source of whom to ask for help.
As we said on this thread, this is a corner of the contrib/ section
that nobody seems to be actively working on, so if you really depend
on it working well, you might have to take the ownership of it ;-)

Thanks.

Re: Bug Report: Subtrees and GPG Signed Commits

[Stephen R Guglielmo]

On Wed, Jan 31, 2018 at 7:33 AM, Stephen R Guglielmo
<srguglielmo@gmail.com> wrote:
> On Tue, Jan 30, 2018 at 6:37 PM, Avery Pennarun <apenwarr@gmail.com> wrote:
>> On Tue, Jan 30, 2018 at 6:24 PM, Junio C Hamano <gitster@pobox.com> wrote:
>>> Stefan Beller <sbeller@google.com> writes:
>>>> There has not been feedback for a while on this thread.
>>>> I think that is because subtrees are not in anyone's hot
>>>> interest area currently.
>>>>
>>>> This is definitely the right place to submit&discuss bugs.
>>>> Looking through "git log --format="%ae %s" -S subtree",
>>>> it seems as if Avery (apenwarr@gmail.com) was mostly
>>>> interested in developing subtrees, though I think he has
>>>> moved on. Originally it was invented by Junio, who is
>>>> the active maintainer of the project in 68faf68938
>>>> (A new merge stragety 'subtree'., 2007-02-15)
>>>
>>> Thanks for trying to help, but I have *NOTHING* to do with the "git
>>> subtree" subcommand (and I personally have no interest in it).  What
>>> I did was a subtree merge strategy (i.e. "git merge -s subtree"),
>>> which is totally a different thing.
>>>
>>> David Greene offered to take it over in 2015, and then we saw some
>>> activity by David Aguilar in 2016, but otherwise the subcommand from
>>> contrib/ has pretty much been dormant these days.
>>
>> Strictly speaking, the 'git subtree' command does in fact use 'git
>> merge -s subtree' under the covers, so Junio is at least partly
>> responsible for giving me the idea :)
>>
>> I actually have never looked into how signed commits work and although
>> I still use git-subtree occasionally (it hasn't needed any
>> maintenance, for my simple use cases), I have never used it with
>> signed commits.
>>
>> git-subtree maintains a cache that maps commit ids in the "original
>> project" with their equivalents in the "merged project."  If there's
>> something magic about how commit ids work with signed commits, I could
>> imagine that causing the "no a valid object name" problems.  Or,
>> git-subtree in --squash mode actually generates new commit objects
>> using some magic of its own.  If it were to accidentally copy a
>> signature into a commit that no longer matches the original, I imagine
>> that new object might get rejected.
>>
>> Unfortunately I don't have time to look into it.  The git-subtree code
>> is pretty straightforward, though, so if Stephen has an hour or two to
>> look deeper it's probably possible to fix it up.  The tool is not
>> actually as magical and difficult as it might seem at first glance :)
>>
>> Sorry I can't help more.
>>
>> Good luck,
>>
>> Avery
>
> Thanks all for the discussion/replies.
>
> We use subtrees extensively in our environment right now. The "sub"
> repos (90+) are located on GitHub, while the "main/parent" repo is
> provided by a vendor on website hosting infrastructure.
>
> I will take a look at:
> git/Documentation/CodingGuidelines
> git/Documentation/SubmittingPatches
> git/contrib/subtree/
>
> Should I follow up in this thread with a patch (it might be a while)?
>
> Thanks!
> Steve

Hi all,

It looks like I've found the cause of the issue. I have
log.showsignature=true in my gitconfig. The toptree_for_commit()
function calls `git log` and passes the output to `git commit-tree` in
new_squash_commit(). Apparently commit-tree doesn't like GPG sigs.

The fix was simple: --no-show-signature. However, I believe this was
added in git v2.10.0, so it's not fully backwards compatible. I'm open
to suggestions on a better fix if this is not acceptable.

Thanks!


https://github.com/srguglielmo/git/commit/822c8a45d049f86ea5c59c0b434303964e4e6f3d



diff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh
index cc033af73..dec085a23 100755
--- a/contrib/subtree/git-subtree.sh
+++ b/contrib/subtree/git-subtree.sh
@@ -475,7 +475,7 @@ squash_msg () {

 toptree_for_commit () {
        commit="$1"
-       git log -1 --pretty=format:'%T' "$commit" -- || exit $?
+       git log --no-show-signature -1 --pretty=format:'%T' "$commit"
-- || exit $?
 }

 subtree_for_commit () {

Re: Bug Report: Subtrees and GPG Signed Commits

[Stephen R Guglielmo]

On Mon, Feb 5, 2018 at 9:30 AM, Stephen R Guglielmo
<srguglielmo@gmail.com> wrote:
> On Wed, Jan 31, 2018 at 7:33 AM, Stephen R Guglielmo
> <srguglielmo@gmail.com> wrote:
>> On Tue, Jan 30, 2018 at 6:37 PM, Avery Pennarun <apenwarr@gmail.com> wrote:
>>> On Tue, Jan 30, 2018 at 6:24 PM, Junio C Hamano <gitster@pobox.com> wrote:
>>>> Stefan Beller <sbeller@google.com> writes:
>>>>> There has not been feedback for a while on this thread.
>>>>> I think that is because subtrees are not in anyone's hot
>>>>> interest area currently.
>>>>>
>>>>> This is definitely the right place to submit&discuss bugs.
>>>>> Looking through "git log --format="%ae %s" -S subtree",
>>>>> it seems as if Avery (apenwarr@gmail.com) was mostly
>>>>> interested in developing subtrees, though I think he has
>>>>> moved on. Originally it was invented by Junio, who is
>>>>> the active maintainer of the project in 68faf68938
>>>>> (A new merge stragety 'subtree'., 2007-02-15)
>>>>
>>>> Thanks for trying to help, but I have *NOTHING* to do with the "git
>>>> subtree" subcommand (and I personally have no interest in it).  What
>>>> I did was a subtree merge strategy (i.e. "git merge -s subtree"),
>>>> which is totally a different thing.
>>>>
>>>> David Greene offered to take it over in 2015, and then we saw some
>>>> activity by David Aguilar in 2016, but otherwise the subcommand from
>>>> contrib/ has pretty much been dormant these days.
>>>
>>> Strictly speaking, the 'git subtree' command does in fact use 'git
>>> merge -s subtree' under the covers, so Junio is at least partly
>>> responsible for giving me the idea :)
>>>
>>> I actually have never looked into how signed commits work and although
>>> I still use git-subtree occasionally (it hasn't needed any
>>> maintenance, for my simple use cases), I have never used it with
>>> signed commits.
>>>
>>> git-subtree maintains a cache that maps commit ids in the "original
>>> project" with their equivalents in the "merged project."  If there's
>>> something magic about how commit ids work with signed commits, I could
>>> imagine that causing the "no a valid object name" problems.  Or,
>>> git-subtree in --squash mode actually generates new commit objects
>>> using some magic of its own.  If it were to accidentally copy a
>>> signature into a commit that no longer matches the original, I imagine
>>> that new object might get rejected.
>>>
>>> Unfortunately I don't have time to look into it.  The git-subtree code
>>> is pretty straightforward, though, so if Stephen has an hour or two to
>>> look deeper it's probably possible to fix it up.  The tool is not
>>> actually as magical and difficult as it might seem at first glance :)
>>>
>>> Sorry I can't help more.
>>>
>>> Good luck,
>>>
>>> Avery
>>
>> Thanks all for the discussion/replies.
>>
>> We use subtrees extensively in our environment right now. The "sub"
>> repos (90+) are located on GitHub, while the "main/parent" repo is
>> provided by a vendor on website hosting infrastructure.
>>
>> I will take a look at:
>> git/Documentation/CodingGuidelines
>> git/Documentation/SubmittingPatches
>> git/contrib/subtree/
>>
>> Should I follow up in this thread with a patch (it might be a while)?
>>
>> Thanks!
>> Steve
>
> Hi all,
>
> It looks like I've found the cause of the issue. I have
> log.showsignature=true in my gitconfig. The toptree_for_commit()
> function calls `git log` and passes the output to `git commit-tree` in
> new_squash_commit(). Apparently commit-tree doesn't like GPG sigs.
>
> The fix was simple: --no-show-signature. However, I believe this was
> added in git v2.10.0, so it's not fully backwards compatible. I'm open
> to suggestions on a better fix if this is not acceptable.
>
> Thanks!
>
>
> https://github.com/srguglielmo/git/commit/822c8a45d049f86ea5c59c0b434303964e4e6f3d
>
>
>
> diff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh
> index cc033af73..dec085a23 100755
> --- a/contrib/subtree/git-subtree.sh
> +++ b/contrib/subtree/git-subtree.sh
> @@ -475,7 +475,7 @@ squash_msg () {
>
>  toptree_for_commit () {
>         commit="$1"
> -       git log -1 --pretty=format:'%T' "$commit" -- || exit $?
> +       git log --no-show-signature -1 --pretty=format:'%T' "$commit"
> -- || exit $?
>  }
>
>  subtree_for_commit () {

Hey again,

Actually, to follow up on this, I added --no-show-signature to several
other locations. The above patch fixes the fatal, however the GPG sig
info is still included in the commit merge message for `subtree pull`.
This fixes that as well.

https://github.com/srguglielmo/git/commit/ebd2f628ddb960931aac5087c45a54b953976e99


diff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh
index cc033af73..8126132dc 100755
--- a/contrib/subtree/git-subtree.sh
+++ b/contrib/subtree/git-subtree.sh
@@ -297,7 +297,7 @@ find_latest_squash () {
        main=
        sub=
        git log --grep="^git-subtree-dir: $dir/*\$" \
-               --pretty=format:'START %H%n%s%n%n%b%nEND%n' HEAD |
+               --no-show-signature --pretty=format:'START
%H%n%s%n%n%b%nEND%n' HEAD |
        while read a b junk
        do
                debug "$a $b $junk"
@@ -341,7 +341,7 @@ find_existing_splits () {
        main=
        sub=
        git log --grep="^git-subtree-dir: $dir/*\$" \
-               --pretty=format:'START %H%n%s%n%n%b%nEND%n' $revs |
+               --no-show-signature --pretty=format:'START
%H%n%s%n%n%b%nEND%n' $revs |
        while read a b junk
        do
                case "$a" in
@@ -382,7 +382,7 @@ copy_commit () {
        # We're going to set some environment vars here, so
        # do it in a subshell to get rid of them safely later
        debug copy_commit "{$1}" "{$2}" "{$3}"
-       git log -1 --pretty=format:'%an%n%ae%n%aD%n%cn%n%ce%n%cD%n%B' "$1" |
+       git log --no-show-signature -1
--pretty=format:'%an%n%ae%n%aD%n%cn%n%ce%n%cD%n%B' "$1" |
        (
                read GIT_AUTHOR_NAME
                read GIT_AUTHOR_EMAIL
@@ -462,8 +462,8 @@ squash_msg () {
                oldsub_short=$(git rev-parse --short "$oldsub")
                echo "Squashed '$dir/' changes from
$oldsub_short..$newsub_short"
                echo
-               git log --pretty=tformat:'%h %s' "$oldsub..$newsub"
-               git log --pretty=tformat:'REVERT: %h %s' "$newsub..$oldsub"
+               git log --no-show-signature --pretty=tformat:'%h %s'
"$oldsub..$newsub"
+               git log --no-show-signature --pretty=tformat:'REVERT:
%h %s' "$newsub..$oldsub"
        else
                echo "Squashed '$dir/' content from commit $newsub_short"
        fi

Re: Bug Report: Subtrees and GPG Signed Commits

[Junio C Hamano]

Stephen R Guglielmo <srguglielmo@gmail.com> writes:

> diff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh
> index cc033af73..dec085a23 100755
> --- a/contrib/subtree/git-subtree.sh
> +++ b/contrib/subtree/git-subtree.sh
> @@ -475,7 +475,7 @@ squash_msg () {
>
>  toptree_for_commit () {
>         commit="$1"
> -       git log -1 --pretty=format:'%T' "$commit" -- || exit $?
> +       git log --no-show-signature -1 --pretty=format:'%T' "$commit"
> -- || exit $?
>  }

Given that all references to this shell function seem to do

	sometree=$(toptree_for_commit $something)

and then $sometree is used as if it were a tree object name, I can
understand why the lack of --no-show-signature in the original
breaks it when the user has show-signature configured.

It probably makes more sense to replace the "git log" with something
more appropirate for the job, like

	git rev-parse --verify "$commit^{tree}"

though.

Re: Bug Report: Subtrees and GPG Signed Commits

[Stephen R Guglielmo]

On Mon, Feb 5, 2018 at 1:45 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Given that all references to this shell function seem to do
>
>         sometree=$(toptree_for_commit $something)
>
> and then $sometree is used as if it were a tree object name, I can
> understand why the lack of --no-show-signature in the original
> breaks it when the user has show-signature configured.
>
> It probably makes more sense to replace the "git log" with something
> more appropirate for the job, like
>
>         git rev-parse --verify "$commit^{tree}"
>
> though.

Junio,

Thanks for the feedback. I read the documentation on `rev-parse` and I
agree it makes more sense for this case. I've modified
`toptree_for_commit()` to use `rev-parse` instead. However, there's a
few other calls to `log` that I'm not sure can be replaced. For those,
I've kept `--no-show-signature`, making this require at least v2.10.0.

Below is the full diff against v2.16.1. Or, for the GitHub-savvy among us:

https://github.com/srguglielmo/git/compare/8279ed033f703d4115bee620dccd32a9ec94d9aa...srguglielmo:4b92b4494da057ed52c2ff8c329457a1c294d135

Thanks,
Steve

diff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh
index dec085a23..9594ca4b5 100755
--- a/contrib/subtree/git-subtree.sh
+++ b/contrib/subtree/git-subtree.sh
@@ -297,7 +297,7 @@ find_latest_squash () {
     main=
     sub=
     git log --grep="^git-subtree-dir: $dir/*\$" \
-        --pretty=format:'START %H%n%s%n%n%b%nEND%n' HEAD |
+        --no-show-signature --pretty=format:'START %H%n%s%n%n%b%nEND%n' HEAD |
     while read a b junk
     do
         debug "$a $b $junk"
@@ -341,7 +341,7 @@ find_existing_splits () {
     main=
     sub=
     git log --grep="^git-subtree-dir: $dir/*\$" \
-        --pretty=format:'START %H%n%s%n%n%b%nEND%n' $revs |
+        --no-show-signature --pretty=format:'START %H%n%s%n%n%b%nEND%n' $revs |
     while read a b junk
     do
         case "$a" in
@@ -382,7 +382,7 @@ copy_commit () {
     # We're going to set some environment vars here, so
     # do it in a subshell to get rid of them safely later
     debug copy_commit "{$1}" "{$2}" "{$3}"
-    git log -1 --pretty=format:'%an%n%ae%n%aD%n%cn%n%ce%n%cD%n%B' "$1" |
+    git log --no-show-signature -1
--pretty=format:'%an%n%ae%n%aD%n%cn%n%ce%n%cD%n%B' "$1" |
     (
         read GIT_AUTHOR_NAME
         read GIT_AUTHOR_EMAIL
@@ -462,8 +462,8 @@ squash_msg () {
         oldsub_short=$(git rev-parse --short "$oldsub")
         echo "Squashed '$dir/' changes from $oldsub_short..$newsub_short"
         echo
-        git log --pretty=tformat:'%h %s' "$oldsub..$newsub"
-        git log --pretty=tformat:'REVERT: %h %s' "$newsub..$oldsub"
+        git log --no-show-signature --pretty=tformat:'%h %s' "$oldsub..$newsub"
+        git log --no-show-signature --pretty=tformat:'REVERT: %h %s'
"$newsub..$oldsub"
     else
         echo "Squashed '$dir/' content from commit $newsub_short"
     fi
@@ -475,7 +475,7 @@ squash_msg () {

 toptree_for_commit () {
     commit="$1"
-    git log -1 --pretty=format:'%T' "$commit" -- || exit $?
+    git rev-parse --verify "$commit^{tree}" || exit $?
 }

 subtree_for_commit () {