HP-UX (git-2.14.0) Depot File / Source Code Build throws SHA1 Error

HP-UX (git-2.14.0) Depot File / Source Code Build throws SHA1 Error

[josephpattara .]

Dear HP-UX Git Maintainers,

Sending this email as I was trying to use the HP-UX depot file which
was recently committed to the
http://hpux.connectinternetsolutions.com/hppd/hpux/Development/Tools/git-2.14.0/.
I tried to use the depot file and also attempted to build the code
from the adapted source code (using native HP-UX compiler as mentioned
in Installation Note) from the website.

But unfortunately both of them failed with an error:

Cloning into xxxx...
remote: Counting objects: 19813, done.
remote: Compressing objects: 100% (9124/9124), done.
remote: Total 19813 (delta 10340), reused 16358 (delta 8293)
Receiving objects: 100% (19813/19813), 6.07 MiB | 9.72 MiB/s, done.
fatal: pack is corrupted (SHA1 mismatch)
fatal: index-pack failed

Looking at the dynamic link I can see the below:

_HP_DLDOPTS="-ldd" /usr/bin/git
  /usr/lib/hpux32/libpthread.so =>        /usr/lib/hpux32/libpthread.so
  libpthread.so.1 =>      /usr/lib/hpux32/libpthread.so.1
  libz.so =>      /usr/local/lib/hpux32/libz.so
  libiconv.so =>  /usr/local/lib/hpux32/libiconv.so
  libintl.so =>   /usr/local/lib/hpux32/libintl.so
  libc.so.1 =>    /usr/lib/hpux32/libc.so.1
  libdl.so.1 =>   /usr/lib/hpux32/libdl.so.1

  Until recently we were using the older GIT version for HP-UX 1.8.5.3
and I can see the older GIT had a dynamic link for libcrypto.so.

_HP_DLDOPTS="-ldd" /usr/local/bin/git
  libz.so =>      /usr/local/lib/hpux32/libz.so
  libiconv.so =>  /usr/local/lib/hpux32/libiconv.so
  libintl.so =>   /usr/local/lib/hpux32/libintl.so
  libcrypto.so => /usr/local/lib/hpux32/libcrypto.so
  libc.so.1 =>    /usr/lib/hpux32/libc.so.1
  libdl.so.1 =>   /usr/lib/hpux32/libdl.so.1

The clarification I have is
1. Is the version 2.14.0 from HPUX connect is it really usable (depot
file and also the source code)
2. Was it deliberate to remove the openssl linkage from the new GIT or
am I really missing any special configure or build flags to enable the
libcrypto.so linkage.
3. Is the SHA1 error which I get is it really to do with the missing
libcrypto.so or I am on a wrong inference track?

Thanks and Regards,
Joseph John

Re: HP-UX (git-2.14.0) Depot File / Source Code Build throws SHA1 Error

[Jeff King]

On Fri, Sep 01, 2017 at 03:04:36PM +0100, josephpattara . wrote:

> Dear HP-UX Git Maintainers,
> 
> Sending this email as I was trying to use the HP-UX depot file which
> was recently committed to the
> http://hpux.connectinternetsolutions.com/hppd/hpux/Development/Tools/git-2.14.0/.
> I tried to use the depot file and also attempted to build the code
> from the adapted source code (using native HP-UX compiler as mentioned
> in Installation Note) from the website.

I don't know anything about HP-UX, but I can tell you a few things from
the Git side...

> But unfortunately both of them failed with an error:
> 
> Cloning into xxxx...
> remote: Counting objects: 19813, done.
> remote: Compressing objects: 100% (9124/9124), done.
> remote: Total 19813 (delta 10340), reused 16358 (delta 8293)
> Receiving objects: 100% (19813/19813), 6.07 MiB | 9.72 MiB/s, done.
> fatal: pack is corrupted (SHA1 mismatch)
> fatal: index-pack failed

Assuming there's no corruption happening on the network, it looks like
there's a bug in the SHA-1 routines of your build. One very basic check
would be to run:

  echo foo | git hash-object --stdin

which should produce:

  257cc5642cb1a054f08cc83f2d943e56fd3ebe99

(If it does that's not a guarantee that there aren't problems with more
complex inputs, but if it produces a different sha1 that's a clear sign
of a sha1 issue).

> Looking at the dynamic link I can see the below:
> 
> _HP_DLDOPTS="-ldd" /usr/bin/git
>   /usr/lib/hpux32/libpthread.so =>        /usr/lib/hpux32/libpthread.so
>   libpthread.so.1 =>      /usr/lib/hpux32/libpthread.so.1
>   libz.so =>      /usr/local/lib/hpux32/libz.so
>   libiconv.so =>  /usr/local/lib/hpux32/libiconv.so
>   libintl.so =>   /usr/local/lib/hpux32/libintl.so
>   libc.so.1 =>    /usr/lib/hpux32/libc.so.1
>   libdl.so.1 =>   /usr/lib/hpux32/libdl.so.1
> 
>   Until recently we were using the older GIT version for HP-UX 1.8.5.3
> and I can see the older GIT had a dynamic link for libcrypto.so.

Git can build against several SHA-1 implementations. In v2.13 and up,
the default is the collision-detecting version publicized as part of the
SHAttered attacks earlier this year. We have had a few problems with it
misbehaving on uncommon platforms or compilers, so it seems a likely
candidate here.

You can try building with "make OPENSSL_SHA1=Yes" to use openssl instead
of the collision-detecting version. You can also try "make BLK_SHA1=Yes"
if you have trouble building against openssl for whatever reason.

> The clarification I have is
> 1. Is the version 2.14.0 from HPUX connect is it really usable (depot
> file and also the source code)

That I don't know. I took a peek at the diff between their source and a
pristine v2.14.0 tarball, and I don't see anything too exotic. So you
could try the upstream code from:

  https://www.kernel.org/pub/software/scm/git/git-2.14.1.tar.gz

but I suspect the problem would still exist.

> 2. Was it deliberate to remove the openssl linkage from the new GIT or
> am I really missing any special configure or build flags to enable the
> libcrypto.so linkage.

Yes, the default flipped but you can still change it. See above.

> 3. Is the SHA1 error which I get is it really to do with the missing
> libcrypto.so or I am on a wrong inference track?

I think it's likely that you're on the right track.

-Peff

Re: HP-UX (git-2.14.0) Depot File / Source Code Build throws SHA1 Error

[josephpattara .]

Hello Jeff,

Your information was very useful.

Firstly, I could see the SHA1 hash was definitely wrong.
-bash-4.3$ echo foo | ./git hash-object --stdin
408935f8873a5ef17ef10cf2a316e51937e90f0b

I attempted to build the code with OPENSSL_SHA1=Yes and the build is
working and I am able to do a git clone from my HP-UX machine. I still
used the adapted source code HPUX connect website.
I am wondering whether the package which was committed to the HPUX
site (http://hpux.connect.org.uk/) did ever worked for any one as it
was the same SHA1 error I was getting when I installed that depot
file. My HPUX architecture is below:

HP-UX hepscott B.11.31 U ia64 2256017764 unlimited-user license

But I can confirm with the adapated source code from HPUX connect and
using the  OPENSSL_SHA1=Yes I am able to build a working git locally.


Thanks a lot.
Kind Regards,
Joseph

On Fri, Sep 1, 2017 at 10:26 PM, Jeff King <peff@peff.net> wrote:
> On Fri, Sep 01, 2017 at 03:04:36PM +0100, josephpattara . wrote:
>
>> Dear HP-UX Git Maintainers,
>>
>> Sending this email as I was trying to use the HP-UX depot file which
>> was recently committed to the
>> http://hpux.connectinternetsolutions.com/hppd/hpux/Development/Tools/git-2.14.0/.
>> I tried to use the depot file and also attempted to build the code
>> from the adapted source code (using native HP-UX compiler as mentioned
>> in Installation Note) from the website.
>
> I don't know anything about HP-UX, but I can tell you a few things from
> the Git side...
>
>> But unfortunately both of them failed with an error:
>>
>> Cloning into xxxx...
>> remote: Counting objects: 19813, done.
>> remote: Compressing objects: 100% (9124/9124), done.
>> remote: Total 19813 (delta 10340), reused 16358 (delta 8293)
>> Receiving objects: 100% (19813/19813), 6.07 MiB | 9.72 MiB/s, done.
>> fatal: pack is corrupted (SHA1 mismatch)
>> fatal: index-pack failed
>
> Assuming there's no corruption happening on the network, it looks like
> there's a bug in the SHA-1 routines of your build. One very basic check
> would be to run:
>
>   echo foo | git hash-object --stdin
>
> which should produce:
>
>   257cc5642cb1a054f08cc83f2d943e56fd3ebe99
>
> (If it does that's not a guarantee that there aren't problems with more
> complex inputs, but if it produces a different sha1 that's a clear sign
> of a sha1 issue).
>
>> Looking at the dynamic link I can see the below:
>>
>> _HP_DLDOPTS="-ldd" /usr/bin/git
>>   /usr/lib/hpux32/libpthread.so =>        /usr/lib/hpux32/libpthread.so
>>   libpthread.so.1 =>      /usr/lib/hpux32/libpthread.so.1
>>   libz.so =>      /usr/local/lib/hpux32/libz.so
>>   libiconv.so =>  /usr/local/lib/hpux32/libiconv.so
>>   libintl.so =>   /usr/local/lib/hpux32/libintl.so
>>   libc.so.1 =>    /usr/lib/hpux32/libc.so.1
>>   libdl.so.1 =>   /usr/lib/hpux32/libdl.so.1
>>
>>   Until recently we were using the older GIT version for HP-UX 1.8.5.3
>> and I can see the older GIT had a dynamic link for libcrypto.so.
>
> Git can build against several SHA-1 implementations. In v2.13 and up,
> the default is the collision-detecting version publicized as part of the
> SHAttered attacks earlier this year. We have had a few problems with it
> misbehaving on uncommon platforms or compilers, so it seems a likely
> candidate here.
>
> You can try building with "make OPENSSL_SHA1=Yes" to use openssl instead
> of the collision-detecting version. You can also try "make BLK_SHA1=Yes"
> if you have trouble building against openssl for whatever reason.
>
>> The clarification I have is
>> 1. Is the version 2.14.0 from HPUX connect is it really usable (depot
>> file and also the source code)
>
> That I don't know. I took a peek at the diff between their source and a
> pristine v2.14.0 tarball, and I don't see anything too exotic. So you
> could try the upstream code from:
>
>   https://www.kernel.org/pub/software/scm/git/git-2.14.1.tar.gz
>
> but I suspect the problem would still exist.
>
>> 2. Was it deliberate to remove the openssl linkage from the new GIT or
>> am I really missing any special configure or build flags to enable the
>> libcrypto.so linkage.
>
> Yes, the default flipped but you can still change it. See above.
>
>> 3. Is the SHA1 error which I get is it really to do with the missing
>> libcrypto.so or I am on a wrong inference track?
>
> I think it's likely that you're on the right track.
>
> -Peff

Re: HP-UX (git-2.14.0) Depot File / Source Code Build throws SHA1 Error

[Junio C Hamano]

"josephpattara ." <josephpattara@gmail.com> writes:

> Firstly, I could see the SHA1 hash was definitely wrong.
> -bash-4.3$ echo foo | ./git hash-object --stdin
> 408935f8873a5ef17ef10cf2a316e51937e90f0b
> ...
> But I can confirm with the adapated source code from HPUX connect and
> using the  OPENSSL_SHA1=Yes I am able to build a working git locally.

I do not know if your HP-UX on Itanium is big/little endian (IIRC it
is selectable at the CPU level), but my blind guess is that this
might help before building without OPENSSL_SHA1=YesPlease?

        cat >>config.mak <<-\EOF
        BASIC_CFLAGS += -DSHA1DC_FORCE_BIGENDIAN=1
        EOF