Git Modifying DLL

Git Modifying DLL

[Jonathon Anderson]

I'm having an issue with git modifying a DLL file and corrupting it.
When I download the original working file, it has a hash starting with
8FE400... I then commit the DLL and push it to our repo. When I
download the file from the repo, the DLL can't be loaded, and it has a
hash starting with E004FB...

Opening the DLL in a hex editor and using the compare feature, there's
a single change to the file. In the original, the byte code starting
at 0x0074 is 2E 0D 0D 0A 24.
In the git file, the byte code starting at 0x0074 is 2E 0D 0A 24

A single carriage return character (0x0D) has been removed, and the
file size has changed from 260,608 bytes to 260,607 bytes.

I ruled out the possibility that the repo server was doing anything to
the file because I deleted the file in my local repository then ran
"git reset --hard HEAD" to restore the file, and the hash had once
again changed to E004FB...

OS: Windows 10.0.19043 pro
git: 2.32.0.windows.1

I have no settings configured for git behavior handling line endings.

The original DLL can be found here:
https://www.powershellgallery.com/packages/PSWindowsUpdate/2.1.1.2

Navigate to "Manual Download", download the nuget package and unzip
it. The file is PSWindowsUpdate.dll

Thank you,
Jonathon

RE: Git Modifying DLL

[Randall S. Becker]

On August 19, 2021 1:59 PM, Jonathon Anderson wrote:
>
>I'm having an issue with git modifying a DLL file and corrupting it.
>When I download the original working file, it has a hash starting with 8FE400... I then commit the DLL and push it to our repo. When I
>download the file from the repo, the DLL can't be loaded, and it has a hash starting with E004FB...
>
>Opening the DLL in a hex editor and using the compare feature, there's a single change to the file. In the original, the byte code starting
>at 0x0074 is 2E 0D 0D 0A 24.
>In the git file, the byte code starting at 0x0074 is 2E 0D 0A 24
>
>A single carriage return character (0x0D) has been removed, and the file size has changed from 260,608 bytes to 260,607 bytes.
>
>I ruled out the possibility that the repo server was doing anything to the file because I deleted the file in my local repository then ran "git
>reset --hard HEAD" to restore the file, and the hash had once again changed to E004FB...
>
>OS: Windows 10.0.19043 pro
>git: 2.32.0.windows.1
>
>I have no settings configured for git behavior handling line endings.
>
>The original DLL can be found here:
>https://www.powershellgallery.com/packages/PSWindowsUpdate/2.1.1.2
>
>Navigate to "Manual Download", download the nuget package and unzip it. The file is PSWindowsUpdate.dll

Have you set up an entry for *.dll as binary in your .gitattributes file?

-Randall

Re: Git Modifying DLL

[Jonathon Anderson]

I had not. I tested that and it worked. I assumed that git would
automatically treat dll files as binary. Thanks for the help!

On Thu, Aug 19, 2021 at 1:13 PM Randall S. Becker
<rsbecker@nexbridge.com> wrote:
>
> On August 19, 2021 1:59 PM, Jonathon Anderson wrote:
> >
> >I'm having an issue with git modifying a DLL file and corrupting it.
> >When I download the original working file, it has a hash starting with 8FE400... I then commit the DLL and push it to our repo. When I
> >download the file from the repo, the DLL can't be loaded, and it has a hash starting with E004FB...
> >
> >Opening the DLL in a hex editor and using the compare feature, there's a single change to the file. In the original, the byte code starting
> >at 0x0074 is 2E 0D 0D 0A 24.
> >In the git file, the byte code starting at 0x0074 is 2E 0D 0A 24
> >
> >A single carriage return character (0x0D) has been removed, and the file size has changed from 260,608 bytes to 260,607 bytes.
> >
> >I ruled out the possibility that the repo server was doing anything to the file because I deleted the file in my local repository then ran "git
> >reset --hard HEAD" to restore the file, and the hash had once again changed to E004FB...
> >
> >OS: Windows 10.0.19043 pro
> >git: 2.32.0.windows.1
> >
> >I have no settings configured for git behavior handling line endings.
> >
> >The original DLL can be found here:
> >https://www.powershellgallery.com/packages/PSWindowsUpdate/2.1.1.2
> >
> >Navigate to "Manual Download", download the nuget package and unzip it. The file is PSWindowsUpdate.dll
>
> Have you set up an entry for *.dll as binary in your .gitattributes file?
>
> -Randall
>

Re: Git Modifying DLL

[Torsten Bögershausen]

First of all, thanks for the report.
(And please, no top-posting in this list, see my comments at the end)

On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:
> I had not. I tested that and it worked. I assumed that git would
> automatically treat dll files as binary. Thanks for the help!
>
> On Thu, Aug 19, 2021 at 1:13 PM Randall S. Becker
> <rsbecker@nexbridge.com> wrote:
> >
> > On August 19, 2021 1:59 PM, Jonathon Anderson wrote:
> > >
> > >I'm having an issue with git modifying a DLL file and corrupting it.
> > >When I download the original working file, it has a hash starting with 8FE400... I then commit the DLL and push it to our repo. When I
> > >download the file from the repo, the DLL can't be loaded, and it has a hash starting with E004FB...
> > >
> > >Opening the DLL in a hex editor and using the compare feature, there's a single change to the file. In the original, the byte code starting
> > >at 0x0074 is 2E 0D 0D 0A 24.
> > >In the git file, the byte code starting at 0x0074 is 2E 0D 0A 24
> > >
> > >A single carriage return character (0x0D) has been removed, and the file size has changed from 260,608 bytes to 260,607 bytes.
> > >
> > >I ruled out the possibility that the repo server was doing anything to the file because I deleted the file in my local repository then ran "git
> > >reset --hard HEAD" to restore the file, and the hash had once again changed to E004FB...
> > >
> > >OS: Windows 10.0.19043 pro
> > >git: 2.32.0.windows.1
> > >
> > >I have no settings configured for git behavior handling line endings.
> > >
> > >The original DLL can be found here:
> > >https://www.powershellgallery.com/packages/PSWindowsUpdate/2.1.1.2
> > >
> > >Navigate to "Manual Download", download the nuget package and unzip it. The file is PSWindowsUpdate.dll
> >
> > Have you set up an entry for *.dll as binary in your .gitattributes file?
> >
> > -Randall
> >

I downloaded the dll.

I can not reproduce the issue here - so I wonder, what is going on.

Could you run the following experiment, please ?
(in git bash or so)

mkdir  dlltest
cd dlltest
git init
cp whereveritis/PSWindowsUpdate.dll .

git ls-files -o --eol

# I get
git ls-files --eol -o PSWindowsUpdate.dll
i/      w/-text attr/			PSWindowsUpdate.dll

This means, that the file is "classified" as "-text" in the working tree.
"-text" means the same as binary.
And git should not change CRLF at all.

Do you have any more .gitattributes in you repo?
Of course, you can run the command inside your repo.
Omit the "-o" for other and simply run
git ls-files --eol

Re: Git Modifying DLL

[Jeff King]

On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:

> I had not. I tested that and it worked. I assumed that git would
> automatically treat dll files as binary. Thanks for the help!

Git doesn't know about any file extensions by default. Its default "is
it binary" test looks for NUL bytes in the first 8k or so of the file.
I'd expect your DLL would probably have such a NUL byte.

Is it possible you have other .gitattributes set which are confusing
things?

You might try:

  git check-attr --all <path>

or:

  git ls-files --stdin | git check-attr --stdin --all

-Peff

Re: Git Modifying DLL

[Jonathon Anderson]

On Fri, Aug 20, 2021 at 1:00 PM Jeff King <peff@peff.net> wrote:
>
> On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:
>
> > I had not. I tested that and it worked. I assumed that git would
> > automatically treat dll files as binary. Thanks for the help!
>
> Git doesn't know about any file extensions by default. Its default "is
> it binary" test looks for NUL bytes in the first 8k or so of the file.
> I'd expect your DLL would probably have such a NUL byte.
>
> Is it possible you have other .gitattributes set which are confusing
> things?
>
> You might try:
>
>   git check-attr --all <path>
>
> or:
>
>   git ls-files --stdin | git check-attr --stdin --all
>
> -Peff

When I remove '*.dll binary" from .gitattributes, I get this:

$ git check-attr --all ./PSWindowsUpdate.dll
./PSWindowsUpdate.dll: text: set
./PSWindowsUpdate.dll: eol: lf



When I add "*.dll binary" to .gitattributes, I get this:

$ git check-attr --all ./PSWindowsUpdate.dll
./PSWindowsUpdate.dll: binary: set
./PSWindowsUpdate.dll: diff: unset
./PSWindowsUpdate.dll: merge: unset
./PSWindowsUpdate.dll: text: unset
./PSWindowsUpdate.dll: eol: lf


I can also confirm that there are null bytes in the first 8k bytes.
These are the first 16 bytes

4D 5A 90 00 03 00 00 00 04 00 00 00 FF FF 00 00

Thanks,
Jonathon

Re: Git Modifying DLL

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1795 bytes --]

On 2021-08-20 at 18:47:02, Jonathon Anderson wrote:
> On Fri, Aug 20, 2021 at 1:00 PM Jeff King <peff@peff.net> wrote:
> >
> > On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:
> >
> > > I had not. I tested that and it worked. I assumed that git would
> > > automatically treat dll files as binary. Thanks for the help!
> >
> > Git doesn't know about any file extensions by default. Its default "is
> > it binary" test looks for NUL bytes in the first 8k or so of the file.
> > I'd expect your DLL would probably have such a NUL byte.
> >
> > Is it possible you have other .gitattributes set which are confusing
> > things?
> >
> > You might try:
> >
> >   git check-attr --all <path>
> >
> > or:
> >
> >   git ls-files --stdin | git check-attr --stdin --all
> >
> > -Peff
> 
> When I remove '*.dll binary" from .gitattributes, I get this:
> 
> $ git check-attr --all ./PSWindowsUpdate.dll
> ./PSWindowsUpdate.dll: text: set
> ./PSWindowsUpdate.dll: eol: lf

Yes, this is definitely not correct.  The flag "text" being set tells
Git to do line-ending conversion and "eol=lf" says to convert line
endings into LF.

You should look for things in your .gitattributes file that say
something like "* text", which you probably don't want.  You could use
"* text=auto", which should be fine for most cases, though.  It's also
possible those aren't in a .gitattributes file in your repository but
one elsewhere on your system.  You can check gitattributes(5) for the
locations of other files that can affect it.

As a note, it is best practice not to check binary dependencies or build
artifacts into the repo.  Those are best stored elsewhere, such as an
artifact server.
-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

RE: Git Modifying DLL

[Randall S. Becker]

On August 20, 2021 3:15 PM, brian m. carlson wrote:
>On 2021-08-20 at 18:47:02, Jonathon Anderson wrote:
>> On Fri, Aug 20, 2021 at 1:00 PM Jeff King <peff@peff.net> wrote:
>> >
>> > On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:
>> >
>> > > I had not. I tested that and it worked. I assumed that git would
>> > > automatically treat dll files as binary. Thanks for the help!
>> >
>> > Git doesn't know about any file extensions by default. Its default
>> > "is it binary" test looks for NUL bytes in the first 8k or so of the file.
>> > I'd expect your DLL would probably have such a NUL byte.
>> >
>> > Is it possible you have other .gitattributes set which are confusing
>> > things?
>> >
>> > You might try:
>> >
>> >   git check-attr --all <path>
>> >
>> > or:
>> >
>> >   git ls-files --stdin | git check-attr --stdin --all
>> >
>> > -Peff
>>
>> When I remove '*.dll binary" from .gitattributes, I get this:
>>
>> $ git check-attr --all ./PSWindowsUpdate.dll
>> ./PSWindowsUpdate.dll: text: set
>> ./PSWindowsUpdate.dll: eol: lf
>
>Yes, this is definitely not correct.  The flag "text" being set tells Git to do line-ending conversion and "eol=lf" says to convert line endings
>into LF.
>
>You should look for things in your .gitattributes file that say something like "* text", which you probably don't want.  You could use
>"* text=auto", which should be fine for most cases, though.  It's also possible those aren't in a .gitattributes file in your repository but one
>elsewhere on your system.  You can check gitattributes(5) for the locations of other files that can affect it.
>
>As a note, it is best practice not to check binary dependencies or build artifacts into the repo.  Those are best stored elsewhere, such as an
>artifact server.

OT: Unless you are using git as an artifact repository to retain the dependency relationship between objects and the source commits that built them for PCI and SWIFT compliance. git is exceptional when it comes to evidence-of-origin forensics of specific installation artifacts when release structures are more complex than "deploy everything" use cases. But that may be a discussion we can have elsewhere.

Just my $20.00.
-Randall

Re: Git Modifying DLL

[Jonathon Anderson]

So, I somehow didn't realize what this was when I was looking at it,
but this is the first line in our .gitattributes

* text eol=lf

It all makes sense now.

And, yea, we're using git to retain dependency relationships and bring
those files under version control so we can audit changes and go
through the full peer review process, etc

On Fri, Aug 20, 2021 at 2:51 PM Randall S. Becker
<rsbecker@nexbridge.com> wrote:
>
> On August 20, 2021 3:15 PM, brian m. carlson wrote:
> >On 2021-08-20 at 18:47:02, Jonathon Anderson wrote:
> >> On Fri, Aug 20, 2021 at 1:00 PM Jeff King <peff@peff.net> wrote:
> >> >
> >> > On Thu, Aug 19, 2021 at 01:21:03PM -0500, Jonathon Anderson wrote:
> >> >
> >> > > I had not. I tested that and it worked. I assumed that git would
> >> > > automatically treat dll files as binary. Thanks for the help!
> >> >
> >> > Git doesn't know about any file extensions by default. Its default
> >> > "is it binary" test looks for NUL bytes in the first 8k or so of the file.
> >> > I'd expect your DLL would probably have such a NUL byte.
> >> >
> >> > Is it possible you have other .gitattributes set which are confusing
> >> > things?
> >> >
> >> > You might try:
> >> >
> >> >   git check-attr --all <path>
> >> >
> >> > or:
> >> >
> >> >   git ls-files --stdin | git check-attr --stdin --all
> >> >
> >> > -Peff
> >>
> >> When I remove '*.dll binary" from .gitattributes, I get this:
> >>
> >> $ git check-attr --all ./PSWindowsUpdate.dll
> >> ./PSWindowsUpdate.dll: text: set
> >> ./PSWindowsUpdate.dll: eol: lf
> >
> >Yes, this is definitely not correct.  The flag "text" being set tells Git to do line-ending conversion and "eol=lf" says to convert line endings
> >into LF.
> >
> >You should look for things in your .gitattributes file that say something like "* text", which you probably don't want.  You could use
> >"* text=auto", which should be fine for most cases, though.  It's also possible those aren't in a .gitattributes file in your repository but one
> >elsewhere on your system.  You can check gitattributes(5) for the locations of other files that can affect it.
> >
> >As a note, it is best practice not to check binary dependencies or build artifacts into the repo.  Those are best stored elsewhere, such as an
> >artifact server.
>
> OT: Unless you are using git as an artifact repository to retain the dependency relationship between objects and the source commits that built them for PCI and SWIFT compliance. git is exceptional when it comes to evidence-of-origin forensics of specific installation artifacts when release structures are more complex than "deploy everything" use cases. But that may be a discussion we can have elsewhere.
>
> Just my $20.00.
> -Randall
>