git fast-export not preserving executable permissions?

git fast-export not preserving executable permissions?

[Doug Glidden]

Hello Git world!

I have run into an issue that I cannot seem to resolve with git
fast-export. When running a fast-export on a repo that contains
scripts with executable permissions (e.g. a gradlew script), the
resulting export does not properly reflect the executable permissions
on the script files.

To illustrate this issue, I created a small sample repo, with one
executable file and one non-executable file. From the output below,
you can see that the mode in the output from fast-export is the same
for both files; according to the documentation for fast-import, the
mode for the executable file should be 100755 instead of 100644.

    $ ls -gG
    total 2
    -rwxr-xr-x 1 106 Apr 29 09:13 executable_script.sh*
    -rw-r--r-- 1  63 Apr 29 09:12 non_executable_file.txt

    $ git fast-export --all
    blob
    mark :1
    data 106
    #!/bin/bash

    # This is a shell script that should be executable.
    echo 'The script executed successfully!'

    blob
    mark :2
    data 63
    This file is a simple text file that should not be executable.

    reset refs/heads/dev
    commit refs/heads/dev
    mark :3
    author Doug <41mortimer@gmail.com> 1588167102 -0400
    committer Doug <41mortimer@gmail.com> 1588167102 -0400
    data 25
    Adding some sample files
    M 100644 :1 executable_script.sh
    M 100644 :2 non_executable_file.txt

Please let me know if there is any further information I can provide
about this issue.

Thank you,
Doug

Re: git fast-export not preserving executable permissions?

[Taylor Blau]

Hi Doug,

On Wed, Apr 29, 2020 at 09:36:31AM -0400, Doug Glidden wrote:
> Hello Git world!
>
> I have run into an issue that I cannot seem to resolve with git
> fast-export. When running a fast-export on a repo that contains
> scripts with executable permissions (e.g. a gradlew script), the
> resulting export does not properly reflect the executable permissions
> on the script files.

Interesting. fast-import and fast-export both understand executable
modes (although Git only understands the modes 644 and 755 for blobs),
so this should be working.

I can not reproduce the issue as-is. Round-tripping a fast-import and
fast-export preserves executable bits for me:

  #!/bin/bash

  set -e

  rm -rf repo client

  git init -q repo
  git init -q client

  (
    cd repo
    printf "x" >x
    printf "y" >y
    chmod +x x
    git add x y
    git commit -q -m "initial commit"
  )

  git -C repo fast-export HEAD | git -C client fast-import

  diff -u <(git -C repo ls-tree HEAD) <(git -C client ls-tree HEAD)

> To illustrate this issue, I created a small sample repo, with one
> executable file and one non-executable file. From the output below,
> you can see that the mode in the output from fast-export is the same
> for both files; according to the documentation for fast-import, the
> mode for the executable file should be 100755 instead of 100644.
>
>     $ ls -gG
>     total 2
>     -rwxr-xr-x 1 106 Apr 29 09:13 executable_script.sh*
>     -rw-r--r-- 1  63 Apr 29 09:12 non_executable_file.txt
>
>     $ git fast-export --all
>     blob
>     mark :1
>     data 106
>     #!/bin/bash
>
>     # This is a shell script that should be executable.
>     echo 'The script executed successfully!'
>
>     blob
>     mark :2
>     data 63
>     This file is a simple text file that should not be executable.
>
>     reset refs/heads/dev
>     commit refs/heads/dev
>     mark :3
>     author Doug <41mortimer@gmail.com> 1588167102 -0400
>     committer Doug <41mortimer@gmail.com> 1588167102 -0400
>     data 25
>     Adding some sample files
>     M 100644 :1 executable_script.sh
>     M 100644 :2 non_executable_file.txt
>
> Please let me know if there is any further information I can provide
> about this issue.

Does Git think that the file is executable? Please run 'git ls-tree
HEAD' to find out.

> Thank you,
> Doug

Thanks,
Taylor

Re: git fast-export not preserving executable permissions?

[Torsten Bögershausen]

On Wed, Apr 29, 2020 at 09:36:31AM -0400, Doug Glidden wrote:
> Hello Git world!
>
> I have run into an issue that I cannot seem to resolve with git
> fast-export. When running a fast-export on a repo that contains
> scripts with executable permissions (e.g. a gradlew script), the
> resulting export does not properly reflect the executable permissions
> on the script files.
>
> To illustrate this issue, I created a small sample repo, with one
> executable file and one non-executable file. From the output below,
> you can see that the mode in the output from fast-export is the same
> for both files; according to the documentation for fast-import, the
> mode for the executable file should be 100755 instead of 100644.
>
>     $ ls -gG
>     total 2
>     -rwxr-xr-x 1 106 Apr 29 09:13 executable_script.sh*
>     -rw-r--r-- 1  63 Apr 29 09:12 non_executable_file.txt

What does
git ls-files -s
give you here ?
We need to know, how Git tracks the files, is it
100644 or 100755 ?

The following works for me:

user@pc:/tmp/fff> git init
Initialized empty Git repository in /private/tmp/fff/.git/
user@pc:/tmp/fff> echo file1 >file1
user@pc:/tmp/fff> git config core.filemode false
user@pc:/tmp/fff> echo file2 >file2
user@pc:/tmp/fff> git add file1 file2
user@pc:/tmp/fff> git ls-files -s
100644 e2129701f1a4d54dc44f03c93bca0a2aec7c5449 0       file1
100644 6c493ff740f9380390d5c9ddef4af18697ac9375 0       file2
user@pc:/tmp/fff> git update-index --chmod=+x file2
user@pc:/tmp/fff> git ls-files -s
100644 e2129701f1a4d54dc44f03c93bca0a2aec7c5449 0       file1
100755 6c493ff740f9380390d5c9ddef4af18697ac9375 0       file2
user@pc:/tmp/fff> git commit -m "Add file1 file2"
[master (root-commit) f75926e] Add file1 file2
 2 files changed, 2 insertions(+)
  create mode 100644 file1
  create mode 100755 file2

user@pc:/tmp/fff> git fast-export --all
blob
mark :1
data 6
file1

blob
mark :2
data 6
file2

reset refs/heads/master
commit refs/heads/master
mark :3
author Torsten Bögershausen <tboegi@web.de> 1588217922 +0200
committer Torsten Bögershausen <tboegi@web.de> 1588217922 +0200
data 16
Add file1 file2
M 100644 :1 file1
M 100755 :2 file2

Re: git fast-export not preserving executable permissions?

[Doug Glidden]

Taylor,

Thanks for your response! It looks like git does not actually
recognize the file as executable:

    $ git ls-tree HEAD
    100644 blob 7d2f57b2381766924e1e4ffcc62615c637bbd784    executable_script.sh
    100644 blob d1d7cf309e091f54f268503b31653d8eba42fe88
non_executable_file.txt

Now you have me wondering if the real problem here is that I'm working
in git-bash on a Windows machine, which means the file permissions
aren't completely native. I'm going to run a similar experiment in a
native Linux environment and see if I get the same results. I'll let
you know what I find.

Thanks,
Doug



On Wed, Apr 29, 2020 at 2:49 PM Taylor Blau <me@ttaylorr.com> wrote:
>
> Hi Doug,
>
> On Wed, Apr 29, 2020 at 09:36:31AM -0400, Doug Glidden wrote:
> > Hello Git world!
> >
> > I have run into an issue that I cannot seem to resolve with git
> > fast-export. When running a fast-export on a repo that contains
> > scripts with executable permissions (e.g. a gradlew script), the
> > resulting export does not properly reflect the executable permissions
> > on the script files.
>
> Interesting. fast-import and fast-export both understand executable
> modes (although Git only understands the modes 644 and 755 for blobs),
> so this should be working.
>
> I can not reproduce the issue as-is. Round-tripping a fast-import and
> fast-export preserves executable bits for me:
>
>   #!/bin/bash
>
>   set -e
>
>   rm -rf repo client
>
>   git init -q repo
>   git init -q client
>
>   (
>     cd repo
>     printf "x" >x
>     printf "y" >y
>     chmod +x x
>     git add x y
>     git commit -q -m "initial commit"
>   )
>
>   git -C repo fast-export HEAD | git -C client fast-import
>
>   diff -u <(git -C repo ls-tree HEAD) <(git -C client ls-tree HEAD)
>
> > To illustrate this issue, I created a small sample repo, with one
> > executable file and one non-executable file. From the output below,
> > you can see that the mode in the output from fast-export is the same
> > for both files; according to the documentation for fast-import, the
> > mode for the executable file should be 100755 instead of 100644.
> >
> >     $ ls -gG
> >     total 2
> >     -rwxr-xr-x 1 106 Apr 29 09:13 executable_script.sh*
> >     -rw-r--r-- 1  63 Apr 29 09:12 non_executable_file.txt
> >
> >     $ git fast-export --all
> >     blob
> >     mark :1
> >     data 106
> >     #!/bin/bash
> >
> >     # This is a shell script that should be executable.
> >     echo 'The script executed successfully!'
> >
> >     blob
> >     mark :2
> >     data 63
> >     This file is a simple text file that should not be executable.
> >
> >     reset refs/heads/dev
> >     commit refs/heads/dev
> >     mark :3
> >     author Doug <41mortimer@gmail.com> 1588167102 -0400
> >     committer Doug <41mortimer@gmail.com> 1588167102 -0400
> >     data 25
> >     Adding some sample files
> >     M 100644 :1 executable_script.sh
> >     M 100644 :2 non_executable_file.txt
> >
> > Please let me know if there is any further information I can provide
> > about this issue.
>
> Does Git think that the file is executable? Please run 'git ls-tree
> HEAD' to find out.
>
> > Thank you,
> > Doug
>
> Thanks,
> Taylor

Re: git fast-export not preserving executable permissions?

[Taylor Blau]

On Fri, May 01, 2020 at 09:35:28AM -0400, Doug Glidden wrote:
> Taylor,

(My full response is below, but please in general do not top-quote mail
here.)

>
> Thanks for your response! It looks like git does not actually
> recognize the file as executable:
>
>     $ git ls-tree HEAD
>     100644 blob 7d2f57b2381766924e1e4ffcc62615c637bbd784    executable_script.sh
>     100644 blob d1d7cf309e091f54f268503b31653d8eba42fe88
> non_executable_file.txt
>
> Now you have me wondering if the real problem here is that I'm working
> in git-bash on a Windows machine, which means the file permissions
> aren't completely native.

I was wondering if that was the case ;-). If you are using NTFS or
FAT32, neither of these filesystems support execute permission bits. (I
am certainly not an expert here, but I know that Dscho (cc'd) would be
able to answer authoritatively here.)

That said, *Git* understands executable permissions, even if your
filesystem doesn't. You can tell Git to mark a file as executable by
the following:

  $ git update-index --chmod=+x /path/to/file

and then committing the result. Round-tripping this through 'git
fast-{im,ex}port' should preserve the permissions from Git's
perspective, and ditto for checking out the contents of a repository on
a filesystem that does support the executable permission bit.

> I'm going to run a similar experiment in a native Linux environment
> and see if I get the same results. I'll let you know what I find.

Sounds good. I'll be very surprised if it doesn't work as you expect.

> Thanks,
> Doug
>
>
>
> On Wed, Apr 29, 2020 at 2:49 PM Taylor Blau <me@ttaylorr.com> wrote:
> >
> > Hi Doug,
> >
> > On Wed, Apr 29, 2020 at 09:36:31AM -0400, Doug Glidden wrote:
> > > Hello Git world!
> > >
> > > I have run into an issue that I cannot seem to resolve with git
> > > fast-export. When running a fast-export on a repo that contains
> > > scripts with executable permissions (e.g. a gradlew script), the
> > > resulting export does not properly reflect the executable permissions
> > > on the script files.
> >
> > Interesting. fast-import and fast-export both understand executable
> > modes (although Git only understands the modes 644 and 755 for blobs),
> > so this should be working.
> >
> > I can not reproduce the issue as-is. Round-tripping a fast-import and
> > fast-export preserves executable bits for me:
> >
> >   #!/bin/bash
> >
> >   set -e
> >
> >   rm -rf repo client
> >
> >   git init -q repo
> >   git init -q client
> >
> >   (
> >     cd repo
> >     printf "x" >x
> >     printf "y" >y
> >     chmod +x x
> >     git add x y
> >     git commit -q -m "initial commit"
> >   )
> >
> >   git -C repo fast-export HEAD | git -C client fast-import
> >
> >   diff -u <(git -C repo ls-tree HEAD) <(git -C client ls-tree HEAD)
> >
> > > To illustrate this issue, I created a small sample repo, with one
> > > executable file and one non-executable file. From the output below,
> > > you can see that the mode in the output from fast-export is the same
> > > for both files; according to the documentation for fast-import, the
> > > mode for the executable file should be 100755 instead of 100644.
> > >
> > >     $ ls -gG
> > >     total 2
> > >     -rwxr-xr-x 1 106 Apr 29 09:13 executable_script.sh*
> > >     -rw-r--r-- 1  63 Apr 29 09:12 non_executable_file.txt
> > >
> > >     $ git fast-export --all
> > >     blob
> > >     mark :1
> > >     data 106
> > >     #!/bin/bash
> > >
> > >     # This is a shell script that should be executable.
> > >     echo 'The script executed successfully!'
> > >
> > >     blob
> > >     mark :2
> > >     data 63
> > >     This file is a simple text file that should not be executable.
> > >
> > >     reset refs/heads/dev
> > >     commit refs/heads/dev
> > >     mark :3
> > >     author Doug <41mortimer@gmail.com> 1588167102 -0400
> > >     committer Doug <41mortimer@gmail.com> 1588167102 -0400
> > >     data 25
> > >     Adding some sample files
> > >     M 100644 :1 executable_script.sh
> > >     M 100644 :2 non_executable_file.txt
> > >
> > > Please let me know if there is any further information I can provide
> > > about this issue.
> >
> > Does Git think that the file is executable? Please run 'git ls-tree
> > HEAD' to find out.
> >
> > > Thank you,
> > > Doug
> >
> > Thanks,
> > Taylor
Thanks,
Taylor

Re: git fast-export not preserving executable permissions?

[Johannes Schindelin]

Hi,

On Fri, 1 May 2020, Taylor Blau wrote:

> If you are using NTFS or FAT32, neither of these filesystems support
> execute permission bits. (I am certainly not an expert here, but I know
> that Dscho (cc'd) would be able to answer authoritatively here.)

On Windows, there are indeed no executable bits. A file is considered
executable if it has a file extension indicating it, e.g. `.exe`.

In Git for Windows' Bash (which comes from MSYS2), `ls -l` will also
consider files whose first line is a hash-bang one (e.g. `#!/bin/sh`) as
executable. This information is however _not_ used by Git.

If you want to mark a file as executable in Git for Windows, you will have
to use `git add --chmod=+x <file>`.

Ciao,
Johannes