Test failure for v2.16.0-rc0 on cygwin

Test failure for v2.16.0-rc0 on cygwin

[Ramsay Jones]

Hi Junio, Adam,

Just a quick note about the failure of the test-suite on cygwin.
In particular, test t5580-clone-push-unc.sh #3, like so:

  $ ./t5580-clone-push-unc.sh -i -v

  ...

  ok 2 - clone
  
  expecting success: 
  	(
  		cd clone &&
  		git checkout -b to-push &&
  		test_commit to-push &&
  		git push origin HEAD
  	) &&
  	rev="$(git -C clone rev-parse --verify refs/heads/to-push)" &&
  	test "$rev" = "$(git rev-parse --verify refs/heads/to-push)"
  
  Switched to a new branch 'to-push'
  [to-push 0adccbe] to-push
   Author: A U Thor <author@example.com>
   1 file changed, 1 insertion(+)
   create mode 100644 to-push.t
  Counting objects: 3, done.
  Delta compression using up to 4 threads.
  Compressing objects: 100% (2/2), done.
  Writing objects: 100% (3/3), 289 bytes | 289.00 KiB/s, done.
  Total 3 (delta 0), reused 0 (delta 0)
  error: remote unpack failed: unable to create temporary object directory
  To file:////localhost/C$/\cygwin64\home\ramsay\git\t\trash directory.t5580-clone-push-unc
   ! [remote rejected] HEAD -> to-push (unpacker error)
  error: failed to push some refs to 'file:////localhost/C$/\cygwin64\home\ramsay\git\t\trash directory.t5580-clone-push-unc'
  not ok 3 - push
  #	
  #		(
  #			cd clone &&
  #			git checkout -b to-push &&
  #			test_commit to-push &&
  #			git push origin HEAD
  #		) &&
  #		rev="$(git -C clone rev-parse --verify refs/heads/to-push)" &&
  #		test "$rev" = "$(git rev-parse --verify refs/heads/to-push)"
  #	
  $ 

My first, knee-jerk, reaction on seeing the backslashes in the 'local' part
of the pathname (despite what it says in commit f21d60b429) was to replace
the 'cygpath -aw .' with 'cygpath -am .'. This leads to exactly the same
error display as above, except '\\' replaced by '/' in the pathname! Hmm.

Note:

  $ ls '//localhost/C$/\cygwin64\home\ramsay\git\t\trash directory.t5580-clone-push-unc'
  clone/  initial.t
  $ ls '//localhost/C$//cygwin64/home/ramsay/git/t/trash directory.t5580-clone-push-unc'
  clone/  initial.t
  $ ls //localhost/C$/\\cygwin64\\home\\ramsay\\git\\t\\trash\ directory.t5580-clone-push-unc
  clone/  initial.t
  $ 
 
So, after some debugging (using the 'cygpath -am .' version), it seems that
the error message is caused by the mkdtemp() call in tmp_objdir_create()
function (tmp-objdir.c:123), failing with an errno of 'permission denied'. 

At the time of the call to mkdtemp, the current working directory is given by
'//localhost/C$/cygwin64/home/ramsay/git/t/trash directory.t5580-clone-push-unc/.git',
and the parameter to mkdtemp is (for example) "./objects/incoming-MgZCkI".

  $ cd trash\ directory.t5580-clone-push-unc/
  $ ls
  clone/  initial.t
  $ ls .git
  branches/       config       HEAD             index  logs/     refs/
  COMMIT_EDITMSG  description  hooks-disabled/  info/  objects/
  $ ls .git/objects
  24/  93/  e7/  info/  pack/
  $ mkdir '//localhost/C$//cygwin64/home/ramsay/git/t/trash directory.t5580-clone-push-unc/.git/./objects/incoming-MgZCkI'
  mkdir: cannot create directory ‘//localhost/C$//cygwin64/home/ramsay/git/t/trash directory.t5580-clone-push-unc/.git/./objects/incoming-MgZCkI’: Permission denied
    $ 

So, when 'doing it by hand', I get the same result. ;-)

Looking at the permissions:

  $ ls -ld '//localhost/C$//cygwin64/home/ramsay/git/t/trash directory.t5580-clone-push-unc/.git/./objects'
  drwxr-xr-x+ 1 ramsay None 0 Dec 30 13:51 '//localhost/C$//cygwin64/home/ramsay/git/t/trash directory.t5580-clone-push-unc/.git/./objects'/

  $ ls -ld '//localhost/C$//cygwin64/home/ramsay/git/t/trash directory.t5580-clone-push-unc/.git'
  drwxr-xr-x+ 1 ramsay None 0 Dec 30 13:51 '//localhost/C$//cygwin64/home/ramsay/git/t/trash directory.t5580-clone-push-unc/.git'/

  $ ls -ld '//localhost/C$//cygwin64/home/ramsay/git/t/trash directory.t5580-clone-push-unc'
  drwxr-xr-x+ 1 ramsay None 0 Dec 30 13:51 '//localhost/C$//cygwin64/home/ramsay/git/t/trash directory.t5580-clone-push-unc'/

  ... and so on, until:

  $ ls -ld '//localhost/C$'
  d---r-x---+ 1 TrustedInstaller TrustedInstaller 0 Oct 28 16:16 '//localhost/C$'/
  $ 

Doing a quick web-search on '//localhost/C$' lead me to several posts which
seem to suggest the problem could be related to UAC filtering on remote
shares (and the fact that I am on Windows 10), along with a registry setting
for 'LocalAccountTokenFilterPolicy'. However, I have not had time to look
into any of that.

Adam, are you running the tests on Windows 10?

I have to go now, so I can't look at this further today. Just FYI.

ATB,
Ramsay Jones

Re: Test failure for v2.16.0-rc0 on cygwin

[Adam Dinwoodie]

On Saturday 30 December 2017 at 02:21 pm +0000, Ramsay Jones wrote:
> Hi Junio, Adam,
> 
> Just a quick note about the failure of the test-suite on cygwin.
> In particular, test t5580-clone-push-unc.sh #3, like so:
> 
> <snip>
>
> Adam, are you running the tests on Windows 10?

I'm only routinely testing on Windows 10 x86_64, but between holidays
various, I've not had the tests running for the past couple of weeks.
I'm kicking off a build now in the name of verifying I see the same
problem.

Adam

Re: Test failure for v2.16.0-rc0 on cygwin

[Adam Dinwoodie]

On Saturday 30 December 2017 at 02:40 pm +0000, Adam Dinwoodie wrote:
> On Saturday 30 December 2017 at 02:21 pm +0000, Ramsay Jones wrote:
> > Hi Junio, Adam,
> > 
> > Just a quick note about the failure of the test-suite on cygwin.
> > In particular, test t5580-clone-push-unc.sh #3, like so:
> > 
> > <snip>
> >
> > Adam, are you running the tests on Windows 10?
> 
> I'm only routinely testing on Windows 10 x86_64, but between holidays
> various, I've not had the tests running for the past couple of weeks.
> I'm kicking off a build now in the name of verifying I see the same
> problem.

I'm not able to reproduce this: t5580 is passing on both my Windows 10
test systems on v2.16.0-rc0.

Looking at your output, it sounds like there's something slightly odd
with your directory permissions.  I agree the mixed slashes look odd,
but given the test is passing on both my systems, I don't think that can
be the problem.

I suspect you're going to need to do some more digging yourself given
this appears to be a permissions issue on your system.  For a start,
when you get to the failing `mkdir` with a UNC path, are you able to
create the equivalent directory using Cygwin's `mkdir` but specifying a
regular non-UNC path, or by opening the UNC path in Explorer and
creating the directory there?

Adam

Re: Test failure for v2.16.0-rc0 on cygwin

[Ramsay Jones]

On 02/01/18 11:36, Adam Dinwoodie wrote:
> On Saturday 30 December 2017 at 02:40 pm +0000, Adam Dinwoodie wrote:
>> On Saturday 30 December 2017 at 02:21 pm +0000, Ramsay Jones wrote:
>>> Hi Junio, Adam,
>>>
>>> Just a quick note about the failure of the test-suite on cygwin.
>>> In particular, test t5580-clone-push-unc.sh #3, like so:
>>>
>>> <snip>
>>>
>>> Adam, are you running the tests on Windows 10?
>>
>> I'm only routinely testing on Windows 10 x86_64, but between holidays
>> various, I've not had the tests running for the past couple of weeks.
>> I'm kicking off a build now in the name of verifying I see the same
>> problem.
> 
> I'm not able to reproduce this: t5580 is passing on both my Windows 10
> test systems on v2.16.0-rc0.

Hmm, interesting. BTW, I should have noted which version I'm on (just
in case it matters): Windows 10 Home, Version 1709, OS Build 16299.125.

I am reasonably up-to-date on cygwin:

  $ uname -a
  CYGWIN_NT-10.0 satellite 2.9.0(0.318/5/3) 2017-09-12 10:18 x86_64 Cygwin
  $ 

[I only recently updated to the creator's update (I'm not signed up to
the 'insider program'), and so could not try out WSL until now. I would
not recommend it to anyone who wants to develop software - a Linux VM
is an _order of magnitude_ faster, so ... ]

> Looking at your output, it sounds like there's something slightly odd
> with your directory permissions.  I agree the mixed slashes look odd,
> but given the test is passing on both my systems, I don't think that can
> be the problem.

The directory permissions look fine to me (except for //localhost/C$).

> I suspect you're going to need to do some more digging yourself given
> this appears to be a permissions issue on your system.  For a start,
> when you get to the failing `mkdir` with a UNC path, are you able to
> create the equivalent directory using Cygwin's `mkdir` but specifying a
> regular non-UNC path, 

yes, this is not a problem.

                        or by opening the UNC path in Explorer and
> creating the directory there?

I didn't get to this because ...

I just tried running the test again by hand, and I can't get it to fail!

Hmm, I have just set off a complete test-suite run, so I won't be able
to report on that for a while ... ;-)

I have an idea: when running the failing tests the other day, I was
remotely logged in via ssh (I have cygwin sshd running on my win10
box), but today I was logged in directly. The sshd daemon is run by
a privileged user, so maybe that could be related ... dunno.

I will have to investigate some more. (If you have any ideas ... :-D )

ATB,
Ramsay Jones

Re: Test failure for v2.16.0-rc0 on cygwin

[Ramsay Jones]

On 02/01/18 15:32, Ramsay Jones wrote:
> On 02/01/18 11:36, Adam Dinwoodie wrote:
>> On Saturday 30 December 2017 at 02:40 pm +0000, Adam Dinwoodie wrote:
>>> On Saturday 30 December 2017 at 02:21 pm +0000, Ramsay Jones wrote:
[snip]
>> I'm not able to reproduce this: t5580 is passing on both my Windows 10
>> test systems on v2.16.0-rc0.
[snip]
> I just tried running the test again by hand, and I can't get it to fail!
> 
> Hmm, I have just set off a complete test-suite run, so I won't be able
> to report on that for a while ... ;-)

Yep, as expected, the test-suite passes no problem now! ;-)

> I have an idea: when running the failing tests the other day, I was
> remotely logged in via ssh (I have cygwin sshd running on my win10
> box), but today I was logged in directly. The sshd daemon is run by
> a privileged user, so maybe that could be related ... dunno.

Also, when logged-in remotely it fails consistently, when logged-in
directly it passes consistently. :-D

ATB,
Ramsay Jones

Re: Test failure for v2.16.0-rc0 on cygwin

[Johannes Schindelin]

Hi,

On Tue, 2 Jan 2018, Ramsay Jones wrote:

> On 02/01/18 15:32, Ramsay Jones wrote:
> > On 02/01/18 11:36, Adam Dinwoodie wrote:
> >> On Saturday 30 December 2017 at 02:40 pm +0000, Adam Dinwoodie wrote:
> >>> On Saturday 30 December 2017 at 02:21 pm +0000, Ramsay Jones wrote:
> [snip]
> >> I'm not able to reproduce this: t5580 is passing on both my Windows 10
> >> test systems on v2.16.0-rc0.
> [snip]
> > I just tried running the test again by hand, and I can't get it to fail!
> > 
> > Hmm, I have just set off a complete test-suite run, so I won't be able
> > to report on that for a while ... ;-)
> 
> Yep, as expected, the test-suite passes no problem now! ;-)
> 
> > I have an idea: when running the failing tests the other day, I was
> > remotely logged in via ssh (I have cygwin sshd running on my win10
> > box), but today I was logged in directly. The sshd daemon is run by
> > a privileged user, so maybe that could be related ... dunno.
> 
> Also, when logged-in remotely it fails consistently, when logged-in
> directly it passes consistently. :-D

You are most likely hitting cmd.exe at some point there. In cmd.exe, there
are some restrictions that are inherited by spawned processes AFAIU. For
example, the current directory cannot be a UNC path.

You are most likely running the interactive Cygwin session in MinTTY? Then
you do not get those restrictions. If you start Cygwin in a cmd.exe
window, you should see the exact same test failures again.

Ciao,
Dscho

Re: Test failure for v2.16.0-rc0 on cygwin

[Ramsay Jones]

On 04/01/18 20:55, Johannes Schindelin wrote:
> On Tue, 2 Jan 2018, Ramsay Jones wrote:
[snip]
>> Also, when logged-in remotely it fails consistently, when logged-in
>> directly it passes consistently. :-D
> 
> You are most likely hitting cmd.exe at some point there. In cmd.exe, there
> are some restrictions that are inherited by spawned processes AFAIU. For
> example, the current directory cannot be a UNC path.
> 
> You are most likely running the interactive Cygwin session in MinTTY? Then
> you do not get those restrictions. If you start Cygwin in a cmd.exe
> window, you should see the exact same test failures again.

I actually don't see a difference when starting cygwin from a cmd.exe, it
passes just fine. The interactive cygwin session(s), either directly, or
most often via the X-server (with ssh-agent in between!), all have their
id's and group membership look like:

  $ who
  $ id
  uid=1001(ramsay) gid=513(None) groups=513(None),545(Users),4(INTERACTIVE),66049(CONSOLE LOGON),11(Authenticated Users),15(This Organization),113(Local account),66048(LOCAL),262154(NTLM Authentication),401408(Medium Mandatory Level)
  $

However, when remotely logged-in over shh, it looks like:

  $ who -H
  NAME     LINE         TIME             COMMENT
  ramsay   pty2         2018-01-02 19:48 (192.168.1.2)
  $ id
  uid=1001(ramsay) gid=513(None) groups=513(None),11(Authenticated Users),66048(LOCAL),66049(CONSOLE LOGON),4(INTERACTIVE),15(This Organization),545(Users),0(root),405504(High Mandatory Level)
  $

So, when remotely logged-in, we have:

  Additional groups: 0(root), 405504(High Mandatory Level)

  Missing groups: 113(Local account), 262154(NTLM Authentication),
                  401408(Medium Mandatory Level)

I haven't thought too much about what that means ...

After reading this[1], I have been meaning to try setting the
'LocalAccountTokenFilterPolicy' registry variable mentioned in
that article, to see if that would make any difference. I haven't
found the time yet ... :-D

ATB,
Ramsay Jones


[1] http://www.tomsitpro.com/articles/windows-10-administrative-shares,2-47.html