[PATCH 0/1] git-p4: remove ticket expiration test

[PATCH 0/1] git-p4: remove ticket expiration test

[Luke Diamand]

As per thread here, this removes the git-p4 ticket expiration
test, since it isn't really that useful.

https://marc.info/?l=git&m=154946136416003&w=2

Luke Diamand (1):
  git-p4: remove ticket expiry test

 t/t9833-errors.sh | 27 ---------------------------
 1 file changed, 27 deletions(-)

-- 
2.20.1.611.gfbb209baf1

[PATCH] git-p4: remove ticket expiry test

[Luke Diamand]

The git-p4 login ticket expiry test causes unreliable test
runs. Since the handling of ticket expiry in git-p4 is far
from polished anyway, let's remove it for now.

A better way to actually run the test is to create a python
"fake" version of "p4" which returns whatever expiry results
the test requires.

Ideally git-p4 would look at the expiry time before starting
any long operations, and cleanup gracefully if there is not
enough time left. But that's quite hard to do.

Signed-off-by: Luke Diamand <luke@diamand.org>
---
 t/t9833-errors.sh | 27 ---------------------------
 1 file changed, 27 deletions(-)

diff --git a/t/t9833-errors.sh b/t/t9833-errors.sh
index 277d347012..47b312e1c9 100755
--- a/t/t9833-errors.sh
+++ b/t/t9833-errors.sh
@@ -45,33 +45,6 @@ test_expect_success 'ticket logged out' '
 	)
 '
 
-test_expect_success 'create group with short ticket expiry' '
-	P4TICKETS="$cli/tickets" &&
-	echo "newpassword" | p4 login &&
-	p4_add_user short_expiry_user &&
-	p4 -u short_expiry_user passwd -P password &&
-	p4 group -i <<-EOF &&
-	Group: testgroup
-	Timeout: 3
-	Users: short_expiry_user
-	EOF
-
-	p4 users | grep short_expiry_user
-'
-
-test_expect_success 'git operation with expired ticket' '
-	P4TICKETS="$cli/tickets" &&
-	P4USER=short_expiry_user &&
-	echo "password" | p4 login &&
-	(
-		cd "$git" &&
-		git p4 sync &&
-		sleep 5 &&
-		test_must_fail git p4 sync 2>errmsg &&
-		grep "failure accessing depot" errmsg
-	)
-'
-
 test_expect_success 'kill p4d' '
 	kill_p4d
 '
-- 
2.20.1.611.gfbb209baf1

Re: [PATCH] git-p4: remove ticket expiry test

[SZEDER Gábor]

On Wed, Feb 06, 2019 at 03:11:53PM +0000, Luke Diamand wrote:
> The git-p4 login ticket expiry test causes unreliable test
> runs. Since the handling of ticket expiry in git-p4 is far
> from polished anyway, let's remove it for now.

I can't judge how far it is from polished, and whether it's worth
having this test or should be removed in the meantime, but I would
like to clarify this part from my previous email that you might have
interpreted as a suggestion to remove these tests (it confused even
myself on second read):

  > I wonder why that failing 'git p4 sync' is
  > necessary in the first place, and whether it's really necessary to
  > test ticket expiration

I was not wondering whether it's necessary to test ticket expiration
_in general_.  What I really meant was whether that first 'git p4
sync' was really necessary in that test 'git operation with expired
ticket'.  After all, what we want to see in this test is that 'git p4
sync' fails with a specific error message when the ticket expired, and
when flakiness hits, then this first 'git p4 sync' does fail with the
expected error message.


> A better way to actually run the test is to create a python
> "fake" version of "p4" which returns whatever expiry results
> the test requires.
> 
> Ideally git-p4 would look at the expiry time before starting
> any long operations, and cleanup gracefully if there is not
> enough time left. But that's quite hard to do.
> 
> Signed-off-by: Luke Diamand <luke@diamand.org>
> ---
>  t/t9833-errors.sh | 27 ---------------------------
>  1 file changed, 27 deletions(-)
> 
> diff --git a/t/t9833-errors.sh b/t/t9833-errors.sh
> index 277d347012..47b312e1c9 100755
> --- a/t/t9833-errors.sh
> +++ b/t/t9833-errors.sh
> @@ -45,33 +45,6 @@ test_expect_success 'ticket logged out' '
>  	)
>  '
>  
> -test_expect_success 'create group with short ticket expiry' '
> -	P4TICKETS="$cli/tickets" &&
> -	echo "newpassword" | p4 login &&
> -	p4_add_user short_expiry_user &&
> -	p4 -u short_expiry_user passwd -P password &&
> -	p4 group -i <<-EOF &&
> -	Group: testgroup
> -	Timeout: 3
> -	Users: short_expiry_user
> -	EOF
> -
> -	p4 users | grep short_expiry_user
> -'
> -
> -test_expect_success 'git operation with expired ticket' '
> -	P4TICKETS="$cli/tickets" &&
> -	P4USER=short_expiry_user &&
> -	echo "password" | p4 login &&
> -	(
> -		cd "$git" &&
> -		git p4 sync &&
> -		sleep 5 &&
> -		test_must_fail git p4 sync 2>errmsg &&
> -		grep "failure accessing depot" errmsg
> -	)
> -'
> -
>  test_expect_success 'kill p4d' '
>  	kill_p4d
>  '
> -- 
> 2.20.1.611.gfbb209baf1
> 

Re: [PATCH 0/1] git-p4: remove ticket expiration test

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 870 bytes --]

Hi Luke,

On Wed, 6 Feb 2019, Luke Diamand wrote:

> As per thread here, this removes the git-p4 ticket expiration
> test, since it isn't really that useful.
> 
> https://marc.info/?l=git&m=154946136416003&w=2

Thank you for the prompt patch!

However, like Gábor, my feeling is that we would want that test case in a
non-flakey form, if possible. If you think that that is only possible with
a mocked p4, so be it, let's remove the test case (because the mocked one
will likely look quite a bit different). But if there are easier ways to
work around the timing issues (such as dropping the first `sync`), then
I'd prefer to have the safety of a regression test.

Thanks,
Dscho

> Luke Diamand (1):
>   git-p4: remove ticket expiry test
> 
>  t/t9833-errors.sh | 27 ---------------------------
>  1 file changed, 27 deletions(-)
> 
> -- 
> 2.20.1.611.gfbb209baf1
> 
> 

Re: [PATCH 0/1] git-p4: remove ticket expiration test

[Luke Diamand]

On Thu, 7 Feb 2019 13:45:18 +0100 (STD)
Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

> Hi Luke,
> 
> On Wed, 6 Feb 2019, Luke Diamand wrote:
> 
> > As per thread here, this removes the git-p4 ticket expiration
> > test, since it isn't really that useful.
> > 
> > https://marc.info/?l=git&m=154946136416003&w=2
> 
> Thank you for the prompt patch!
> 
> However, like Gábor, my feeling is that we would want that test case in a
> non-flakey form, if possible. If you think that that is only possible with
> a mocked p4, so be it, let's remove the test case (because the mocked one
> will likely look quite a bit different). But if there are easier ways to
> work around the timing issues (such as dropping the first `sync`), then
> I'd prefer to have the safety of a regression test.

I've got a mocked-up p4 wrapper which returns whatever expiration time the test needs. I'll submit it tomorrow.

It's just a few lines of python script to generate the marshalled data, so it's not very complicated.

> 
> Thanks,
> Dscho
> 
> > Luke Diamand (1):
> >   git-p4: remove ticket expiry test
> > 
> >  t/t9833-errors.sh | 27 ---------------------------
> >  1 file changed, 27 deletions(-)
> > 
> > -- 
> > 2.20.1.611.gfbb209baf1
> > 
> > 


-- 
Luke Diamand <luke@diamand.org>

Re: [PATCH 0/1] git-p4: remove ticket expiration test

[Johannes Schindelin]

Hi Luke,

On Thu, 7 Feb 2019, Luke Diamand wrote:

> I've got a mocked-up p4 wrapper which returns whatever expiration time
> the test needs. I'll submit it tomorrow.

Great!
Dscho