[PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

[PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

[SZEDER Gábor]

Before installing the necessary dependencies, our OSX build jobs run
'brew update --quiet'.  This is problematic for two reasons:

  - This '--quiet' flag apparently broke overnight, resulting in
    errored builds:

      +brew update --quiet
      ==> Downloading https://homebrew.bintray.com/bottles-portable-ruby/portable-ruby-2.3.7.mavericks.bottle.tar.gz
      ######################################################################## 100.0%
      ==> Pouring portable-ruby-2.3.7.mavericks.bottle.tar.gz
      Usage: brew update_report [--preinstall]
      The Ruby implementation of brew update. Never called manually.
              --preinstall                 Run in 'auto-update' mode (faster, less
                                           output).
          -f, --force                      Override warnings and enable potentially
                                           unsafe operations.
          -d, --debug                      Display any debugging information.
          -v, --verbose                    Make some output more verbose.
          -h, --help                       Show this message.
      Error: invalid option: --quiet
      The command "ci/install-dependencies.sh" failed and exited with 1 during .

    I belive that this breakage will be noticed and fixed soon-ish, so
    we could probably just wait a bit for this issue to solve itself,
    but:

  - 'brew update --quiet' wasn't really quiet in the first place, as
    it listed over about 2000 lines worth of available packages that
    we absolutely don't care about, see e.g. one of the latest
    'master' builds:

      https://travis-ci.org/git/git/jobs/486134962#L113

So drop this '--quiet' option and redirect 'brew update's standard
output to /dev/null to make it really quiet, thereby making the OSX
builds work again despite the above mentioned breakage.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
---

Notes:
    There is no conflict with Dscho's Azure Pipelines patch series; the
    patch contexts overlap a bit, but the auto-merging results look good
    to me.

 ci/install-dependencies.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 06c3546e1e..5968efdbbe 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -34,7 +34,7 @@ linux-clang|linux-gcc)
 	popd
 	;;
 osx-clang|osx-gcc)
-	brew update --quiet
+	brew update >/dev/null
 	# Uncomment this if you want to run perf tests:
 	# brew install gnu-time
 	brew install git-lfs gettext
-- 
2.20.1.642.gc55a771460

Re: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 3512 bytes --]

Hi Gábor,

On Sat, 2 Feb 2019, SZEDER Gábor wrote:

> Before installing the necessary dependencies, our OSX build jobs run
> 'brew update --quiet'.  This is problematic for two reasons:
> 
>   - This '--quiet' flag apparently broke overnight, resulting in
>     errored builds:
> 
>       +brew update --quiet
>       ==> Downloading https://homebrew.bintray.com/bottles-portable-ruby/portable-ruby-2.3.7.mavericks.bottle.tar.gz
>       ######################################################################## 100.0%
>       ==> Pouring portable-ruby-2.3.7.mavericks.bottle.tar.gz
>       Usage: brew update_report [--preinstall]
>       The Ruby implementation of brew update. Never called manually.
>               --preinstall                 Run in 'auto-update' mode (faster, less
>                                            output).
>           -f, --force                      Override warnings and enable potentially
>                                            unsafe operations.
>           -d, --debug                      Display any debugging information.
>           -v, --verbose                    Make some output more verbose.
>           -h, --help                       Show this message.
>       Error: invalid option: --quiet
>       The command "ci/install-dependencies.sh" failed and exited with 1 during .
> 
>     I belive that this breakage will be noticed and fixed soon-ish, so
>     we could probably just wait a bit for this issue to solve itself,
>     but:
> 
>   - 'brew update --quiet' wasn't really quiet in the first place, as
>     it listed over about 2000 lines worth of available packages that
>     we absolutely don't care about, see e.g. one of the latest
>     'master' builds:
> 
>       https://travis-ci.org/git/git/jobs/486134962#L113
> 
> So drop this '--quiet' option and redirect 'brew update's standard
> output to /dev/null to make it really quiet, thereby making the OSX
> builds work again despite the above mentioned breakage.
> 
> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
> ---
> 
> Notes:
>     There is no conflict with Dscho's Azure Pipelines patch series; the
>     patch contexts overlap a bit, but the auto-merging results look good
>     to me.

And this indeed also affects Azure Pipelines.

Since Junio's workflow is very different from GitHub Flow (where this
issue would be worked around with a simple, single Pull Request), we have
no prayer at a workaround on our side, though: pretty much *all* of the
next builds of Junio's branches will be broken, unless he chooses to
backport your patch to all of the base commits he chose for those
branches.

So our best bet at not getting overwhelmed with failed builds is to help
Homebrew get `brew update --quiet` to work again. I just opened a ticket
to that end:

	https://github.com/Homebrew/brew/issues/5666

I do like your patch, though, and am very much in favor of fast-tracking
it all the way down to `maint`.

Ciao,
Dscho

>  ci/install-dependencies.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
> index 06c3546e1e..5968efdbbe 100755
> --- a/ci/install-dependencies.sh
> +++ b/ci/install-dependencies.sh
> @@ -34,7 +34,7 @@ linux-clang|linux-gcc)
>  	popd
>  	;;
>  osx-clang|osx-gcc)
> -	brew update --quiet
> +	brew update >/dev/null
>  	# Uncomment this if you want to run perf tests:
>  	# brew install gnu-time
>  	brew install git-lfs gettext
> -- 
> 2.20.1.642.gc55a771460
> 
> 
> 

Re: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

[SZEDER Gábor]

On Mon, Feb 04, 2019 at 10:49:05AM +0100, Johannes Schindelin wrote:
> Hi Gábor,
> 
> On Sat, 2 Feb 2019, SZEDER Gábor wrote:
> 
> > Before installing the necessary dependencies, our OSX build jobs run
> > 'brew update --quiet'.  This is problematic for two reasons:
> > 
> >   - This '--quiet' flag apparently broke overnight, resulting in
> >     errored builds:
> > 
> >       +brew update --quiet
> >       ==> Downloading https://homebrew.bintray.com/bottles-portable-ruby/portable-ruby-2.3.7.mavericks.bottle.tar.gz
> >       ######################################################################## 100.0%
> >       ==> Pouring portable-ruby-2.3.7.mavericks.bottle.tar.gz
> >       Usage: brew update_report [--preinstall]
> >       The Ruby implementation of brew update. Never called manually.
> >               --preinstall                 Run in 'auto-update' mode (faster, less
> >                                            output).
> >           -f, --force                      Override warnings and enable potentially
> >                                            unsafe operations.
> >           -d, --debug                      Display any debugging information.
> >           -v, --verbose                    Make some output more verbose.
> >           -h, --help                       Show this message.
> >       Error: invalid option: --quiet
> >       The command "ci/install-dependencies.sh" failed and exited with 1 during .
> > 
> >     I belive that this breakage will be noticed and fixed soon-ish, so
> >     we could probably just wait a bit for this issue to solve itself,
> >     but:
> > 
> >   - 'brew update --quiet' wasn't really quiet in the first place, as
> >     it listed over about 2000 lines worth of available packages that
> >     we absolutely don't care about, see e.g. one of the latest
> >     'master' builds:
> > 
> >       https://travis-ci.org/git/git/jobs/486134962#L113
> > 
> > So drop this '--quiet' option and redirect 'brew update's standard
> > output to /dev/null to make it really quiet, thereby making the OSX
> > builds work again despite the above mentioned breakage.
> > 
> > Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
> > ---
> > 
> > Notes:
> >     There is no conflict with Dscho's Azure Pipelines patch series; the
> >     patch contexts overlap a bit, but the auto-merging results look good
> >     to me.
> 
> And this indeed also affects Azure Pipelines.
> 
> Since Junio's workflow is very different from GitHub Flow (where this
> issue would be worked around with a simple, single Pull Request),

I doubt that a single PR could be sufficient, because it won't help
contributors who want to run CI builds in their own forks; they would
have to rebase their branches on top of the fix, or cherry-pick it, or
whatnot.

> we have
> no prayer at a workaround on our side, though: pretty much *all* of the
> next builds of Junio's branches will be broken, unless he chooses to
> backport your patch to all of the base commits he chose for those
> branches.
> 
> So our best bet at not getting overwhelmed with failed builds ...

Oh, in my long-time for-fun semi-automated CI experiment [1] all
integration branches and cooking topics are still just fine :) they
all have been including this patch (among many others) for quite some
time now (since when the list of available Homebrew packages was
"only" about 1700 lines... about a year ago).

Unfortunately, I never upstreamed it, because that long list of
packages is folded up anyway when viewing the logs, so why bother...
In hindsight, I should have.  Oh, well.

[1] https://travis-ci.org/szeder/git-cooking-topics-for-travis-ci/branches

> is to help
> Homebrew get `brew update --quiet` to work again. I just opened a ticket
> to that end:
> 
> 	https://github.com/Homebrew/brew/issues/5666
> 
> I do like your patch, though, and am very much in favor of fast-tracking
> it all the way down to `maint`.
> 
> Ciao,
> Dscho
> 
> >  ci/install-dependencies.sh | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
> > index 06c3546e1e..5968efdbbe 100755
> > --- a/ci/install-dependencies.sh
> > +++ b/ci/install-dependencies.sh
> > @@ -34,7 +34,7 @@ linux-clang|linux-gcc)
> >  	popd
> >  	;;
> >  osx-clang|osx-gcc)
> > -	brew update --quiet
> > +	brew update >/dev/null
> >  	# Uncomment this if you want to run perf tests:
> >  	# brew install gnu-time
> >  	brew install git-lfs gettext
> > -- 
> > 2.20.1.642.gc55a771460

Re: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 1241 bytes --]

Hi Gábor,

On Mon, 4 Feb 2019, SZEDER Gábor wrote:

> On Mon, Feb 04, 2019 at 10:49:05AM +0100, Johannes Schindelin wrote:
> 
> > So our best bet at not getting overwhelmed with failed builds ...
> 
> Oh, in my long-time for-fun semi-automated CI experiment [1] all
> integration branches and cooking topics are still just fine :) they
> all have been including this patch (among many others) for quite some
> time now (since when the list of available Homebrew packages was
> "only" about 1700 lines... about a year ago).
> 
> Unfortunately, I never upstreamed it, because that long list of
> packages is folded up anyway when viewing the logs, so why bother...
> In hindsight, I should have.  Oh, well.
> 
> [1] https://travis-ci.org/szeder/git-cooking-topics-for-travis-ci/branches

Heh, I feel ya. 70 branches in Git for Windows' patch thicket are waiting
(for the most part) to be upstreamed.

> > is to help Homebrew get `brew update --quiet` to work again. I just
> > opened a ticket to that end:
> > 
> > 	https://github.com/Homebrew/brew/issues/5666

And this ticket has been closed with a fix! Yay! I checked, and on Travis,
`pu` is now only broken for the FUNNYNAMES issue in Elijah's t4038 that I
already reported.

Ciao,
Dscho

Re: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

[Junio C Hamano]

SZEDER Gábor <szeder.dev@gmail.com> writes:

> Before installing the necessary dependencies, our OSX build jobs run
> 'brew update --quiet'.  This is problematic for two reasons:
>
>   - This '--quiet' flag apparently broke overnight, resulting in
>     errored builds:
> ...
>     I belive that this breakage will be noticed and fixed soon-ish, so
>     we could probably just wait a bit for this issue to solve itself,

Yuck.  Well, an external influence that can break the automated
build job overnight should be capable of fixing it overnight ;-)

>     but:
>
>   - 'brew update --quiet' wasn't really quiet in the first place, as
>     it listed over about 2000 lines worth of available packages that
>     we absolutely don't care about, see e.g. one of the latest
>     'master' builds:
>
>       https://travis-ci.org/git/git/jobs/486134962#L113
>
> So drop this '--quiet' option and redirect 'brew update's standard
> output to /dev/null to make it really quiet, thereby making the OSX
> builds work again despite the above mentioned breakage.

Sounds good.

>     There is no conflict with Dscho's Azure Pipelines patch series; the
>     patch contexts overlap a bit, but the auto-merging results look good
>     to me.

Thanks for extra careful.

If this is truly urgent, I could merge this to 'maint' and merge the
result upwards to 'pu' and it would hide the issue on my four
integration branches.  But one thing that makes me wonder is if we
can (or want to) do anything to help other people who test build
with pull requests.  I guess they need to rebase on top of whatever
commit that has this fix?  That sounds more like a tail wagging a
dog, though.  I dunno.

>  ci/install-dependencies.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
> index 06c3546e1e..5968efdbbe 100755
> --- a/ci/install-dependencies.sh
> +++ b/ci/install-dependencies.sh
> @@ -34,7 +34,7 @@ linux-clang|linux-gcc)
>  	popd
>  	;;
>  osx-clang|osx-gcc)
> -	brew update --quiet
> +	brew update >/dev/null
>  	# Uncomment this if you want to run perf tests:
>  	# brew install gnu-time
>  	brew install git-lfs gettext

Re: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

[SZEDER Gábor]

On Mon, Feb 04, 2019 at 10:26:29AM -0800, Junio C Hamano wrote:
> SZEDER Gábor <szeder.dev@gmail.com> writes:
> 
> >   - This '--quiet' flag apparently broke overnight, resulting in
> >     errored builds:
> > ...
> >     I belive that this breakage will be noticed and fixed soon-ish, so
> >     we could probably just wait a bit for this issue to solve itself,
> 
> Yuck.  Well, an external influence that can break the automated
> build job overnight should be capable of fixing it overnight ;-)

> If this is truly urgent, I could merge this to 'maint' and merge the
> result upwards to 'pu' and it would hide the issue on my four
> integration branches.  But one thing that makes me wonder is if we
> can (or want to) do anything to help other people who test build
> with pull requests.  I guess they need to rebase on top of whatever
> commit that has this fix?  That sounds more like a tail wagging a
> dog, though.  I dunno.

Contributors don't have to rebase to make their PR builds work.  When
Travis CI builds a PR, it doesn't build the branch on its own, but the
merge of that branch into 'master' (or whatever the default branch is
called), assuming can be it merged cleanly.  So as soon as this fix
lands in master, the PR builds should be fine.  I suppose.

However, they will need to rebase their WIP/not PR-ed/not upstreamable
branches on top of this fix if they want to run Travis CI builds in
their own forks.

Unless, of course, the external influence does manage to fix itself
overnight :)  Under Dscho's bugreport it looks like they already
merged a one-liner fix, but how long will it take to tickle down to
Travis CI, I have no idea.

Re: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 427 bytes --]

Hi Gábor,

On Mon, 4 Feb 2019, SZEDER Gábor wrote:

> Under Dscho's bugreport it looks like they already merged a one-liner
> fix, but how long will it take to tickle down to Travis CI, I have no
> idea.

Since the fix affected a Homebrew package, it was not so much about
trickling down to Travis, but about trickling down into the public
Homebrew repositories. And it seems that that trickling is complete now.

Ciao,
Dscho

Re: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On Mon, 4 Feb 2019, SZEDER Gábor wrote:
>
>> Under Dscho's bugreport it looks like they already merged a one-liner
>> fix, but how long will it take to tickle down to Travis CI, I have no
>> idea.
>
> Since the fix affected a Homebrew package, it was not so much about
> trickling down to Travis, but about trickling down into the public
> Homebrew repositories. And it seems that that trickling is complete now.

Good news.  Thanks both for reporting and monitoring.

I still think the >/dev/null thing is a good idea, so unless I hear
otherwise, let's merge it down to 'next', etc.

Re: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 773 bytes --]

Hi Junio,

On Tue, 5 Feb 2019, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > On Mon, 4 Feb 2019, SZEDER Gábor wrote:
> >
> >> Under Dscho's bugreport it looks like they already merged a one-liner
> >> fix, but how long will it take to tickle down to Travis CI, I have no
> >> idea.
> >
> > Since the fix affected a Homebrew package, it was not so much about
> > trickling down to Travis, but about trickling down into the public
> > Homebrew repositories. And it seems that that trickling is complete now.
> 
> Good news.  Thanks both for reporting and monitoring.
> 
> I still think the >/dev/null thing is a good idea, so unless I hear
> otherwise, let's merge it down to 'next', etc.

Yes, it is a very good idea.

Ciao,
Dscho