From: Chris Torek <chris.torek@gmail.com>
To: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Cc: Jan Kara <jack@suse.cz>, Git List <git@vger.kernel.org>
Subject: Re: Stochastic bisection support
Date: Fri, 19 Nov 2021 23:54:12 -0800 [thread overview]
Message-ID: <CAPx1GvfT36SvJ6Lwf1-2KUebVXkCkNvYTUw=FU+dHBy76VN5RQ@mail.gmail.com> (raw)
In-Reply-To: <nycvar.QRO.7.76.6.2111191731400.63@tvgsbejvaqbjf.bet>
On Fri, Nov 19, 2021 at 8:51 AM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> An interesting problem, for sure!
>
> It is slightly related to a scenario that has been described to me
> recently: in a gigantic project whose full test suite is too large to run
> with every Pull Request, where tests are more likely to become flaky
> rather than simply break, a stochastic CI regime was introduced where a
> semi-random subset of the test suite is run with every CI build. That team
> also came up with the concept of attaching confidences as you describe.
>
> I only had time to look at the first patch closely so far. I hope to find
> more time next week to review further.
I only scanned for obvious items myself, but the idea of a
probabilistic test is indeed interesting.
I do wonder why you (Jan Kara, not Dscho :-) ) used fixed-point
arithmetic here though.
Chris
next prev parent reply other threads:[~2021-11-20 7:54 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-18 16:49 Stochastic bisection support Jan Kara
2021-11-18 16:49 ` [PATCH 01/27] bisect: Fixup test rev-list-bisect/02 Jan Kara
2021-11-18 20:08 ` Chris Torek
2021-11-19 16:31 ` Johannes Schindelin
2021-11-22 12:48 ` Jan Kara
2021-11-18 16:49 ` [PATCH 02/27] bisect: Fixup bisect-porcelain/17 Jan Kara
2021-11-18 22:05 ` Taylor Blau
2021-11-22 12:27 ` Jan Kara
2021-11-18 16:49 ` [PATCH 03/27] bisect: Fixup test bisect-porcelain/20 Jan Kara
2021-11-18 20:13 ` Chris Torek
2021-11-18 22:10 ` Taylor Blau
2021-11-22 12:49 ` Jan Kara
2021-11-18 16:49 ` [PATCH 04/27] bisect: Fixup bisect-porcelain/32 Jan Kara
2021-11-18 16:49 ` [PATCH 05/27] bisect: Fixup bisect-porcelain/34 Jan Kara
2021-11-18 16:49 ` [PATCH 06/27] bisect: Fixup bisect-porcelain/40 Jan Kara
2021-11-18 16:49 ` [PATCH 07/27] bisect: Remove duplicated bisect-porcelain/48 Jan Kara
2021-11-18 16:49 ` [PATCH 08/27] bisect: Fixup bisect-porcelain/50 Jan Kara
2021-11-18 16:49 ` [PATCH 09/27] bisect: Fixup bisect-porcelain/54 Jan Kara
2021-11-18 16:49 ` [PATCH 10/27] bisect: Fixup bisect-porcelain/58 Jan Kara
2021-11-18 16:49 ` [PATCH 11/27] bisect: Fix bisection debugging Jan Kara
2021-11-18 16:49 ` [PATCH 12/27] bisect: Accept and store confidence with each decision Jan Kara
2021-11-18 16:49 ` [PATCH 13/27] bisect: Allow specifying desired result confidence Jan Kara
2021-11-18 16:49 ` [PATCH 14/27] bisect: Use void * for commit_weight Jan Kara
2021-11-18 16:49 ` [PATCH 15/27] bisect: Rename clear_distance() to clear_counted_flag() Jan Kara
2021-11-18 16:49 ` [PATCH 16/27] bisect: Separate commit list reversal Jan Kara
2021-11-18 16:49 ` [PATCH 17/27] bisect: Allow more complex commit weights Jan Kara
2021-11-18 16:49 ` [PATCH 18/27] bisect: Terminate early if there are no eligible commits Jan Kara
2021-11-18 16:49 ` [PATCH 19/27] bisect: Compute reachability of tested revs Jan Kara
2021-11-18 16:49 ` [PATCH 20/27] bisect: Compute probability a particular commit is bad Jan Kara
2021-11-18 16:49 ` [PATCH 21/27] bisect: Reorganize commit weight computation Jan Kara
2021-11-18 16:49 ` [PATCH 22/27] bisect: Move count_distance() Jan Kara
2021-11-18 16:49 ` [PATCH 23/27] bisect: Find bisection point for stochastic weights Jan Kara
2021-11-18 16:49 ` [PATCH 24/27] bisect: Stop bisection when we are confident about bad commit Jan Kara
2021-11-18 16:49 ` [PATCH 25/27] bisect: Report commit with the highest probability Jan Kara
2021-11-18 16:49 ` [PATCH 26/27] bisect: Debug stochastic bisection Jan Kara
2021-11-18 16:49 ` [PATCH 27/27] bisect: Allow bisection debugging of approx_halfway() Jan Kara
2021-11-18 22:49 ` Stochastic bisection support Taylor Blau
2021-11-22 12:13 ` Jan Kara
2021-11-19 16:39 ` Johannes Schindelin
2021-11-20 7:54 ` Chris Torek [this message]
2021-11-22 11:57 ` Jan Kara
2021-11-22 12:55 ` Christian Couder
2021-11-22 13:31 ` Jan Kara
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAPx1GvfT36SvJ6Lwf1-2KUebVXkCkNvYTUw=FU+dHBy76VN5RQ@mail.gmail.com' \
--to=chris.torek@gmail.com \
--cc=Johannes.Schindelin@gmx.de \
--cc=git@vger.kernel.org \
--cc=jack@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).