From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS31976 209.132.180.0/23 X-Spam-Status: No, score=-4.0 required=3.0 tests=AWL,BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED, SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 59EC51F463 for ; Fri, 13 Dec 2019 11:16:04 +0000 (UTC) DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:cc:subject:references:date:in-reply-to :message-id:mime-version:content-type:content-transfer-encoding; q=dns; s=default; b=WC0xnsnaSNjmK4PNmPwKpKEoY+ZYTp5wXFwzw1brFvv +Um/xm9c6l3xM78xPC077PtuuNL9vJMQlznNrixPOIdY2Uxle9HfZlTjbM5hqwgr nDlhataEsdNhIZg6+yQha3LjAkwbBcwZp6qk1CS4ZMMaT9yoRGfGYjYu3qh2VJhc = DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:cc:subject:references:date:in-reply-to :message-id:mime-version:content-type:content-transfer-encoding; s=default; bh=JTB03npRmd3eAHedwFj/OiI9e8E=; b=t6cqvxNNnIH6dYuan 2ryWCAotuh+8VwhhtMNz2tEa9HGV6Zm94FBVr9QGQUbAv/u/5wYn5KgyBFxn8AeE Ha4EZLLRtOlsVrLdI0DpTnBajSoSYUKTGa5lkvYK7qXNX7OSyIxXko994v0xf9QB YmzHiY6XUQMFwUMarDYZ6UyXpM= Received: (qmail 17482 invoked by alias); 13 Dec 2019 11:15:46 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Received: (qmail 17415 invoked by uid 89); 13 Dec 2019 11:15:40 -0000 Authentication-Results: sourceware.org; auth=none X-HELO: us-smtp-delivery-1.mimecast.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1576235736; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s/nOCcIpSn8etgQa/+3TbWeX7+Md0VjfkbdoQMbB39g=; b=UbsFgqdU17OZm0T8M100GBN8QKa/Jdxn2SxQqD3S8cZOrWsLlNnc50zZ/WkdOZC7fh0JKI 1oKGNdGEbQOt+xqEbCUYoIVpwsQQSuLMr0oXeDGnUE/iSagG7el6JwO+ctcwvyu09qH1DZ Z4+QyURMG+XrPoxPESmXI5MFqG0icLc= From: Florian Weimer To: Carlos O'Donell Cc: Stefan Liebler , GNU C Library Subject: Re: How to handle long running tests in nptl. References: <0ba7efb0-9e71-1764-bce8-686c8f452285@linux.ibm.com> Date: Fri, 13 Dec 2019 12:15:27 +0100 In-Reply-To: (Carlos O'Donell's message of "Thu, 12 Sep 2019 10:02:15 -0400") Message-ID: <87wob079yo.fsf@oldenburg2.str.redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 Content-Type: text/plain Content-Transfer-Encoding: quoted-printable * Carlos O'Donell: > On 9/12/19 9:21 AM, Stefan Liebler wrote: >> Hi, >>=20 >> the nptl tests are currently running in sequence. Some of them are runni= ng very long: >> -tst-cond16: 20s >> -tst-cond17: 20s >> -tst-cond18: 20s >> -tst-rwlock-tryrdlock-stall: 20s >> -tst-mutex10: 16s >> -tst-rwlock20: 10s >> -tst-rwlock-trywrlock-stall: 10s >> -tst-rwlock-pwn: 10s >>=20 >> The listed tests are responsible for over two minutes of runtime of "mak= e check". They all are running a test in a loop for a large amount of itera= tions or seconds in order to trigger e.g. a race condition. >>=20 >> How to handle those long running tests with respect of "make check" runt= ime? >> - reduce runtime by reducing number of iterations or seconds >> - move those tests to "make xcheck" >> - reduce runtime while running "make check" and rerun with unchanged run= time in "make xcheck" >> - change nothing >> - other ideas? > > The use of 'make xcheck' is for tests that need specific persmissions > to run, like root. It's also used for tests that require many minutes to run or require special firewall settings, e.g., resolv/tst-resolv-qtypes. Unfortunately, we do not capture data of legitimate test failures, so it's hard to tell how valuable such tests are. Many of the timeout-heavy nptl tests do not actually need a quiet system, though, so they could be moved to a separate subdirectory that contains only such tests. Or we could add some markup to the tests and add a more intelligent test scheduler. Thanks, Florian