From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS3215 2.6.0.0/16 X-Spam-Status: No, score=-3.4 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by dcvr.yhbt.net (Postfix) with ESMTP id E81791F8C5 for ; Tue, 22 Mar 2022 18:23:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240286AbiCVSY5 (ORCPT ); Tue, 22 Mar 2022 14:24:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240282AbiCVSYz (ORCPT ); Tue, 22 Mar 2022 14:24:55 -0400 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 18B411F623 for ; Tue, 22 Mar 2022 11:23:14 -0700 (PDT) Received: by mail-wm1-x329.google.com with SMTP id v130-20020a1cac88000000b00389d0a5c511so3810022wme.5 for ; Tue, 22 Mar 2022 11:23:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=8V1FoXCb86xqZ2XOaLWrGr4avTXqor6TX1WNeRZ9avw=; b=J42ZzPVvrZZyqS7Bunm/ZXZula4c8FXnn50t4fnm+4IYEnCZpO3haeZSGpzBndt3i/ xoWmVNcyEBZMkIaOQCnOlaGJS3TPQjfKh2D22xYkQn9uEUjK3DMaSmfGa8k7hLMjlNnm CoOnMPxlGp+KrxUygs8TC60JCrKZuvFOLNkDgvv9vt5NMVRaAN78DTavyON7DSDJCEqM kI1eAtxphCzVqA6Os7yt38HTSXYT49LeHscQK5i7bOzThK+RSH+BWyMIV6b1+/XDKYmb 9BOc62NRHUEKSEqJ++cPn00QJVs0VfyUv4zLvcdv4h8GomX2rqZN0nUrXb+ECYzr2BW1 m1zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=8V1FoXCb86xqZ2XOaLWrGr4avTXqor6TX1WNeRZ9avw=; b=KHEhEyEdFelI7wCudvC0rpqJCrdUdQvXqG02DgFwXbfsW+t7aJfTrxSIzRzT/48TGk VzD7HEx0nyD4apI940dWjTlFX8Rx1p1lQU0QXeAhqdrpFgxpc/miir0qx3+CWV8IHY4K JFUPVEMeFXtMKOKOGOp6My7yLwWQCIaSC4iXDcGuK3WtdSH5McNwacr9JYHFbpaZ/H7z tPxVO0PE58dWncwS4gHsD6rYSB4LWFiPDxP4DlRn8sZtvPM/FXfhZarX5sISVjaJ4v9L SvncctTClL4zhwdJiwyvyAudnGAHYyFsacKBWSyVK+nsMI1PCpsks260hF1Ueq56FU+L Ydng== X-Gm-Message-State: AOAM530vaCRQ0CiF9fqNag832g2D3GfP62HmIku6HltwhX9W3UoNfty2 8OJ7MyIcj0MXhfr7j3yroeuIqmEFgLc= X-Google-Smtp-Source: ABdhPJxkFiYMMbnzRdXzYLLP38v2nF5myKw8mKK5YqkJPD+JPaIyAEOqqqg3HL1gabh7DB54L0i/tQ== X-Received: by 2002:a7b:c8d5:0:b0:38c:9eb9:129 with SMTP id f21-20020a7bc8d5000000b0038c9eb90129mr5264547wml.65.1647973392489; Tue, 22 Mar 2022 11:23:12 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id c24-20020a7bc018000000b0038a18068cf5sm2306801wmb.15.2022.03.22.11.23.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Mar 2022 11:23:11 -0700 (PDT) Message-Id: <901fa32f6eacab7ba76aedbb020855c8f9718943.1647973380.git.gitgitgadget@gmail.com> In-Reply-To: References: From: "Jeff Hostetler via GitGitGadget" Date: Tue, 22 Mar 2022 18:22:36 +0000 Subject: [PATCH v3 03/27] t/helper/fsmonitor-client: create stress test Fcc: Sent Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Derrick Stolee , =?UTF-8?Q?=C3=86var_Arnfj=C3=B6r=C3=B0?= Bjarmason , Torsten =?unknown-8bit?Q?B=C3=B6gershausen?= , rsbecker@nexbridge.com, Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler Create a stress test to hammer on the fsmonitor daemon. Create a client-side thread pool of n threads and have each of them make m requests as fast as they can. We do not currently inspect the contents of the response. We're only interested in placing a heavy request load on the daemon. This test is useful for interactive testing and various experimentation. For example, to place additional load on the daemon while another test is running. We currently do not have a test script that actually uses this helper. We might add such a test in the future. Signed-off-by: Jeff Hostetler --- t/helper/test-fsmonitor-client.c | 106 +++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) diff --git a/t/helper/test-fsmonitor-client.c b/t/helper/test-fsmonitor-client.c index 3062c8a3c2b..54a4856c48c 100644 --- a/t/helper/test-fsmonitor-client.c +++ b/t/helper/test-fsmonitor-client.c @@ -7,6 +7,8 @@ #include "cache.h" #include "parse-options.h" #include "fsmonitor-ipc.h" +#include "thread-utils.h" +#include "trace2.h" #ifndef HAVE_FSMONITOR_DAEMON_BACKEND int cmd__fsmonitor_client(int argc, const char **argv) @@ -79,20 +81,121 @@ static int do_send_flush(void) return 0; } +struct hammer_thread_data +{ + pthread_t pthread_id; + int thread_nr; + + int nr_requests; + const char *token; + + int sum_successful; + int sum_errors; +}; + +static void *hammer_thread_proc(void *_hammer_thread_data) +{ + struct hammer_thread_data *data = _hammer_thread_data; + struct strbuf answer = STRBUF_INIT; + int k; + int ret; + + trace2_thread_start("hammer"); + + for (k = 0; k < data->nr_requests; k++) { + strbuf_reset(&answer); + + ret = fsmonitor_ipc__send_query(data->token, &answer); + if (ret < 0) + data->sum_errors++; + else + data->sum_successful++; + } + + strbuf_release(&answer); + trace2_thread_exit(); + return NULL; +} + +/* + * Start a pool of client threads that will each send a series of + * commands to the daemon. + * + * The goal is to overload the daemon with a sustained series of + * concurrent requests. + */ +static int do_hammer(const char *token, int nr_threads, int nr_requests) +{ + struct hammer_thread_data *data = NULL; + int k; + int sum_join_errors = 0; + int sum_commands = 0; + int sum_errors = 0; + + if (!token || !*token) + token = get_token_from_index(); + if (nr_threads < 1) + nr_threads = 1; + if (nr_requests < 1) + nr_requests = 1; + + CALLOC_ARRAY(data, nr_threads); + + for (k = 0; k < nr_threads; k++) { + struct hammer_thread_data *p = &data[k]; + p->thread_nr = k; + p->nr_requests = nr_requests; + p->token = token; + + if (pthread_create(&p->pthread_id, NULL, hammer_thread_proc, p)) { + warning("failed to create thread[%d] skipping remainder", k); + nr_threads = k; + break; + } + } + + for (k = 0; k < nr_threads; k++) { + struct hammer_thread_data *p = &data[k]; + + if (pthread_join(p->pthread_id, NULL)) + sum_join_errors++; + sum_commands += p->sum_successful; + sum_errors += p->sum_errors; + } + + fprintf(stderr, "HAMMER: [threads %d][requests %d] [ok %d][err %d][join %d]\n", + nr_threads, nr_requests, sum_commands, sum_errors, sum_join_errors); + + free(data); + + /* + * Return an error if any of the _send_query requests failed. + * We don't care about thread create/join errors. + */ + return sum_errors > 0; +} + int cmd__fsmonitor_client(int argc, const char **argv) { const char *subcmd; const char *token = NULL; + int nr_threads = 1; + int nr_requests = 1; const char * const fsmonitor_client_usage[] = { "test-tool fsmonitor-client query []", "test-tool fsmonitor-client flush", + "test-tool fsmonitor-client hammer [] [] []", NULL, }; struct option options[] = { OPT_STRING(0, "token", &token, "token", "command token to send to the server"), + + OPT_INTEGER(0, "threads", &nr_threads, "number of client threads"), + OPT_INTEGER(0, "requests", &nr_requests, "number of requests per thread"), + OPT_END() }; @@ -111,6 +214,9 @@ int cmd__fsmonitor_client(int argc, const char **argv) if (!strcmp(subcmd, "flush")) return !!do_send_flush(); + if (!strcmp(subcmd, "hammer")) + return !!do_hammer(token, nr_threads, nr_requests); + die("Unhandled subcommand: '%s'", subcmd); } #endif -- gitgitgadget