From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS3215 2.6.0.0/16 X-Spam-Status: No, score=-4.0 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,RCVD_IN_DNSWL_MED,SPF_HELO_PASS, SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 4244A1F8C6 for ; Sat, 31 Jul 2021 10:48:13 +0000 (UTC) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 4798D386F83B for ; Sat, 31 Jul 2021 10:48:10 +0000 (GMT) Received: from dog.birch.relay.mailchannels.net (dog.birch.relay.mailchannels.net [23.83.209.48]) by sourceware.org (Postfix) with ESMTPS id B3B52385801E for ; Sat, 31 Jul 2021 10:47:57 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org B3B52385801E Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=gotplt.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gotplt.org X-Sender-Id: dreamhost|x-authsender|siddhesh@gotplt.org Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id C2E699219C0; Sat, 31 Jul 2021 10:47:55 +0000 (UTC) Received: from pdx1-sub0-mail-a16.g.dreamhost.com (100-98-55-223.trex.outbound.svc.cluster.local [100.98.55.223]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 63BAF9217D1; Sat, 31 Jul 2021 10:47:54 +0000 (UTC) X-Sender-Id: dreamhost|x-authsender|siddhesh@gotplt.org Received: from pdx1-sub0-mail-a16.g.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384) by 100.98.55.223 (trex/6.3.3); Sat, 31 Jul 2021 10:47:55 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|siddhesh@gotplt.org X-MailChannels-Auth-Id: dreamhost X-Wipe-Callous: 3d53c4407fa813f4_1627728475555_2359650808 X-MC-Loop-Signature: 1627728475554:1467942928 X-MC-Ingress-Time: 1627728475554 Received: from pdx1-sub0-mail-a16.g.dreamhost.com (localhost [127.0.0.1]) by pdx1-sub0-mail-a16.g.dreamhost.com (Postfix) with ESMTP id 249C97F953; Sat, 31 Jul 2021 03:47:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gotplt.org; h=subject:to :cc:references:from:message-id:date:mime-version:in-reply-to :content-type:content-transfer-encoding; s=gotplt.org; bh=9eZLOZ WAioyNsor6fTneBksbzbc=; b=Qb6MUmxQhE3txbrThDQdb3IiKpbpfp6vEifJ1R QmoJOe6VBK5c4L2lveQ9EeFoPMd9FLz3DhLwfRpmcfzljPdZHALy7b8UygXVnoQM O5ssSJFUXqFFyBj6/b7ri8hfiT2kOF6UWHJ7VKuUUv9/0QL8II2/ADZMwSggzs0s 0/ruU= Received: from [192.168.1.158] (unknown [1.186.101.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: siddhesh@gotplt.org) by pdx1-sub0-mail-a16.g.dreamhost.com (Postfix) with ESMTPSA id A80C88B665; Sat, 31 Jul 2021 03:47:49 -0700 (PDT) Subject: Re: [PATCH v2 3/5] benchtests: Add a script to convert benchout string JSON to CSV To: "naohirot@fujitsu.com" , 'Joseph Myers' References: <20210713082214.307529-1-naohirot@fujitsu.com> <20210720063540.362366-1-naohirot@fujitsu.com> X-DH-BACKEND: pdx1-sub0-mail-a16 From: Siddhesh Poyarekar Message-ID: <434f9a46-fd69-34a2-93fd-f4a64aab0828@gotplt.org> Date: Sat, 31 Jul 2021 16:17:44 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=iso-2022-jp; format=flowed; delsp=yes Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "libc-alpha@sourceware.org" , Wilco Dijkstra Errors-To: libc-alpha-bounces+e=80x24.org@sourceware.org Sender: "Libc-alpha" On 7/30/21 12:35 PM, naohirot@fujitsu.com wrote: > Hi Siddhesh, > > Thank you for the advice! > >> Most people in the community who work in string function improvements >> tend to use (and improve wherever it is lacking) >> benchtests/scripts/compare_strings.py for their result analysis. Adding >> a flag to dump csv to that script ought to be trivial if that's what you >> need. > > I see. I didn't use compare_strings.py daily, but plot_strings.py. > >> The script is under-documented though, so perhaps a wiki page describing >> what the script does and various example uses would go a very long way. > > I found the wiki page. > https://sourceware.org/glibc/wiki/benchmarking/benchmarks Yeah that needs to improve :/ >> The reason for emitting json is precisely to allow developers to >> implement their own analysis tools around them when their use cases are >> niche. Your specific use case is not niche and could be added as a flag >> to compare_strings.py if needed. You only need a new flag --csv (or -o >> csv, tab, etc.) to print in csv instead of the current output, which is >> meant for reading on the terminal. > > Yes, converting to CSV in Python will be easy. > But comparing two string benchout results directly between "before" and "after" > was not so easy AFAIK. > And creating graphs in spreadsheet manually is tolerable in a few times, but not > in frequent times. > That's the reason I created another 'jq' script, merge_strings4graph.sh". > Dose most people compare the two results indirectly through a common base ifunc > using "--base" option of compare_strings.py or "--baseline" option of plot_strings.py? If you need to compare two results then just set one as the base using the --base/--baseline and see how the other compares. That's usually sufficient to justify addition of new variants to glibc. Siddhesh