From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS22989 209.51.188.0/24 X-Spam-Status: No, score=-3.7 required=3.0 tests=AWL,BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 25D881F463 for ; Mon, 6 Jan 2020 12:47:07 +0000 (UTC) Received: from localhost ([::1]:51400 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ioRmj-0008Ey-LC for normalperson@yhbt.net; Mon, 06 Jan 2020 07:47:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:56574) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ioRme-0008Eq-E8 for bug-gnulib@gnu.org; Mon, 06 Jan 2020 07:47:01 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ioRmd-0005UP-4o for bug-gnulib@gnu.org; Mon, 06 Jan 2020 07:47:00 -0500 Received: from mo6-p00-ob.smtp.rzone.de ([2a01:238:20a:202:5300::3]:24348) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ioRmc-0005SN-Ln for bug-gnulib@gnu.org; Mon, 06 Jan 2020 07:46:59 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1578314815; s=strato-dkim-0002; d=clisp.org; h=References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: X-RZG-CLASS-ID:X-RZG-AUTH:From:Subject:Sender; bh=EUYMJFt7B1BVpNEPdJtqswT1Ft/bcAT+bH4n9EIWTB4=; b=ULiUgZJL4KcKLPO5Vpm26FB40G7p66znEVcoNyaSg1WBHzQMoE4U0alKbfEwR47UBK oicFgPTxUYfJgDSAWuH/Zv2dLw56PkZDiDq3UaEJqen3+LV/TuktKs+64xfDyv8N+gGF GiBkhn6j83j6ulQH+3tJdJmLKBeIPa5jvhwwfxE3PowqfiMx9RtmjyS561DS8rhd/4zl fD+OeC59aQ0X7ujKN7JfsIWE95oytw6lFYp0a5vDOae/4yjIrMazTKJnkRURuvMLcWUN 4T01p/C2zocJ85d/dT1ji3Mq4mCLHwBUA9/Bd94CzOWMt/0tOhAaIWzaJpRO1es3VRky NMRw== X-RZG-AUTH: ":Ln4Re0+Ic/6oZXR1YgKryK8brlshOcZlIWs+iCP5vnk6shH+AHjwLuWOH6fzxfs=" X-RZG-CLASS-ID: mo00 Received: from bruno.haible.de by smtp.strato.de (RZmta 46.1.3 DYNA|AUTH) with ESMTPSA id R06a06w06CksVTL (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (curve X9_62_prime256v1 with 256 ECDH bits, eq. 3072 bits RSA)) (Client did not present a certificate); Mon, 6 Jan 2020 13:46:54 +0100 (CET) From: Bruno Haible To: Tim =?ISO-8859-1?Q?R=FChsen?= Subject: Re: string types Date: Mon, 06 Jan 2020 13:46:53 +0100 Message-ID: <8097572.G4CVgrrz2P@omega> User-Agent: KMail/5.1.3 (Linux/4.4.0-170-generic; KDE/5.18.0; x86_64; ; ) In-Reply-To: <9ac48e4c-9470-d64b-e14e-a3f66c9c00e1@gmx.de> References: <175192568.e2XXTFFdkW@omega> <7775995.zBxFH0rR4k@omega> <9ac48e4c-9470-d64b-e14e-a3f66c9c00e1@gmx.de> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a01:238:20a:202:5300::3 X-BeenThere: bug-gnulib@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Gnulib discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: ag , bug-gnulib@gnu.org Errors-To: bug-gnulib-bounces+normalperson=yhbt.net@gnu.org Sender: "bug-gnulib" Hi Tim, > >>> - providing primitives for string allocation reduces the amount of buffer > >>> overflow bugs that otherwise occur in this area. [1] > >>> [1] https://lists.gnu.org/archive/html/bug-gnulib/2019-09/msg00031.html > > ... > We created a 'catch them all' string/buffer type plus API. It is a good > compromise for all kinds of situations, works like a memory buffer but > is guaranteed 0-terminated, allows custom stack buffers with fallback to > heap if to small. > > https://gitlab.com/gnuwget/wget2/blob/master/libwget/buffer.c > > > There also is a sprintf functionality (glibc compatible) using these > buffers - and the operation is well faster than glibc's sprintf-like > functions for all format strings tested (tested back a few years). The > code is also much smaller (380 C code lines), the return values are > size_t. It doesn't support float/double. > > https://gitlab.com/gnuwget/wget2/blob/master/libwget/buffer_printf.c > > If there is serious interest, I could prepare modules for gnulib. It is interesting that your solution does not only cover the simple cases (string concatenation, etc.), but also the more complex one, possibly with if()s in the generation logic, and all this without blatant potential for buffer overflow bugs. So, the solution would consists of the following parts: (A) A growable buffer type, with up to N (128 or 1024 or so) bytes on the stack. (B) A set of functions for appending to such a growable buffer. (C) A function for creating a heap-allocated 'char *' from a growable buffer. (D) Short-hand functions for the simple cases (like string concatenation). It would be good to have all these well integrated (in terms of function names and calling conventions). So far, in gnulib, we have only pieces of it: - Module 'scratch_buffer' is (A) without (B), (C), (D). - Modules 'vasnprintf', 'asprintf' are (B), (C), (D) but without (A). Before you start writing the code, it's worth looking at the following questions: * Should the module 'scratch_buffer' be reused for (A)? Or is this not possible? If not, can it still have a memory-leak prevention like described in lib/malloc/scratch_buffer.h? * What about reusing the complete vasnprintf.c for (B), rather than adding another, limited printf-like implementation? * Is it best to implement (D) based on (A), (B), (C), or directly from scratch? Bruno