git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
From: Ramsay Jones <ramsay@ramsayjones.plus.com>
To: Adam Dinwoodie <git@dinwoodie.org>,
	Olga Telezhnaya <olyatelezhnaya@gmail.com>
Cc: Git Mailing List <git@vger.kernel.org>, Jeff King <peff@peff.net>,
	Christian Couder <christian.couder@gmail.com>
Subject: Re: [PATCH v3 22/23] cat-file: tests for new atoms added
Date: Sun, 18 Feb 2018 22:55:29 +0000	[thread overview]
Message-ID: <58b2bdcd-d621-fd21-ab4d-6a9478319b19@ramsayjones.plus.com> (raw)
In-Reply-To: <CA+kUOamJowkxp0xAPf5FA+wBkiOjQeYzW1sKMwcatQBBA1qWpw@mail.gmail.com>



On 16/02/18 14:55, Adam Dinwoodie wrote:
> On 12 February 2018 at 08:08, Olga Telezhnaya wrote:
>> Add some tests for new formatting atoms from ref-filter.
>> Some of new atoms are supported automatically,
>> some of them are expanded into empty string
>> (because they are useless for some types of objects),
>> some of them could be supported later in other patches.
> 
> This commit appears to be introducing failures in t1006 on Cygwin.
> Specifically, tests 15, 36, 58 and 89, all titled "Check %(refname)
> gives empty output", are failing on the pu branch at 21937aad4, and
> git bisect identifies this commit, 3c1571744 ("cat-file: tests for new
> atoms added", 2018-02-12), as the culprit.

Hi Adam, Olga,

Sparse has been complaining about this 'ot/cat-batch-format' branch for
a while, so I had already noted (apart from a symbol which should be
marked as static) something that looked somewhat off - namely, the on
stack initialisation of a 'struct ref_array_item' (builtin/cat-file.c,
line 246, in function batch_object_write()).

In particular, the 'struct ref_array_item' ends with a [FLEX_ARRAY] field
for the refname, so it clearly isn't meant to be declared on the stack.
The 'ref-filter' code uses a 'constructor' function called 'new_ref_array_\
item() which, among others, takes a 'refname' parameter with which to
initialise the refname FLEX_ARRAY field.

So, on Linux, if you build with SANITIZE=address and then run that test:

  $ ./t1006-cat-file.sh -i -v
  Initialized empty Git repository in /home/ramsay/git/t/trash directory.t1006-cat-file/.git/
  expecting success: 
  	echo_without_newline "$hello_content" > hello &&
  	git update-index --add hello
  
  ok 1 - setup
  
  ...
  
  =================================================================
  ==12556==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fff132f04a8 at pc 0x7f1c1b3ca20b bp 0x7fff132f00f0 sp 0x7fff132ef898
  READ of size 4 at 0x7fff132f04a8 thread T0
      #0 0x7f1c1b3ca20a in __interceptor_strlen (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x7020a)
      #1 0x6798cc in strbuf_addstr /home/ramsay/git/strbuf.h:273
      #2 0x6798cc in quote_formatting /home/ramsay/git/ref-filter.c:496
      #3 0x68325d in format_ref_array_item /home/ramsay/git/ref-filter.c:2238
      #4 0x68358a in show_ref_array_item /home/ramsay/git/ref-filter.c:2262
      #5 0x429866 in batch_object_write builtin/cat-file.c:252
      #6 0x42b095 in batch_one_object builtin/cat-file.c:306
      #7 0x42b095 in batch_objects builtin/cat-file.c:407
      #8 0x42b095 in cmd_cat_file builtin/cat-file.c:528
      #9 0x40d3cb in run_builtin /home/ramsay/git/git.c:346
      #10 0x40d3cb in handle_builtin /home/ramsay/git/git.c:556
      #11 0x40f8ae in run_argv /home/ramsay/git/git.c:608
      #12 0x40f8ae in cmd_main /home/ramsay/git/git.c:685
      #13 0x40b667 in main /home/ramsay/git/common-main.c:43
      #14 0x7f1c1ab7982f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
      #15 0x40ce38 in _start (/home/ramsay/git/git+0x40ce38)
  
  Address 0x7fff132f04a8 is located in stack of thread T0 at offset 328 in frame
      #0 0x4296ff in batch_object_write builtin/cat-file.c:245
  
    This frame has 4 object(s):
      [32, 36) 'type'
      [96, 104) 'contents'
      [160, 168) 'size'
      [224, 328) 'item' <== Memory access at offset 328 overflows this variable
  HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
        (longjmp and C++ exceptions *are* supported)
  SUMMARY: AddressSanitizer: stack-buffer-overflow ??:0 __interceptor_strlen
  Shadow bytes around the buggy address:
    0x100062656040: 00 00 f3 f3 f3 f3 f3 f3 f3 f3 00 00 00 00 00 00
    0x100062656050: 00 00 00 00 f1 f1 f1 f1 00 00 00 f4 f3 f3 f3 f3
    0x100062656060: 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1 f1 f1
    0x100062656070: 04 f4 f4 f4 f2 f2 f2 f2 00 f4 f4 f4 f2 f2 f2 f2
    0x100062656080: 00 f4 f4 f4 f2 f2 f2 f2 00 00 00 00 00 00 00 00
  =>0x100062656090: 00 00 00 00 00[f4]f4 f4 f3 f3 f3 f3 00 00 00 00
    0x1000626560a0: 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1 f1 f1
    0x1000626560b0: 04 f4 f4 f4 f2 f2 f2 f2 04 f4 f4 f4 f2 f2 f2 f2
    0x1000626560c0: 04 f4 f4 f4 f2 f2 f2 f2 00 f4 f4 f4 f2 f2 f2 f2
    0x1000626560d0: 00 f4 f4 f4 f2 f2 f2 f2 00 00 f4 f4 f2 f2 f2 f2
    0x1000626560e0: 00 00 00 f4 f2 f2 f2 f2 00 00 00 f4 f2 f2 f2 f2
  Shadow byte legend (one shadow byte represents 8 application bytes):
    Addressable:           00
    Partially addressable: 01 02 03 04 05 06 07 
    Heap left redzone:       fa
    Heap right redzone:      fb
    Freed heap region:       fd
    Stack left redzone:      f1
    Stack mid redzone:       f2
    Stack right redzone:     f3
    Stack partial redzone:   f4
    Stack after return:      f5
    Stack use after scope:   f8
    Global redzone:          f9
    Global init order:       f6
    Poisoned by user:        f7
    Container overflow:      fc
    Array cookie:            ac
    Intra object redzone:    bb
    ASan internal:           fe
  ==12556==ABORTING
  Aborted (core dumped)
  not ok 15 - Check %(refname) gives empty output
  #	
  #		echo "$expected" >expect &&
  #		echo $sha1 | git cat-file --batch-check="$atoms" >actual &&
  #		test_cmp expect actual
  #	    
  $ 
 
... you get a stack-overflow at that very stackframe. Incidentally, the
output from the failed test #15 on cygwin always looked the same: 
  
  $ xxd actual
  00000000: a0c4 ffff 0a                             .....
  $ 
 
So, just as an experiment, I tried changing that variable to a heap
variable and initialising it with an empty ("") refname:
 
  diff --git a/builtin/cat-file.c b/builtin/cat-file.c
  index c9d83ceff..12a743034 100644
  --- a/builtin/cat-file.c
  +++ b/builtin/cat-file.c
  @@ -243,19 +243,20 @@ static void print_object_or_die(struct batch_options *opt, struct expand_data *d
   static void batch_object_write(const char *obj_name, struct batch_options *opt,
   			       struct expand_data *data)
   {
  -	struct ref_array_item item = {0};
  +	struct ref_array_item *item;
   
  -	item.oid = data->oid;
  -	item.rest = data->rest;
  -	item.objectname = obj_name;
  +	FLEX_ALLOC_STR(item, refname, "");
  +	item->oid = data->oid;
  +	item->rest = data->rest;
  +	item->objectname = obj_name;
   
  -	if (show_ref_array_item(&item, &opt->format))
  +	if (show_ref_array_item(item, &opt->format))
   		return;
   	if (!opt->buffer_output)
   		fflush(stdout);
   
   	if (opt->print_contents) {
  -		data->type = item.type;
  +		data->type = item->type;
   		print_object_or_die(opt, data);
   		batch_write(opt, "\n", 1);
   	}
  -- 

... and now this test passes on cygwin (and the SANITIZE build on Linux).

Of course, this is not a real fix, since this has probably only changed
the stack-overflow into an un-diagnosed heap-overflow bug! ;-)

However, the above should provide enough info for someone more familiar
with the code to implement a correct fix.

[BTW, the symbol that should be marked static is: cat_file_info, in file
ref-filter.c, line 103.]

ATB,
Ramsay Jones



  reply	other threads:[~2018-02-18 22:55 UTC|newest]

Thread overview: 115+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-01-26 19:43 [PATCH RFC 01/24] ref-filter: get rid of goto Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 21/24] ref-filter: work with objectsize:disk Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 16/24] ref-filter: make cat_file_info independent Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 02/24] ref-filter: add return value to some functions Olga Telezhnaya
2018-01-26 21:05   ` Junio C Hamano
2018-01-29  7:36     ` Оля Тележная
2018-01-26 19:43 ` [PATCH RFC 09/24] cat-file: start use ref_array_item struct Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 15/24] cat-file: move skip_object_info into ref-filter Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 14/24] ref_filter: add is_rest_atom_used function Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 06/24] cat-file: move struct expand_data into ref-filter Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 10/24] ref-filter: make populate_value global Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 13/24] ref-filter: add is_cat flag Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 07/24] cat-file: start migrating to ref-filter Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 24/24] cat-file: update of docs Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 03/24] cat-file: split expand_atom into 2 functions Olga Telezhnaya
2018-01-26 21:46   ` Junio C Hamano
2018-01-29  7:26     ` Оля Тележная
2018-01-26 19:43 ` [PATCH RFC 12/24] ref-filter: get rid of expand_atom_into_fields Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 04/24] cat-file: reuse struct ref_format Olga Telezhnaya
2018-01-26 22:18   ` Junio C Hamano
2018-01-26 19:43 ` [PATCH RFC 08/24] ref-filter: reuse parse_ref_filter_atom Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 11/24] cat-file: start reusing populate_value Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 05/24] ref-filter: make valid_atom as function parameter Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 18/24] ref-filter: make valid_atom general again Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 22/24] for-each-ref: tests for new atoms added Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 20/24] ref-filter: unifying formatting of cat-file opts Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 19/24] ref-filter: make populate_value internal again Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 17/24] cat-file: reuse printing logic from ref-filter Olga Telezhnaya
2018-01-26 19:43 ` [PATCH RFC 23/24] cat-file: tests for new atoms added Olga Telezhnaya
2018-01-26 20:19 ` [PATCH RFC 01/24] ref-filter: get rid of goto Junio C Hamano
2018-01-29  7:13   ` Оля Тележная
2018-01-30 20:49     ` Junio C Hamano
2018-01-31  6:39       ` Оля Тележная
2018-02-05 11:27 ` [PATCH RFC v2 01/25] " Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 22/25] ref-filter: work with objectsize:disk Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 12/25] cat-file: start reusing populate_value() Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 04/25] ref-filter: make valid_atom as function parameter Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 03/25] cat-file: reuse struct ref_format Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 18/25] ref-filter: make valid_atom general again Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 23/25] for-each-ref: tests for new atoms added Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 19/25] cat-file: reuse printing logic from ref-filter Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 02/25] ref-filter: add return value to some functions Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 21/25] ref-filter: unifying formatting of cat-file opts Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 13/25] ref-filter: get rid of mark_atom_in_object_info() Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 09/25] cat-file: start use ref_array_item struct Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 24/25] cat-file: tests for new atoms added Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 10/25] ref-filter: make populate_value() global Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 25/25] cat-file: update of docs Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 06/25] cat-file: split expand_atom() into 2 functions Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 05/25] cat-file: move struct expand_data into ref-filter Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 11/25] ref-filter: rename field in ref_array_item stuct Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 20/25] ref-filter: make populate_value() internal again Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 16/25] cat-file: move skip_object_info into ref-filter Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 14/25] ref-filter: add is_cat flag Olga Telezhnaya
2018-02-05 21:19     ` Junio C Hamano
2018-02-05 11:27   ` [PATCH RFC v2 17/25] ref-filter: make cat_file_info independent Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 15/25] ref_filter: add is_atom_used function Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 08/25] ref-filter: reuse parse_ref_filter_atom() Olga Telezhnaya
2018-02-05 11:27   ` [PATCH RFC v2 07/25] cat-file: start migrating formatting to ref-filter Olga Telezhnaya
2018-02-12  8:08   ` [PATCH v3 01/23] ref-filter: get rid of goto Olga Telezhnaya
2018-02-12  8:08     ` [PATCH v3 05/23] cat-file: move struct expand_data into ref-filter Olga Telezhnaya
2018-02-15  5:26       ` Jeff King
2018-02-12  8:08     ` [PATCH v3 11/23] ref-filter: rename field in ref_array_item stuct Olga Telezhnaya
2018-02-12  8:08     ` [PATCH v3 20/23] ref-filter: unifying formatting of cat-file opts Olga Telezhnaya
2018-02-15  5:56       ` Jeff King
2018-02-15 10:34         ` Оля Тележная
2018-02-15 18:31           ` Junio C Hamano
2018-02-12  8:08     ` [PATCH v3 12/23] cat-file: start reusing populate_value() Olga Telezhnaya
2018-02-15  5:42       ` Jeff King
2018-02-12  8:08     ` [PATCH v3 04/23] ref-filter: make valid_atom as function parameter Olga Telezhnaya
2018-02-15  5:23       ` Jeff King
2018-02-15 10:03         ` Оля Тележная
2018-02-12  8:08     ` [PATCH v3 16/23] ref-filter: make cat_file_info independent Olga Telezhnaya
2018-02-15  5:53       ` Jeff King
2018-02-15 10:33         ` Оля Тележная
2018-02-16 21:54           ` Jeff King
2018-02-12  8:08     ` [PATCH v3 22/23] cat-file: tests for new atoms added Olga Telezhnaya
2018-02-16 14:55       ` Adam Dinwoodie
2018-02-18 22:55         ` Ramsay Jones [this message]
2018-02-19  1:04           ` Ramsay Jones
2018-02-12  8:08     ` [PATCH v3 10/23] ref-filter: make populate_value() global Olga Telezhnaya
2018-02-12  8:08     ` [PATCH v3 07/23] cat-file: start migrating formatting to ref-filter Olga Telezhnaya
2018-02-15  5:32       ` Jeff King
2018-02-12  8:08     ` [PATCH v3 15/23] cat-file: move skip_object_info into ref-filter Olga Telezhnaya
2018-02-15  5:51       ` Jeff King
2018-02-15 10:27         ` Оля Тележная
2018-02-12  8:08     ` [PATCH v3 14/23] ref_filter: add is_atom_used function Olga Telezhnaya
2018-02-15  5:49       ` Jeff King
2018-02-15 10:20         ` Оля Тележная
2018-02-12  8:08     ` [PATCH v3 18/23] cat-file: reuse printing logic from ref-filter Olga Telezhnaya
2018-02-12  8:08     ` [PATCH v3 17/23] ref-filter: make valid_atom general again Olga Telezhnaya
2018-02-15  5:53       ` Jeff King
2018-02-12  8:08     ` [PATCH v3 21/23] for-each-ref: tests for new atoms added Olga Telezhnaya
2018-02-15  5:57       ` Jeff King
2018-02-15 10:38         ` Оля Тележная
2018-02-12  8:08     ` [PATCH v3 03/23] cat-file: reuse struct ref_format Olga Telezhnaya
2018-02-15  5:18       ` Jeff King
2018-02-12  8:08     ` [PATCH v3 09/23] cat-file: start use ref_array_item struct Olga Telezhnaya
2018-02-15  5:40       ` Jeff King
2018-02-15 10:13         ` Оля Тележная
2018-02-12  8:08     ` [PATCH v3 23/23] cat-file: update of docs Olga Telezhnaya
2018-02-15  6:00       ` Jeff King
2018-02-12  8:08     ` [PATCH v3 13/23] ref-filter: get rid of mark_atom_in_object_info() Olga Telezhnaya
2018-02-15  5:45       ` Jeff King
2018-02-15 10:19         ` Оля Тележная
2018-02-12  8:08     ` [PATCH v3 08/23] ref-filter: reuse parse_ref_filter_atom() Olga Telezhnaya
2018-02-15  5:37       ` Jeff King
2018-02-15 10:11         ` Оля Тележная
2018-02-12  8:08     ` [PATCH v3 06/23] cat-file: split expand_atom() into 2 functions Olga Telezhnaya
2018-02-15  5:28       ` Jeff King
2018-02-12  8:08     ` [PATCH v3 19/23] ref-filter: make populate_value() internal again Olga Telezhnaya
2018-02-12  8:08     ` [PATCH v3 02/23] ref-filter: add return value to some functions Olga Telezhnaya
2018-02-15  5:16       ` Jeff King
2018-02-15  9:59         ` Оля Тележная
2018-02-15  5:08     ` [PATCH v3 01/23] ref-filter: get rid of goto Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=58b2bdcd-d621-fd21-ab4d-6a9478319b19@ramsayjones.plus.com \
    --to=ramsay@ramsayjones.plus.com \
    --cc=christian.couder@gmail.com \
    --cc=git@dinwoodie.org \
    --cc=git@vger.kernel.org \
    --cc=olyatelezhnaya@gmail.com \
    --cc=peff@peff.net \
    --subject='Re: [PATCH v3 22/23] cat-file: tests for new atoms added' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 git git/ https://public-inbox.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/inbox.comp.version-control.git
	nntp://ie5yzdi7fg72h7s4sdcztq5evakq23rdt33mfyfcddc5u3ndnw24ogqd.onion/inbox.comp.version-control.git
	nntp://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/inbox.comp.version-control.git
	nntp://news.gmane.io/gmane.comp.version-control.git
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git