git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
From: Derrick Stolee <stolee@gmail.com>
To: Damien Robert <damien.olivier.robert@gmail.com>,
	Jeff King <peff@peff.net>,
	git@vger.kernel.org, Junio C Hamano <gitster@pobox.com>,
	Derrick Stolee <dstolee@microsoft.com>,
	William Baker <William.Baker@microsoft.com>
Cc: Damien Robert <damien.olivier.robert+git@gmail.com>
Subject: Re: [PATCH v2 1/1] midx.c: fix an integer overflow
Date: Thu, 12 Mar 2020 14:28:11 -0400
Message-ID: <a538c497-a79d-43be-3d00-c7a619acc4e6@gmail.com> (raw)
In-Reply-To: <20200312173520.2401776-1-damien.olivier.robert+git@gmail.com>

On 3/12/2020 1:35 PM, Damien Robert wrote:
> When verifying a midx index with 0 objects, the
>     m->num_objects - 1
> overflows to 4294967295.
> 
> Fix this both by checking that the midx contains at least one oid,
> and also that we don't write any midx when there is no packfiles.
> 
> Signed-off-by: Damien Robert <damien.olivier.robert+git@gmail.com>
> ---
> Should I add a test? It is a bit troublesome to generate a zero object midx
> file since this patch prevents it from using 'midx write'...

I'm glad that your patch makes it impossible to generate a zero-object
multi-pack-index, and that makes a test hard to implement. I'm not sure
what history Git has for storing explicit binary content into the test
suite. There really is only one "empty" multi-pack-index, but it is
unfortunately still a bit big for a test case to write explicitly due
to the 256-word fanout table.

I _think_ the t/tXXXX directories are used for this kind of data storage,
so you could generate an empty multi-pack-index from an older version of
Git then store it there. Please wait for someone else on-list to say that
this is a good idea, though. It may not be worth the pain of a binary file
in the patch.

>  midx.c | 35 +++++++++++++++++++++++------------
>  1 file changed, 23 insertions(+), 12 deletions(-)
> 
> diff --git a/midx.c b/midx.c
> index 1527e464a7..2cece7f9ea 100644
> --- a/midx.c
> +++ b/midx.c
> @@ -923,6 +923,12 @@ static int write_midx_internal(const char *object_dir, struct multi_pack_index *
>  	cur_chunk = 0;
>  	num_chunks = large_offsets_needed ? 5 : 4;
>  
> +	if (packs.nr - dropped_packs == 0) {
> +		error(_("no pack files to index."));

nit: I would use "pack-files" here. Second best is "packfiles".

> +		result = 1;
> +		goto cleanup;
> +	}
> +
>  	written = write_midx_header(f, num_chunks, packs.nr - dropped_packs);
>  
>  	chunk_ids[cur_chunk] = MIDX_CHUNKID_PACKNAMES;
> @@ -1124,22 +1130,27 @@ int verify_midx_file(struct repository *r, const char *object_dir, unsigned flag
>  				    i, oid_fanout1, oid_fanout2, i + 1);
>  	}
>  
> -	if (flags & MIDX_PROGRESS)
> -		progress = start_sparse_progress(_("Verifying OID order in multi-pack-index"),
> -						 m->num_objects - 1);
> -	for (i = 0; i < m->num_objects - 1; i++) {
> -		struct object_id oid1, oid2;
> +	if (m->num_objects == 0)
> +		midx_report(_("Warning: the midx contains no oid."));

Should this "Warning: " be here? The other calls to midx_report() do not have such prefix.

It could be valuable to add "warning: %s\n" to the fprintf inside midx_report(), but that should be done as its own patch.

Also, it may be valuable to return from this block so you do not need to put the block below in a tabbed block, reducing the complexity of this patch.

> +	else
> +	{
> +		if (flags & MIDX_PROGRESS)
> +			progress = start_sparse_progress(_("Verifying OID order in multi-pack-index"),
> +							 m->num_objects - 1);
> +		for (i = 0; i < m->num_objects - 1; i++) {
> +			struct object_id oid1, oid2;
>  
> -		nth_midxed_object_oid(&oid1, m, i);
> -		nth_midxed_object_oid(&oid2, m, i + 1);
> +			nth_midxed_object_oid(&oid1, m, i);
> +			nth_midxed_object_oid(&oid2, m, i + 1);
>  
> -		if (oidcmp(&oid1, &oid2) >= 0)
> -			midx_report(_("oid lookup out of order: oid[%d] = %s >= %s = oid[%d]"),
> -				    i, oid_to_hex(&oid1), oid_to_hex(&oid2), i + 1);
> +			if (oidcmp(&oid1, &oid2) >= 0)
> +				midx_report(_("oid lookup out of order: oid[%d] = %s >= %s = oid[%d]"),
> +					    i, oid_to_hex(&oid1), oid_to_hex(&oid2), i + 1);
>  
> -		midx_display_sparse_progress(progress, i + 1);
> +			midx_display_sparse_progress(progress, i + 1);
> +		}
> +		stop_progress(&progress);
>  	}
> -	stop_progress(&progress);
>  
>  	/*
>  	 * Create an array mapping each object to its packfile id.  Sort it
> 

Thanks for digging into this!

-Stolee

  parent reply index

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-28 16:24 [PATCH " Damien Robert
2020-02-28 18:55 ` Jeff King
2020-02-28 20:39   ` Junio C Hamano
2020-02-29 17:15     ` Damien Robert
2020-02-29 15:38   ` Damien Robert
2020-03-12 17:35 ` [PATCH v2 " Damien Robert
2020-03-12 18:24   ` Damien Robert
2020-03-12 18:28   ` Derrick Stolee [this message]
2020-03-12 21:41     ` Damien Robert
2020-03-23 22:25   ` [PATCH v3 " Damien Robert
2020-03-24  6:01     ` Jeff King
2020-03-24 18:48       ` Junio C Hamano
2020-03-26 21:35   ` [PATCH v4 " Damien Robert
2020-03-26 23:27     ` Junio C Hamano
2020-03-28 22:23       ` Damien Robert
2020-03-28 23:51         ` Junio C Hamano
2020-03-28 22:18   ` [PATCH 1/1] midx.c: fix an integer underflow Damien Robert

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a538c497-a79d-43be-3d00-c7a619acc4e6@gmail.com \
    --to=stolee@gmail.com \
    --cc=William.Baker@microsoft.com \
    --cc=damien.olivier.robert+git@gmail.com \
    --cc=damien.olivier.robert@gmail.com \
    --cc=dstolee@microsoft.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror http://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.io/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git