git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Jeff Hostetler <git@jeffhostetler.com>
To: git@vger.kernel.org
Cc: gitster@pobox.com, peff@peff.net, jonathantanmy@google.com,
	Jeff Hostetler <jeffhost@microsoft.com>
Subject: [PATCH v2 0/6] Partial clone part 1: object filtering
Date: Thu,  2 Nov 2017 17:50:07 +0000	[thread overview]
Message-ID: <20171102175013.3371-1-git@jeffhostetler.com> (raw)

From: Jeff Hostetler <jeffhost@microsoft.com>

Here is V2 of the list-object filtering. It replaces [1]
and reflect a refactoring and simplification of the original.

After much discussion on the "list-object-filter-map" I've replaced
it with a regular oidset -- the only need for the map was to store
the first observed pathname for each blob, but that itself was of
questionable value.

I've extended oidmap and oidset to have iterators.  These 2 commits
could be pulled out and applied on their own, but for now I need
them here.

There were also several comments on the layout of the filtering
API and the layout of the filter source code.  I've restructured
the filtering routines to put them in the same source file, and
made them all static.  These are now hidden behind a "factory-like"
function with a vtable.  This greatly simplifies the code in
traverse_commit_list_filtered().

I've added "--filter-ignore-missing" parameter to rev-list and
pack-objects to ignore missing objects rather than error out.
This allows this patch series to better stand on its own eliminates
the need in part 1 for "patch 9" from V1.

This is a brute force ignore all missing objects.  Later, in part
2 or part 3 when --exclude-promisor-objects is introduced, we will
be able to ignore EXPECTED missing objects.

Finally, patch 1 in this series is the same [2] which is currently
cooking in next.

[1] https://public-inbox.org/git/20171024185332.57261-1-git@jeffhostetler.com/

[2] * jh/dir-add-exclude-from-blob (2017-10-27) 1 commit
    - dir: allow exclusions from blob in addition to file


Jeff Hostetler (6):
  dir: allow exclusions from blob in addition to file
  oidmap: add oidmap iterator methods
  oidset: add iterator methods to oidset
  list-objects: filter objects in traverse_commit_list
  rev-list: add list-objects filtering support
  pack-objects: add list-objects filtering

 Documentation/git-pack-objects.txt     |  12 +-
 Documentation/git-rev-list.txt         |   6 +-
 Documentation/rev-list-options.txt     |  34 +++
 Makefile                               |   2 +
 builtin/pack-objects.c                 |  28 ++-
 builtin/rev-list.c                     |  75 +++++-
 dir.c                                  | 132 ++++++++---
 dir.h                                  |   3 +
 list-objects-filter-options.c          | 119 ++++++++++
 list-objects-filter-options.h          |  55 +++++
 list-objects-filter.c                  | 408 +++++++++++++++++++++++++++++++++
 list-objects-filter.h                  |  84 +++++++
 list-objects.c                         |  95 ++++++--
 list-objects.h                         |   2 +-
 oidmap.h                               |  22 ++
 oidset.c                               |  10 +
 oidset.h                               |  36 +++
 t/t5317-pack-objects-filter-objects.sh | 369 +++++++++++++++++++++++++++++
 t/t6112-rev-list-filters-objects.sh    | 225 ++++++++++++++++++
 19 files changed, 1664 insertions(+), 53 deletions(-)
 create mode 100644 list-objects-filter-options.c
 create mode 100644 list-objects-filter-options.h
 create mode 100644 list-objects-filter.c
 create mode 100644 list-objects-filter.h
 create mode 100755 t/t5317-pack-objects-filter-objects.sh
 create mode 100755 t/t6112-rev-list-filters-objects.sh

-- 
2.9.3


             reply	other threads:[~2017-11-02 17:50 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-11-02 17:50 Jeff Hostetler [this message]
2017-11-02 17:50 ` [PATCH v2 1/6] dir: allow exclusions from blob in addition to file Jeff Hostetler
2017-11-02 17:50 ` [PATCH v2 2/6] oidmap: add oidmap iterator methods Jeff Hostetler
2017-11-02 17:50 ` [PATCH v2 3/6] oidset: add iterator methods to oidset Jeff Hostetler
2017-11-02 17:50 ` [PATCH v2 4/6] list-objects: filter objects in traverse_commit_list Jeff Hostetler
2017-11-02 19:32   ` Jonathan Tan
2017-11-03 11:54     ` Johannes Schindelin
2017-11-03 13:37       ` Jeff Hostetler
2017-11-07 18:54     ` Jeff Hostetler
2017-11-06 17:51   ` Jeff Hostetler
2017-11-06 18:08     ` Jonathan Tan
2017-11-02 17:50 ` [PATCH v2 5/6] rev-list: add list-objects filtering support Jeff Hostetler
2017-11-02 17:50 ` [PATCH v2 6/6] pack-objects: add list-objects filtering Jeff Hostetler
2017-11-02 19:44 ` [PATCH v2 0/6] Partial clone part 1: object filtering Jonathan Tan
2017-11-03 13:43   ` Jeff Hostetler
2017-11-03 15:05     ` Junio C Hamano
2017-11-03 18:34       ` Jeff Hostetler
2017-11-08  0:41         ` Jonathan Tan
2017-11-08  0:54           ` Junio C Hamano
2017-11-08 14:39             ` Jeff Hostetler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20171102175013.3371-1-git@jeffhostetler.com \
    --to=git@jeffhostetler.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jeffhost@microsoft.com \
    --cc=jonathantanmy@google.com \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).