git@vger.kernel.org mailing list mirror (one of many)
 help / Atom feed
From: Brandon Williams <bmwill@google.com>
To: git@vger.kernel.org
Cc: Johannes.Schindelin@gmx.de, gitster@pobox.com, peff@peff.net, sbeller@google.com, jrnieder@gmail.com, pclouds@gmail.com, Brandon Williams <bmwill@google.com>
Subject: [WIP/RFC 00/23] repository object
Date: Thu, 18 May 2017 16:21:11 -0700
Message-ID: <20170518232134.163059-1-bmwill@google.com> (raw)

When I first started working on the git project I found it very difficult to
understand parts of the code base because of the inherently global nature of
our code.  It also made working on submodules very difficult.  Since we can
only open up a single repository per process, you need to launch a child
process in order to process a submodule.  But you also need to be able to
communicate other stateful information to the children processes so that the
submodules know how best to format their output or match against a
pathspec...it ends up feeling like layering on hack after hack.  What I would
really like to do, is to have the ability to have a repository object so that I
can open a submodule in-process.

Before this becomes a reality for all commands, much of the library code would
need to be refactored in order to work purely on handles instead of global
state.  As it turned out, ls-files is a pretty simple command and doesn't have
*too* many dependencies.  The biggest thing that needed to be changed was
piping through an index into a couple library routines so that they don't
inherently rely on 'the_index'.  A few of these changes I've sent out and can
be found at 'origin/bw/pathspec-sans-the-index' and
'origin/bw/dir-c-stops-relying-on-the-index' which this series is based on.

Patches 1-16 are refactorings to prepare either library code or ls-files itself
to be ready to handle passing around an index struct.  Patches 17-22 introduce
a repository struct and change a couple of things about how submodule caches
work (getting submodule information from .gitmodules).  And Patch 23 converts
ls-files to use a repository struct. 

The most interesting part of the series is from 17-23.  And 1-16 could be taken
as is without the rest of the series.

This is still very much in a WIP state, though it does pass all tests.  What
I'm hoping for here is to get a discussion started about the feasibility of a
change like this and hopefully to get the ball rolling.  Is this a direction we
want to move in?  Is it worth the pain?

Thanks for taking the time to look at this and entertain my insane ideas :)

Brandon Williams (23):
  convert: convert get_cached_convert_stats_ascii to take an index
  convert: convert crlf_to_git to take an index
  convert: convert convert_to_git_filter_fd to take an index
  convert: convert convert_to_git to take an index
  convert: convert renormalize_buffer to take an index
  tree: convert read_tree to take an index parameter
  ls-files: convert overlay_tree_on_cache to take an index
  ls-files: convert write_eolinfo to take an index
  ls-files: convert show_killed_files to take an index
  ls-files: convert show_other_files to take an index
  ls-files: convert show_ru_info to take an index
  ls-files: convert ce_excluded to take an index
  ls-files: convert prune_cache to take an index
  ls-files: convert show_files to take an index
  ls-files: factor out debug info into a function
  ls-files: factor out tag calculation
  repo: introduce new repository object
  repo: add index_state to struct repo
  repo: add per repo config
  submodule-config: refactor to allow for multiple submodule_cache's
  repo: add repo_read_gitmodules
  submodule: add is_submodule_active
  ls-files: use repository object

 Makefile                               |   1 +
 apply.c                                |   2 +-
 builtin/blame.c                        |   2 +-
 builtin/commit.c                       |   3 +-
 builtin/ls-files.c                     | 348 ++++++++++++++++-----------------
 cache.h                                |   4 +-
 combine-diff.c                         |   2 +-
 config.c                               |   2 +-
 convert.c                              |  31 +--
 convert.h                              |  19 +-
 diff.c                                 |   6 +-
 dir.c                                  |   2 +-
 git.c                                  |   2 +-
 ll-merge.c                             |   2 +-
 merge-recursive.c                      |   4 +-
 repo.c                                 | 112 +++++++++++
 repo.h                                 |  22 +++
 sha1_file.c                            |   6 +-
 submodule-config.c                     |  40 +++-
 submodule-config.h                     |  10 +
 submodule.c                            |  51 +++++
 submodule.h                            |   2 +
 t/t3007-ls-files-recurse-submodules.sh |  39 ++++
 tree.c                                 |  28 ++-
 tree.h                                 |   3 +-
 25 files changed, 513 insertions(+), 230 deletions(-)
 create mode 100644 repo.c
 create mode 100644 repo.h

-- 
2.13.0.303.g4ebf302169-goog


             reply index

Thread overview: 41+ messages in thread (expand / mbox.gz / Atom feed / [top])
2017-05-18 23:21 Brandon Williams [this message]
2017-05-18 23:21 ` [WIP/RFC 01/23] convert: convert get_cached_convert_stats_ascii to take an index Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 02/23] convert: convert crlf_to_git " Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 03/23] convert: convert convert_to_git_filter_fd " Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 04/23] convert: convert convert_to_git " Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 05/23] convert: convert renormalize_buffer " Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 06/23] tree: convert read_tree to take an index parameter Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 07/23] ls-files: convert overlay_tree_on_cache to take an index Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 08/23] ls-files: convert write_eolinfo " Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 09/23] ls-files: convert show_killed_files " Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 10/23] ls-files: convert show_other_files " Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 11/23] ls-files: convert show_ru_info " Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 12/23] ls-files: convert ce_excluded " Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 13/23] ls-files: convert prune_cache " Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 14/23] ls-files: convert show_files " Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 15/23] ls-files: factor out debug info into a function Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 16/23] ls-files: factor out tag calculation Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 17/23] repo: introduce new repository object Brandon Williams
2017-05-20 21:25   ` Stefan Beller
2017-05-23 17:35     ` Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 18/23] repo: add index_state to struct repo Brandon Williams
2017-05-20 21:27   ` Stefan Beller
2017-05-18 23:21 ` [WIP/RFC 19/23] repo: add per repo config Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 20/23] submodule-config: refactor to allow for multiple submodule_cache's Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 21/23] repo: add repo_read_gitmodules Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 22/23] submodule: add is_submodule_active Brandon Williams
2017-05-18 23:21 ` [WIP/RFC 23/23] ls-files: use repository object Brandon Williams
2017-05-19 12:25 ` [WIP/RFC 00/23] " Jeff Hostetler
2017-05-19 18:28 ` Ben Peart
2017-05-23 17:29   ` Brandon Williams
2017-05-20 21:37 ` Stefan Beller
2017-05-22 13:03   ` Johannes Schindelin
2017-05-21  8:23 ` Jacob Keller
2017-05-21 16:28 ` brian m. carlson
2017-05-22 19:35 ` Jeff King
2017-05-23 17:26   ` Brandon Williams
2017-05-24  1:57     ` Junio C Hamano
2017-05-29 10:36   ` Duy Nguyen
2017-05-29 11:23     ` Ævar Arnfjörð Bjarmason
2017-05-29 11:31       ` Duy Nguyen
2017-05-30 17:12         ` Brandon Williams

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply to all the recipients using the --to, --cc,
  and --in-reply-to switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170518232134.163059-1-bmwill@google.com \
    --to=bmwill@google.com \
    --cc=Johannes.Schindelin@gmx.de \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jrnieder@gmail.com \
    --cc=pclouds@gmail.com \
    --cc=peff@peff.net \
    --cc=sbeller@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org mailing list mirror (one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/
       or Tor2web: https://www.tor2web.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox