git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Christian Couder <christian.couder@gmail.com>
To: Junio C Hamano <gitster@pobox.com>
Cc: git@vger.kernel.org, John Cai <johncai86@gmail.com>,
	Jonathan Tan <jonathantanmy@google.com>,
	Jonathan Nieder <jrnieder@gmail.com>,
	Taylor Blau <me@ttaylorr.com>, Derrick Stolee <stolee@gmail.com>
Subject: Re: [PATCH v3 0/2] Implement filtering repacks
Date: Wed, 21 Dec 2022 04:53:50 +0100	[thread overview]
Message-ID: <CAP8UFD2_ZbrOh4Tto2V80wn4ki+3j6Giegx+QyrLqd7uH6U4GQ@mail.gmail.com> (raw)
In-Reply-To: <xmqqa64ipl4r.fsf@gitster.g>

On Wed, Nov 23, 2022 at 1:31 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> Christian Couder <christian.couder@gmail.com> writes:
>
> > In the discussions with Junio and Taylor following the first and
> > second versions, Junio suggested adding `--filter=<filter-spec>` to
> > `git gc` and I am still Ok with doing it, either later in a followup
> > patch or in a v4. I haven't done it yet, as it's not clear how to
> > implement it efficiently only in `git gc`.
>
> Did I make such a suggestion?  I only said --filter=<filter-spec>
> does not seem like a good option to surface and stress at the
> end-user level for "git pack-objects".

In https://lore.kernel.org/git/xmqqilkm9wv6.fsf@gitster.g/ you wrote:

-> Unlike prune-packed, pruning objects that could be refetched has
-> negative performance impact.  So adding an option to enable/disable
-> such pruning is needed.  If the frontmost UI entry point were "gc",
-> it needs an opt-in option to invoke the "filtering repack", in other
-> words.  "pack-objects" should not need any more work than what you
-> have here (with the "terminal" and "force" discarded), as "--filter"
-> is such an opt-in option already.

I understood the above to mean that you would be Ok with adding
a "--filter" flag to `git gc`.

> The similarity of "git prune-packed" with this new operation was
> brought up to illustrate the point that users of "git gc" do not
> want to even know about having to remove redundantly available
> objects via "prune-packed" by giving an extra option to "git gc"
> (hence --prune is the default), and honoring the filter spec when
> objects are known to be available via the configured promisor remote
> is likely what users at the "git gc" level would want to see happen
> automatically.  IOW, adding more options to "gc" would be the last
> thing I would want to see.

I still think that users should be able to control if `git gc` performs
such a filtering when repacking, and I don't see how it could be done
without any new option at all. I think that for example only checking
the 'remote.<name>.partialclonefilter' config variable to decide if it
should be done or not is just too dangerous for people already using
this variable.

In the v4 I will send really soon now, I tentatively implemented a
'gc.repackFilter' config option for the purpose of telling `git gc` that it
should perform filtering repacks with the specified filter. I hope that
this will help move the discussion forward.

BTW in the latest "What's cooking in git.git" emails there is the
following about this patch series:

-> Seems to break CI.
-> cf. https://github.com/git/git/actions/runs/3560918726

I took a look at the failures in the different failing tests and they
don't seem related to this patch
series and seem quite random:

=== Failed test: t3106-ls-tree-pattern ===
The full logs are in the 'print test failures' step below.
See also the 'failed-tests-*' artifacts attached to this run.
Error: failed: t3106.3 combine with "--object-only"
failure: t3106.3 combine with "--object-only"
Error: failed: t3106.5 combine with "--long"
failure: t3106.5 combine with "--long"

=== Failed test: t5601-clone ===
The full logs are in the 'print test failures' step below.
See also the 'failed-tests-*' artifacts attached to this run.
skip: t5601.60 clone c:temp is dos drive
skip: t5601.101 colliding file detection
Error: failed: t5601.110 auto-discover bundle URI from HTTP clone
failure: t5601.110 auto-discover bundle URI from HTTP clone
Error: failed: t5601.111 auto-discover multiple bundles from HTTP clone
failure: t5601.111 auto-discover multiple bundles from HTTP clone

=== Failed test: t5556-http-auth ===
The full logs are in the 'print test failures' step below.
See also the 'failed-tests-*' artifacts attached to this run.
Error: failed: t5556.2 http auth anonymous no challenge
failure: t5556.2 http auth anonymous no challenge
Error: failed: t5556.3 http auth www-auth headers to credential helper
bearer valid
failure: t5556.3 http auth www-auth headers to credential helper bearer valid

=== Failed test: t4203-mailmap ===
The full logs are in the 'print test failures' step below.
See also the 'failed-tests-*' artifacts attached to this run.
Error: failed: t4203.66 git cat-file -s returns correct size with --use-mailmap
failure: t4203.66 git cat-file -s returns correct size with --use-mailmap
Error: failed: t4203.67 git cat-file -s returns correct size with
--use-mailmap for tag objects
failure: t4203.67 git cat-file -s returns correct size with
--use-mailmap for tag objects

etc

I tried different build flags but can't reproduce locally.

  reply	other threads:[~2022-12-21  3:54 UTC|newest]

Thread overview: 37+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-10-12 13:51 [PATCH 0/3] Implement filtering repacks Christian Couder
2022-10-12 13:51 ` [PATCH 1/3] pack-objects: allow --filter without --stdout Christian Couder
2022-10-12 13:51 ` [PATCH 2/3] repack: add --filter=<filter-spec> option Christian Couder
2022-10-12 13:51 ` [PATCH 3/3] repack: introduce --force to force filtering Christian Couder
2022-10-14 16:46 ` [PATCH 0/3] Implement filtering repacks Junio C Hamano
2022-10-20 11:23   ` Christian Couder
2022-10-28 19:49     ` Taylor Blau
2022-10-28 20:26       ` Junio C Hamano
2022-11-07  9:12         ` Christian Couder
2022-11-07  9:00       ` Christian Couder
2022-10-25 12:28 ` [PATCH v2 0/2] " Christian Couder
2022-10-25 12:28   ` [PATCH v2 1/2] pack-objects: allow --filter without --stdout Christian Couder
2022-10-25 12:28   ` [PATCH v2 2/2] repack: add --filter=<filter-spec> option Christian Couder
2022-10-28 19:54   ` [PATCH v2 0/2] Implement filtering repacks Taylor Blau
2022-11-07  9:29     ` Christian Couder
2022-11-22 17:51   ` [PATCH v3 " Christian Couder
2022-11-22 17:51     ` [PATCH v3 1/2] pack-objects: allow --filter without --stdout Christian Couder
2022-11-22 17:51     ` [PATCH v3 2/2] repack: add --filter=<filter-spec> option Christian Couder
2022-11-23  0:31     ` [PATCH v3 0/2] Implement filtering repacks Junio C Hamano
2022-12-21  3:53       ` Christian Couder [this message]
2022-11-23  0:35     ` Junio C Hamano
2022-12-21  4:04     ` [PATCH v4 0/3] " Christian Couder
2022-12-21  4:04       ` [PATCH v4 1/3] pack-objects: allow --filter without --stdout Christian Couder
2023-01-04 14:56         ` Patrick Steinhardt
2022-12-21  4:04       ` [PATCH v4 2/3] repack: add --filter=<filter-spec> option Christian Couder
2023-01-04 14:56         ` Patrick Steinhardt
2023-01-05  1:39           ` Junio C Hamano
2022-12-21  4:04       ` [PATCH v4 3/3] gc: add gc.repackFilter config option Christian Couder
2023-01-04 14:57         ` Patrick Steinhardt
2024-05-15 13:25 ` [PATCH v2 0/3] upload-pack: support a missing-action Christian Couder
2024-05-15 13:25   ` [PATCH v2 1/3] rev-list: refactor --missing=<missing-action> Christian Couder
2024-05-15 16:16     ` Junio C Hamano
2024-05-15 13:25   ` [PATCH v2 2/3] pack-objects: use the missing action API Christian Couder
2024-05-15 16:46     ` Junio C Hamano
2024-05-15 13:25   ` [PATCH v2 3/3] upload-pack: allow configuring a missing-action Christian Couder
2024-05-15 17:08     ` Junio C Hamano
2024-05-15 13:59   ` [PATCH v2 0/3] upload-pack: support " Christian Couder

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAP8UFD2_ZbrOh4Tto2V80wn4ki+3j6Giegx+QyrLqd7uH6U4GQ@mail.gmail.com \
    --to=christian.couder@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=johncai86@gmail.com \
    --cc=jonathantanmy@google.com \
    --cc=jrnieder@gmail.com \
    --cc=me@ttaylorr.com \
    --cc=stolee@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).