Split commit graphs and commit-graph read

Split commit graphs and commit-graph read

[Bryan Turner]

Just a quick question about a behavior I've noticed with the commit
graph. (Amazing feature, by the way!)

If the _very first_ write done is split:
git commit-graph write --reachable --split

You end up with something like this:
.../objects$ ls -R info
info:
commit-graphs  packs

info/commit-graphs:
commit-graph-chain  graph-6612fcc8fd04d3af2cc268a6bd9161ae40f5fcbf.graph

info/commit-graph doesn't exist, but I have a 1-graph "chain" in
place. (And subsequent write --split calls write additional ones; I've
got a few now in this repository, but still no info/commit-graph.)

git commit-graph verify seems happy:
.../objects$ git commit-graph verify
Verifying commits in commit graph: 100% (98768/98768), done.

But git commit-graph read isn't:
.../objects$ git commit-graph read
fatal: Could not open commit-graph
'/path/to/repository/objects/info/commit-graph': No such file or
directory

Running some tests with commands like git for-each-ref and git
rev-list shows that the "split" commit graph is being used (setting
core.commitGraph=false makes commands noticeably slower), so
functionally all seems well. But should git commit-graph read be
handling this better?

Best regards,
Bryan Turner

Re: Split commit graphs and commit-graph read

[Derrick Stolee]

On 11/8/2019 6:41 PM, Bryan Turner wrote:

Hi Bryan,

> Just a quick question about a behavior I've noticed with the commit
> graph. (Amazing feature, by the way!)
 
> If the _very first_ write done is split:
> git commit-graph write --reachable --split
> 
> You end up with something like this:
> .../objects$ ls -R info
> info:
> commit-graphs  packs
> 
> info/commit-graphs:
> commit-graph-chain  graph-6612fcc8fd04d3af2cc268a6bd9161ae40f5fcbf.graph
> 
> info/commit-graph doesn't exist, but I have a 1-graph "chain" in
> place. (And subsequent write --split calls write additional ones; I've
> got a few now in this repository, but still no info/commit-graph.)
> 
> git commit-graph verify seems happy:
> .../objects$ git commit-graph verify
> Verifying commits in commit graph: 100% (98768/98768), done.

This workflow seems expected.

> But git commit-graph read isn't:
> .../objects$ git commit-graph read
> fatal: Could not open commit-graph
> '/path/to/repository/objects/info/commit-graph': No such file or
> directory
> 
> Running some tests with commands like git for-each-ref and git
> rev-list shows that the "split" commit graph is being used (setting
> core.commitGraph=false makes commands noticeably slower), so
> functionally all seems well. But should git commit-graph read be
> handling this better?

Unfortunately, you're running into an issue because I designed the
"read" subcommand poorly (and also forgot to update it for
incremental commit-graph files). The biggest issue is that "read"
is not really meant for end-users. It really should have been built
as a test-tool. This point was corrected when I got around to writing
the multi-pack-index since it uses "test-tool read-midx" instead of
add.

To fix this issue, I would probably go about it by removing the "read"
subcommand and creating a "test-tool read-commit-graph" for the tests
that need that output.

If others on-list think that the better thing to do is to update the
"read" subcommand to provide the same output, but iterate over each
layer of an incremental commit-graph, then I can do that work instead.

Thanks,
-Stolee

Re: Split commit graphs and commit-graph read

[Junio C Hamano]

Derrick Stolee <stolee@gmail.com> writes:

> To fix this issue, I would probably go about it by removing the "read"
> subcommand and creating a "test-tool read-commit-graph" for the tests
> that need that output.
>
> If others on-list think that the better thing to do is to update the
> "read" subcommand to provide the same output, but iterate over each
> layer of an incremental commit-graph, then I can do that work instead.

I think moving it to test-tool is a good idea, whether the output
presents a single result that consolidates all layers together, or
allows to name just a single layer and shows its contents.  I do not
have a strong opinion between the two (both sounds usable debugging
aid).

Thanks.

Re: Split commit graphs and commit-graph read

[Jeff King]

On Sun, Nov 10, 2019 at 08:19:20PM -0500, Derrick Stolee wrote:

> > Running some tests with commands like git for-each-ref and git
> > rev-list shows that the "split" commit graph is being used (setting
> > core.commitGraph=false makes commands noticeably slower), so
> > functionally all seems well. But should git commit-graph read be
> > handling this better?
> 
> Unfortunately, you're running into an issue because I designed the
> "read" subcommand poorly (and also forgot to update it for
> incremental commit-graph files). The biggest issue is that "read"
> is not really meant for end-users. It really should have been built
> as a test-tool. This point was corrected when I got around to writing
> the multi-pack-index since it uses "test-tool read-midx" instead of
> add.
> 
> To fix this issue, I would probably go about it by removing the "read"
> subcommand and creating a "test-tool read-commit-graph" for the tests
> that need that output.
> 
> If others on-list think that the better thing to do is to update the
> "read" subcommand to provide the same output, but iterate over each
> layer of an incremental commit-graph, then I can do that work instead.

In theory I suppose one could use it to debug a commit-graph file "in
the field" as it were, where somebody does not necessarily have the
test-tool programs. But in practice, I have not ever done that (I didn't
even know "commit-graph read" was there), and it's not that big a deal
to just have a build of git.git handy.

I'd be much more likely to use "commit-graph verify". And perhaps it
could grow a "--verbose" flag if somebody really wants that (but I think
it would be fine to punt until somebody cares enough to do so).

I guess dropping the sub-command is technically a backwards incompatible
change, but since it didn't do anything that normal users would find
useful in the first place, I wouldn't be sorry to see it go.

-Peff

Re: Split commit graphs and commit-graph read

[Bryan Turner]

On Sun, Nov 10, 2019 at 5:19 PM Derrick Stolee <stolee@gmail.com> wrote:
>
> > But git commit-graph read isn't:
> > .../objects$ git commit-graph read
> > fatal: Could not open commit-graph
> > '/path/to/repository/objects/info/commit-graph': No such file or
> > directory
> >
> > Running some tests with commands like git for-each-ref and git
> > rev-list shows that the "split" commit graph is being used (setting
> > core.commitGraph=false makes commands noticeably slower), so
> > functionally all seems well. But should git commit-graph read be
> > handling this better?
>
> Unfortunately, you're running into an issue because I designed the
> "read" subcommand poorly (and also forgot to update it for
> incremental commit-graph files). The biggest issue is that "read"
> is not really meant for end-users. It really should have been built
> as a test-tool. This point was corrected when I got around to writing
> the multi-pack-index since it uses "test-tool read-midx" instead of
> add.
>
> To fix this issue, I would probably go about it by removing the "read"
> subcommand and creating a "test-tool read-commit-graph" for the tests
> that need that output.
>
> If others on-list think that the better thing to do is to update the
> "read" subcommand to provide the same output, but iterate over each
> layer of an incremental commit-graph, then I can do that work instead.

I was only running "git commit-graph read" as another way of verifying
that everything looked good (to Git, that is). I'd already run "git
commit-graph verify" and just found the discrepancy in their behavior
surprising/interesting. For what it's worth, I personally don't see
anything amiss with removing the "read" subcommand if that's simpler
than updating it to handle incremental graphs. (Put differently, it's
not a command I'd ever have Bitbucket Server run, or that I'm likely
to try and run going forward. In this case I just completed some
commit graph-related changes to Bitbucket Server and was looking for
every possible avenue I could pursue to try and ensure what I did was
producing the "correct" output after my changes got deployed to our
internal dogfooding server.)

Thanks for taking the time to look into it and clarify the behavior! I
really appreciate it.

Best regards,
Bryan Turner

>
> Thanks,
> -Stolee