Re: Question relate to collaboration on git monorepo

[Elijah Newren <newren@gmail.com>]

On Wed, Sep 21, 2022 at 8:22 AM ZheNing Hu <adlternative@gmail.com> wrote:
>
> Emily Shaffer <emilyshaffer@google.com> 于2022年9月21日周三 02:53写道：
> >
> > On Tue, Sep 20, 2022 at 5:42 AM ZheNing Hu <adlternative@gmail.com> wrote:
> > >
> > > Hey, guys,
> > >
> > > If two users of git monorepo are working on different sub project
> > > /project1 and /project2 by partial-clone and sparse-checkout ,
> > > if user one push first, then user two want to push too, he must
> > > pull some blob which pushed by user one. I guess their repo size
> > > will gradually increase by other's project's objects, so is there any way
> > > to delete unnecessary blobs out of working project (sparse-checkout
> > > filterspec), or just git pull don't really fetch these unnecessary blobs?
> >
> > This is exactly what the combination of partial clone and sparse
> > checkout is for!
> >
> > Dev A is working on project1/, and excludes project2/ from her sparse
> > filter; she also cloned with `--filter=blob:none`.
> > Dev B is working on project2/, and excludes project1/ from his sparse
> > filter, and similarly  is using blob:none partial clone filter.
> >
> > Assuming everybody is contributing by direct push, and not using a
> > code review tool or something else which handles the push for them...
> > Dev A finishes first, and pushes.
> > Dev B needs to pull, like you say - but during that pull he doesn't
> > need to fetch the objects in project1, because they're excluded by the
> > combination of his partial clone filter and his sparse checkout
> > pattern. The pull needs to happen because there is a new commit which
> > Dev B's commit needs to treat as a parent, and so Dev B's client needs
> > to know the ID of that commit.
> >
>
> I don't agree here, it indeed fetches the blobs during git pull. So I
> do a little
> change in the previous test:
>
> (
>   cd m2
>   git cat-file --batch-check --batch-all-objects | grep blob | wc -l >
> blob_count1
> #  git push
> #  git -c pull.rebase=false pull --no-edit #no conflict
>   git fetch origin main
>   git cat-file --batch-check --batch-all-objects | grep blob | wc -l >
> blob_count2
>   git merge --no-edit origin/main
>   git cat-file --batch-check --batch-all-objects | grep blob | wc -l >
> blob_count3
>   printf "blob_count1=%s\n" $(cat blob_count1)
>   printf "blob_count2=%s\n" $(cat blob_count2)
>   printf "blob_count3=%s\n" $(cat blob_count3)
> )
>
> warning: This repository uses promisor remotes. Some objects may not be loaded.
> remote: Enumerating objects: 32, done.
> remote: Counting objects: 100% (32/32), done.
> remote: Compressing objects: 100% (20/20), done.
> remote: Total 30 (delta 0), reused 0 (delta 0), pack-reused 0
> Receiving objects: 100% (30/30), 2.61 KiB | 2.61 MiB/s, done.
> From /Users/adl/./mono-repo
>  * branch            main       -> FETCH_HEAD
>    a6a17f2..16a8585  main       -> origin/main
> warning: This repository uses promisor remotes. Some objects may not be loaded.
> Merge made by the 'ort' strategy.

Note: The merge completed successfully, and we see no evidence of
additional blobs being downloaded before this point.

> remote: Enumerating objects: 1, done.
> remote: Counting objects: 100% (1/1), done.
> remote: Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
> Receiving objects: 100% (1/1), 87 bytes | 87.00 KiB/s, done.

Here, we do have an object download, which occurred after the merge
completed, so there must be something happening after the merge which
needs the extra blob; if we keep reading...

>  project1/file1 | 10 ++++++++++
>  1 file changed, 10 insertions(+)

Ah, the 'helpful' diffstat.  It downloads blobs from a promisor remote
just so we can see what has changed, including in the area of the
project we don't care about.

(This is yet another reason it'd be nice to have a --restrict mode for
grep/diff/log/etc. for sparse-checkout uses, and an ability to make it
the default in some repo, so you could get just the diffstat within
the region of the project that you care about.  We're discussing such
an idea, but it isn't implemented yet.)

> warning: This repository uses promisor remotes. Some objects may not be loaded.
> blob_count1=11
> blob_count2=11
> blob_count3=12
>
> The result shows that blob count doesn't change in git fetch, but in git merge.

If you add --no-stat to your merge command (or set merge.stat to
false), the extra blob will not be downloaded.