Does git load index file into memory?

Does git load index file into memory?

[Farhan Khan]

Hi all,

Does git load the entire index file into memory when it wants to
edit/view it? I ask because I wonder if this can become a problem with
the index file becomes arbitrarily large, like for the Linux kernel.

Thanks,
--
Farhan Khan
PGP Fingerprint: B28D 2726 E2BC A97E 3854 5ABE 9A9F 00BC D525 16EE

Re: Does git load index file into memory?

[Thomas Gummerer]

On 10/12, Farhan Khan wrote:
> Hi all,
> 
> Does git load the entire index file into memory when it wants to
> edit/view it? I ask because I wonder if this can become a problem with
> the index file becomes arbitrarily large, like for the Linux kernel.

Yes, currently git always loads the entire index file for any
operation on it.  This is not particularly a problem for projects like
the linux kernel, as the index file for it is relatively small, ~6MB
at the moment.

It is more of a problem for larger repositories, such as the windows
repository, which has and index file of ~300-400MB, iirc, where
loading the index has a significant cost.  There's some patch series
in progress to improve the performance, e.g. Ben Pearts series to load
the index in parallel [*1*].

For writing the index to disk again, the split index feature can help
improve performance.  See also 'man git-update-index' and
"core.splitIndex" in 'man git-config'.

[*1*]: https://public-inbox.org/git/20181010155938.20996-1-peartben@gmail.com/

> Thanks,
> --
> Farhan Khan
> PGP Fingerprint: B28D 2726 E2BC A97E 3854 5ABE 9A9F 00BC D525 16EE