* git gc ineffective @ 2020-11-09 9:20 Michal Suchánek 2020-11-09 9:49 ` Andreas Schwab 0 siblings, 1 reply; 8+ messages in thread From: Michal Suchánek @ 2020-11-09 9:20 UTC (permalink / raw) To: Git List Hello, I am running git 2.29.1 I noticed I am running out of disk space, and one repository taking up about 38G. Did git gc --aggressive, and the used space *raised* to 42G, and git would report it does gc after every commit. Did gc without --aggressive, and the used space almost *doubled* to 75G (about 1G is checkout and some utracked files). The expected size is about 20G. Is there some way to debug gc to figure out what went wrong? Thanks Michal ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: git gc ineffective 2020-11-09 9:20 git gc ineffective Michal Suchánek @ 2020-11-09 9:49 ` Andreas Schwab 2020-11-09 10:17 ` Michal Suchánek 0 siblings, 1 reply; 8+ messages in thread From: Andreas Schwab @ 2020-11-09 9:49 UTC (permalink / raw) To: Michal Suchánek; +Cc: Git List On Nov 09 2020, Michal Suchánek wrote: > I noticed I am running out of disk space, and one repository taking up > about 38G. Did git gc --aggressive, and the used space *raised* to 42G, > and git would report it does gc after every commit. Do you have a lot of loose objects? Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: git gc ineffective 2020-11-09 9:49 ` Andreas Schwab @ 2020-11-09 10:17 ` Michal Suchánek 2020-11-09 10:43 ` Michal Suchánek 0 siblings, 1 reply; 8+ messages in thread From: Michal Suchánek @ 2020-11-09 10:17 UTC (permalink / raw) To: Andreas Schwab; +Cc: Git List On Mon, Nov 09, 2020 at 10:49:21AM +0100, Andreas Schwab wrote: > On Nov 09 2020, Michal Suchánek wrote: > > > I noticed I am running out of disk space, and one repository taking up > > about 38G. Did git gc --aggressive, and the used space *raised* to 42G, > > and git would report it does gc after every commit. > > Do you have a lot of loose objects? { for i in .git/objects/?? ; do ls $i ; done ; } | wc -l 53392 > > Andreas. > > -- > Andreas Schwab, schwab@linux-m68k.org > GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 > "And now for something completely different." ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: git gc ineffective 2020-11-09 10:17 ` Michal Suchánek @ 2020-11-09 10:43 ` Michal Suchánek 2020-11-09 10:48 ` Michal Suchánek 2020-11-09 11:17 ` Andreas Schwab 0 siblings, 2 replies; 8+ messages in thread From: Michal Suchánek @ 2020-11-09 10:43 UTC (permalink / raw) To: Andreas Schwab; +Cc: Git List On Mon, Nov 09, 2020 at 11:17:38AM +0100, Michal Suchánek wrote: > On Mon, Nov 09, 2020 at 10:49:21AM +0100, Andreas Schwab wrote: > > On Nov 09 2020, Michal Suchánek wrote: > > > > > I noticed I am running out of disk space, and one repository taking up > > > about 38G. Did git gc --aggressive, and the used space *raised* to 42G, > > > and git would report it does gc after every commit. > > > > Do you have a lot of loose objects? > { for i in .git/objects/?? ; do ls $i ; done ; } | wc -l > 53392 And in the double-size repository it's doubled, too: { for i in .git/objects/?? ; do ls $i ; done ; } | wc -l 101167 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: git gc ineffective 2020-11-09 10:43 ` Michal Suchánek @ 2020-11-09 10:48 ` Michal Suchánek 2020-11-09 11:17 ` Andreas Schwab 1 sibling, 0 replies; 8+ messages in thread From: Michal Suchánek @ 2020-11-09 10:48 UTC (permalink / raw) To: Andreas Schwab; +Cc: Git List On Mon, Nov 09, 2020 at 11:43:27AM +0100, Michal Suchánek wrote: > On Mon, Nov 09, 2020 at 11:17:38AM +0100, Michal Suchánek wrote: > > On Mon, Nov 09, 2020 at 10:49:21AM +0100, Andreas Schwab wrote: > > > On Nov 09 2020, Michal Suchánek wrote: > > > > > > > I noticed I am running out of disk space, and one repository taking up > > > > about 38G. Did git gc --aggressive, and the used space *raised* to 42G, > > > > and git would report it does gc after every commit. > > > > > > Do you have a lot of loose objects? > > { for i in .git/objects/?? ; do ls $i ; done ; } | wc -l > > 53392 > And in the double-size repository it's doubled, too: > { for i in .git/objects/?? ; do ls $i ; done ; } | wc -l > 101167 And server-side bare repository has 16G with 30836 loose objects and git 2.26.2. Thanks Michal ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: git gc ineffective 2020-11-09 10:43 ` Michal Suchánek 2020-11-09 10:48 ` Michal Suchánek @ 2020-11-09 11:17 ` Andreas Schwab 2021-01-07 18:35 ` Michal Suchánek 1 sibling, 1 reply; 8+ messages in thread From: Andreas Schwab @ 2020-11-09 11:17 UTC (permalink / raw) To: Michal Suchánek; +Cc: Git List On Nov 09 2020, Michal Suchánek wrote: > On Mon, Nov 09, 2020 at 11:17:38AM +0100, Michal Suchánek wrote: >> On Mon, Nov 09, 2020 at 10:49:21AM +0100, Andreas Schwab wrote: >> > On Nov 09 2020, Michal Suchánek wrote: >> > >> > > I noticed I am running out of disk space, and one repository taking up >> > > about 38G. Did git gc --aggressive, and the used space *raised* to 42G, >> > > and git would report it does gc after every commit. >> > >> > Do you have a lot of loose objects? >> { for i in .git/objects/?? ; do ls $i ; done ; } | wc -l >> 53392 > And in the double-size repository it's doubled, too: > { for i in .git/objects/?? ; do ls $i ; done ; } | wc -l > 101167 git count-objects also shows the size. Does it help to prune them --exprire now? Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: git gc ineffective 2020-11-09 11:17 ` Andreas Schwab @ 2021-01-07 18:35 ` Michal Suchánek 2021-01-07 21:48 ` Johannes Sixt 0 siblings, 1 reply; 8+ messages in thread From: Michal Suchánek @ 2021-01-07 18:35 UTC (permalink / raw) To: Andreas Schwab; +Cc: Git List Hello, On Mon, Nov 09, 2020 at 12:17:57PM +0100, Andreas Schwab wrote: > On Nov 09 2020, Michal Suchánek wrote: > > > On Mon, Nov 09, 2020 at 11:17:38AM +0100, Michal Suchánek wrote: > >> On Mon, Nov 09, 2020 at 10:49:21AM +0100, Andreas Schwab wrote: > >> > On Nov 09 2020, Michal Suchánek wrote: > >> > > >> > > I noticed I am running out of disk space, and one repository taking up > >> > > about 38G. Did git gc --aggressive, and the used space *raised* to 42G, > >> > > and git would report it does gc after every commit. > >> > > >> > Do you have a lot of loose objects? > >> { for i in .git/objects/?? ; do ls $i ; done ; } | wc -l > >> 53392 > > And in the double-size repository it's doubled, too: > > { for i in .git/objects/?? ; do ls $i ; done ; } | wc -l > > 101167 > > git count-objects also shows the size. $ git count-objects 59853 objects, 43249880 kilobytes $ du -hs .git 48G .git $ git gc --aggressive Enumerating objects: 1825080, done. Counting objects: 100% (1825080/1825080), done. Delta compression using up to 4 threads Compressing objects: 100% (1803925/1803925), done. Writing objects: 100% (1825080/1825080), done. Total 1825080 (delta 1234005), reused 587969 (delta 0), pack-reused 0 Removing duplicate objects: 100% (256/256), done. Checking connectivity: 2003814, done. Expanding reachable commits in commit graph: 337512, done. $ du -hs .git 172G .git $ git count-objects 178734 objects, 175309572 kilobytes > Does it help to prune them --exprire now? $ git prune Checking connectivity: 1825478, done. $ du -hs .git 3.9G .git $ git --version git version 2.26.2 So it is my wrong expectation that 'gc' comand removes garbage. It creates it en masse. It just makes is in a way that the 'prune' command that really reoves garbage can now remove it. Thanks Michal ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: git gc ineffective 2021-01-07 18:35 ` Michal Suchánek @ 2021-01-07 21:48 ` Johannes Sixt 0 siblings, 0 replies; 8+ messages in thread From: Johannes Sixt @ 2021-01-07 21:48 UTC (permalink / raw) To: Michal Suchánek; +Cc: Git List, Andreas Schwab Am 07.01.21 um 19:35 schrieb Michal Suchánek: > Hello, > > On Mon, Nov 09, 2020 at 12:17:57PM +0100, Andreas Schwab wrote: >> On Nov 09 2020, Michal Suchánek wrote: >> >>> On Mon, Nov 09, 2020 at 11:17:38AM +0100, Michal Suchánek wrote: >>>> On Mon, Nov 09, 2020 at 10:49:21AM +0100, Andreas Schwab wrote: >>>>> On Nov 09 2020, Michal Suchánek wrote: >>>>> >>>>>> I noticed I am running out of disk space, and one repository taking up >>>>>> about 38G. Did git gc --aggressive, and the used space *raised* to 42G, >>>>>> and git would report it does gc after every commit. >>>>> >>>>> Do you have a lot of loose objects? >>>> { for i in .git/objects/?? ; do ls $i ; done ; } | wc -l >>>> 53392 >>> And in the double-size repository it's doubled, too: >>> { for i in .git/objects/?? ; do ls $i ; done ; } | wc -l >>> 101167 >> >> git count-objects also shows the size. > $ git count-objects > 59853 objects, 43249880 kilobytes > $ du -hs .git > 48G .git > $ git gc --aggressive > Enumerating objects: 1825080, done. > Counting objects: 100% (1825080/1825080), done. > Delta compression using up to 4 threads > Compressing objects: 100% (1803925/1803925), done. > Writing objects: 100% (1825080/1825080), done. > Total 1825080 (delta 1234005), reused 587969 (delta 0), pack-reused 0 > Removing duplicate objects: 100% (256/256), done. > Checking connectivity: 2003814, done. > Expanding reachable commits in commit graph: 337512, done. > $ du -hs .git > 172G .git > $ git count-objects > 178734 objects, 175309572 kilobytes > >> Does it help to prune them --exprire now? > > $ git prune > Checking connectivity: 1825478, done. > $ du -hs .git > 3.9G .git > $ git --version > git version 2.26.2 > > So it is my wrong expectation that 'gc' comand removes garbage. It > creates it en masse. > > It just makes is in a way that the 'prune' command that really reoves > garbage can now remove it. It's an unfortunate default behavior of `git gc`. Set gc.pruneExpire to 'now' to countermand it, but watch out for the caveats. See https://stackoverflow.com/questions/55414916/how-to-avoid-that-git-gc-generates-garbage-loose-objects for more details. -- Hannes ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-01-07 21:52 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-11-09 9:20 git gc ineffective Michal Suchánek 2020-11-09 9:49 ` Andreas Schwab 2020-11-09 10:17 ` Michal Suchánek 2020-11-09 10:43 ` Michal Suchánek 2020-11-09 10:48 ` Michal Suchánek 2020-11-09 11:17 ` Andreas Schwab 2021-01-07 18:35 ` Michal Suchánek 2021-01-07 21:48 ` Johannes Sixt
Code repositories for project(s) associated with this public inbox https://80x24.org/mirrors/git.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).