Re: [RFC \ WISH] Add -o option to git-rev-list

[Linus Torvalds <torvalds@osdl.org>]

On Sun, 10 Dec 2006, Linus Torvalds wrote:
> 
> So don't worry too much about low-memory situations.

Btw, I should obviously clarify that. You should _always_ worry about 
low-memory situations, but the only real issue is really "working set 
size", not "data in files or in memory".

If the file representation is very dense (like the git pack-files are, for 
example), then it may well make sense to keep the data in a file, just 
because it's smaller there than if you keep it in expanded form in memory.

Also, it's nice to keep stuff in the filesystem rather than in process VM 
memory, because filesystem data that is cached in memory is useful for 
_other_ processes, ie it has a lifetime that is longer than the process 
itself. However, that's obviously only true for long-lived files that are 
shared among processes, it's _not_ true for temporary files.

For temporary files, the memory footprint of a temp-file is usually 
_larger_ than the memory footprint of the same data kept in memory. You 
have things like page-cache alignment, and often the issue of marshalling 
data into ASCII etc. So temp-files are almost never a better solution than 
keeping things in memory (unless you use those temp-files to truly 
_share_ data between processes, ie you do a shared mmap and they can 
re-use the same pages actively in a way they couldn't otherwise).