[PATCH 0/17] Sliding window mmap for packfiles.

["Shawn O. Pearce" <spearce@spearce.org>]

This 17 patch series implements my much discussed, but never produced
(until now), 'mmap sliding window' for packfile data access.

The key idea behind this topic is to mmap large non-contiguous
segments of a packfile rather than the entire file.  If available
virtual memory is getting low we unmap the least recently used
packfile segment to free up address space for the currently needed
segment.

This series also permits accessing packfiles up to 4 GiB in size,
even on systems which permit only 2 GiB of virtual memory within
a single process (e.g. Windows and some older UNIXes).  Of course
4 GiB is still the upper limit on packfile size due to the current
format of the index file.

This series is 'pu' ready, but it may be too early to bring it
directly into 'next'.

Patch summary
-------------

  1 - Replace unpack_entry_gently with unpack_entry.
  2 - Introduce new config option for mmap limit.
  3 - Refactor packed_git to prepare for sliding mmap windows.
  4 - Use off_t for index and pack file lengths.
  5 - Create read_or_die utility routine.
  6 - Refactor how we open pack files to prepare for multiple windows.

    Most of the above changes are incremental refactorings to help
    get the code in a state where we can start to implement and make
    use of the struct pack_window concept this series introduces.

  7 - Replace use_packed_git with window cursors.
  8 - Loop over pack_windows when inflating/accessing data.
  9 - Document why header parsing won't exceed a window.
 10 - Unmap individual windows rather than entire files.
 11 - Fully activate the sliding window pack access.

    These commits actually implement the core of the mmap sliding
    window implementation and the necessary garbage collection to
    support unmapping the least recently used window.

 12 - Load core configuration in git-verify-pack.
 13 - Ensure core.packedGitWindowSize cannot be less than 2 pages.
 14 - Improve error message when packfile mmap fails.
 15 - Support unmapping windows on 'temporary' packfiles.

    The above sequence of commits are bug fixes on top of the initial
    commits.  I did not fold these back into the earlier commits
    as I felt the bug fix commit messages provided useful details.

 16 - Create pack_report() as a debugging aid.
 17 - Test suite for sliding window mmap implementation.

    These provide debugging and testing tools.