git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / Atom feed
6b041f94eea02644ee851496ef216cc2369a2635 blob 10347 bytes (raw)

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
 
Git commit graph format
=======================

The Git commit graph stores a list of commit OIDs and some associated
metadata, including:

- The generation number of the commit. Commits with no parents have
  generation number 1; commits with parents have generation number
  one more than the maximum generation number of its parents. We
  reserve zero as special, and can be used to mark a generation
  number invalid or as "not computed".

- The root tree OID.

- The commit date.

- The parents of the commit, stored using positional references within
  the graph file.

These positional references are stored as unsigned 32-bit integers
corresponding to the array position within the list of commit OIDs. Due
to some special constants we use to track parents, we can store at most
(1 << 30) + (1 << 29) + (1 << 28) - 1 (around 1.8 billion) commits.

== Commit graph files have the following format:

In order to allow extensions that add extra data to the graph, we organize
the body into "chunks" and provide a binary lookup table at the beginning
of the body. The header includes certain values, such as number of chunks
and hash type.

All multi-byte numbers are in network byte order.

HEADER:

  4-byte signature:
      The signature is: {'C', 'G', 'P', 'H'}

  1-byte version number:
      Currently, the only valid version is 1.

  1-byte Hash Version (1 = SHA-1)
      We infer the hash length (H) from this value.

  1-byte number (C) of "chunks"

  1-byte number (B) of base commit-graphs
      We infer the length (H*B) of the Base Graphs chunk
      from this value.

CHUNK LOOKUP:

  (C + 1) * 12 bytes listing the table of contents for the chunks:
      First 4 bytes describe the chunk id. Value 0 is a terminating label.
      Other 8 bytes provide the byte-offset in current file for chunk to
      start. (Chunks are ordered contiguously in the file, so you can infer
      the length using the next chunk position if necessary.) Each chunk
      ID appears at most once.

  The remaining data in the body is described one chunk at a time, and
  these chunks may be given in any order. Chunks are required unless
  otherwise specified.

CHUNK DATA:

  OID Fanout (ID: {'O', 'I', 'D', 'F'}) (256 * 4 bytes)
      The ith entry, F[i], stores the number of OIDs with first
      byte at most i. Thus F[255] stores the total
      number of commits (N).

  OID Lookup (ID: {'O', 'I', 'D', 'L'}) (N * H bytes)
      The OIDs for all commits in the graph, sorted in ascending order.

  Commit Data (ID: {'C', 'D', 'A', 'T' }) (N * (H + 16) bytes)
    * The first H bytes are for the OID of the root tree.
    * The next 8 bytes are for the positions of the first two parents
      of the ith commit. Stores value 0x7000000 if no parent in that
      position. If there are more than two parents, the second value
      has its most-significant bit on and the other bits store an array
      position into the Extra Edge List chunk.
    * The next 8 bytes store the generation number of the commit and
      the commit time in seconds since EPOCH. The generation number
      uses the higher 30 bits of the first 4 bytes, while the commit
      time uses the 32 bits of the second 4 bytes, along with the lowest
      2 bits of the lowest byte, storing the 33rd and 34th bit of the
      commit time.

  Extra Edge List (ID: {'E', 'D', 'G', 'E'}) [Optional]
      This list of 4-byte values store the second through nth parents for
      all octopus merges. The second parent value in the commit data stores
      an array position within this list along with the most-significant bit
      on. Starting at that array position, iterate through this list of commit
      positions for the parents until reaching a value with the most-significant
      bit on. The other bits correspond to the position of the last parent.

  Base Graphs List (ID: {'B', 'A', 'S', 'E'}) [Optional]
      This list of H-byte hashes describe a set of B commit-graph files that
      form a commit-graph chain. The graph position for the ith commit in this
      file's OID Lookup chunk is equal to i plus the number of commits in all
      base graphs.  If B is non-zero, this chunk must exist.

  Modified Path Bloom Filter Index (ID: {'M', 'P', 'B', 'I'}) [Optional]
    2 + (N * 8) bytes

    * 1-byte number (K) of hashes per path

    * An array of 8 byte entries, one for each N commits stored in the
      commit-graph, with the ith entry associated to the ith commit in the
      OID Lookup chunk.
      Each entry should be interpreted as follows:

      - If all bits in the 8 bytes are set, then there is no modified path
	Bloom filter stored for this commit.

	Thou shalt not store a modified path Bloom filter for a commit that
	modifies a submodule (gitlink file)!

      - If the most significant bit of the first byte is set, then the
	remaining 63 bits represent the bit array of an "embedded" Bloom
	filter containing the set of paths that were modified between the
	associated commit and its first parent, or in case of a root commit
	between the associated commit and the empty tree.  All embedded
	modified path Bloom filters use the same hashing scheme that is
	used in the Modified Path Bloom Filters chunk, see below.

      - If the second most significant bit of the first byte is set, then
	the last four bytes form an array position into the Modified Path
	Bloom Filter Merge Index chunk.  The remaining bits in the first
	four bytes should be set to 0.  This can optionally be used to
	store Bloom filters for all parents of merge commits.
	[TODO: Only the last four bytes?  Shouldn't that be last 62 bits?!]

      - If the two most significant bits of the first byte are unset, then
	the entry represents an 8 byte offset pointing to a Bloom filter
	in the Modified Path Bloom Filters chunk, which contains the set
	of paths that were modified between the associated commit and its
	first parent, or in case of a root commit between the associated
	commit and the empty tree.  This offset is relative to the start
	of the Modified Path Bloom Filters chunk.  Multiple entries can
	point to the same offset.

  Modified Path Bloom Filter Merge Index (ID: {'M', 'P', 'B', 'M'}) [Optional]
      An array of 8 byte entries.
      If a merge commit's entry in the Modified Path Bloom Filter Index
      chunk stores an array position into this chunk, then the entry at
      that position is associated with that merge commit and its first
      parent, the next entry with that merge commit and its second parent,
      etc. for all parents of that merge commit.

      Each entry should be interpreted as follows, similar to the entries
      in the Modified Path Bloom Filter Index chunk:

      - If all bits in the 8 bytes are set, then there is no modified path
	Bloom filter stored for the associated merge commit and its
	corresponding parent.

      - If the MSB of the first byte is set, then the remaining 63 bits
	represent the bit array of an "embedded" Bloom filter containing
	the set of paths that were modified between the associated merge
	commit and its corresponding parent.  All embedded modified path
	Bloom filters use the same hashing scheme that is used in the
	Modified Path Bloom Filters chunk, see below.

      - If the most significant bit of the first byte is unset, then the
	entry represents an 8 byte offset pointing to a Bloom filter in
	the Modified Path Bloom Filters chunk, which contains the set of
	paths that were modified between the associated merge commit and
	its corresponding parent.  This offset is relative to the start of
	the Modified Path Bloom Filters chunk.  Multiple entries can point
	to the same offset.

      This chunk should not exist if the commit-graph file doesn't contain
      a Modified Path Bloom Filter Index chunk.  This chunk can be omitted
      if the Modified Path Bloom Filter Index doesn't contain any array
      indexes pointing into it.  This chunk can be omitted even when the
      commit-graph contains merge commits.

  Modified Path Bloom Filters (ID: {'M', 'P', 'B', 'F'}) [Optional]
      A number of consecuive modified path Bloom filters of varying sizes.
      Each Bloom filter consists of 4 + M bytes:

      - The first 4 bytes specify the number of m bits in the Bloom
	filter's bit array.

      - The next M bytes hold the Bloom filter's bit array of m bits.

      The bits in the array are indexed in network byte order, i.e. the
      array's 0th bit is the least significant bit of the last byte, and
      the (m-1)th bit is in the first byte.  If m is not a multiple of 8,
      then the unused leading bits should be set to 0.

      For each path (including leading directories) modified between a
      commit and its parent K bit positions should be set using the
      following hashing scheme:

	for i in [0, K)
	  bit_pos[i] = (hash1 + i * hash2 + (i * i * i - i) / 6) % m

      where hash1 and hash2 are the 32 bit MurmurHash3 hashes of the path
      with seeds 0xe83c5163 and 0x3b376b0c, respectively.  These bit
      positions should be calculated using 32 bit unsigned integer
      arithmetic.  The directory separator is a single '/'.  Directories
      should be added without a trailing '/'.

      The order of Bloom filters in this chunk is unspecified.  Multiple
      entries in the Modified Path Bloom Filter Index or Modified Path
      Bloom Filter Merge Index chunks can point to the same Bloom filter
      in this chunk.

      This chunk should not exist if the commit-graph doesn't contain a
      Modified Path Bloom Filter Index chunk.  This chunk can be omitted
      if neither the Modified Path Bloom Filter Index nor the Modified
      Path Bloom Filter Merge Index chunks contain any offsets pointing
      into it.

  Modified Path Bloom Filter Excludes (ID: {'M', 'P', 'B', 'X'}) [Optional]
      A number of consecutive null terminated strings in memcmp() order.
      Paths in these strings should not be added to any modified path
      Bloom filters.
      [TODO: Clarify!  Only literal matches, or should we support
      globbing/patterns as well?  Only full match, or leading directories
      as well?]

      This chunk should not exist if the commit-graph doesn't contain a
      Modified Path Bloom Filter Index chunk.

TRAILER:

	H-byte HASH-checksum of all of the above.
debug log:

solving 6b041f94ee ...
found 6b041f94ee in https://public-inbox.org/git/20200529085038.26008-16-szeder.dev@gmail.com/
found efd0f99d8b in https://public-inbox.org/git/20200529085038.26008-4-szeder.dev@gmail.com/
found a4f17441ae in https://80x24.org/mirrors/git.git
preparing index
index prepared:
100644 a4f17441aed30f14c036a4bed6a911c86cf31ce5	Documentation/technical/commit-graph-format.txt

applying [1/2] https://public-inbox.org/git/20200529085038.26008-4-szeder.dev@gmail.com/
diff --git a/Documentation/technical/commit-graph-format.txt b/Documentation/technical/commit-graph-format.txt
index a4f17441ae..efd0f99d8b 100644


applying [2/2] https://public-inbox.org/git/20200529085038.26008-16-szeder.dev@gmail.com/
diff --git a/Documentation/technical/commit-graph-format.txt b/Documentation/technical/commit-graph-format.txt
index efd0f99d8b..6b041f94ee 100644

Checking patch Documentation/technical/commit-graph-format.txt...
Applied patch Documentation/technical/commit-graph-format.txt cleanly.
Checking patch Documentation/technical/commit-graph-format.txt...
Applied patch Documentation/technical/commit-graph-format.txt cleanly.

index at:
100644 6b041f94eea02644ee851496ef216cc2369a2635	Documentation/technical/commit-graph-format.txt

git@vger.kernel.org list mirror (unofficial, one of many)

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 git git/ https://public-inbox.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.io/gmane.comp.version-control.git
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for the project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git