git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Linus Torvalds <torvalds@osdl.org>
To: Junio C Hamano <junkio@cox.net>
Cc: git@vger.kernel.org
Subject: Re: [PATCH 3/2] merge-trees script for Linus git
Date: Fri, 15 Apr 2005 23:26:17 -0700 (PDT)	[thread overview]
Message-ID: <Pine.LNX.4.58.0504152256520.7211@ppc970.osdl.org> (raw)
In-Reply-To: <Pine.LNX.4.58.0504152152580.7211@ppc970.osdl.org>



On Fri, 15 Apr 2005, Linus Torvalds wrote:
> 
> Actually, it turns out that I have a cunning plan.

Damn, my cunning plan is some good stuff. 

Or maybe it is _so_ cunning that I just confuse even myself. But it looks 
like it is actually working, and that it allows pretty much instantaenous 
merges.

The plan goes like this:

 - each "index" entry has two bits worth of "stage" state. stage 0 is the 
   normal one, and is the only one you'd see in any kind of normal use.

 - however, when you do "read-tree" with multiple trees, the "stage" 
   starts out at 0, but increments for each tree you read. And in 
   particular, the old "-m" flag (which used to be "merge with old state")  
   has a new meaning: it now means "start at stage 1" instead.

 - this means that you can do

	read-tree -m <tree1> <tree2> <tree3>

   and you will end up with an index with all of the <tree1> entries in 
   "stage1", all of the <tree2> entries in "stage2" and all of the <tree3>
   entries in "stage3".

 - furthermore, "read-tree" has this special-case logic that says: if you 
   see a file that matches in all respects in all three states, it 
   "collapses" back to "stage0".

 - write-tree refuses to write a nonsensical tree, so write-tree will 
   complain about unmerged entries if it sees a single entry that is not
   stage 0".

Ok, this all sounds like a collection of totally nonsensical rules, but
it's actually exactly what you want in order to do a fast merge. The 
differnt stages represent the "result tree" (stage 0, aka "merged"), the 
original tree (stage 1, aka "orig"), and the two trees you are trying to 
merge (stage 2 and 3 respectively).

In fact, the way "read-tree" works, it's entirely agnostic about how you
assign the stages, and you could really assign them any which way, and the
above is just a suggested way to do it (except since "write-tree" refuses
to write anything but stage0 entries, it makes sense to always consider
stage 0 to be the "full merge" state).

So what happens? Try it out. Select the original tree, and two trees to 
merge, and look how it works:

 - if a file exists in identical format in all three trees, it will 
   automatically collapse to "merged" state by the new read-tree.

 - a file that has _any_ difference what-so-ever in the three trees will 
   stay as separate entries in the index. It's up to "script policy" to 
   determine how to remove the non-0 stages, and insert a merged version. 
   But since the index is always sorted, they're easy to find: they'll be
   clustered together.

 - the index file saves and restores with all this information, so you can 
   merge things incrementally, but as long as it has entries in stages
   1/2/3 (ie "unmerged entries") you can't write the result.

So now the merge algorithm ends up being really simple:

 - you walk the index in order, and ignore all entries of stage 0, since 
   they've already been done.
 - if you find a "stage1", but no matching "stage2" or "stage3", you know 
   it's been removed from both trees (it only existed in the original 
   tree), and you remove that entry.
 - if you find a matching "stage2" and "stage3" tree, you remove one of 
   them, and turn the other into a "stage0" entry. Remove any matching
   "stage1" entry if it exists too.
  .. all the normal trivial rules ..

NOTE NOTE NOTE! I could make "read-tree" do some of these nontrivial 
merges, but I ended up deciding that only the "matches in all three 
states" thing collapses by default. Why? Because even though there are 
other trivial cases ("matches in both merge trees but not in the original 
one"), those cases might actually be interesting for the merge logic to 
know about, so I thought I'd leave all that information around. I expect 
it to be fairly rare anyway, so writing out a few extra index entries to 
disk so that others can decide to annotate the merge a bit more sounded 
like a fair deal.

I should make "ls-files" have a "-l" format, which shows the index and the 
mode for each file too. Right now it's very hard to see what the contents 
of the index is. But all my tests seem to say that not only does this 
work, it's pretty efficient too. And it's dead _simple_, thanks to having 
all the merge information in just one place, the same index we always use 
anyway.

Btw, it also means that you don't even have to have a separate 
subdirectory for this. All the information literally is in the index file, 
which is a temporary thing anyway. We don't need to worry about what is in 
the working directory, since we'll never show it, and we'll never need to 
use it.

Damn, I'm good.

(On the other hand, it is Friday evening at 11PM, and I'm sitting in front
of the computer. I'm a sad case. I will now go take a beer, and relax. I
think this is another of my "Really Good Ideas" (tm), and is worth the
beer.  This "feels" right).

		Linus

  reply	other threads:[~2005-04-16  6:21 UTC|newest]

Thread overview: 130+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2005-04-14  0:29 Merge with git-pasky II Petr Baudis
2005-04-13 21:25 ` Christopher Li
2005-04-14  0:45   ` Petr Baudis
2005-04-13 22:00     ` Christopher Li
2005-04-14  3:51     ` Linus Torvalds
2005-04-14  1:23       ` Christopher Li
2005-04-14  5:03         ` Paul Jackson
2005-04-14  2:16           ` Christopher Li
2005-04-14  6:16             ` Paul Jackson
2005-04-14  7:05       ` Junio C Hamano
2005-04-14  8:06         ` Linus Torvalds
2005-04-14  8:39           ` Junio C Hamano
2005-04-14  9:10             ` Linus Torvalds
2005-04-14 11:14               ` Junio C Hamano
2005-04-14 12:16                 ` Petr Baudis
2005-04-14 18:12                   ` Junio C Hamano
2005-04-14 18:36                     ` Linus Torvalds
2005-04-14 19:59                       ` Junio C Hamano
2005-04-14 20:20                         ` Petr Baudis
2005-04-15  0:42                         ` Linus Torvalds
2005-04-15  2:33                           ` Barry Silverman
2005-04-15 10:02                           ` David Woodhouse
2005-04-15 15:32                             ` Linus Torvalds
2005-04-15 16:01                               ` David Woodhouse
2005-04-15 16:31                                 ` C. Scott Ananian
2005-04-15 17:11                                   ` Linus Torvalds
2005-04-16 15:33                                 ` Johannes Schindelin
2005-04-17 13:14                                   ` David Woodhouse
2005-04-15 19:20                               ` Paul Jackson
2005-04-16  1:44                               ` Simon Fowler
2005-04-16 12:19                                 ` David Lang
2005-04-16 15:55                                   ` Simon Fowler
2005-04-16 16:03                                     ` Petr Baudis
2005-04-16 16:26                                       ` Simon Fowler
2005-04-16 16:26                                       ` Linus Torvalds
2005-04-16 23:02                                         ` David Lang
2005-04-17 14:52                                         ` Ingo Molnar
2005-04-17 15:08                                           ` Brad Roberts
2005-04-17 15:18                                             ` Ingo Molnar
2005-04-17 15:28                                           ` Ingo Molnar
2005-04-17 17:34                                             ` Linus Torvalds
2005-04-17 22:12                                               ` Herbert Xu
2005-04-17 22:35                                                 ` Linus Torvalds
2005-04-17 23:29                                                   ` Herbert Xu
2005-04-17 23:34                                                     ` Petr Baudis
2005-04-17 23:53                                                       ` Kenneth Johansson
2005-04-18  0:49                                                       ` Herbert Xu
2005-04-18  0:55                                                         ` Petr Baudis
2005-04-17 23:50                                                     ` Linus Torvalds
2005-04-18  4:16                                               ` Sanjoy Mahajan
2005-04-18  7:42                                               ` Ingo Molnar
2005-04-16 20:29                               ` Sanjoy Mahajan
2005-04-16 20:41                                 ` Linus Torvalds
2005-04-15  2:21                       ` [Patch] ls-tree enhancements Junio C Hamano
2005-04-15 16:13                         ` Petr Baudis
2005-04-15 18:25                           ` Junio C Hamano
2005-04-15  9:14                       ` Merge with git-pasky II David Woodhouse
2005-04-15  9:36                         ` Ingo Molnar
2005-04-15 10:05                           ` David Woodhouse
2005-04-15 14:53                             ` Ingo Molnar
2005-04-15 15:09                               ` David Woodhouse
2005-04-15 12:03                         ` Johannes Schindelin
2005-04-15 10:22                           ` Theodore Ts'o
2005-04-15 14:53                         ` Linus Torvalds
2005-04-15 15:29                           ` David Woodhouse
2005-04-15 15:51                             ` Linus Torvalds
2005-04-15 15:54                           ` Paul Jackson
2005-04-15 16:30                             ` C. Scott Ananian
2005-04-15 18:29                               ` Paul Jackson
2005-04-14 18:51                     ` Christopher Li
2005-04-14 19:35                     ` Petr Baudis
2005-04-14 20:01                       ` Live Merging from remote repositories Barry Silverman
2005-04-14 23:22                         ` Junio C Hamano
2005-04-15  1:07                           ` Question about git process model Barry Silverman
2005-04-14 20:23                       ` Re: Merge with git-pasky II Erik van Konijnenburg
2005-04-14 20:24                         ` Petr Baudis
2005-04-14 23:12                       ` Junio C Hamano
2005-04-14 20:24                         ` Christopher Li
2005-04-14 23:31                         ` Petr Baudis
2005-04-14 20:30                           ` Christopher Li
2005-04-14 20:37                             ` Christopher Li
2005-04-14 20:50                               ` Christopher Li
2005-04-15  0:58                           ` Junio C Hamano
2005-04-14 22:30                             ` Christopher Li
2005-04-15  7:43                               ` Junio C Hamano
2005-04-15  6:28                                 ` Christopher Li
2005-04-15 11:11                                   ` Junio C Hamano
     [not found]                                     ` <7vaco0i3t9.fsf_-_@assigned-by-dhcp.cox.net>
2005-04-15 18:44                                       ` write-tree is pasky-0.4 Linus Torvalds
2005-04-15 18:56                                         ` Petr Baudis
2005-04-15 20:13                                           ` Linus Torvalds
2005-04-15 22:36                                             ` Petr Baudis
2005-04-16  0:22                                               ` Linus Torvalds
2005-04-16  1:13                                                 ` Daniel Barkalow
2005-04-16  2:18                                                   ` Linus Torvalds
2005-04-16  2:49                                                     ` Daniel Barkalow
2005-04-16  3:13                                                       ` Linus Torvalds
2005-04-16  3:56                                                         ` Daniel Barkalow
2005-04-16  6:59                                                         ` Paul Jackson
2005-04-16 15:34                                                 ` Re: Re: " Petr Baudis
2005-04-15 20:10                                         ` Junio C Hamano
2005-04-15 20:58                                           ` C. Scott Ananian
2005-04-15 21:22                                             ` Petr Baudis
2005-04-15 23:16                                             ` Junio C Hamano
2005-04-15 21:48                                           ` [PATCH 1/2] merge-trees script for Linus git Junio C Hamano
2005-04-15 21:54                                             ` [PATCH 2/2] " Junio C Hamano
2005-04-15 23:33                                             ` [PATCH 3/2] " Junio C Hamano
2005-04-16  1:02                                               ` Linus Torvalds
2005-04-16  4:10                                                 ` Junio C Hamano
2005-04-16  5:02                                                   ` Linus Torvalds
2005-04-16  6:26                                                     ` Linus Torvalds [this message]
2005-04-16  8:12                                                     ` Junio C Hamano
2005-04-16  9:27                                                       ` [PATCH] Byteorder fix for read-tree, new -m semantics version Junio C Hamano
2005-04-16 10:35                                                       ` [PATCH 1/2] Add --stage to show-files for new stage dircache Junio C Hamano
2005-04-16 10:42                                                         ` [PATCH 2/2] " Junio C Hamano
2005-04-16 14:03                                                       ` Issues with higher-order stages in dircache Junio C Hamano
2005-04-17  5:11                                                         ` Junio C Hamano
2005-04-17  5:31                                                           ` Linus Torvalds
2005-04-17  6:01                                                             ` Junio C Hamano
2005-04-17 10:00                                                         ` Summary of "read-tree -m O A B" mechanism Junio C Hamano
2005-04-16 15:28                                                       ` [PATCH 3/2] merge-trees script for Linus git Linus Torvalds
2005-04-16 16:36                                                         ` Linus Torvalds
2005-04-16 17:14                                                           ` Junio C Hamano
2005-04-15 19:54                             ` Re: Merge with git-pasky II Petr Baudis
2005-04-15 10:22                           ` Junio C Hamano
2005-04-15 20:40                             ` Petr Baudis
2005-04-15 22:41                               ` Junio C Hamano
2005-04-15 19:57           ` Junio C Hamano
2005-04-15 20:45             ` Linus Torvalds
2005-04-14  0:30 ` Petr Baudis
2005-04-14 22:11 ` git merge Petr Baudis

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=Pine.LNX.4.58.0504152256520.7211@ppc970.osdl.org \
    --to=torvalds@osdl.org \
    --cc=git@vger.kernel.org \
    --cc=junkio@cox.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).