Updated git HOWTO for kernel hackers

Updated git HOWTO for kernel hackers

[Jeff Garzik]

Things in git-land are moving at lightning speed, and usability has 
improved a lot since my post a month ago:  http://lkml.org/lkml/2005/5/26/11



1) installing git

git requires bootstrapping, since you must have git installed in order 
to check out git.git (git repo), and linux-2.6.git (kernel repo).  I 
have put together a bootstrap tarball of today's git repository.

Download tarball from:
http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-20050622.tar.bz2

tarball build-deps:  zlib, libcurl, libcrypto (openssl)

install tarball:  unpack && make && sudo make prefix=/usr/local install

jgarzik helper scripts, not in official git distribution:
http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-new-branch
http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-changes-script

After reading the rest of this document, come back and update your copy 
of git to the latest:
rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git.git


2) download a linux kernel tree for the very first time

$ mkdir -p linux-2.6/.git
$ cd linux-2.6
$ rsync -a --delete --verbose --stats --progress \
rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ 
\          <- word-wrapped backslash; sigh
     .git/


3) update local kernel tree to latest 2.6.x upstream ("fast-forward merge")

$ cd linux-2.6
$ git-pull-script \
rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git


4) check out files from the git repository into the working directory

$ git checkout -f


5) check in your own modifications (e.g. do some hacking, or apply a patch)

# go to repo
$ cd linux-2.6

# make some modifications
$ patch -sp1 < /tmp/my.patch
$ diffstat -p1 < /tmp/my.patch

# NOTE: add '--add' and/or '--remove' if files were added or removed
$ git-update-cache <list of all files changed>

# check in changes
$ git commit


6) List all changes in working dir, in diff format.

$ git-diff-cache -p HEAD


7) List all changesets (i.e. show each cset's description text) in local 
branch of local tree, that are not present in remote tree.

$ cd my-kernel-tree-2.6
$ git-changes-script -L ../linux-2.6 | less


8) List all changesets:

$ git-whatchanged


9) apply all patches in a Berkeley mbox-format file

First, download and add to your PATH Linus's git tools:
rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git-tools.git

$ cd my-kernel-tree-2.6
$ dotest /path/to/mbox  # yes, Linus has no taste in naming scripts


10) don't forget to download tags from time to time.

git-pull-script only downloads sha1-indexed object data, and the 
requested remote head.  This misses updates to the .git/refs/tags/ and 
.git/refs/heads directories.  It is advisable to update your kernel .git 
directories periodically with a full rsync command, to make sure you got 
everything:

$ cd linux-2.6
$ rsync -a --delete --verbose --stats --progress \
rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
\          <- word-wrapped backslash; sigh
     .git/


11) list all branches, such as those found in my netdev-2.6 or 
libata-dev trees.

Download
rsync://rsync.kernel.org/pub/scm/linux/kernel/git/jgarzik/netdev-2.6.git
	or
rsync://rsync.kernel.org/pub/scm/linux/kernel/git/jgarzik/libata-dev.git


$ cd netdev-2.6
$ ls .git/refs/heads/

{ these are the current netdev-2.6 branches }
> 8139cp       forcedeth    master     qeth           smc91x         we18
> 8139too-iomap  for-linus    natsemi      r8169      smc91x-eeprom  wifi
> airo           hdlc         ns83820      register-netdev  starfire
> atmel          ieee80211    orinoco      remove-drivers   tlan
> chelsio        iff-running  orinoco-hch  sis900           veth
> dm9000         janitor      ppp          skge             viro


12) make desired branch current in working directory

$ git checkout -f $branch


13) create a new branch, and make it current

$ cp .git/refs/heads/master .git/refs/heads/my-new-branch-name
$ git checkout -f my-new-branch-name


14) examine which branch is current

$ ls -l .git/HEAD


15) undo all local modifications (same as checkout):

$ git checkout -f


16) obtain a diff between current branch, and master branch

In most trees WITH BRANCHES, .git/refs/heads/master contains the current 
'vanilla' upstream tree, for easy diffing and merging.  (in trees 
without branches, 'master' simply contains your latest changes)

$ git-diff-tree -p master HEAD

Re: Updated git HOWTO for kernel hackers

[Dave Jones]

On Wed, Jun 22, 2005 at 06:24:54PM -0400, Jeff Garzik wrote:
 > 
 > Things in git-land are moving at lightning speed, and usability has 
 > improved a lot since my post a month ago:  http://lkml.org/lkml/2005/5/26/11
 > 
 > 
 > 
 > 1) installing git
 > 
 > git requires bootstrapping, since you must have git installed in order 
 > to check out git.git (git repo), and linux-2.6.git (kernel repo).  I 
 > have put together a bootstrap tarball of today's git repository.
 > 
 > Download tarball from:
 > http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-20050622.tar.bz2

<blatant self-promotion>
daily snapshots (refreshed once an hour) are available at:
http://www.codemonkey.org.uk/projects/git-snapshots/git/
</blatant self-promotion>

 > tarball build-deps:  zlib, libcurl, libcrypto (openssl)
 > 
 > install tarball:  unpack && make && sudo make prefix=/usr/local install

the sudo thing isn't necessary. make install by itself installs it
in ~/bin/ just fine.

 > After reading the rest of this document, come back and update your copy 
 > of git to the latest:
 > rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git.git

See above, which allows you to skip this step ;)

		Dave

Re: Updated git HOWTO for kernel hackers

[Jeff Garzik]

Dave Jones wrote:
> On Wed, Jun 22, 2005 at 06:24:54PM -0400, Jeff Garzik wrote:
>  > 
>  > Things in git-land are moving at lightning speed, and usability has 
>  > improved a lot since my post a month ago:  http://lkml.org/lkml/2005/5/26/11
>  > 
>  > 
>  > 
>  > 1) installing git
>  > 
>  > git requires bootstrapping, since you must have git installed in order 
>  > to check out git.git (git repo), and linux-2.6.git (kernel repo).  I 
>  > have put together a bootstrap tarball of today's git repository.
>  > 
>  > Download tarball from:
>  > http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-20050622.tar.bz2
> 
> <blatant self-promotion>
> daily snapshots (refreshed once an hour) are available at:
> http://www.codemonkey.org.uk/projects/git-snapshots/git/
> </blatant self-promotion>
> 
>  > tarball build-deps:  zlib, libcurl, libcrypto (openssl)
>  > 
>  > install tarball:  unpack && make && sudo make prefix=/usr/local install
> 
> the sudo thing isn't necessary. make install by itself installs it
> in ~/bin/ just fine.

Clearly this does not work if installing in /usr/local, as I and others 
do (and as the example shows).


>  > After reading the rest of this document, come back and update your copy 
>  > of git to the latest:
>  > rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git.git
> 
> See above, which allows you to skip this step ;)

huh?  Nothing allows you to skip that step.  Regardless of when you suck 
the tarball, even from your snapshots, the users should not skip this step.

	Jeff

Re: Updated git HOWTO for kernel hackers

[Dave Jones]

On Wed, Jun 22, 2005 at 06:47:03PM -0400, Jeff Garzik wrote:

 > > > install tarball:  unpack && make && sudo make prefix=/usr/local install
 > >
 > >the sudo thing isn't necessary. make install by itself installs it
 > >in ~/bin/ just fine.
 > 
 > Clearly this does not work if installing in /usr/local, as I and others 
 > do (and as the example shows).

Sure, it just seemed to imply that it doesn't work with a non-root install,
which isn't true.

 > > > After reading the rest of this document, come back and update your copy 
 > > > of git to the latest:
 > > > rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git.git
 > >
 > >See above, which allows you to skip this step ;)
 > 
 > huh?  Nothing allows you to skip that step.  Regardless of when you suck 
 > the tarball, even from your snapshots, the users should not skip this step.

At worse, users will have tools 59 minutes old.  If a situation arises
where git from an hour ago isn't new enough to pull from the repository,
we have bigger problems.

You seem to be proposing that everyone needs the shiniest newest things,
which clearly isn't true, and suggesting so just complicates things
further imo.

		Dave

Re: Updated git HOWTO for kernel hackers

[Jeff Garzik]

Dave Jones wrote:
> At worse, users will have tools 59 minutes old.  If a situation arises
> where git from an hour ago isn't new enough to pull from the repository,
> we have bigger problems.
> 
> You seem to be proposing that everyone needs the shiniest newest things,
> which clearly isn't true, and suggesting so just complicates things
> further imo.

For the purposes of the these instructions, it is highly recommended.

However, I was unaware that your snapshots were updated hourly.  Yeah, 
that's quite fine.

	Jeff

Re: Updated git HOWTO for kernel hackers

[Jeff Garzik]

Dave Jones wrote:
> On Wed, Jun 22, 2005 at 06:24:54PM -0400, Jeff Garzik wrote:
>  > 
>  > Things in git-land are moving at lightning speed, and usability has 
>  > improved a lot since my post a month ago:  http://lkml.org/lkml/2005/5/26/11
>  > 
>  > 
>  > 
>  > 1) installing git
>  > 
>  > git requires bootstrapping, since you must have git installed in order 
>  > to check out git.git (git repo), and linux-2.6.git (kernel repo).  I 
>  > have put together a bootstrap tarball of today's git repository.
>  > 
>  > Download tarball from:
>  > http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-20050622.tar.bz2
> 
> <blatant self-promotion>
> daily snapshots (refreshed once an hour) are available at:
> http://www.codemonkey.org.uk/projects/git-snapshots/git/
> </blatant self-promotion>

I was about to link to this, but a problem arose:  your snapshots don't 
include the .git/objects directory.

Also, a git-latest.tar.gz symlink would be nice.

	Jeff

Re: Updated git HOWTO for kernel hackers

[Dave Jones]

On Fri, Jun 24, 2005 at 11:33:38PM -0400, Jeff Garzik wrote:
 > Dave Jones wrote:
 > >On Wed, Jun 22, 2005 at 06:24:54PM -0400, Jeff Garzik wrote:
 > > > 
 > > > Things in git-land are moving at lightning speed, and usability has 
 > > > improved a lot since my post a month ago:  
 > > http://lkml.org/lkml/2005/5/26/11
 > > > 
 > > > 
 > > > 
 > > > 1) installing git
 > > > 
 > > > git requires bootstrapping, since you must have git installed in order 
 > > > to check out git.git (git repo), and linux-2.6.git (kernel repo).  I 
 > > > have put together a bootstrap tarball of today's git repository.
 > > > 
 > > > Download tarball from:
 > > > 
 > > http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-20050622.tar.bz2
 > >
 > ><blatant self-promotion>
 > >daily snapshots (refreshed once an hour) are available at:
 > >http://www.codemonkey.org.uk/projects/git-snapshots/git/
 > ></blatant self-promotion>
 > 
 > I was about to link to this, but a problem arose:  your snapshots don't 
 > include the .git/objects directory.

This is intentional.  Why is this a problem ?
In the same way, the bitkeeper snapshots I used to do never included
Bitkeeper/, and CVS snapshots don't include the CVS/ dirs.

 > Also, a git-latest.tar.gz symlink would be nice.

That's doable.

		Dave

Re: Updated git HOWTO for kernel hackers

[Greg KH]

On Wed, Jun 22, 2005 at 06:24:54PM -0400, Jeff Garzik wrote:
> 10) don't forget to download tags from time to time.
> 
> git-pull-script only downloads sha1-indexed object data, and the 
> requested remote head.  This misses updates to the .git/refs/tags/ and 
> .git/refs/heads directories.  It is advisable to update your kernel .git 
> directories periodically with a full rsync command, to make sure you got 
> everything:
> 
> $ cd linux-2.6
> $ rsync -a --delete --verbose --stats --progress \
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
> \          <- word-wrapped backslash; sigh
>     .git/

Ok, this is annoying.  Is there some reason why git doesn't pull the
tags in properly when doing a merge?  Chris and I just hit this when I
pulled his 2.6.12.1 tree and and was wondering where the tag went.

thanks,

greg k-h

Re: Updated git HOWTO for kernel hackers

[Linus Torvalds]

On Wed, 22 Jun 2005, Greg KH wrote:
> 
> Ok, this is annoying.  Is there some reason why git doesn't pull the
> tags in properly when doing a merge?  Chris and I just hit this when I
> pulled his 2.6.12.1 tree and and was wondering where the tag went.

Tags are private in git (the same way branches are), which means that you
can have a million of your own tags and never disturb anybody else.

But, like branches, it means that if you want a tag, you need to know the 
tag you want, and download it the same way you download a branch.

		Linus

Re: Updated git HOWTO for kernel hackers

[Jeff Garzik]

Linus Torvalds wrote:
> 
> On Wed, 22 Jun 2005, Greg KH wrote:
> 
>>Ok, this is annoying.  Is there some reason why git doesn't pull the
>>tags in properly when doing a merge?  Chris and I just hit this when I
>>pulled his 2.6.12.1 tree and and was wondering where the tag went.
> 
> 
> Tags are private in git (the same way branches are), which means that you
> can have a million of your own tags and never disturb anybody else.
> 
> But, like branches, it means that if you want a tag, you need to know the 
> tag you want, and download it the same way you download a branch.

Still -- that's interesting data that no script currently tracks.  You 
gotta fall back to rsync.

	Jeff

Re: Updated git HOWTO for kernel hackers

[Linus Torvalds]

On Wed, 22 Jun 2005, Jeff Garzik wrote:
>
> > But, like branches, it means that if you want a tag, you need to know the 
> > tag you want, and download it the same way you download a branch.
> 
> Still -- that's interesting data that no script currently tracks.  You 
> gotta fall back to rsync.

Something like

	git-ssh/http-pull -w tags/<tagname> tags/<tagname> <url>

_should_ hopefully work now (and the "-a" flag should mean that you also 
get all the objects needed for the tag).

I've not tested it, as usual, but it should work as of today thanks to 
Daniel Barkalow fixing the pulling of arbitrary objects.

		Linus

Re: Updated git HOWTO for kernel hackers

[Jeff Garzik]

Linus Torvalds wrote:
> 
> On Wed, 22 Jun 2005, Jeff Garzik wrote:
> 
>>>But, like branches, it means that if you want a tag, you need to know the 
>>>tag you want, and download it the same way you download a branch.
>>
>>Still -- that's interesting data that no script currently tracks.  You 
>>gotta fall back to rsync.
> 
> 
> Something like
> 
> 	git-ssh/http-pull -w tags/<tagname> tags/<tagname> <url>
> 
> _should_ hopefully work now (and the "-a" flag should mean that you also 
> get all the objects needed for the tag).

The problem isn't pulling tags, the problem is that nothing 
automatically downloads the 41-byte tag files themselves.  Pulling 
linux-2.6.git after the 2.6.12 release did not cause refs/tags/v2.6.12 
to be downloaded.

With BK, tags came with each pull.  With git, you have to go "outside 
the system" (rsync) just get the new tags.

	Jeff

Re: Updated git HOWTO for kernel hackers

[Linus Torvalds]

On Wed, 22 Jun 2005, Jeff Garzik wrote:
>
> With BK, tags came with each pull.  With git, you have to go "outside 
> the system" (rsync) just get the new tags.

You don't have to use rsync, and you don't have to go outside the system. 
That was my point: you can use "git-ssh-pull" to pull the tags.

But yes, you have to explicitly ask for them by name, ie the other side 
has to let you know: "Oh, btw, I created a 'xyz' tag for you". And having 
another helper script to hide the details of how git-*-pull handles tags 
is obviously also a good idea, although it's pretty low on my list of 
things to worry about.

		Linus

Re: Updated git HOWTO for kernel hackers

[Jeff Garzik]

Linus Torvalds wrote:
> 
> On Wed, 22 Jun 2005, Jeff Garzik wrote:
> 
>>With BK, tags came with each pull.  With git, you have to go "outside 
>>the system" (rsync) just get the new tags.
> 
> 
> You don't have to use rsync, and you don't have to go outside the system. 
> That was my point: you can use "git-ssh-pull" to pull the tags.

OK, understood.


> But yes, you have to explicitly ask for them by name, ie the other side 
> has to let you know: "Oh, btw, I created a 'xyz' tag for you". And having 
> another helper script to hide the details of how git-*-pull handles tags 
> is obviously also a good idea, although it's pretty low on my list of 
> things to worry about.

The problem is still that nothing says "oh, btw, I created 'xyz' tag for 
you" AFAICS?

IMO the user (GregKH and me, at least) just wants to know their set of 
tags and heads is up-to-date on local disk.  Wants to know what tags are 
out there.  It's quite annoying when two data sets are out of sync 
(.git/objects and .git/refs/tags).

Asking for the tag by name isn't useful at all, in that regard, because 
that requires that the user already know what tags are available.  To 
get that info, one must use rsync, gitweb, or a subscription to Psychic 
Friends Network.

	Jeff

Re: Updated git HOWTO for kernel hackers

[Linus Torvalds]

On Wed, 22 Jun 2005, Jeff Garzik wrote:
> 
> The problem is still that nothing says "oh, btw, I created 'xyz' tag for 
> you" AFAICS?
> 
> IMO the user (GregKH and me, at least) just wants to know their set of 
> tags and heads is up-to-date on local disk.  Wants to know what tags are 
> out there.  It's quite annoying when two data sets are out of sync 
> (.git/objects and .git/refs/tags).

Well, I really think this is the exact same issue as when you write any 
annoucement, and say "please pull from branch xyz of repo abc".

What I'm saying is that for a tagged release, that really translates to
"please pull tag xyz from repo abc" and the tools like git-ssh-pull will 
just do the right thing: they'll pull the tag itself _and_ they'll pull 
the objects it points to.

Of course, right now "git fetch" is hardcoded to always write FETCH_HEAD 
(not the tag name), but I'm saying ythat _literally_ you can do this 
already:

	git fetch repo-name tags/xyz &&
		( cat .git/FETCH_HEAD > .git/tags/xyz )

and it should do exactly what you want. Hmm?

So if we script this (maybe teach "git-fetch-script" to take "tag" as its 
first argument and do this on its own), and people learn to just do

	git fetch tag v2.6.18.5

when Chris or Greg make an announcement about "v2.6.18.5", then you're all
done, no?

The change to "git-fetch-script" would look something like the appended.. 
Totally untested, of course. Give it a try,

			Linus

---
diff --git a/git-fetch-script b/git-fetch-script
--- a/git-fetch-script
+++ b/git-fetch-script
@@ -1,5 +1,12 @@
 #!/bin/sh
 #
+destination=FETCH_HEAD
+
+if [ "$1" = "tag" ]; then
+	shift
+	destination="refs/tags/$2"
+fi
+
 merge_repo=$1
 merge_name=${2:-HEAD}
 
@@ -35,7 +42,7 @@ download_objects () {
 }
 
 echo "Getting remote $merge_name"
-download_one "$merge_repo/$merge_name" "$GIT_DIR"/FETCH_HEAD || exit 1
+download_one "$merge_repo/$merge_name" "$GIT_DIR/$dest" || exit 1
 
 echo "Getting object database"
-download_objects "$merge_repo" "$(cat "$GIT_DIR"/FETCH_HEAD)" || exit 1
+download_objects "$merge_repo" "$(cat "$GIT_DIR/$dest")" || exit 1

Re: Updated git HOWTO for kernel hackers

[Jeff Garzik]

Linus Torvalds wrote:
> What I'm saying is that for a tagged release, that really translates to
> "please pull tag xyz from repo abc" and the tools like git-ssh-pull will 
> just do the right thing: they'll pull the tag itself _and_ they'll pull 
> the objects it points to.

Yes, everything does the right there here.


> Of course, right now "git fetch" is hardcoded to always write FETCH_HEAD 
> (not the tag name), but I'm saying ythat _literally_ you can do this 
> already:
> 
> 	git fetch repo-name tags/xyz &&
> 		( cat .git/FETCH_HEAD > .git/tags/xyz )
> 
> and it should do exactly what you want. Hmm?

No, not at all.  This sub-thread is all about tags/ dir updates.  Users 
should be able to do

	git pull-more rsync://...

and get ALL of .git/refs/tags/* that have appeared since their last update.

Concrete example:  I have a git tree on local disk.  I need to find out 
where, between 2.6.12-rc1 and 2.6.12, a driver broke.  This requires 
that I have -ALL- linux-2.6.git/refs/tags on disk already, so that I can 
bounce quickly and easily between tags.

It is valuable to have a local copy of -all- tags, -before- you need 
them.  That is why people like me and GregKH use rsync directly.  We 
want EVERYTHING in the kernel.org linux-2.6.git tree, not just what we 
know we need right now.

	Jeff

Re: Updated git HOWTO for kernel hackers

[Linus Torvalds]

On Wed, 22 Jun 2005, Jeff Garzik wrote:
> 
> Concrete example:  I have a git tree on local disk.  I need to find out 
> where, between 2.6.12-rc1 and 2.6.12, a driver broke.  This requires 
> that I have -ALL- linux-2.6.git/refs/tags on disk already, so that I can 
> bounce quickly and easily between tags.

Absolutely not.

I might have my private tags in my kernel, and you might have your private 
tags ("tested") in your kernel, so there is no such thing as "ALL".

The fact that BK had it was a BK deficiency, and just meant that you 
basically couldn't use tags at all with BK, the "official ones" excepted. 
It basically meant that nobody else than me could ever tag a tree. Do you 
not see how that violates the very notion of "distributed"?

This is _exactly_ the same thing as if you said "I want to merge with ALL
BRANCHES".  That notion doesn't exist. You can rsync the whole repository,
and you'll get all branches from that repository, that's really by virtue
of doing a filesystem operation, not because you asked git to get you all
branches.

A tag is even _implemented_ exactly like a branch, except it allows (but
does not require) that extra step of signing an object. The only
difference is literally whether it is in refs/branches or refs/tags.

> It is valuable to have a local copy of -all- tags, -before- you need 
> them.

You seem to not realize that "all tags" is a nonsensical statement in a 
distributed system.

If you want to have a list of official tags, why not just do exactly that? 
What's so hard with saying "ok, that place has a list of 'official' tags, 
let's fetch them".

How would you fetch them? You might use rsync, for example. Or maybe wget. 
Or whatever. The point is that this works already. You're asking for 
something nonsensical, outside of just a script that does

	rsync -r --ignore-existing repo/refs/tags/ .git/refs/tags/

See? What's your complaint with just doing that?

			Linus

Re: Updated git HOWTO for kernel hackers

[Jeff Garzik]

Linus Torvalds wrote:
> 	rsync -r --ignore-existing repo/refs/tags/ .git/refs/tags/
> 
> See? What's your complaint with just doing that?

No complaint with that operation.  The complaint is that it's an 
additional operation.  Re-read what Greg said:

> Is there some reason why git doesn't pull the
> tags in properly when doing a merge?  Chris and I just hit this when I
> pulled his 2.6.12.1 tree and and was wondering where the tag went.

Multiple users -- not just me -- would prefer that git-pull-script 
pulled the tags, too.

Suggested solution:  add '--tags' to git-pull-script 
(git-fetch-script?), which calls
	rsync -r --ignore-existing repo/refs/tags/ .git/refs/tags/


> You seem to not realize that "all tags" is a nonsensical statement in a 
> distributed system.
> 
> If you want to have a list of official tags, why not just do exactly that? 
> What's so hard with saying "ok, that place has a list of 'official' tags, 
> let's fetch them".

I know how tags work, and I like the new flexibility above and beyond BK.

Kernel hackers are surprised when the tags aren't pulled, along with the 
objects.  BK and CVS trained us that tags came with the repo, no 
additional steps needed.  Why not give us the OPTION of working like 
we've always worked?

Let the kernel hacker say "yes, I really do want to download the tags 
Linus publicly posted in linux-2.6.git/refs/tags" because this was a 
common operation in the previous workflow, a common operation that we 
-made use of-.

	Jeff

Re: Updated git HOWTO for kernel hackers

[Linus Torvalds]

On Thu, 23 Jun 2005, Jeff Garzik wrote:
>
> No complaint with that operation.  The complaint is that it's an 
> additional operation.  Re-read what Greg said:

Please re-read what I said.

Pulling a regular head _cannot_ and _must_not_ update tags. Tags are not 
associated with the tree, and they _cannot_ and _must_not_ be so, exactly 
because that would make them global instead of private, and it would 
fundamentally make them not be distributed, and would mean that they'd be 
pointless as anything but "Linus' official tags".

That's what we had in BK _AND IT DOES NOT WORK_!

Does it help when I scream?

> > Is there some reason why git doesn't pull the
> > tags in properly when doing a merge?  Chris and I just hit this when I
> > pulled his 2.6.12.1 tree and and was wondering where the tag went.

And I suggested that if you want that, then you pull on the TAG. You take 
my modification, you test it, and you see if

	git fetch tag ..repo.. tagname

works.

That solves exactly the case that Greg is complaining about, and it solves
it in a _sane_ manner: you tell git that you want a tag, and git fetches
it for you. It's that simple, and it does not introduce the _BROKEN_
notion that tags are associated directly with the commit itself and
somehow visible to all.

> Multiple users -- not just me -- would prefer that git-pull-script 
> pulled the tags, too.

And multiple users -- clearly including you -- aren't listening to me. 
Tags are separate from the source they tag, and they HAVE TO BE. There is 
no "you automatically get the tags when you get the tree", because the two 
don't have a 1:1 relationship.

And not making them separate breaks a lot of things. As mentioned, it
fundamentally breaks the distributed nature, but that also means that it
breaks whenever two people use the same name for a tag, for example. You
can't "merge" tags. BK had a very strange form of merging, which was (I
think) to pick the one last in the BK ChangeSet file, but that didn't make
it "right". You just never noticed, because Linux could never use tags at
all due to the lack of privacy, except for big releases..

> Suggested solution:  add '--tags' to git-pull-script 
> (git-fetch-script?), which calls
> 	rsync -r --ignore-existing repo/refs/tags/ .git/refs/tags/

How is this AT ALL different from just having a separate script that does
this? You've introduced nothing but syntactic fluff, and you've made it
less flexible at the same time. First off, you might want to get new tags
_without_ fetching anything else, and you might indeed want to get the 
tags _first_ in order to decide what you want to fetch. In fact, in many 
cases that's exactly what you want, namely you want to fetch the data 
based on the tag.

Secondly, if your worry is that you forget, then hell, write a small shell 
function, and be done with it.

BUT DO NOT MESS UP THINGS FOR OTHER PEOPLE.

When I fetch somebody elses head, I had better not fetch his tags. His
tags may not even make _sense_ in what I have - he may tag things in other
branches that I'm not fetching at all. In fact, his tag-namespace might be
_different_ from mine, ie he might have tagged something "broken" in his
tree, and I tagged something _else_ "broken" in mine, just because it
happens to be a very useful tag for when you want to mark "ok, that was a
broken tree".

It is wrong, wrong, _wrong_ to think that fetching somebody elses tree
means that you should fetch his tags. The _only_ reason you think it's
right is because you've only ever seen centralized tags: tags were the one
thing that BK kept centralized.

But once people realize that they can use tags in their own trees, and 
nobody else will ever notice, they'll slowly start using them. Maybe it 
takes a few months or even longer. But it will happen. And I refuse to 
make stupid decisions that makes it not work.

And thinking that "fetching a tree fetches all the tags from that tree"  
really _is_ a stupid decision. It's missing the big picture. It's missing
the fact that tags _should_ be normal every-day things that you just use
as "book-marks", and that the kind of big "synchronization point for many
people" tag should actually be the _rare_ case.

The fact that global tags make that private "bookmark" usage impossible
should be a big red blinking sign saying "don't do global tags".

> Let the kernel hacker say "yes, I really do want to download the tags 
> Linus publicly posted in linux-2.6.git/refs/tags" because this was a 
> common operation in the previous workflow, a common operation that we 
> -made use of-.

And I already suggested a trivial script. Send me the script patch,
instead of arguing for stupid things.

			Linus

Re: Updated git HOWTO for kernel hackers

[Greg KH]

On Wed, Jun 22, 2005 at 10:58:13PM -0700, Linus Torvalds wrote:
> And I suggested that if you want that, then you pull on the TAG. You take 
> my modification, you test it, and you see if
> 
> 	git fetch tag ..repo.. tagname
> 
> works.

Hm, that doesn't work right now.  Both:
  git fetch rsync://rsync.kernel.org/pub/scm/linux/kernel/git/chrisw/linux-2.6.12.y.git tag v2.6.12.1
or
  git fetch tag rsync://rsync.kernel.org/pub/scm/linux/kernel/git/chrisw/linux-2.6.12.y.git v2.6.12.1

die.  Or am I just trying to take a point you were making about not
pulling all tags (which I can live with, just was not aware it was this
way, and I agree that it does offer up a lot of possiblities of me using
local tags in the future), and taking it literally?

thanks,

greg k-h

Re: Updated git HOWTO for kernel hackers

[Linus Torvalds]

On Wed, 22 Jun 2005, Greg KH wrote:
> 
> Hm, that doesn't work right now.

Yeah, my suggested mod sucks.

Try the following slightly modified version instead, with

	git fetch rsync://rsync.kernel.org/pub/scm/linux/kernel/git/chrisw/linux-2.6.12.y.git tag v2.6.12.1

and now it should work.

		Linus

---
diff --git a/git-fetch-script b/git-fetch-script
--- a/git-fetch-script
+++ b/git-fetch-script
@@ -1,7 +1,13 @@
 #!/bin/sh
 #
+destination=FETCH_HEAD
+
 merge_repo=$1
 merge_name=${2:-HEAD}
+if [ "$2" = "tag" ]; then
+	merge_name="refs/tags/$3"
+	destination="$merge_name"
+fi
 
 : ${GIT_DIR=.git}
 : ${GIT_OBJECT_DIRECTORY="${SHA1_FILE_DIRECTORY-"$GIT_DIR/objects"}"}
@@ -35,7 +41,7 @@ download_objects () {
 }
 
 echo "Getting remote $merge_name"
-download_one "$merge_repo/$merge_name" "$GIT_DIR"/FETCH_HEAD || exit 1
+download_one "$merge_repo/$merge_name" "$GIT_DIR/$destination" || exit 1
 
 echo "Getting object database"
-download_objects "$merge_repo" "$(cat "$GIT_DIR"/FETCH_HEAD)" || exit 1
+download_objects "$merge_repo" "$(cat "$GIT_DIR/$destination")" || exit 1

Re: Updated git HOWTO for kernel hackers

[Greg KH]

On Wed, Jun 22, 2005 at 11:51:40PM -0700, Linus Torvalds wrote:
> 
> 
> On Wed, 22 Jun 2005, Greg KH wrote:
> > 
> > Hm, that doesn't work right now.
> 
> Yeah, my suggested mod sucks.
> 
> Try the following slightly modified version instead, with
> 
> 	git fetch rsync://rsync.kernel.org/pub/scm/linux/kernel/git/chrisw/linux-2.6.12.y.git tag v2.6.12.1
> 
> and now it should work.

Yes, that patch works for me.

thanks,

greg k-h

Re: Updated git HOWTO for kernel hackers

[Jeff Garzik]

Linus Torvalds wrote:
> Pulling a regular head _cannot_ and _must_not_ update tags. Tags are not 
> associated with the tree, and they _cannot_ and _must_not_ be so, exactly 

For general git implementation, strongly agreed.


> And not making them separate breaks a lot of things. As mentioned, it
> fundamentally breaks the distributed nature, but that also means that it
> breaks whenever two people use the same name for a tag, for example. You
> can't "merge" tags. BK had a very strange form of merging, which was (I
> think) to pick the one last in the BK ChangeSet file, but that didn't make
> it "right". You just never noticed, because Linux could never use tags at
> all due to the lack of privacy, except for big releases..

Agreed.


> How is this AT ALL different from just having a separate script that does
> this? You've introduced nothing but syntactic fluff, and you've made it
> less flexible at the same time. First off, you might want to get new tags
> _without_ fetching anything else, and you might indeed want to get the 
> tags _first_ in order to decide what you want to fetch.

That's a fair point.  A separate script would be better.


> because that would make them global instead of private, and it would 
> fundamentally make them not be distributed, and would mean that they'd be 
> pointless as anything but "Linus' official tags".
[...]
> the fact that tags _should_ be normal every-day things that you just use
> as "book-marks", and that the kind of big "synchronization point for many
> people" tag should actually be the _rare_ case.

For my use, I require all "Linus official tags" to be present in all my 
kernel trees, precisely because it is a big sync point for many people.

User A sends me a patch against 2.6.12-rc2, user B sends me a patch 
against 2.6.12-rc3, user C sends me a patch against 2.6.12...  I create 
a branch with
	cp .git/refs/tags/$kversion .git/refs/heads/foo-net-drvr
	git checkout -f foo-net-drvr
apply the patch, then pull linux-2.6.git to merge up to the latest version.

So in my case, the rare case is the 99% common case :)

I suppose this usage is just highly specific to me.

	Jeff

Re: Updated git HOWTO for kernel hackers

[Petr Baudis]

Dear diary, on Thu, Jun 23, 2005 at 07:58:13AM CEST, I got a letter
where Linus Torvalds <torvalds@osdl.org> told me that...
> Does it help when I scream?

Nope. I still think you are wrong. :-) (BTW, Cogito always fetches all
the tags now - but it's not that I would have a huge problem with
changing that to some better behaviour.)

> > Multiple users -- not just me -- would prefer that git-pull-script 
> > pulled the tags, too.
> 
> And multiple users -- clearly including you -- aren't listening to me. 
> Tags are separate from the source they tag, and they HAVE TO BE. There is 
> no "you automatically get the tags when you get the tree", because the two 
> don't have a 1:1 relationship.
> 
> And not making them separate breaks a lot of things. As mentioned, it
> fundamentally breaks the distributed nature, but that also means that it
> breaks whenever two people use the same name for a tag, for example. You
> can't "merge" tags. BK had a very strange form of merging, which was (I
> think) to pick the one last in the BK ChangeSet file, but that didn't make
> it "right". You just never noticed, because Linux could never use tags at
> all due to the lack of privacy, except for big releases..

I think there should simply be two namespaces - public tags and private
tags. Private tags for stuff like "broken", "merged", or "funnychange".
Other people don't care about those, and they certainly shouldn't get
them by default (but they should have a way to get them explicitly, if
you tell them). But then there are the official tags, like "v2.6.13" or
even "v2.6.12-ck2" - if you merge with those branches, you should always
get those precisely for what Jeff says - they are big syncing points for
a lot of people and you should be always able to refer to v2.6.13 if you
have the commit in your tree.

Since there should be _few_ of those tags, you might even want to get
tags only from branches marked "tagtrusted" (Cogito's origin branch
would be by default), or want to interactively confirm new tag additions
during a pull. Also, ideally there would be no or only extremely rare
tag conflicts.

I think it would be simplest to use a special prefix for the private
tags. ~ and ! might get touched by shell, so what about %?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
<Espy> be careful, some twit might quote you out of context..

Re: Updated git HOWTO for kernel hackers

[Martin Langhoff]

On 6/23/05, Petr Baudis <pasky@ucw.cz> wrote:
> I think there should simply be two namespaces - public tags and private
> tags. Private tags for stuff like "broken", "merged", or "funnychange".

I guess that public tags would also probably be in a different
location from the actual tree. With the split Linus advocates, several
people could be publishing sets of "public" tags, as well as having
the official tags hosted separately from the .git repo.

cheers,


martin

Re: Updated git HOWTO for kernel hackers

[Vojtech Pavlik]

On Wed, Jun 22, 2005 at 10:58:13PM -0700, Linus Torvalds wrote:

> And thinking that "fetching a tree fetches all the tags from that tree"  
> really _is_ a stupid decision. It's missing the big picture. It's missing
> the fact that tags _should_ be normal every-day things that you just use
> as "book-marks", and that the kind of big "synchronization point for many
> people" tag should actually be the _rare_ case.
> 
> The fact that global tags make that private "bookmark" usage impossible
> should be a big red blinking sign saying "don't do global tags".

Maybe it'd make sense to differentiate between the two types of tags?
To have local tags which don't propagate, and global (version) tags
which do? They could live in different namespaces and thus wouldn't
interfere.

-- 
Vojtech Pavlik
SuSE Labs, SuSE CR

Re: Updated git HOWTO for kernel hackers

[Horst von Brand]

Jeff Garzik <jgarzik@pobox.com> said:
> Linus Torvalds wrote:
> > 	rsync -r --ignore-existing repo/refs/tags/ .git/refs/tags/
> > 
> > See? What's your complaint with just doing that?
> 
> No complaint with that operation.  The complaint is that it's an 
> additional operation.  Re-read what Greg said:
> 
> > Is there some reason why git doesn't pull the
> > tags in properly when doing a merge?  Chris and I just hit this when I
> > pulled his 2.6.12.1 tree and and was wondering where the tag went.
> 
> Multiple users -- not just me -- would prefer that git-pull-script 
> pulled the tags, too.
> 
> Suggested solution:  add '--tags' to git-pull-script 
> (git-fetch-script?), which calls
> 	rsync -r --ignore-existing repo/refs/tags/ .git/refs/tags/

I don't think either is really a solution. IMHO there should be a
distinction between "official tags" (that get passed around together with
everything else) and "private tags" for everybodys own home use (that could
be passed around, but only explicitly). Plus the possibility to erase,
move, &c private tags, and perhaps upgrade them to official status (thus
setting them in stone).
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: Updated git HOWTO for kernel hackers

[Linus Torvalds]

On Wed, 22 Jun 2005, Jeff Garzik wrote:
>
> 2) download a linux kernel tree for the very first time
> 
> $ mkdir -p linux-2.6/.git
> $ cd linux-2.6
> $ rsync -a --delete --verbose --stats --progress \
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ 
> \          <- word-wrapped backslash; sigh
>      .git/

Gaah. I should do a "git-clone-script" or something that does this, and 
then you could just do

	git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git linux-2.6
	
Anybody?

> # make some modifications
> $ patch -sp1 < /tmp/my.patch
> $ diffstat -p1 < /tmp/my.patch
> 
> # NOTE: add '--add' and/or '--remove' if files were added or removed
> $ git-update-cache <list of all files changed>
> 
> # check in changes
> $ git commit

A few notes on these things:

	git-apply --index /tmp/my.patch

will not only apply the patch (unified patches only!), but will do the
index updates for you while it's at it, so if the patch contains new files
(or it deletes files), you don't need to worry about it.

Also, you can do

	git commit <list-of-files-to-commit>

as a shorthand for

	git-update-cache <list-of-files-to-commit>
	git commit

which some people will probably find more natural.

> 6) List all changes in working dir, in diff format.
> 
> $ git-diff-cache -p HEAD

Or, perhaps preferably:

	git diff HEAD

since that is shorter ad will also show renames.

> 8) List all changesets:
> 
> $ git-whatchanged

No, if you just want the changesets listed, then

	git log

is a lot better, since it shows merges.

"git-whatchanged" is useful if you actually want to see what the commits 
_changed_, and then you often want to use the "-p" flag to see it as 
patches. Also, it's worth pointing out the fact that you can limit it to 
certain subdirectories (or individual files) etc, ie:

	git-whatchanged -p drivers/net

since that is often what people want.

But if you just want the log, "git log" is faster and simpler and more 
correct.

> 16) obtain a diff between current branch, and master branch
> 
> In most trees WITH BRANCHES, .git/refs/heads/master contains the current 
> 'vanilla' upstream tree, for easy diffing and merging.  (in trees 
> without branches, 'master' simply contains your latest changes)
> 
> $ git-diff-tree -p master HEAD

Again, I think is possibly more naturally expressed with "git diff":

	git diff master..HEAD

which just says "show the differences from 'master' to 'HEAD'" and will
also show renames etc.

(A plain "git diff" will show just the difference to the index file, in 
case you care).

		Linus

Re: Updated git HOWTO for kernel hackers

[Jeff Garzik]

Linus Torvalds wrote:
[snip]

Thanks, this is all good stuff.  I'll update it and post another, in a 
few days or so.

git-clone-script would indeed be nice, even if its only a 2-line script.

	Jeff

Re: Updated git HOWTO for kernel hackers

[Linus Torvalds]

On Wed, 22 Jun 2005, Jeff Garzik wrote:
> 
> git-clone-script would indeed be nice, even if its only a 2-line script.

Ok, added. You can update your tutorial to make the initial setup of a 
kernel archive slightly less scary, ie it's now

	git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git linux-2.6
	cd linux-2.6
	git checkout

which looks almost user-friendly.

(Of course, since the rsync protocol doesn't know anything about git
consistency, if the mirroring is half-way, you'll end up with something
less than wonderful, and confusing. Details, details)

		Linus

Re: Updated git HOWTO for kernel hackers

[Jeff Garzik]

Linus Torvalds wrote:
> (Of course, since the rsync protocol doesn't know anything about git
> consistency, if the mirroring is half-way, you'll end up with something
> less than wonderful, and confusing. Details, details)

Would it make sense to add an fsck step to git-clone-script?

	Jeff

Re: Updated git HOWTO for kernel hackers

[Linus Torvalds]

On Thu, 23 Jun 2005, Jeff Garzik wrote:
>
> Linus Torvalds wrote:
> > (Of course, since the rsync protocol doesn't know anything about git
> > consistency, if the mirroring is half-way, you'll end up with something
> > less than wonderful, and confusing. Details, details)
> 
> Would it make sense to add an fsck step to git-clone-script?

Well, it's going to be slow. Of course, it's not as slow as pulling the 
stuff over a DSL line or whatever, but still..

I think I need to make something that just verifies the top <n> commits or
whatever - I need that for "pull" anyway, so that you can do a 

	git fsck ORIG_HEAD..

and it will fsck only the new stuff that arrived as a result of the pull.

And we need to improve the git-ssh-pull/git-http-pull scripts so that they
do pipelined requests: right now it's usually a lot faster to do "rsync"  
than it is to do git-ssh-pull (unless you do a small pull), because even
though the rsync ends up needing to compare the full directory contents,
it then transfers the data much faster.

		Linus

Re: Updated git HOWTO for kernel hackers

[Linus Torvalds]

On Wed, 22 Jun 2005, Linus Torvalds wrote:
> 
> A few notes on these things:
> 
> 	git-apply --index /tmp/my.patch
> 
> will not only apply the patch (unified patches only!), but will do the
> index updates for you while it's at it, so if the patch contains new files
> (or it deletes files), you don't need to worry about it.

Btw, if the patch contains rename/copy-patches or mode updates, you _need_
to use git-apply, since regular "patch" doesn't know about file modes and
can't handle file renames or copies.

Now, the rename/copy patches are easy to avoid by just not asking git to
generate them (so they'll show up as just straight file creates, with a
delete of the old file for a rename), but the file mode part in particular
is useful as more than just a way to create smaller (and more
human-readable) patches.

		Linus

Re: Updated git HOWTO for kernel hackers

[Jeff Garzik]

Linus Torvalds wrote:
> A few notes on these things:
> 
> 	git-apply --index /tmp/my.patch
> 
> will not only apply the patch (unified patches only!), but will do the
> index updates for you while it's at it, so if the patch contains new files
> (or it deletes files), you don't need to worry about it.

The output isn't terribly helpful:

[jgarzik@pretzel netdev-2.6]$ git apply --index \
	~/tmp/linux-2.6.12-rc4-cxgb2.1.1.patch
Fragment applied at offset 11

That is worse than no message at all...  fragment?  offset 11?  did it 
work?  Did it apply only a "fragment" of my patch, not the whole thing? 
  I'm worried! </mental monologue>

Outputting the following (stolen from 'git commit') would be far more 
useful:

       modified: Documentation/networking/cxgb.txt
       modified: drivers/net/chelsio/Makefile
       deleted:  drivers/net/chelsio/ch_ethtool.h
       modified: drivers/net/chelsio/common.h
       modified: drivers/net/chelsio/cphy.h
       modified: drivers/net/chelsio/cpl5_cmd.h
       modified: drivers/net/chelsio/cxgb2.c
       deleted:  drivers/net/chelsio/cxgb2.h
       modified: drivers/net/chelsio/elmer0.h
       modified: drivers/net/chelsio/espi.c
       modified: drivers/net/chelsio/espi.h
       modified: drivers/net/chelsio/gmac.h
       modified: drivers/net/chelsio/mv88x201x.c
       deleted:  drivers/net/chelsio/osdep.h
       modified: drivers/net/chelsio/pm3393.c
       modified: drivers/net/chelsio/regs.h
       modified: drivers/net/chelsio/sge.c
       modified: drivers/net/chelsio/sge.h
       modified: drivers/net/chelsio/subr.c
       modified: drivers/net/chelsio/suni1x10gexp_regs.h
       deleted:  drivers/net/chelsio/tp.c
       deleted:  drivers/net/chelsio/tp.h
       modified: include/linux/pci_ids.h


> Also, you can do
> 
> 	git commit <list-of-files-to-commit>
> 
> as a shorthand for
> 
> 	git-update-cache <list-of-files-to-commit>
> 	git commit
> 
> which some people will probably find more natural.

It would be natural if it functioned like 'bk citool' ;-)

	git commit --figure-out-for-me-what-files-changed

'git diff' can do this, so it's certainly feasible.

Obviously added/removed files would still require git-update-cache or 
git-commit<list of files>.


> "git-whatchanged" is useful if you actually want to see what the commits 
> _changed_, and then you often want to use the "-p" flag to see it as 
> patches. Also, it's worth pointing out the fact that you can limit it to 
> certain subdirectories (or individual files) etc, ie:
> 
> 	git-whatchanged -p drivers/net
> 
> since that is often what people want.
> 
> But if you just want the log, "git log" is faster and simpler and more 
> correct.

I usually want just two things:

1) browse the log

2) list changes in local tree that are not in $remote_tree, a la
	bk changes -L ../linux-2.6

I agree that seeing the merge csets is useful, that is why [being 
ignorant of 'git log'] I used git-changes-script.

	Jeff

Re: Updated git HOWTO for kernel hackers

[Linus Torvalds]

On Wed, 22 Jun 2005, Jeff Garzik wrote:
> 
> The output isn't terribly helpful:

Yeah. The good news is that if something bad happens, it surely lets you 
know, and it's very verbose about that.

I should probably remove the "Fragment applied at offset xx" thing, it was 
basically a debugging message to make sure that I applied patch fragments 
correctly even if the line offset given in the patch was sligthly off..

> Outputting the following (stolen from 'git commit') would be far more 
> useful:
> 
>        modified: Documentation/networking/cxgb.txt
>        modified: drivers/net/chelsio/Makefile
>        deleted:  drivers/net/chelsio/ch_ethtool.h
>        modified: drivers/net/chelsio/common.h
>        modified: drivers/net/chelsio/cphy.h
>        modified: drivers/net/chelsio/cpl5_cmd.h
>        modified: drivers/net/chelsio/cxgb2.c
>        deleted:  drivers/net/chelsio/cxgb2.h
>        modified: drivers/net/chelsio/elmer0.h
>        modified: drivers/net/chelsio/espi.c
>        modified: drivers/net/chelsio/espi.h
>        modified: drivers/net/chelsio/gmac.h
>        modified: drivers/net/chelsio/mv88x201x.c
>        deleted:  drivers/net/chelsio/osdep.h
>        modified: drivers/net/chelsio/pm3393.c
>        modified: drivers/net/chelsio/regs.h
>        modified: drivers/net/chelsio/sge.c
>        modified: drivers/net/chelsio/sge.h
>        modified: drivers/net/chelsio/subr.c
>        modified: drivers/net/chelsio/suni1x10gexp_regs.h
>        deleted:  drivers/net/chelsio/tp.c
>        deleted:  drivers/net/chelsio/tp.h
>        modified: include/linux/pci_ids.h

How about this patch? Then you can say

	git-apply --stat --summary --apply --index /tmp/my.patch

and it will not only apply the patch, but also give a diffstat and a
summary or renames etc..

This also removes the "Fragment.." debugging message.

Btw, "--stat" and "--summary" normally turn off the "apply" flag, so 
"--apply" has to come _after_ the stat/summary thing, fwiw:

diff --git a/apply.c b/apply.c
--- a/apply.c
+++ b/apply.c
@@ -860,7 +860,6 @@ static int find_offset(const char *buf, 
 		n = (i >> 1)+1;
 		if (i & 1)
 			n = -n;
-		fprintf(stderr, "Fragment applied at offset %d\n", n);
 		return try;
 	}
 
@@ -1434,6 +1433,10 @@ int main(int argc, char **argv)
 			check_index = 1;
 			continue;
 		}
+		if (!strcmp(arg, "--apply")) {
+			apply = 1;
+			continue;
+		}
 		if (!strcmp(arg, "--show-files")) {
 			show_files = 1;
 			continue;

> > Also, you can do
> > 
> > 	git commit <list-of-files-to-commit>
> > 
> > as a shorthand for
> > 
> > 	git-update-cache <list-of-files-to-commit>
> > 	git commit
> > 
> > which some people will probably find more natural.
> 
> It would be natural if it functioned like 'bk citool' ;-)
> 
> 	git commit --figure-out-for-me-what-files-changed
> 
> 'git diff' can do this, so it's certainly feasible.

Well, it _does_ do that. That's what the "git status" thing does, and look 
at the initial commit message comments that it prepares for you: it tells 
you which files are modified but haven't been marked for check-in etc.

But the thing is, you need to have a graphical tool for that. I don't want
to have some silly command line that asks for each modified file whether
you want to include that file in the commit or not.

So this is where "git" ends, and a nice user interface (written by 
somebody else than me) begins. Ie this is a cogito-like thing.

> > "git-whatchanged" is useful if you actually want to see what the commits 
> > _changed_, and then you often want to use the "-p" flag to see it as 
> > patches. Also, it's worth pointing out the fact that you can limit it to 
> > certain subdirectories (or individual files) etc, ie:
> > 
> > 	git-whatchanged -p drivers/net
> > 
> > since that is often what people want.
> > 
> > But if you just want the log, "git log" is faster and simpler and more 
> > correct.
> 
> I usually want just two things:
> 
> 1) browse the log
> 
> 2) list changes in local tree that are not in $remote_tree, a la
> 	bk changes -L ../linux-2.6
> 
> I agree that seeing the merge csets is useful, that is why [being 
> ignorant of 'git log'] I used git-changes-script.

For (1) "bk log" is good. For (2) you'll have to use your own script, or
just have the remote tree as a branch in the same tree, in which case you
can do

	git log remotebranch..mybranch

and it will do what you expect. In fact, since "HEAD" is the default 
branch for the final one, you can do

	git log remotebranch..

and you'll get the log of everything that is in your HEAD but it _not_ in 
the "remotebranch" branch.

Btw, if you have the remote as a branch in your own tree, you can also do

	gitk remotebranch..mybranch

which is a really nice way of graphically seeing "what is in 'mybranch' 
that is not in 'remotebranch'".

			Linus

Re: Updated git HOWTO for kernel hackers

[Adam Kropelin]

Linus Torvalds wrote:
> On Wed, 22 Jun 2005, Jeff Garzik wrote:
>> git commit --figure-out-for-me-what-files-changed
>
> Well, it _does_ do that. That's what the "git status" thing does, and
> look at the initial commit message comments that it prepares for you: 
> it
> tells you which files are modified but haven't been marked for 
> check-in
> etc.
>
> But the thing is, you need to have a graphical tool for that. I don't
> want to have some silly command line that asks for each modified file
> whether you want to include that file in the commit or not.

I know I shouldn't invoke this particular acronym, but I rather like 
CVS's approach. If the user does not specify any files on the command 
line, assume he wants to check in everything that has changed (added and 
removed files excluded). When you see the initial commit message you can 
review the list of affected files and you can always abort and specify 
files explictly if you realize you want to exclude some.

I like that method because it gives you a kick in the pants for having 
mixed multiple unrelated changes in your working directory. "Oh, you 
were lazy and changed six unrelated things without comitting, eh? You 
will now pay for your lack of rigor by typing filenames..." On the flip 
side, you get rewarded with less typing if you keep your working 
directory clean.

--Adam

Re: Updated git HOWTO for kernel hackers

[Linus Torvalds]

On Wed, 22 Jun 2005, Adam Kropelin wrote:
> 
> I know I shouldn't invoke this particular acronym, but I rather like 
> CVS's approach.

The problem I have with "git commit" committing everything dirty by
default is that it encourages exactly the wrong kind of behaviour, ie the 
"commit it all in one go without thinking about it".

Also, CVS really doesn't have much choice, since CVS doesn't _have_ the 
notion of marking files for commits. In contrast, in git the index file 
really does end up beign a good way to say which files are ready to be 
committed.

And "git status" really isn't that hard to type, and it will tell you 
exactly what you've already marked for commit, and what you have dirty in 
the tree but isn't marked for commit yet.

So I think the "git commit <file-list>" thing is very convenient, but it's
convenient exactly because it's concise yet still precise and doesn't 
encourage the "just commit whatever random dirty state I have right now" 
mentality.

And if you have more than a few files dirty in your tree, I really think
it's much better to do "git status" and think about it a bit and select
the files you do want to commit than it is to just do "git commit" and let
it rip.

Now, I could well imagine adding an "--all" flag (and not even allow the 
shorthane version) to both git-update-cache and "git commit". So that you 
could say "commit all the dirty state", but you'd at least have to think 
about it before you did so.

		Linus

Re: Updated git HOWTO for kernel hackers

[Jeff Garzik]

Linus Torvalds wrote:
> The problem I have with "git commit" committing everything dirty by
> default is that it encourages exactly the wrong kind of behaviour, ie the 
> "commit it all in one go without thinking about it".

100% agreed


> And "git status" really isn't that hard to type, and it will tell you 
> exactly what you've already marked for commit, and what you have dirty in 
> the tree but isn't marked for commit yet.

Having found about it recently, 'git status' is quite useful.


> So I think the "git commit <file-list>" thing is very convenient, but it's
> convenient exactly because it's concise yet still precise and doesn't 
> encourage the "just commit whatever random dirty state I have right now" 
> mentality.
> 
> And if you have more than a few files dirty in your tree, I really think
> it's much better to do "git status" and think about it a bit and select
> the files you do want to commit than it is to just do "git commit" and let
> it rip.

For me at least, providing a file list is a pain, because I am so 
precise [read: obsessive] about keeping an otherwise clean working dir 
:)  Except in rare occasions, I know precisely that the changes in the 
working dir comprise 100% of what I plan to commit.

Locally I have scripted

      git-diff-cache -p HEAD | diffstat -p1 | awk '{print $1}' > /tmp/lst
      git-update-cache `cat /tmp/lst`

because of this.

[again, clearly doesn't work with remove/add/mode change]


> Now, I could well imagine adding an "--all" flag (and not even allow the 
> shorthane version) to both git-update-cache and "git commit". So that you 
> could say "commit all the dirty state", but you'd at least have to think 
> about it before you did so.

That's pretty much what I suggested when I said

	git commit --figure-out-for-me-what-files-changed

:)

So I certainly agree there.

	Jeff

Re: Updated git HOWTO for kernel hackers

[Linus Torvalds]

On Thu, 23 Jun 2005, Jeff Garzik wrote:
> 
> Locally I have scripted
> 
>       git-diff-cache -p HEAD | diffstat -p1 | awk '{print $1}' > /tmp/lst
>       git-update-cache `cat /tmp/lst`
> 
> because of this.

Btw, that's some extremely convoluted computation.

This is exactly when you do _not_ want the diff in "patch" form, and you
really want the native git format (which is just a strange "this file
changed from this mode/sha1 to that mode/sha1" format).

So instead, try to do just

	git-diff-cache HEAD | cut -f2

and now it's going to be a whole lot simpler and faster - it won't turn 
things into a diff only to do a "diffstat" on it to turn it into a name 
again. I bet it's more reliable too.

> [again, clearly doesn't work with remove/add/mode change]

Well, it actually can work with removes, and rewriting it to be a bit 
more clean (and handle files that start with "-") gives you:

	git-update-cache --remove -- $(git-diff-cache HEAD | cut -f2)

which should actually work fine for files that you have removed. But yes,
it fundamentally _cannot_ work for new files, of course, since git will
never even try to look for files you haven't told it about. So you always 
have to add files by hand some way.

Note how the "--remove" parameter to git-update-cache really only means
"it's ok if some of the files mentioned don't exist any more, and that
means you should remove them from the cache".

Without the "--remove" flag, a filename that is listed but that doesn't
exist in the working tree is either considered an error, or is ignored
(depending on the "--ignore-missing" flag).

That's actually what "--add" means too: it means "it's ok if some of the
filenames on the command line don't currently exist in the index: if they
exist in the working directory, you should add them".

So even if it looks a bit strange, in a script it actually makes perfect
sense to write something that seems as _apparently_ senseless as:

	git-update-cache --add --remove --refresh -- "$@"

and it will refresh all existing files, and add or remove any files 
explicitly mentioned that either exist or have been removed in the working 
directory.

			Linus

Re: Updated git HOWTO for kernel hackers

[Miles Bader]

Linus Torvalds <torvalds@osdl.org> writes:
> And if you have more than a few files dirty in your tree, I really think
> it's much better to do "git status" and think about it a bit and select
> the files you do want to commit than it is to just do "git commit" and let
> it rip.
>
> Now, I could well imagine adding an "--all" flag (and not even allow the 
> shorthane version) to both git-update-cache and "git commit". So that you 
> could say "commit all the dirty state", but you'd at least have to think 
> about it before you did so.

I think both modes of operation are useful -- sometimes I want to hack
in the tree and later decide what to commit, and sometimes I know
exactly what sequence of commits I want to make and do a series of
"change-some-files then commit everything" steps.

In the latter case, it's very convenient to have commit just grab
everything and clear the slate for my next step.  Morever, I use the
latter style enough that I think even the requirement of a long option
seems annoying and artificial; a short option would be fine though...

-Miles
-- 
Any man who is a triangle, has thee right, when in Cartesian Space, to
have angles, which when summed, come to know more, nor no less, than
nine score degrees, should he so wish.  [TEMPLE OV THEE LEMUR]

Re: Updated git HOWTO for kernel hackers

[Jeff Garzik]

[-- Attachment #1: Type: text/plain, Size: 1427 bytes --]

Linus Torvalds wrote:
> How about this patch? Then you can say
> 
> 	git-apply --stat --summary --apply --index /tmp/my.patch
> 
> and it will not only apply the patch, but also give a diffstat and a
> summary or renames etc..

Quite nice.



>>I usually want just two things:
>>
>>1) browse the log
>>
>>2) list changes in local tree that are not in $remote_tree, a la
>>	bk changes -L ../linux-2.6
>>
>>I agree that seeing the merge csets is useful, that is why [being 
>>ignorant of 'git log'] I used git-changes-script.
> 
> 
> For (1) "bk log" is good.

Chuckle.  What does one call a Freudian slip, in computer-land?


> For (2) you'll have to use your own script, or
> just have the remote tree as a branch in the same tree, in which case you
> can do
> 
> 	git log remotebranch..mybranch

Very neat.  That makes some things a bit easier, since I usually carry a 
'vanilla' branch as .git/refs/heads/master, and do all my modifications 
on other branches.

FWIW, git-changes-script (attached) facilitates #2 for me right now.  I 
use it just like BK's '-L' feature:

	cd netdev-2.6
	git checkout -f ieee80211
	git-changes-script -L ../linux-2.6 | less

That will produce the same output as the feature you just taught me,

	git log master..ieee80211

WARNING:  You have previously called git-changes-script quite ugly (not 
surprising), and this 'git log x..y' will probably replace it in my 
usage, long term.

	Jeff



[-- Attachment #2: git-changes-script --]
[-- Type: text/plain, Size: 2373 bytes --]

#!/bin/bash
#
# Make a log of changes in a GIT branch.
#
# This script was originally written by (c) Ross Vandegrift.
# Adapted to his scripts set by (c) Petr Baudis, 2005.
# Major optimizations by (c) Phillip Lougher.
# Rendered trivial by Linus Torvalds.
# Added -L|-R option by James Bottomley
#
# options:
# script [-L <dir> | -R <dir> |-r <from_sha1> [ -r <to_sha1] ] [<sha1>]
#
# With no options shows all the revisions from HEAD to the root
# -L shows all the changes in the local tree compared to the tree at <dir>
# -R shows all the changes in the remote tree at <dir> compared to the local
# -r shows all the changes in one commit or between two

tmpfile=/tmp/git_changes.$$
r1=
r2=

showcommit() {
	commit="$1"
	echo commit ${commit%:*};
	git-cat-file commit $commit | \
		while read key rest; do
			case "$key" in
			"author"|"committer")
				date=(${rest#*> })
				sec=${date[0]}; tz=${date[1]}
				dtz=${tz/+/+ }; dtz=${dtz/-/- }
				pdate="$(date -Rud "1970-01-01 UTC + $sec sec $dtz" 2>/dev/null)"
				if [ "$pdate" ]; then
					echo $key $rest | sed "s/>.*/> ${pdate/+0000/$tz}/"
				else
					echo $key $rest
				fi
				;;
			"")
				echo; cat
				;;
			*)
				echo $key $rest
				;;
			esac

		done
}

while true; do
	case "$1" in
		-R)	shift;
			diffsearch=+
			remote="$1"
			shift;;
		-L)	shift;
			diffsearch=-
			remote="$1"
			shift;;
		-r)	shift;
			if [ -z "$r1" ]; then
				r1="$1"
			else
				r2="$1"
			fi
			shift;;
		*)	base="$1"
			break;;
	esac
done

if [ -n "$r1" ]; then
	if [ -z "$r2" ]; then
		showcommit $r1
		exit 0
	fi
	diffsearch=+
	remote=`pwd`;
	tobase="$r2";
	base="$r1"
fi
	
if [ -z "$base" ]; then
	base=$(cat .git/HEAD) || exit 1
fi

git-rev-tree $base | sort -rn  > ${tmpfile}.base
if [ -n "$remote" ]; then
	[ -d $remote/.git ] || exit 1
	if [ -z "$tobase" ]; then
		tobase=$(cat $remote/.git/HEAD) || exit 1
	fi
	pushd $remote > /dev/null
	git-rev-tree $tobase | sort -rn > ${tmpfile}.remote
	diff -u ${tmpfile}.base ${tmpfile}.remote | grep "^${diffsearch}[^${diffsearch}]" | cut -c 1- > ${tmpfile}.diff
	rm -f ${tmpfile}.base ${tmpfile}.remote
	mv ${tmpfile}.diff ${tmpfile}.base
	if [ $diffsearch = "-" ]; then
		popd > /dev/null
	fi
fi

[ -s "${tmpfile}.base" ] || exit 0

cat ${tmpfile}.base | while read time commit parents; do
	showcommit $commit
	echo -e "\n--------------------------"

done
rm -f ${tmpfile}.base

Re: Updated git HOWTO for kernel hackers

[Linus Torvalds]

On Thu, 23 Jun 2005, Jeff Garzik wrote:
> 
> Chuckle.  What does one call a Freudian slip, in computer-land?

A "Knuthian slip"?

> WARNING:  You have previously called git-changes-script quite ugly (not 
> surprising), and this 'git log x..y' will probably replace it in my 
> usage, long term.

Even short-term, you could actually make it prettier. 

You can actually use git across multiple directories by setting the
GIT_ALTERNATE_OBJECT_DIRECTORIES environment variable to point to the
alternate ones, so you should be able to do a "compare with remote" with 
something like this:

	export GIT_ALTERNATE_OBJECT_DIRECTORIES=$remote/.git/objects
	remote_head=$(cat $remote/.git/HEAD)
	git log $remote_head..

which should literally give a nice log of what is in your HEAD but not in 
$remote_head. And if you want to see it the other way? Just change the 
last line to

	git log HEAD..$remote_head

and voila, you're done.

The nice thing about this approach is that this works with other git
programs too, ie you can replace "git log" with "gitk", and suddenly you
see graphically the commits that are in your tree but not in the remote
HEAD or vice versa.

Yeah, yeah, totally untested and maybe I'm talking through by *ss, but it 
should work in theory.

		Linus

Re: Updated git HOWTO for kernel hackers

[Anton Altaparmakov]

On Wed, 22 Jun 2005, Linus Torvalds wrote:
> On Wed, 22 Jun 2005, Jeff Garzik wrote:
> > 2) download a linux kernel tree for the very first time
> > 
> > $ mkdir -p linux-2.6/.git
> > $ cd linux-2.6
> > $ rsync -a --delete --verbose --stats --progress \
> > rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ 
> > \          <- word-wrapped backslash; sigh
> >      .git/
> 
> Gaah. I should do a "git-clone-script" or something that does this, and 
> then you could just do
> 
> 	git clone rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git linux-2.6
> 	
> Anybody?

What's wrong with Pasky's cogito scripts?  There is a cg-pull as well as a 
cg-clone in there already.  If nothing else you could just copy the 
relevant scripts and rename them to git-blah...

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

Re: Updated git HOWTO for kernel hackers

[Daniel Barkalow]

On Wed, 22 Jun 2005, Jeff Garzik wrote:

> 5) check in your own modifications (e.g. do some hacking, or apply a patch)
> 
> # go to repo
> $ cd linux-2.6
> 
> # make some modifications
> $ patch -sp1 < /tmp/my.patch
> $ diffstat -p1 < /tmp/my.patch
> 
> # NOTE: add '--add' and/or '--remove' if files were added or removed
> $ git-update-cache <list of all files changed>

There's actually "git add" for when you add a file (if you're actually
developing with git, rather than just applying patching with it). No
script, so far as I can tell, for removing a file, though.

	-Daniel
*This .sig left intentionally blank*

Re: Updated git HOWTO for kernel hackers

[Dave Airlie]

> jgarzik helper scripts, not in official git distribution:
> http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-new-branch
> http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-changes-script
> 

Do you have some sort of magic bk-make-sum type util at all... 

For sending trees to Linus I used to run bk-make-sum and gcapatch and
then just throw my own stuff in the top of the mail ....

I'm being lazy I probably could write it myself, but bk-make-sum was a
very useful script for me...

Dave.

Mercurial vs Updated git HOWTO for kernel hackers

[Matt Mackall]

On Wed, Jun 22, 2005 at 06:24:54PM -0400, Jeff Garzik wrote:
> 
> Things in git-land are moving at lightning speed, and usability has 
> improved a lot since my post a month ago:  http://lkml.org/lkml/2005/5/26/11

And here's a quick comparison with the current state of Mercurial..

> 1) installing git
> 
> git requires bootstrapping, since you must have git installed in order 
> to check out git.git (git repo), and linux-2.6.git (kernel repo).  I 
> have put together a bootstrap tarball of today's git repository.
> 
> Download tarball from:
> http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-20050622.tar.bz2
> 
> tarball build-deps:  zlib, libcurl, libcrypto (openssl)
> 
> install tarball:  unpack && make && sudo make prefix=/usr/local install
> 
> jgarzik helper scripts, not in official git distribution:
> http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-new-branch
> http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-changes-script
> 
> After reading the rest of this document, come back and update your copy 
> of git to the latest:
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git.git

Download from: http://selenic.com/mercurial/mercurial-snapshot.tar.gz
Build-deps: Python 2.3
Install: unpack && python setup.py install [--home=/usr/local]

> 2) download a linux kernel tree for the very first time
> 
> $ mkdir -p linux-2.6/.git
> $ cd linux-2.6
> $ rsync -a --delete --verbose --stats --progress \
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ 
> \          <- word-wrapped backslash; sigh
>     .git/

$ mkdir linux-2.6
$ cd linux-2.6
$ hg init http://www.kernel.org/hg/    # obviously you can also browse this

This downloads about 125M of data, which include the whole kernel history
back to 2.4.0 and everything in Linus' git repo as well.

> 3) update local kernel tree to latest 2.6.x upstream ("fast-forward merge")
> 
> $ cd linux-2.6
> $ git-pull-script \
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git

$ hg pull        # defaults to where you originally pulled from

It takes about 4M of transfer and well under a minute to pull the
entire git history, starting from a base of 2.6.12-rc2.

> 4) check out files from the git repository into the working directory
> 
> $ git checkout -f

$ hg update      # or up or checkout or co, depending on your SCM habits

> 5) check in your own modifications (e.g. do some hacking, or apply a patch)
> 
> # go to repo
> $ cd linux-2.6
> 
> # make some modifications
> $ patch -sp1 < /tmp/my.patch
> $ diffstat -p1 < /tmp/my.patch
> 
> # NOTE: add '--add' and/or '--remove' if files were added or removed
> $ git-update-cache <list of all files changed>
> 
> # check in changes
> $ git commit

$ hg commit [files]    # check in everything changed or just the named files

5.1) undo the last commit or pull

$ hg undo

> 6) List all changes in working dir, in diff format.
> 
> $ git-diff-cache -p HEAD

$ hg status            # show changed files

> 7) List all changesets (i.e. show each cset's description text) in local 
> branch of local tree, that are not present in remote tree.
> 
> $ cd my-kernel-tree-2.6
> $ git-changes-script -L ../linux-2.6 | less

$ hg history | less         # How does git know what's not in the
                            # remote tree? Psychic?

> 8) List all changesets:
> 
> $ git-whatchanged

$ hg history | less

> 9) apply all patches in a Berkeley mbox-format file
> 
> First, download and add to your PATH Linus's git tools:
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git-tools.git
> 
> $ cd my-kernel-tree-2.6
> $ dotest /path/to/mbox  # yes, Linus has no taste in naming scripts

hg doesn't do mboxes directly, but you can do:

$ cat patch-list | xargs hg import

> 10) don't forget to download tags from time to time.
> 
> git-pull-script only downloads sha1-indexed object data, and the 
> requested remote head.  This misses updates to the .git/refs/tags/ and 
> .git/refs/heads directories.  It is advisable to update your kernel .git 
> directories periodically with a full rsync command, to make sure you got 
> everything:
>
> $ cd linux-2.6
> $ rsync -a --delete --verbose --stats --progress \
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
> \          <- word-wrapped backslash; sigh
>     .git/

Tags in mercurial are properly version controlled and come along for
the ride with pulls. Also, the right thing happens with merges.
 
> 11) list all branches, such as those found in my netdev-2.6 or 
> libata-dev trees.
> 
> Download
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/jgarzik/netdev-2.6.git
> 	or
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/jgarzik/libata-dev.git
> 
> 
> $ cd netdev-2.6
> $ ls .git/refs/heads/
> 
> { these are the current netdev-2.6 branches }
> >8139cp       forcedeth    master     qeth           smc91x         we18
> >8139too-iomap  for-linus    natsemi      r8169      smc91x-eeprom  wifi
> >airo           hdlc         ns83820      register-netdev  starfire
> >atmel          ieee80211    orinoco      remove-drivers   tlan
> >chelsio        iff-running  orinoco-hch  sis900           veth
> >dm9000         janitor      ppp          skge             viro

$ hg heads   # Has Andrew mentioned your git forest gives him a headache?

> 12) make desired branch current in working directory
> 
> $ git checkout -f $branch

$ hg update -C <rev or id or tag>

> 13) create a new branch, and make it current
> 
> $ cp .git/refs/heads/master .git/refs/heads/my-new-branch-name
> $ git checkout -f my-new-branch-name

Since the hg repo is lightweight, this is usually done by just having
different directories. Thus we don't explicitly name branches.

$ mkdir new-branch
$ cd new-branch
$ hg init -u ../linux   # makes hardlinks and does a checkout

> 14) examine which branch is current
> 
> $ ls -l .git/HEAD

$ echo $PWD

> 15) undo all local modifications (same as checkout):
> 
> $ git checkout -f

$ hg update -C

> 16) obtain a diff between current branch, and master branch
> 
> In most trees WITH BRANCHES, .git/refs/heads/master contains the current 
> 'vanilla' upstream tree, for easy diffing and merging.  (in trees 
> without branches, 'master' simply contains your latest changes)
> 
> $ git-diff-tree -p master HEAD

$ hg diff -r <rev> -r <rev> 

17) run a browsable, pullable repo server of the current repo on your
local machine

$ hg serve

18) push your changes to a remote server

$ hg push ssh://user@host/path/  # aliases and defaults in .hgrc

19) get per-file history

$ hg log <file> | less

20) get annotated file contents

$ hg annotate [file]

21) record that a file has been copied or renamed for the next commit

$ hg copy <source> <dest>

22) get online help

$ hg help [command]


More info at http://selenic.com/mercurial/

-- 
Mathematics is the supreme nostalgia of our time.

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Petr Baudis]

Dear diary, on Fri, Jun 24, 2005 at 01:56:34AM CEST, I got a letter
where Matt Mackall <mpm@selenic.com> told me that...
> On Wed, Jun 22, 2005 at 06:24:54PM -0400, Jeff Garzik wrote:
> > 
> > Things in git-land are moving at lightning speed, and usability has 
> > improved a lot since my post a month ago:  http://lkml.org/lkml/2005/5/26/11
> 
> And here's a quick comparison with the current state of Mercurial..

And here's a quick back-comparison with Cogito. ;-)

> > 1) installing git
> > 
> > git requires bootstrapping, since you must have git installed in order 
> > to check out git.git (git repo), and linux-2.6.git (kernel repo).  I 
> > have put together a bootstrap tarball of today's git repository.
> > 
> > Download tarball from:
> > http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-20050622.tar.bz2
> > 
> > tarball build-deps:  zlib, libcurl, libcrypto (openssl)
> > 
> > install tarball:  unpack && make && sudo make prefix=/usr/local install
> > 
> > jgarzik helper scripts, not in official git distribution:
> > http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-new-branch
> > http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-changes-script
> > 
> > After reading the rest of this document, come back and update your copy 
> > of git to the latest:
> > rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git.git
> 
> Download from: http://selenic.com/mercurial/mercurial-snapshot.tar.gz
> Build-deps: Python 2.3
> Install: unpack && python setup.py install [--home=/usr/local]

Download from: http://www.kernel.org/pub/software/scm/cogito/
Deps: git's + bash + reasonable shell environment
Install: edit Makefile, make + make install

> > 2) download a linux kernel tree for the very first time
> > 
> > $ mkdir -p linux-2.6/.git
> > $ cd linux-2.6
> > $ rsync -a --delete --verbose --stats --progress \
> > rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ 
> > \          <- word-wrapped backslash; sigh
> >     .git/
> 
> $ mkdir linux-2.6
> $ cd linux-2.6
> $ hg init http://www.kernel.org/hg/    # obviously you can also browse this

$ cg-clone \
rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/

(that will checkout to linux-2.6/ directory; you can specify the target
directory as the optional second parameter)

> > 3) update local kernel tree to latest 2.6.x upstream ("fast-forward merge")
> > 
> > $ cd linux-2.6
> > $ git-pull-script \
> > rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
> 
> $ hg pull        # defaults to where you originally pulled from

$ cg-update	# defaults to where you originally pulled from

(cg-pull just gets the changes to your repository, but won't merge them
into your branch)

> > 4) check out files from the git repository into the working directory
> > 
> > $ git checkout -f
> 
> $ hg update      # or up or checkout or co, depending on your SCM habits

In Cogito, all files are always checked out.

> > 5) check in your own modifications (e.g. do some hacking, or apply a patch)
> > 
> > # go to repo
> > $ cd linux-2.6
> > 
> > # make some modifications
> > $ patch -sp1 < /tmp/my.patch
> > $ diffstat -p1 < /tmp/my.patch
> > 
> > # NOTE: add '--add' and/or '--remove' if files were added or removed
> > $ git-update-cache <list of all files changed>
> > 
> > # check in changes
> > $ git commit
> 
> $ hg commit [files]    # check in everything changed or just the named files

$ cg-commit [-m"Message"...] [files] # check in everything changed or just
                                     # the named files

If you pass multiple -m arguments, they get formatted as separate
paragraphs in the log message. It is customary for the first -m argument
to contain a short one-line summary.

Note that you must add/remove files by

$ cg-add files...

and

$ cg-rm files...

> 5.1) undo the last commit or pull
> 
> $ hg undo

$ cg-admin-uncommit

Note that you should never do this if you already pushed the changes
out, or someone might get them. (That holds for regular Git too.) See

$ cg-help cg-admin-uncommit   # (or cg-admin-uncommit --help)

for details. (That's another Cogito's cool feature. Handy docs! ;-)

> > 6) List all changes in working dir, in diff format.
> > 
> > $ git-diff-cache -p HEAD
> 
> $ hg status            # show changed files

$ cg-status		# show changed files
$ cg-diff [-c] [files]	# show the diffs, -c colourfully

> > 7) List all changesets (i.e. show each cset's description text) in local 
> > branch of local tree, that are not present in remote tree.
> > 
> > $ cd my-kernel-tree-2.6
> > $ git-changes-script -L ../linux-2.6 | less
> 
> $ hg history | less         # How does git know what's not in the
>                             # remote tree? Psychic?

# -c colourfully, -s prints only summaries, one line per changeset
$ cg-log [-c] [-s] -m -r linux-2.6 # List changes only in linux-2.6

Note that | less is unnecessary (even undesirable with -c).

> > 8) List all changesets:
> > 
> > $ git-whatchanged
> 
> $ hg history | less

$ cg-log [-c] [-s]

8.1) List all changesets in the origin branch:

$ cg-log [-c] [-s] -r origin

8.2) List all changesets concerning files CREDITS and fs/inode.c:

$ cg-log [-c] [-s] CREDITS fs/inode.c

> > 9) apply all patches in a Berkeley mbox-format file
> > 
> > First, download and add to your PATH Linus's git tools:
> > rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git-tools.git
> > 
> > $ cd my-kernel-tree-2.6
> > $ dotest /path/to/mbox  # yes, Linus has no taste in naming scripts
> 
> hg doesn't do mboxes directly, but you can do:
> 
> $ cat patch-list | xargs hg import

Theoretically, dotest should work just fine even if you use Cogito.
Anyone tested it?

> > 10) don't forget to download tags from time to time.
> > 
> > git-pull-script only downloads sha1-indexed object data, and the 
> > requested remote head.  This misses updates to the .git/refs/tags/ and 
> > .git/refs/heads directories.  It is advisable to update your kernel .git 
> > directories periodically with a full rsync command, to make sure you got 
> > everything:
> >
> > $ cd linux-2.6
> > $ rsync -a --delete --verbose --stats --progress \
> > rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
> > \          <- word-wrapped backslash; sigh
> >     .git/
> 
> Tags in mercurial are properly version controlled and come along for
> the ride with pulls. Also, the right thing happens with merges.

cg-update and cg-pull takes fetches new tags during a pull.

> > 11) list all branches, such as those found in my netdev-2.6 or 
> > libata-dev trees.
> > 
> > Download
> > rsync://rsync.kernel.org/pub/scm/linux/kernel/git/jgarzik/netdev-2.6.git
> > 	or
> > rsync://rsync.kernel.org/pub/scm/linux/kernel/git/jgarzik/libata-dev.git
> > 
> > 
> > $ cd netdev-2.6
> > $ ls .git/refs/heads/
> > 
> > { these are the current netdev-2.6 branches }
> > >8139cp       forcedeth    master     qeth           smc91x         we18
> > >8139too-iomap  for-linus    natsemi      r8169      smc91x-eeprom  wifi
> > >airo           hdlc         ns83820      register-netdev  starfire
> > >atmel          ieee80211    orinoco      remove-drivers   tlan
> > >chelsio        iff-running  orinoco-hch  sis900           veth
> > >dm9000         janitor      ppp          skge             viro
> 
> $ hg heads   # Has Andrew mentioned your git forest gives him a headache?

$ cg-branch-ls

# Note that Cogito supports only remote  branches properly now; that
# will yet evolve (in some backwards-compatible way).

> > 12) make desired branch current in working directory
> > 
> > $ git checkout -f $branch
> 
> $ hg update -C <rev or id or tag>

You can check the desired branch out into another directory:

$ cg-clone path/to/linux-2.6/.git#branch anotherdir

Switching branches in place will be supported soon (although I have
doubts about its usefulness).

> > 13) create a new branch, and make it current
> > 
> > $ cp .git/refs/heads/master .git/refs/heads/my-new-branch-name
> > $ git checkout -f my-new-branch-name
> 
> Since the hg repo is lightweight, this is usually done by just having
> different directories. Thus we don't explicitly name branches.
> 
> $ mkdir new-branch
> $ cd new-branch
> $ hg init -u ../linux   # makes hardlinks and does a checkout

$ mkdir new-branch
$ cd new-branch
$ cg-clone -s ../linux-2.6

(Note that cg-clone given local path will do hardlinks too.)

We don't explicitly name branches either. You can make the branch
visible from the other tree by

$ cg-branch-add new-branch ../new-branch

and then refer to it as new-branch.

> > 14) examine which branch is current
> > 
> > $ ls -l .git/HEAD
> 
> $ echo $PWD

Always the "master" branch.

> > 15) undo all local modifications (same as checkout):
> > 
> > $ git checkout -f
> 
> $ hg update -C

$ cg-cancel

> > 16) obtain a diff between current branch, and master branch
> > 
> > In most trees WITH BRANCHES, .git/refs/heads/master contains the current 
> > 'vanilla' upstream tree, for easy diffing and merging.  (in trees 
> > without branches, 'master' simply contains your latest changes)
> > 
> > $ git-diff-tree -p master HEAD
> 
> $ hg diff -r <rev> -r <rev> 

$ cg-diff -r <rev> -r <rev>

> 17) run a browsable, pullable repo server of the current repo on your
> local machine
> 
> $ hg serve

Make it accessible over HTTP, SSH, rsync, or for the local users if you
just want them to access it.

> 18) push your changes to a remote server
> 
> $ hg push ssh://user@host/path/  # aliases and defaults in .hgrc

Will be supported Real Soon (tm) (well, probably sometimes next week).

> 19) get per-file history
> 
> $ hg log <file> | less

$ cg-log [-c] [-s] <file>

> 20) get annotated file contents
> 
> $ hg annotate [file]

Planned.

> 22) get online help
> 
> $ hg help [command]

$ cg-help [command]

Cool. Except where the concepts are just different, Cogito mostly
appears at least equally simple to use as Mercurial. Yes, some features
are missing yet. I hope to fix that soon. :-)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
<Espy> be careful, some twit might quote you out of context..

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Christopher Li]

On Fri, Jun 24, 2005 at 08:41:01AM +0200, Petr Baudis wrote:
> > 5.1) undo the last commit or pull
> > 
> > $ hg undo
> 
> $ cg-admin-uncommit
> 
> Note that you should never do this if you already pushed the changes
> out, or someone might get them. (That holds for regular Git too.) See
> 
> $ cg-help cg-admin-uncommit   # (or cg-admin-uncommit --help)
> 
> for details. (That's another Cogito's cool feature. Handy docs! ;-)
> 

Does it still works if the last commit was interrupted  or due to error for some
reason?  Undo pull is pretty cool because you might pull a lot of commit
in one blow. Get rid of commit one by one is going to be painful. Some times
the object you pull has more than one chain of history it will be very nasty
if you want to clean it up.

Mercurial's undo is taking a snapshot of all the changed file's repo file length
at every commit or pull.  It just truncate the file to original size and undo 
is done.

Chris

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Petr Baudis]

Dear diary, on Fri, Jun 24, 2005 at 02:38:19PM CEST, I got a letter
where Christopher Li <hg@chrisli.org> told me that...
> On Fri, Jun 24, 2005 at 08:41:01AM +0200, Petr Baudis wrote:
> > > 5.1) undo the last commit or pull
> > > 
> > > $ hg undo
> > 
> > $ cg-admin-uncommit
> > 
> > Note that you should never do this if you already pushed the changes
> > out, or someone might get them. (That holds for regular Git too.) See
> > 
> > $ cg-help cg-admin-uncommit   # (or cg-admin-uncommit --help)
> > 
> > for details. (That's another Cogito's cool feature. Handy docs! ;-)
> > 
> 
> Does it still works if the last commit was interrupted  or due to error for some
> reason?

If the last commit was interrupted, it didn't happen, so your tree stays
in the same state as before doing the commit, as well as the repository.
You can just try again. If you want to get rid of dirty stuff,
cg-cancel.

> Undo pull is pretty cool because you might pull a lot of commit
> in one blow. Get rid of commit one by one is going to be painful. Some times
> the object you pull has more than one chain of history it will be very nasty
> if you want to clean it up.

If it was a tree merge, cg-admin-uncommit will undo it. If it was
fast-forward merge, there is no direct way to uncommit it, but you can
find the first fast-forwarded commit and pass it as argument to
cg-admin-uncommit; it will then rewind all the commits up to (including)
the given commit.

> Mercurial's undo is taking a snapshot of all the changed file's repo file length
> at every commit or pull.  It just truncate the file to original size and undo 
> is done.

"Trunactes"? That sounds very wrong... you mean replace with old
version? Anyway, what if the file has same length? It just doesn't make
much sense to me.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
<Espy> be careful, some twit might quote you out of context..

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Andrew Thompson]

[-- Attachment #1: Type: text/plain, Size: 585 bytes --]

Petr Baudis wrote:
>>Mercurial's undo is taking a snapshot of all the changed file's repo file length
>>at every commit or pull.  It just truncate the file to original size and undo 
>>is done.
> 
> "Trunactes"? That sounds very wrong... you mean replace with old
> version? Anyway, what if the file has same length? It just doesn't make
> much sense to me.

I believe this works because the files stored in a binary format that 
appends new changesets onto the end. Thus, truncating the new stuff from 
the end effectively removes the commit.

-- 
Andrew Thompson
http://aktzero.com/

[-- Attachment #2: andrewkt.vcf --]
[-- Type: text/x-vcard, Size: 153 bytes --]

begin:vcard
fn:Andrew Thompson
n:Thompson;Andrew
email;internet:andrewkt@aktzero.com
x-mozilla-html:FALSE
url:http://aktzero.com/
version:2.1
end:vcard


[-- Attachment #3: Type: text/plain, Size: 139 bytes --]

_______________________________________________
Mercurial mailing list
Mercurial@selenic.com
http://selenic.com/mailman/listinfo/mercurial

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Petr Baudis]

Dear diary, on Tue, Jun 28, 2005 at 05:10:47PM CEST, I got a letter
where Andrew Thompson <andrewkt@aktzero.com> told me that...
> Petr Baudis wrote:
> >>Mercurial's undo is taking a snapshot of all the changed file's repo file 
> >>length
> >>at every commit or pull.  It just truncate the file to original size and 
> >>undo is done.
> >
> >"Trunactes"? That sounds very wrong... you mean replace with old
> >version? Anyway, what if the file has same length? It just doesn't make
> >much sense to me.
> 
> I believe this works because the files stored in a binary format that 
> appends new changesets onto the end. Thus, truncating the new stuff from 
> the end effectively removes the commit.

Yes, I'm sorry - I missed the "repo" part and thought that was what it
was doing with the checked out files. ;-)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
<Espy> be careful, some twit might quote you out of context..

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Horst von Brand]

Andrew Thompson <andrewkt@aktzero.com> wrote:
> Petr Baudis wrote:

> >>Mercurial's undo is taking a snapshot of all the changed file's repo
> >>file length at every commit or pull.  It just truncate the file to
> >>original size and undo is done.

> > "Trunactes"? That sounds very wrong... you mean replace with old
> > version? Anyway, what if the file has same length? It just doesn't make
> > much sense to me.

> I believe this works because the files stored in a binary format that
> appends new changesets onto the end. Thus, truncating the new stuff
> from the end effectively removes the commit.

And is exactly the wrong way around. Even RCS stored the _last_ version and
differences to earlier ones (you'll normally want the last one (or
something near), and so occasionally having to reconstruct earlier ones by
going back isn't a big deal; having to build up the current version by
starting from /dev/null and applying each and every patch that ever touched
the file each time is expensive given enough history, besides that any
error in the file is guaranteed to destroy the current version, not
(hopefully) just making old versions unavailable).  It also means that
losing old history (what you'll want to do once in a while, e.g. forget
everything before 2.8) is simple: Chop off at the right point.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Christopher Li]

On Tue, Jun 28, 2005 at 05:54:14PM -0400, Horst von Brand wrote:
> 
> And is exactly the wrong way around. Even RCS stored the _last_ version and
> differences to earlier ones (you'll normally want the last one (or
> something near), and so occasionally having to reconstruct earlier ones by
> going back isn't a big deal; having to build up the current version by
> starting from /dev/null and applying each and every patch that ever touched
> the file each time is expensive given enough history, besides that any

Mercurial store a full text node when it detect the delta gets too long
to reach certain point. So what you describe here will not happen on
mercurial. Having it append only is a very nice feature.

> error in the file is guaranteed to destroy the current version, not
> (hopefully) just making old versions unavailable).  It also means that
> losing old history (what you'll want to do once in a while, e.g. forget
> everything before 2.8) is simple: Chop off at the right point.

You can still chop of the history before the full node, but rebuilding the
repositories. Mercurial save some much space that you would wonder why do you
what to chop the history if you can keep it.

Chris

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Kyle Moffett]

On Jun 28, 2005, at 17:54:14, Horst von Brand wrote:
> Andrew Thompson <andrewkt@aktzero.com> wrote:
>> I believe this works because the files stored in a binary format that
>> appends new changesets onto the end. Thus, truncating the new stuff
>> from the end effectively removes the commit.
>
> And is exactly the wrong way around. Even RCS stored the _last_  
> version and
> differences to earlier ones (you'll normally want the last one (or
> something near), and so occasionally having to reconstruct earlier  
> ones by
> going back isn't a big deal; having to build up the current version by
> starting from /dev/null and applying each and every patch that ever  
> touched
> the file each time is expensive given enough history, besides that any
> error in the file is guaranteed to destroy the current version, not
> (hopefully) just making old versions unavailable).  It also means that
> losing old history (what you'll want to do once in a while, e.g.  
> forget
> everything before 2.8) is simple: Chop off at the right point.

If we have versions A through A+N, Mercurial will create a new revlog  
file and
store a new full version when the total size of the changes between A  
and A+N
is greater than a certain amount, effectively ensuring that  
retrieving the
latest version of a file is O(size-of-file) instead of O(size-of- 
file*revisions).
This is the same speed as RCS for the tip, and significantly faster  
than RCS
for non-tip, which is crucial for merges.

Cheers,
Kyle Moffett

--
There are two ways of constructing a software design. One way is to  
make it so simple that there are obviously no deficiencies. And the  
other way is to make it so complicated that there are no obvious  
deficiencies.
   -- C.A.R. Hoare

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Matt Mackall]

On Tue, Jun 28, 2005 at 05:00:27PM +0200, Petr Baudis wrote:
> > Mercurial's undo is taking a snapshot of all the changed file's repo file length
> > at every commit or pull.  It just truncate the file to original size and undo 
> > is done.
> 
> "Trunactes"? That sounds very wrong... you mean replace with old
> version? Anyway, what if the file has same length? It just doesn't make
> much sense to me.

Everything in Mercurial is an append-only log. A transaction journal
records the original length of each log so that it can be restored on
failure.

-- 
Mathematics is the supreme nostalgia of our time.

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Kyle Moffett]

On Jun 28, 2005, at 14:01:57, Matt Mackall wrote:
> Everything in Mercurial is an append-only log. A transaction journal
> records the original length of each log so that it can be restored on
> failure.

Does this mean that (excepting the "undo" feature) one could set the
ext3 "append-only" attribute on the repository files to avoid losing
data due to user account compromise?

Cheers,
Kyle Moffett

--
I lost interest in "blade servers" when I found they didn't throw  
knives at people who weren't supposed to be in your machine room.
   -- Anthony de Boer

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Sean]

On Tue, June 28, 2005 4:27 pm, Kyle Moffett said:
> On Jun 28, 2005, at 14:01:57, Matt Mackall wrote:
>> Everything in Mercurial is an append-only log. A transaction journal
>> records the original length of each log so that it can be restored on
>> failure.
>
> Does this mean that (excepting the "undo" feature) one could set the
> ext3 "append-only" attribute on the repository files to avoid losing
> data due to user account compromise?
>

Probably.  In Git, which is a bit more flexible than Mecurial you can
chmod your objects to read-only or use the ext3 immutable setting to
protect your existing objects.   You can even have a setup where objects
are archived onto write-once media like DVD and still participate in a
live repository, where new objects are written to hard disk, but older
object are (automatically) sourced from the DVD.

Sean

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Matt Mackall]

On Tue, Jun 28, 2005 at 04:45:12PM -0400, Sean wrote:
> On Tue, June 28, 2005 4:27 pm, Kyle Moffett said:
> > On Jun 28, 2005, at 14:01:57, Matt Mackall wrote:
> >> Everything in Mercurial is an append-only log. A transaction journal
> >> records the original length of each log so that it can be restored on
> >> failure.
> >
> > Does this mean that (excepting the "undo" feature) one could set the
> > ext3 "append-only" attribute on the repository files to avoid losing
> > data due to user account compromise?
> >
> 
> Probably.  In Git, which is a bit more flexible than Mecurial you can
> chmod your objects to read-only or use the ext3 immutable setting to
> protect your existing objects.

You can do this in Mercurial as well.

> You can even have a setup where objects
> are archived onto write-once media like DVD and still participate in a
> live repository, where new objects are written to hard disk, but older
> object are (automatically) sourced from the DVD.

Have fun with that. It's an excellent way to destroy your DVD drive.

Git's completely structureless filename hashing pretty much guarantees
that disk layout will degrade to worst-case random access behavior
over time. Just walking through the 2000 commit blobs in the current
tree can take minutes cold cache on a fast hard disk. Walking the 1700
tree blobs in a given version takes quite a while too.

Put that on a DVD and that could easily turn into hours of continuous
seeking for a simple operation like checking out tip of tree.

And as far as I know, ISO9660 and UDF don't really handle huge
directories well. So if you try and put the whole kernel history (200k
files, some huge number of directory blobs, and 30k-60k commit blobs)
on a DVD, you'll be really hurting.

Meanwhile the whole history (>30k changesets) with Mercurial fits on a
regular CD, with reasonable directory sizes and I/O patterns.

Not that it's really worth the trouble. It takes longer for me to burn
an ISO image to disc than to download a complete kernel repo from
kernel.org.

-- 
Mathematics is the supreme nostalgia of our time.

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Sean]

On Tue, June 28, 2005 6:14 pm, Matt Mackall said:
>> You can even have a setup where objects
>> are archived onto write-once media like DVD and still participate in a
>> live repository, where new objects are written to hard disk, but older
>> object are (automatically) sourced from the DVD.
>
> Have fun with that. It's an excellent way to destroy your DVD drive.

Oh come on, stop the FUD.   You pack all the objects up into a single pack
file (see new feature in Git) and you burn it _once_ to dvd or cdrom.

>
> Git's completely structureless filename hashing pretty much guarantees
> that disk layout will degrade to worst-case random access behavior
> over time. Just walking through the 2000 commit blobs in the current
> tree can take minutes cold cache on a fast hard disk. Walking the 1700
> tree blobs in a given version takes quite a while too.
>
> Put that on a DVD and that could easily turn into hours of continuous
> seeking for a simple operation like checking out tip of tree.
>
> And as far as I know, ISO9660 and UDF don't really handle huge
> directories well. So if you try and put the whole kernel history (200k
> files, some huge number of directory blobs, and 30k-60k commit blobs)
> on a DVD, you'll be really hurting.
>
> Meanwhile the whole history (>30k changesets) with Mercurial fits on a
> regular CD, with reasonable directory sizes and I/O patterns.
>
> Not that it's really worth the trouble. It takes longer for me to burn
> an ISO image to disc than to download a complete kernel repo from
> kernel.org.
>

Git is still developing, there will be new ways to seek and cache things
etc eventually that remove any performance issue.  Git gets this right, it
concentrates on what is important, stays flexible and trusts that down the
road as things mature any performance problems can be dealt with.

It already has some tools that are better than BK ever had (gitk, gitweb,
etc..)

Cheers,
Sean

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Kyle Moffett]

On Jun 28, 2005, at 18:23:09, Sean wrote:
> Git is still developing, there will be new ways to seek and cache  
> things
> etc eventually that remove any performance issue.  Git gets this  
> right, it
> concentrates on what is important, stays flexible and trusts that  
> down the
> road as things mature any performance problems can be dealt with.

Have you tried (or even looked at) Mercurial?  I'm now using it for four
different projects that used to be in CVS and I'm loving it.

> It already has some tools that are better than BK ever had (gitk,  
> gitweb,
> etc..)

Likewise for Mercurial, except that IMHO, a from-scratch Mercurial  
pull via
HTTP + Mercurial checkout is faster than a BK or GIT checkout alone.   
And
then there's the fact that it stores the whole mess in a fraction of the
space used by git.

Please, just _try_ it first.  You'll like it, I promise.  (It's also a
much smaller codebase too)

Cheers,
Kyle Moffett

--
I lost interest in "blade servers" when I found they didn't throw  
knives at people who weren't supposed to be in your machine room.
   -- Anthony de Boer

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Matt Mackall]

On Tue, Jun 28, 2005 at 06:23:09PM -0400, Sean wrote:
> On Tue, June 28, 2005 6:14 pm, Matt Mackall said:
> >> You can even have a setup where objects
> >> are archived onto write-once media like DVD and still participate in a
> >> live repository, where new objects are written to hard disk, but older
> >> object are (automatically) sourced from the DVD.
> >
> > Have fun with that. It's an excellent way to destroy your DVD drive.
> 
> Oh come on, stop the FUD.   You pack all the objects up into a single pack
> file (see new feature in Git) and you burn it _once_ to dvd or cdrom.

And even as one big file, it will _still_ be layed out on disk in
pessimal order.

> > Git's completely structureless filename hashing pretty much guarantees
> > that disk layout will degrade to worst-case random access behavior
> > over time. Just walking through the 2000 commit blobs in the current
> > tree can take minutes cold cache on a fast hard disk. Walking the 1700
> > tree blobs in a given version takes quite a while too.
> >
> > Put that on a DVD and that could easily turn into hours of continuous
> > seeking for a simple operation like checking out tip of tree.
> >
> > And as far as I know, ISO9660 and UDF don't really handle huge
> > directories well. So if you try and put the whole kernel history (200k
> > files, some huge number of directory blobs, and 30k-60k commit blobs)
> > on a DVD, you'll be really hurting.
> >
> > Meanwhile the whole history (>30k changesets) with Mercurial fits on a
> > regular CD, with reasonable directory sizes and I/O patterns.
> >
> > Not that it's really worth the trouble. It takes longer for me to burn
> > an ISO image to disc than to download a complete kernel repo from
> > kernel.org.
> >
> 
> Git is still developing, there will be new ways to seek and cache things
> etc eventually that remove any performance issue.

Again, have fun with that. Mercurial already went down this path a
month ago, discovered it couldn't reasonably be fixed without
abandoning the hashes as file name scheme, and changed repo layout.

Git's going to have a much harder time as it's pretty solidly tied to
lookup by contents hash. If you throw that out, you might as well use
Mercurial.

-- 
Mathematics is the supreme nostalgia of our time.

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Sean]

On Tue, June 28, 2005 6:49 pm, Matt Mackall said:

> Again, have fun with that. Mercurial already went down this path a
> month ago, discovered it couldn't reasonably be fixed without
> abandoning the hashes as file name scheme, and changed repo layout.
>
> Git's going to have a much harder time as it's pretty solidly tied to
> lookup by contents hash. If you throw that out, you might as well use
> Mercurial.
>

By the sounds of it, git could just use Mecurial or some variation thereof
as a back end.  Git is not tied to it's back end.   Afterall, Mecurial
just took the basic ideas from Linus' and adapted them to a different back
end.  But there are very few situation where Git performance is a
practical problem, and where it is things are being addressed.   Git is
already so much better for the things I do than BK ever was, I'll stick
with it.

Sean.

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Kyle Moffett]

On Jun 28, 2005, at 18:59:28, Sean wrote:
> By the sounds of it, git could just use Mecurial or some variation  
> thereof
> as a back end.

Umm, you seem to miss the point, sir.  If you use Mercurial, there is no
reason you should layer any part of Git on top of it.  It already does
everything that git does anyways.

> Git is already so much better for the things I do than BK ever was,  
> I'll
> stick with it.

This is like saying "Windows 3.1 is already so much better for the  
things
I do than DOS ever was, I'll stick with it."  :-D

Cheers,
Kyle Moffett

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a18 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$
L++++(+++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+
PGP+++ t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$  
r  !y?(-)
------END GEEK CODE BLOCK------

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Sean]

On Tue, June 28, 2005 7:25 pm, Kyle Moffett said:
> On Jun 28, 2005, at 18:59:28, Sean wrote:
>> By the sounds of it, git could just use Mecurial or some variation
>> thereof
>> as a back end.
>
> Umm, you seem to miss the point, sir.  If you use Mercurial, there is no
> reason you should layer any part of Git on top of it.  It already does
> everything that git does anyways.

No, you seem to miss the point.  Git already does everything Mercurial
does, and does it pretty well too.  The _point_ was that if the big
"feature" of Mercurial is it's on disk format, Git is perfectly capable of
copying it at any point.   The on disk format just ISN'T CLOSE TO BEING
THE MOST IMPORTANT THING AT THE MOMENT.

>
>> Git is already so much better for the things I do than BK ever was,
>> I'll
>> stick with it.
>
> This is like saying "Windows 3.1 is already so much better for the
> things
> I do than DOS ever was, I'll stick with it."  :-D

Yes, so what's your point?  Mercurial is trying to solve a problem that is
already perfectly well handled for me by Git.   Therefore I have _zero_
motivation to direct my efforts elsewhere.

Cheers,
Sean

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Kyle Moffett]

On Jun 28, 2005, at 19:37:36, Sean wrote:
> No, you seem to miss the point.  Git already does everything Mercurial
> does, and does it pretty well too.  The _point_ was that if the big
> "feature" of Mercurial is it's on disk format, Git is perfectly  
> capable of
> copying it at any point.   The on disk format just ISN'T CLOSE TO  
> BEING
> THE MOST IMPORTANT THING AT THE MOMENT.

Firstly, no need to shout, we can all hear you :-D.

Git and Mercurial have all of the same core functionality.  The only
significant remaining difference is that Mercurial uses 1/20th the
network bandwidth and disk space.  If you happen to be interested in
that advantage (as I am, due to my aging equipment and poor internet
connection), then there are two options: (1) fix git, or (2) just use
Mercurial.  From my point of view, option 2 is much more productive.
You may (and probably do) have different priorities and requirements
than I do, but in my view, Mercurial is an excellent tool.

> Yes, so what's your point?  Mercurial is trying to solve a problem  
> that is
> already perfectly well handled for me by Git.   Therefore I have  
> _zero_
> motivation to direct my efforts elsewhere.

Actually, Mercurial solved some of the problems first, before git did;
distributed merge is one example that comes to mind.  In any case, I'm
not trying to tell you what to use, I'm just pointing out alternatives
that are available and explaining why I like them, in case you haven't
seen them or tried them before.

Cheers,
Kyle Moffett

-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a18 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$
L++++(+++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+
PGP+++ t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$  
r  !y?(-)
------END GEEK CODE BLOCK------

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Sean]

On Tue, June 28, 2005 8:08 pm, Kyle Moffett said:

> Firstly, no need to shout, we can all hear you :-D.

ok

>
> Git and Mercurial have all of the same core functionality.  The only
> significant remaining difference is that Mercurial uses 1/20th the
> network bandwidth and disk space.  If you happen to be interested in
> that advantage (as I am, due to my aging equipment and poor internet
> connection), then there are two options: (1) fix git, or (2) just use
> Mercurial.  From my point of view, option 2 is much more productive.
> You may (and probably do) have different priorities and requirements
> than I do, but in my view, Mercurial is an excellent tool.

well the feature set for both are changing rapidly.  i like the emphasis
placed on functionality over performance shown by the git developers (not
that git is slow, it's _way_ faster than bk ever was).  also the web
interface that i looked at for mecurial (admittedly four or fivve weeks
back) didn't come close to gitweb.   and the work done by jon seymour and
others on the history lineralization is just great.   it's something bk
lacked and was always a thorn in my side.

> Actually, Mercurial solved some of the problems first, before git did;
> distributed merge is one example that comes to mind.  In any case, I'm
> not trying to tell you what to use, I'm just pointing out alternatives
> that are available and explaining why I like them, in case you haven't
> seen them or tried them before.

there will be a price to pay if the linux community fragments over choice
of scm.  the good news is that we're no longer locked into the whims of
some proprietary system.  so it should be straight forward for those who
choose any tool to work with those who've chosen another.  this is already
evidenced by the fact that the git repository is pulled and re-exeported
with mecurial.

anyway, all the best, just wish you guys would spend less time trying to
convert git users and more time advancing your own tool.

sean

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Kyle Moffett]

On Jun 28, 2005, at 20:25:02, Sean wrote:
> there will be a price to pay if the linux community fragments over  
> choice
> of scm.

I don't agree.  With the current set of SCMs, I don't think it will  
be long
before somebody invents a gitweb/Mercurial/whatever gateway, such  
that I can
"hg serve" from my Mercurial repository and have Linus "git pull" from a
multiprotocol bridge.

> the good news is that we're no longer locked into the whims of
> some proprietary system.  so it should be straight forward for  
> those who
> choose any tool to work with those who've chosen another.  this is  
> already
> evidenced by the fact that the git repository is pulled and re- 
> exeported
> with mecurial.

I agree completely!  Cheers to the end of proprietary revision storage!

> anyway, all the best, just wish you guys would spend less time  
> trying to
> convert git users and more time advancing your own tool.

A project with no users isn't much of a project, now is it?  In any  
case,
this thread has long since passed its usefulness, so let's let it  
die, ok?

Cheers,
Kyle Moffett

--
I lost interest in "blade servers" when I found they didn't throw  
knives at people who weren't supposed to be in your machine room.
   -- Anthony de Boer

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Vojtech Pavlik]

On Tue, Jun 28, 2005 at 11:53:47PM -0400, Kyle Moffett wrote:

> On Jun 28, 2005, at 20:25:02, Sean wrote:
> >there will be a price to pay if the linux community fragments over
> >choice of scm.
> 
> I don't agree.  With the current set of SCMs, I don't think it will
> be long before somebody invents a gitweb/Mercurial/whatever gateway,
> such that I can "hg serve" from my Mercurial repository and have
> Linus "git pull" from a multiprotocol bridge.

I hope this will happen sooner than later, since that way the
competition between git and mercurial will give us the best tools while
keeping interoperability.

-- 
Vojtech Pavlik
SuSE Labs, SuSE CR

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Matt Mackall]

On Tue, Jun 28, 2005 at 06:59:28PM -0400, Sean wrote:
> Afterall, Mecurial just took the basic ideas from Linus' and adapted
> them to a different back end.

??!

Just for the record, Mercurial had working distributed merge before
Git had any sort of merge at all. So it's hardly a case of me copying
git.

Linus and I both freely borrowed ideas from Monotone and others and
thus there is some rough similarity between the two.

-- 
Mathematics is the supreme nostalgia of our time.

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Thomas Arendsen Hein]

(repost to all lists who received the original mail)

* Kyle Moffett <mrmacman_g4@mac.com> [20050628 22:28]:
> On Jun 28, 2005, at 14:01:57, Matt Mackall wrote:
> >Everything in Mercurial is an append-only log. A transaction journal
> >records the original length of each log so that it can be restored on
> >failure.
> 
> Does this mean that (excepting the "undo" feature) one could set the
> ext3 "append-only" attribute on the repository files to avoid losing
> data due to user account compromise?

This will break Mercurial's journaling. If 'hg pull' fails it
truncates the already appended-to files to the last known state.

Thomas

-- 
Email: thomas@intevation.de
http://intevation.de/~thomas/

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Andrea Arcangeli]

On Fri, Jun 24, 2005 at 08:41:01AM +0200, Petr Baudis wrote:
> Cool. Except where the concepts are just different, Cogito mostly
> appears at least equally simple to use as Mercurial. Yes, some features
> are missing yet. I hope to fix that soon. :-)

The user interface and network protocol isn't the big deal, the big deal
is the more efficient on-disk storage format IMHO.

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Theodore Ts'o]

On Fri, Jun 24, 2005 at 03:06:04PM +0200, Andrea Arcangeli wrote:
> On Fri, Jun 24, 2005 at 08:41:01AM +0200, Petr Baudis wrote:
> > Cool. Except where the concepts are just different, Cogito mostly
> > appears at least equally simple to use as Mercurial. Yes, some features
> > are missing yet. I hope to fix that soon. :-)
> 
> The user interface and network protocol isn't the big deal, the big deal
> is the more efficient on-disk storage format IMHO.

E2fsprogs with the full revision history imported into git is 100
megs, and that's with deltas.  E2fsprogs imported into Mercurial is 17
megs (and actually, the imported repository was just a tad bit smaller
than e2fsprogs' BK repository).

Which do you think is going to be faster to operate from a cold start
using 4200 rpm laptop drives?  :-)

						- Ted

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Paolo Ciarrocchi]

2005/6/24, Theodore Ts'o <tytso@mit.edu>:
> On Fri, Jun 24, 2005 at 03:06:04PM +0200, Andrea Arcangeli wrote:
> > On Fri, Jun 24, 2005 at 08:41:01AM +0200, Petr Baudis wrote:
> > > Cool. Except where the concepts are just different, Cogito mostly
> > > appears at least equally simple to use as Mercurial. Yes, some features
> > > are missing yet. I hope to fix that soon. :-)
> >
> > The user interface and network protocol isn't the big deal, the big deal
> > is the more efficient on-disk storage format IMHO.
> 
> E2fsprogs with the full revision history imported into git is 100
> megs, and that's with deltas.  E2fsprogs imported into Mercurial is 17
> megs (and actually, the imported repository was just a tad bit smaller
> than e2fsprogs' BK repository).
> 
> Which do you think is going to be faster to operate from a cold start
> using 4200 rpm laptop drives?  :-)
> 
>                                                - Ted

That's quite intersting, what the rational behind such a difference in
terms of disk occupation ?

-- 
Paolo

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Christopher Li]

On Fri, Jun 24, 2005 at 03:46:21PM +0200, Paolo Ciarrocchi wrote:
> > 
> > Which do you think is going to be faster to operate from a cold start
> > using 4200 rpm laptop drives?  :-)
> > 
> >                                                - Ted
> 
> That's quite intersting, what the rational behind such a difference in
> terms of disk occupation ?
>

Let me see. Mercurial using delta or full storage for the repository.
It insert a full node when it detect that delta it need to reach
certain node is too big. It just like MPEG movies, most of the frame
is delta to the previous frame.  Once a while you have full frame to
allow you seek to.

But git has delta as well right? Another factor is that all file has
same path in mercurial using the same storage file. So in mercurial
it has far less file to store in the repository. Each file has two repository
files, the data storage file and the index file. Remember that file system
like ext3 is using blocks, if you store very small stuff on a file, it is
still going to take at least one block on disk. So that will defeat the delta
compression if the delta is always on a new file.


Chris

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Kevin Smith]

Andrea Arcangeli wrote:
 > On Fri, Jun 24, 2005 at 08:41:01AM +0200, Petr Baudis wrote:
 >
 >>Cool. Except where the concepts are just different, Cogito mostly
 >>appears at least equally simple to use as Mercurial. Yes, some
 >>features are missing yet. I hope to fix that soon. :-)
 >
 >
 > The user interface and network protocol isn't the big deal, the
 > big deal is the more efficient on-disk storage format IMHO.

For me, efficient storage is not very important, because I mostly deal 
with small projects. Likewise, speed isn't a factor for me, since both 
tools are plenty fast on small repos.

For me, the big advantage of mercurial is that it is written in python, 
instead of shell scripts. I know for some people that's a DISadvantage, 
but I see the following benefits as a result:

- Can run on (native) MS Windows
   (necessary for me because I often work on cross-platform projects)
- Python code can be more clear and expressive (IMHO)

In the long run, I think the python code base will be easier to maintain 
and enhance. A rewrite of cogito in python or ruby would be cool.

One advantage that cogito has is that git viewing/browsing tools can 
operate directly on cogito repos. But a psychological drawback is the 
ongoing confusion between git and cogito. Questions: Would a git-based 
tool that writes to the repo (such as StGIT) mess up a cogito repo? Can 
you switch a repo between git and cogito or back, at any time?

Mercurial's tags use a radical approach, whereas cogito's are more 
conventional. I haven't yet used mercurial's versioned-tags enough yet 
to judge whether they are better, worse, or just different.

I am impressed with the vibrancy of the development communities of both 
projects. Both are able to serve repos on a plain http server. Both are 
easy to use and have decent basic feature sets. Both projects are 
developing test suites.

Mostly, I'm thrilled with this new wave of lightweight distributed SCM 
systems. Most of the established tools tended to be too heavy on 
features and complexity, and have taken a long time to develop. I love 
that a single developer or small team can now create a simple but usable 
distributed SCM in a couple months.

Kevin

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Matt Mackall]

On Fri, Jun 24, 2005 at 09:57:00AM -0400, Kevin Smith wrote:
> Mercurial's tags use a radical approach, whereas cogito's are more 
> conventional. I haven't yet used mercurial's versioned-tags enough yet 
> to judge whether they are better, worse, or just different.

FYI, after reading Linus' rant about tags, I added a second kind of
tags to Mercurial. So now it has both 'official' tags, which are
properly version controlled, and 'private' tags (like git's) as well.

To add a local tag, add a section like this to .hg/hgrc:

[tags]
tested = d6ac88a738c4b3afea56ff09e449b91d85abff68

(This was a bit of a no-brainer, as it was two lines of code)
-- 
Mathematics is the supreme nostalgia of our time.

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Petr Baudis]

Dear diary, on Fri, Jun 24, 2005 at 03:57:00PM CEST, I got a letter
where Kevin Smith <yarcs@qualitycode.com> told me that...
> - Can run on (native) MS Windows
>   (necessary for me because I often work on cross-platform projects)

I'd expect everything to work fine with Cygwin (or with only minor
problems easy to fix) or just any working bash + GNU coreutils
installation. Any issue with that?

> - Python code can be more clear and expressive (IMHO)
> 
> In the long run, I think the python code base will be easier to maintain 
> and enhance. A rewrite of cogito in python or ruby would be cool.

I've planned to rewrite Cogito in Perl, but it turned out the shell
scripts really are far from the PITA they appeared to be - they work
just fine and feel quite comfortable from the maintenance standpoint as
well. It surprised me too but I don't plan to rewrite Cogito in a
different language soon.

> One advantage that cogito has is that git viewing/browsing tools can 
> operate directly on cogito repos. But a psychological drawback is the 
> ongoing confusion between git and cogito. Questions: Would a git-based 
> tool that writes to the repo (such as StGIT) mess up a cogito repo? Can 
> you switch a repo between git and cogito or back, at any time?

Cogito's only unusual requirement (well, expectation) is that HEAD is a
symlink to .git/refs/heads/master, and .git/refs/heads/master should
reflect your current head. I will try to ease up this restriction so
that things will mostly work even if you just have HEAD. I think that
most auxiliary commands (e.g. cg-log - you just have to love it) should
work on any sensible git tree (but I didn't test it - yet).

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
<Espy> be careful, some twit might quote you out of context..

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Sven Verdoolaege]

On Tue, Jun 28, 2005 at 05:07:52PM +0200, Petr Baudis wrote:
> Dear diary, on Fri, Jun 24, 2005 at 03:57:00PM CEST, I got a letter
> where Kevin Smith <yarcs@qualitycode.com> told me that...
> > - Can run on (native) MS Windows
> >   (necessary for me because I often work on cross-platform projects)
> 
> I'd expect everything to work fine with Cygwin (or with only minor
> problems easy to fix) or just any working bash + GNU coreutils
> installation. Any issue with that?

The code is full of Unixisms.  You'd almost think that the authors
of git have some kind of affinity towards Unix.

i386-mingw32-gcc -g -O2 -Wall '-DSHA1_HEADER=<openssl/sha.h>'   -c -o read-cache.o read-cache.c
In file included from read-cache.c:6:
cache.h:14:22: sys/mman.h: No such file or directory
cache.h:16:24: netinet/in.h: No such file or directory
In file included from read-cache.c:6:
cache.h: In function `create_ce_mode':
cache.h:104: warning: implicit declaration of function `S_ISLNK'
cache.h:105: warning: implicit declaration of function `htonl'
cache.h:105: error: `S_IFLNK' undeclared (first use in this function)
cache.h:105: error: (Each undeclared identifier is reported only once
cache.h:105: error: for each function it appears in.)
[..]
read-cache.c:376: warning: implicit declaration of function `mmap'
read-cache.c:376: error: `PROT_READ' undeclared (first use in this function)
read-cache.c:376: error: `PROT_WRITE' undeclared (first use in this function)
read-cache.c:376: error: `MAP_PRIVATE' undeclared (first use in this function)
read-cache.c:376: warning: assignment makes pointer from integer without a cast
read-cache.c:399: warning: implicit declaration of function `munmap'
[..]

skimo

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Petr Baudis]

Dear diary, on Tue, Jun 28, 2005 at 05:15:22PM CEST, I got a letter
where Sven Verdoolaege <skimo@kotnet.org> told me that...
> On Tue, Jun 28, 2005 at 05:07:52PM +0200, Petr Baudis wrote:
> > Dear diary, on Fri, Jun 24, 2005 at 03:57:00PM CEST, I got a letter
> > where Kevin Smith <yarcs@qualitycode.com> told me that...
> > > - Can run on (native) MS Windows
> > >   (necessary for me because I often work on cross-platform projects)
> > 
> > I'd expect everything to work fine with Cygwin (or with only minor
> > problems easy to fix) or just any working bash + GNU coreutils
> > installation. Any issue with that?
> 
> The code is full of Unixisms.  You'd almost think that the authors
> of git have some kind of affinity towards Unix.

Ah. Well, I was speaking just of Cogito and didn't realize you might
want git too so that you can actually do anything with Cogito then. ;-)

> read-cache.c:376: warning: implicit declaration of function `mmap'

Well, I'm actually not very clueful about MS Windows environment, but
doesn't it provide any POSIX API? You might have more luck with that.
Or I guess it should work on Cygwin.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
<Espy> be careful, some twit might quote you out of context..

Cygwin and Native MS Windows (was: Mercurial vs Updated git HOWTO for kernel hackers)

[Kevin Smith]

Petr Baudis wrote:
 > Dear diary, on Fri, Jun 24, 2005 at 03:57:00PM CEST, I got a letter
 > where Kevin Smith <yarcs@qualitycode.com> told me that...
 >
 >>- Can run on (native) MS Windows
 >>  (necessary for me because I often work on cross-platform projects)
 >
 >
 > I'd expect everything to work fine with Cygwin (or with only minor
 > problems easy to fix) or just any working bash + GNU coreutils
 > installation. Any issue with that?

I don't see cygwin as being a reasonable option for a core tool like 
SCM. Cygwin is too large and invasive to expect Windows users to install 
and use it. For my own projects, native Windows operation is a requirement.

Kevin

Cogito vs. Git (was: Mercurial vs Updated git HOWTO for kernel hackers)

[Kevin Smith]

Petr Baudis wrote:
> Cogito's only unusual requirement (well, expectation) is that HEAD is a
> symlink to .git/refs/heads/master, and .git/refs/heads/master should
> reflect your current head. I will try to ease up this restriction so
> that things will mostly work even if you just have HEAD. I think that
> most auxiliary commands (e.g. cg-log - you just have to love it) should
> work on any sensible git tree (but I didn't test it - yet).

So you're saying that aside from that one solveable issue, I could use 
low-level git tools, third-party over-git tools, and cogito, 
interchangebly on a single repo without the tools becoming confused? 
That's cool, and should be made more clear in the cogito readme. I was 
under the impression that cogito was tracking all kinds of extra meta 
magic stuff that git tools wouldn't keep updated.

If I were using cogito, I probably wouldn't want to use the low-level 
git stuff directly, but I might want to use (or maybe even write) some 
other over-git tools.

You might also consider removing the "Core GIT" section from the README, 
because I think it increases the confusion between the two.

Cheers,

Kevin

Re: Cogito vs. Git (was: Mercurial vs Updated git HOWTO for kernel hackers)

[Petr Baudis]

Dear diary, on Tue, Jun 28, 2005 at 06:51:24PM CEST, I got a letter
where Kevin Smith <yarcs@qualitycode.com> told me that...
> Petr Baudis wrote:
> >Cogito's only unusual requirement (well, expectation) is that HEAD is a
> >symlink to .git/refs/heads/master, and .git/refs/heads/master should
> >reflect your current head. I will try to ease up this restriction so
> >that things will mostly work even if you just have HEAD. I think that
> >most auxiliary commands (e.g. cg-log - you just have to love it) should
> >work on any sensible git tree (but I didn't test it - yet).
> 
> So you're saying that aside from that one solveable issue, I could use 
> low-level git tools, third-party over-git tools, and cogito, 
> interchangebly on a single repo without the tools becoming confused? 

Yes, given that you use the toolset "atomically" - if you start doing
some large-scale operation using one tool, you should finish it using
the same tool. That is, if you start merging by Cogito and then try to
commit by Linus' git plumbing, it won't work (and vice versa). cg-seek
combined with third-party over-git tools might have unknown consequences
as well.

> That's cool, and should be made more clear in the cogito readme. I was 
> under the impression that cogito was tracking all kinds of extra meta 
> magic stuff that git tools wouldn't keep updated.

It is tracking only seeks and merges in way the git tools wouldn't keep
updated. I will try to actually make it use ~/.git/MERGE_HEAD instead of
~/.git/merging as well, to make people's live even easier.

> If I were using cogito, I probably wouldn't want to use the low-level 
> git stuff directly, but I might want to use (or maybe even write) some 
> other over-git tools.
> 
> You might also consider removing the "Core GIT" section from the README, 
> because I think it increases the confusion between the two.

I might consider throwing away Git from Cogito distribution altogether,
but that would require Git to have some actual release cycle. ;-)

FYI, I will merge the pile of queued Cogito
patches during the weekend and next week, preparing 0.11.4 or 0.12
(depends on how I feel about it). I will also merge the Linus' stuff,
but only up to:

7323aa11af1527d5a786d93ee34401c72c5df051
	Jan Harkes  2005-06-25 13:41
	[PATCH] git-write-tree doesn't check alternate directories

The next patch is

c323ac7d9c573c5ee8b45b9b9def92a4d4d8204d
	Linus Torvalds  2005-06-25 14:42
	git-pack-objects: create a packed object representation

and I'm not sure if I'm ready to already take the packed stuff; I will
see how tested it gets (and if fsck will get taught about it etc). If it
won't be ready by the time I'm ready for the release, I will merge the
rest right after the release.


-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
<Espy> be careful, some twit might quote you out of context..

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Matthias Urlichs]

Hi, Petr Baudis wrote:

> Switching branches in place will be supported soon (although I have
> doubts about its usefulness).

Well, I don't. Main reason: It's simply a lot faster to create+switch to a
branch locally for doing independent work, than to hardlink the whole
Linux directory tree into a clone tree.

Having one tree also simpifies the "what do I have that's not merged yet"
question -- just call "gitk $(cat .git/refs/heads/*)". ;-)

The only problem I have with it is that "git-read-tree -m -u"
doesn't delete files yet. To repeat my question from last week:
>> Would it be safe to add all files for which
>> read_tree.c:merge_cache:fn() returns zero to a "delete me" list?
(files on which which then actually get deleted, of course, if g-r-t
doesn't find any problems.)

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
Politics -- the gentle art of getting votes from the poor and campaign
funds from the rich by promising to protect each from the other.
		-- Oscar Ameringer

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Linus Torvalds]

On Fri, 24 Jun 2005, Matthias Urlichs wrote:
> 
> Well, I don't. Main reason: It's simply a lot faster to create+switch to a
> branch locally for doing independent work, than to hardlink the whole
> Linux directory tree into a clone tree.
> 
> Having one tree also simpifies the "what do I have that's not merged yet"
> question -- just call "gitk $(cat .git/refs/heads/*)". ;-)

Actually, I think that gets close to another real advantage of branches:  
that is also what allows you to edit things that failed a merge.

For example, let's say that a merge fail. You've got the HEAD and the
MERGE_HEAD, but a file type conflict (like a symlink that has turned into
a directory) or something like that means that you can't resolve them
sanely at all.

So this is merge problem where you can't just do a three-way merge and fix 
up the result and commit: you have to fix things up before you can even 
really do the merge. This is when switching to the MERGE_HEAD thing and 
fixing it up there, committing it, and then doing the merge with the 
original HEAD and the new MERGE_HEAD is really convenient.

(No, the scripts don't help you in cases like this, and we don't do the
MERGE_HEAD as a real branch right now, but the point is that we _can_, and
that this is more than an efficiency issue, it's a fundamental issue of
working with multiple end-points together. You _could_ clone the other 
head into a totally new repository, fix it there, and then try the merge 
anew, but now you're working around a limitation, not just doing 
something slower).

I still think you can go a bit too far on your branch usage (ie Jeff), but 
hey, what's the difference between three branches and fifty, really?

(I'm kidding. The difference between three and fifty is how well you can 
keep track of them in your head, but maybe Jeff just has a bigger head 
than most people do. Jeff, do people go "Boy, you've got a big head" the 
first time they meet you?)

			Linus

Re: Mercurial vs Updated git HOWTO for kernel hackers

[John W. Linville]

On Fri, Jun 24, 2005 at 12:00:33PM -0700, Linus Torvalds wrote:

> Jeff, do people go "Boy, you've got a big head" the 
> first time they meet you?)

FWIW, Jeff _does_ have quite the melon riding around on his shoulders...
-- 
John W. Linville
linville@tuxdriver.com

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Jeff Garzik]

Linus Torvalds wrote:
> I still think you can go a bit too far on your branch usage (ie Jeff), but 
> hey, what's the difference between three branches and fifty, really?

LOL  ;-)


> (I'm kidding. The difference between three and fifty is how well you can 
> keep track of them in your head, but maybe Jeff just has a bigger head 
> than most people do. Jeff, do people go "Boy, you've got a big head" the 
> first time they meet you?)

50 branches is just easier than 50 different trees, for my usage :)

Poor kernel.org would melt if I had 50 trees :)

	Jeff

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Daniel Barkalow]

On Fri, 24 Jun 2005, Matthias Urlichs wrote:

> Hi, Petr Baudis wrote:
> 
> > Switching branches in place will be supported soon (although I have
> > doubts about its usefulness).
> 
> Well, I don't. Main reason: It's simply a lot faster to create+switch to a
> branch locally for doing independent work, than to hardlink the whole
> Linux directory tree into a clone tree.

There's another option, which I may be the only person currently
using: have a repo (.git directory) not in a working directory, and have
the objects/ and refs/ subdirectories of the .git directories in your
working directories symlinked to those subdirectories of the repo. Then
you can have any number of functionally identical working directories,
each of which is set to some branch. If you have a reasonably small number
of branches at any time, they can each have a working directory switched
to them. On the other hand, all of the branches are accessible from any of
them.

	-Daniel
*This .sig left intentionally blank*

Should "git-read-tree -m -u" delete files?

[Junio C Hamano]

>>>>> "MU" == Matthias Urlichs <smurf@smurf.noris.de> writes:

MU> The only problem I have with it is that "git-read-tree -m -u"
MU> doesn't delete files yet. To repeat my question from last week:
>>> Would it be safe to add all files for which
>>> read_tree.c:merge_cache:fn() returns zero to a "delete me" list?
MU> (files on which which then actually get deleted, of course, if g-r-t
MU> doesn't find any problems.)

As the guilty party for the "read-tree two-way semantics table"
you quoted in your "question from last week" message, I should
have replied sooner but could not.  Sorry about that [*1*].

Anyway, here are my answers.

 (1) No, merge_function[] returning zero just means "I did not
     cause anything to change the number of already processed
     entries".  When it wants to delete an entry, it explicitly
     marks the entry to be deleted by calling deleted_entry(),
     and the deletion is processed at the very end by
     check_updates() function.  Note that we do _not_ return
     zero in this case.

 (2) The part you quoted in your "last week" message is case 10;
     the current code does delete the path with -u [*2*].

 (3) There could be cases where twoway_merge() does not delete a
     clean path that _should_ be removed.  If that is the case
     then you have spotted a bug --- I would appreciate it if
     you can show a reproduction recipe.  I have looked at the
     function briefly again while writing this reply and did not
     find suspicious code that would just return 0 without
     calling deleted_entry(), though.

 (4) Using --emu23 (followed by git-merge-cache, of course),
     instead of doing "git-read-tree -m -u H M", should remove
     deleted paths as well.


[Footnote]

*1* I was on a crazy travel schedule, going just for 3 days last
week and then for another 2 days this week to Japan from US west
coast, two 10-hour roundtrip flights X-<.  Now I am back and
hopefully fully functional ;-).

*2* The part you quoted in your previous message was this.  I am
re-quoting from the original to give it a bit more context:

    Two Tree Merge
    ~~~~~~~~~~~~~~
    ...
    In this case, the "git-read-tree -m $H $M" command makes sure
    that no local change is lost as the result of this "merge".
    Here are the "carry forward" rules:

            I (index)           H        M        Result
           -------------------------------------------------------
         ...
            clean I==H  I==M
           ------------------
         ...
         10 yes   yes   N/A     exists   nothing  remove path from cache

This case is covered by this test in t1002:

    test_expect_success \
        '10 - path removed.' \
        'rm -f .git/index &&
         echo rezrov >rezrov &&
         git-update-cache --add rezrov &&
         git-read-tree -m -u $treeH $treeM &&
         git-ls-files --stage >10.out &&
         cmp M.out 10.out &&
         sha1sum -c M.sha1'

Where paths involved are:

	path		treeH		treeM
       -----------------------------------------------------
        bozbar		exists		modified from treeH
        frotz		does not exist	added in treeM
        nitfol		exists		same as in treeH
        rezrov		exists		deleted in treeM

and after this test runs, you can see that the path "rezrov"
gets removed from your work tree.  Insert "exit" just before the
next test, run "cd t && sh t1002-*.sh -i -v", and inspect what
is in the "t/trash" directory.

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Joel Becker]

On Fri, Jun 24, 2005 at 08:41:01AM +0200, Petr Baudis wrote:
> Theoretically, dotest should work just fine even if you use Cogito.
> Anyone tested it?

	When I update the OCFS2 git tree, I clone with Cogito and patch
with dotest.  Been doing it for a month or two, no problems.

Joel

-- 

"I think it would be a good idea."  
        - Mahatma Ghandi, when asked what he thought of Western
          civilization

Joel Becker
Senior Member of Technical Staff
Oracle
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Kyle Moffett]

On Jun 23, 2005, at 19:56:34, Matt Mackall wrote:
> On Wed, Jun 22, 2005 at 06:24:54PM -0400, Jeff Garzik wrote:
>> Things in git-land are moving at lightning speed, and usability has
>> improved a lot since my post a month ago:  http://lkml.org/lkml/ 
>> 2005/5/26/11
>
> And here's a quick comparison with the current state of Mercurial..

Oooh!!! Nice!!! This is my new official source keeper for my next  
project.  I
especially like the fast operation and low disk storage  
requirements.  Many
kudos to Matt Mackall for his awesome program!  The efficient network  
protocol
is great too!

Cheers,
Kyle Moffett

--
There are two ways of constructing a software design. One way is to  
make it so simple that there are obviously no deficiencies. And the  
other way is to make it so complicated that there are no obvious  
deficiencies.
  -- C.A.R. Hoare

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Pavel Machek]

Hi!

> > Things in git-land are moving at lightning speed, and usability has 
> > improved a lot since my post a month ago:  http://lkml.org/lkml/2005/5/26/11
> 
> And here's a quick comparison with the current state of Mercurial..
> 
> > 1) installing git
> > 
> > git requires bootstrapping, since you must have git installed in order 
> > to check out git.git (git repo), and linux-2.6.git (kernel repo).  I 
> > have put together a bootstrap tarball of today's git repository.
> > 
> > Download tarball from:
> > http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-20050622.tar.bz2
> > 
> > tarball build-deps:  zlib, libcurl, libcrypto (openssl)
> > 
> > install tarball:  unpack && make && sudo make prefix=/usr/local install
> > 
> > jgarzik helper scripts, not in official git distribution:
> > http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-new-branch
> > http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-changes-script
> > 
> > After reading the rest of this document, come back and update your copy 
> > of git to the latest:
> > rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git.git
> 
> Download from: http://selenic.com/mercurial/mercurial-snapshot.tar.gz
> Build-deps: Python 2.3
> Install: unpack && python setup.py install [--home=/usr/local]

Did that... (had to install python2.3-dev, first), but got...

> $ mkdir linux-2.6
> $ cd linux-2.6
> $ hg init http://www.kernel.org/hg/    # obviously you can also browse this
> 
> This downloads about 125M of data, which include the whole kernel history
> back to 2.4.0 and everything in Linus' git repo as well.

byte-compiling /usr/local/lib/python/mercurial/fancyopts.py to
fancyopts.pyc
byte-compiling /usr/local/lib/python/mercurial/hg.py to hg.pyc
byte-compiling /usr/local/lib/python/mercurial/hgweb.py to hgweb.pyc
byte-compiling /usr/local/lib/python/mercurial/httprangereader.py to
httprangereader.pyc
byte-compiling /usr/local/lib/python/mercurial/lock.py to lock.pyc
byte-compiling /usr/local/lib/python/mercurial/mdiff.py to mdiff.pyc
byte-compiling /usr/local/lib/python/mercurial/revlog.py to revlog.pyc
byte-compiling /usr/local/lib/python/mercurial/transaction.py to
transaction.pyc
byte-compiling /usr/local/lib/python/mercurial/ui.py to ui.pyc
byte-compiling /usr/local/lib/python/mercurial/util.py to util.pyc
byte-compiling /usr/local/lib/python/mercurial/version.py to
version.pyc
running install_scripts
copying build/scripts-2.3/hg -> /usr/local/bin
copying build/scripts-2.3/hgmerge -> /usr/local/bin
changing mode of /usr/local/bin/hg to 755
changing mode of /usr/local/bin/hgmerge to 755
running install_data
creating /usr/local/lib/python/mercurial/templates
copying templates/map -> /usr/local/lib/python/mercurial/templates
copying templates/map-raw -> /usr/local/lib/python/mercurial/templates
copying templates/changelog.tmpl ->
/usr/local/lib/python/mercurial/templates
copying templates/changelogentry.tmpl ->
/usr/local/lib/python/mercurial/templates
copying templates/changeset-raw.tmpl ->
/usr/local/lib/python/mercurial/templates
copying templates/changeset.tmpl ->
/usr/local/lib/python/mercurial/templates
copying templates/fileannotate.tmpl ->
/usr/local/lib/python/mercurial/templates
copying templates/filediff-raw.tmpl ->
/usr/local/lib/python/mercurial/templates
copying templates/filediff.tmpl ->
/usr/local/lib/python/mercurial/templates
copying templates/filelog.tmpl ->
/usr/local/lib/python/mercurial/templates
copying templates/filelogentry.tmpl ->
/usr/local/lib/python/mercurial/templates
copying templates/filerevision-raw.tmpl ->
/usr/local/lib/python/mercurial/templates
copying templates/filerevision.tmpl ->
/usr/local/lib/python/mercurial/templates
copying templates/footer.tmpl ->
/usr/local/lib/python/mercurial/templates
copying templates/header-raw.tmpl ->
/usr/local/lib/python/mercurial/templates
copying templates/header.tmpl ->
/usr/local/lib/python/mercurial/templates
copying templates/manifest.tmpl ->
/usr/local/lib/python/mercurial/templates
copying templates/tags.tmpl ->
/usr/local/lib/python/mercurial/templates
root@Elf:/data/tmp/hg/mercurial-snapshot# Use "exit" to leave the
shell.
root@Elf:/data/tmp/hg/mercurial-snapshot# exit
pavel@Elf:/data/tmp/hg/mercurial-snapshot$ cd ../..
pavel@Elf:/data/tmp$ mkdir linux-hg
pavel@Elf:/data/tmp$ cd linux-hg/
pavel@Elf:/data/tmp/linux-hg$ hg init http://www.kernel.org/hg/
Traceback (most recent call last):
  File "/usr/local/bin/hg", line 11, in ?
    from mercurial import commands
ImportError: No module named mercurial
pavel@Elf:/data/tmp/linux-hg$
							Pavel
-- 
teflon -- maybe it is a trademark, but it should not be.

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Kyle Moffett]

On Jun 27, 2005, at 14:31:18, Pavel Machek wrote:
> pavel@Elf:/data/tmp/linux-hg$ hg init http://www.kernel.org/hg/
> Traceback (most recent call last):
>   File "/usr/local/bin/hg", line 11, in ?
>     from mercurial import commands
> ImportError: No module named mercurial

Apparently your Python does not automatically look in /usr/local/lib/ 
python
and you don't have PYTHONPATH=/usr/local/lib/python.  Try adding the  
following
to your .bash_profile, then logging out and back in again:

if [ -z "$PYTHONPATH" ]; then
     PYTHONPATH=/usr/local/lib/python
else
     PYTHONPATH="$PYTHONPATH:/usr/local/lib/python"
fi
export PYTHONPATH

Cheers,
Kyle Moffett

--
Somone asked me why I work on this free (http://www.fsf.org/philosophy/)
software stuff and not get a real job. Charles Shultz had the best  
answer:

"Why do musicians compose symphonies and poets write poems? They do it
because life wouldn't have any meaning for them if they didn't.  
That's why
I draw cartoons. It's my life."
-- Charles Shultz

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Matt Mackall]

On Mon, Jun 27, 2005 at 08:31:18PM +0200, Pavel Machek wrote:
> Hi!
> 
> > > Things in git-land are moving at lightning speed, and usability has 
> > > improved a lot since my post a month ago:  http://lkml.org/lkml/2005/5/26/11
> > 
> > And here's a quick comparison with the current state of Mercurial..
> > 
> > > 1) installing git
> > > 
> > > git requires bootstrapping, since you must have git installed in order 
> > > to check out git.git (git repo), and linux-2.6.git (kernel repo).  I 
> > > have put together a bootstrap tarball of today's git repository.
> > > 
> > > Download tarball from:
> > > http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-20050622.tar.bz2
> > > 
> > > tarball build-deps:  zlib, libcurl, libcrypto (openssl)
> > > 
> > > install tarball:  unpack && make && sudo make prefix=/usr/local install
> > > 
> > > jgarzik helper scripts, not in official git distribution:
> > > http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-new-branch
> > > http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-changes-script
> > > 
> > > After reading the rest of this document, come back and update your copy 
> > > of git to the latest:
> > > rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git.git
> > 
> > Download from: http://selenic.com/mercurial/mercurial-snapshot.tar.gz
> > Build-deps: Python 2.3
> > Install: unpack && python setup.py install [--home=/usr/local]
> 
> Did that... (had to install python2.3-dev, first), but got...
> Traceback (most recent call last):
>   File "/usr/local/bin/hg", line 11, in ?
>     from mercurial import commands
> ImportError: No module named mercurial

>From the README:

 To install system-wide:

 $ python setup.py install   # change python to python2.3 if 2.2 is default

 To install in your home directory (~/bin and ~/lib, actually), run:

 $ python2.3 setup.py install --home=~
 $ export PYTHONPATH=${HOME}/lib/python  # add this to your .bashrc
 $ export PATH=${HOME}/bin:$PATH         # 

 And finally:

 $ hg                                    # test installation, show help

 If you get complaints about missing modules, you probably haven't set
 PYTHONPATH correctly.

-- 
Mathematics is the supreme nostalgia of our time.

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Benjamin LaHaise]

On Mon, Jun 27, 2005 at 12:40:31PM -0700, Matt Mackall wrote:
>  $ export PYTHONPATH=${HOME}/lib/python  # add this to your .bashrc

This needs to be ${HOME}/lib64/python on x86-64.

		-ben

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Matt Mackall]

On Mon, Jun 27, 2005 at 03:51:34PM -0400, Benjamin LaHaise wrote:
> On Mon, Jun 27, 2005 at 12:40:31PM -0700, Matt Mackall wrote:
> >  $ export PYTHONPATH=${HOME}/lib/python  # add this to your .bashrc
> 
> This needs to be ${HOME}/lib64/python on x86-64.

Thanks, I'll add this to the README.

-- 
Mathematics is the supreme nostalgia of our time.

Re: Mercurial vs Updated git HOWTO for kernel hackers

[Ed Tomlinson]

On Monday 27 June 2005 15:51, Benjamin LaHaise wrote:
> On Mon, Jun 27, 2005 at 12:40:31PM -0700, Matt Mackall wrote:
> >  $ export PYTHONPATH=${HOME}/lib/python  # add this to your .bashrc
> 
> This needs to be ${HOME}/lib64/python on x86-64.

Be careful.  This is not true on debian.

Ed Tomlinson

Re: Updated git HOWTO for kernel hackers

[Amin Azez]

Thanks for the HOWTO, Jeff, but it gives me problems in step 4.
I checked out your latest git source today and "make install"ed it as 
part of your instructions and at step 4 I get:

$ git checkout -f
error: cannot map sha1 file f8640c306db2d583b9a30f2e52f8fb0a4cf624e0
fatal: failed to unpack tree object a92b7b80579fe68fe229892815c750f6652eb6a9

$ cat .git/HEAD
a92b7b80579fe68fe229892815c750f6652eb6a9

Naturally I have no idea what f8640c306db2d583b9a30f2e52f8fb0a4cf624e0 
refers to.

Step 3:
$ git-pull-script \
rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git

said I was already up to date.

Variations on step 4:

$ git-read-tree -m HEAD
or
$ git-read-tree a92b7b80579fe68fe229892815c750f6652eb6a9
also fail in the same way.

My linux-2.6 directory only has one entry, .git, containing about 75M of 
files.

Sam

Jeff Garzik wrote:
> 
> Things in git-land are moving at lightning speed, and usability has 
> improved a lot since my post a month ago:  
> http://lkml.org/lkml/2005/5/26/11
> 
> 
> 
> 1) installing git
> 
> git requires bootstrapping, since you must have git installed in order 
> to check out git.git (git repo), and linux-2.6.git (kernel repo).  I 
> have put together a bootstrap tarball of today's git repository.
> 
> Download tarball from:
> http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-20050622.tar.bz2
> 
> tarball build-deps:  zlib, libcurl, libcrypto (openssl)
> 
> install tarball:  unpack && make && sudo make prefix=/usr/local install
> 
> jgarzik helper scripts, not in official git distribution:
> http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-new-branch
> http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-changes-script
> 
> After reading the rest of this document, come back and update your copy 
> of git to the latest:
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git.git
> 
> 
> 2) download a linux kernel tree for the very first time
> 
> $ mkdir -p linux-2.6/.git
> $ cd linux-2.6
> $ rsync -a --delete --verbose --stats --progress \
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/ 
> \          <- word-wrapped backslash; sigh
>     .git/
> 
> 
> 3) update local kernel tree to latest 2.6.x upstream ("fast-forward merge")
> 
> $ cd linux-2.6
> $ git-pull-script \
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
> 
> 
> 4) check out files from the git repository into the working directory
> 
> $ git checkout -f
> 
> 
> 5) check in your own modifications (e.g. do some hacking, or apply a patch)
> 
> # go to repo
> $ cd linux-2.6
> 
> # make some modifications
> $ patch -sp1 < /tmp/my.patch
> $ diffstat -p1 < /tmp/my.patch
> 
> # NOTE: add '--add' and/or '--remove' if files were added or removed
> $ git-update-cache <list of all files changed>
> 
> # check in changes
> $ git commit
> 
> 
> 6) List all changes in working dir, in diff format.
> 
> $ git-diff-cache -p HEAD
> 
> 
> 7) List all changesets (i.e. show each cset's description text) in local 
> branch of local tree, that are not present in remote tree.
> 
> $ cd my-kernel-tree-2.6
> $ git-changes-script -L ../linux-2.6 | less
> 
> 
> 8) List all changesets:
> 
> $ git-whatchanged
> 
> 
> 9) apply all patches in a Berkeley mbox-format file
> 
> First, download and add to your PATH Linus's git tools:
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git-tools.git
> 
> $ cd my-kernel-tree-2.6
> $ dotest /path/to/mbox  # yes, Linus has no taste in naming scripts
> 
> 
> 10) don't forget to download tags from time to time.
> 
> git-pull-script only downloads sha1-indexed object data, and the 
> requested remote head.  This misses updates to the .git/refs/tags/ and 
> .git/refs/heads directories.  It is advisable to update your kernel .git 
> directories periodically with a full rsync command, to make sure you got 
> everything:
> 
> $ cd linux-2.6
> $ rsync -a --delete --verbose --stats --progress \
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
> \          <- word-wrapped backslash; sigh
>     .git/
> 
> 
> 11) list all branches, such as those found in my netdev-2.6 or 
> libata-dev trees.
> 
> Download
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/jgarzik/netdev-2.6.git
>     or
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/jgarzik/libata-dev.git
> 
> 
> $ cd netdev-2.6
> $ ls .git/refs/heads/
> 
> { these are the current netdev-2.6 branches }
> 
>> 8139cp       forcedeth    master     qeth           smc91x         we18
>> 8139too-iomap  for-linus    natsemi      r8169      smc91x-eeprom  wifi
>> airo           hdlc         ns83820      register-netdev  starfire
>> atmel          ieee80211    orinoco      remove-drivers   tlan
>> chelsio        iff-running  orinoco-hch  sis900           veth
>> dm9000         janitor      ppp          skge             viro
> 
> 
> 
> 12) make desired branch current in working directory
> 
> $ git checkout -f $branch
> 
> 
> 13) create a new branch, and make it current
> 
> $ cp .git/refs/heads/master .git/refs/heads/my-new-branch-name
> $ git checkout -f my-new-branch-name
> 
> 
> 14) examine which branch is current
> 
> $ ls -l .git/HEAD
> 
> 
> 15) undo all local modifications (same as checkout):
> 
> $ git checkout -f
> 
> 
> 16) obtain a diff between current branch, and master branch
> 
> In most trees WITH BRANCHES, .git/refs/heads/master contains the current 
> 'vanilla' upstream tree, for easy diffing and merging.  (in trees 
> without branches, 'master' simply contains your latest changes)
> 
> $ git-diff-tree -p master HEAD
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: Updated git HOWTO for kernel hackers

[Amin Azez]

Dave Jones daily snapshot of git solved the problem, available from:
http://www.codemonkey.org.uk/projects/git-snapshots/git/

I realise that Jeff's howto suggested updating git using git, but it
suggested doing this after following the intermediate steps. I also find
it ironic that the version of git Jeff provides doesn't work with his
instructions; however, still many thanks to Jeff for his HOWTO and to
Dave for git.

Azez

Amin Azez wrote:
> Thanks for the HOWTO, Jeff, but it gives me problems in step 4.
> I checked out your latest git source today and "make install"ed it as
> part of your instructions and at step 4 I get:
> 
> $ git checkout -f
> error: cannot map sha1 file f8640c306db2d583b9a30f2e52f8fb0a4cf624e0
> fatal: failed to unpack tree object
> a92b7b80579fe68fe229892815c750f6652eb6a9
> 
> $ cat .git/HEAD
> a92b7b80579fe68fe229892815c750f6652eb6a9
> 
> Naturally I have no idea what f8640c306db2d583b9a30f2e52f8fb0a4cf624e0
> refers to.
> 
> Step 3:
> $ git-pull-script \
> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
> 
> said I was already up to date.
> 
> Variations on step 4:
> 
> $ git-read-tree -m HEAD
> or
> $ git-read-tree a92b7b80579fe68fe229892815c750f6652eb6a9
> also fail in the same way.
> 
> My linux-2.6 directory only has one entry, .git, containing about 75M of
> files.
> 
> Sam
> 
> Jeff Garzik wrote:
> 
>>
>> Things in git-land are moving at lightning speed, and usability has
>> improved a lot since my post a month ago: 
>> http://lkml.org/lkml/2005/5/26/11
>>
>>
>>
>> 1) installing git
>>
>> git requires bootstrapping, since you must have git installed in order
>> to check out git.git (git repo), and linux-2.6.git (kernel repo).  I
>> have put together a bootstrap tarball of today's git repository.
>>
>> Download tarball from:
>> http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-20050622.tar.bz2
>>
>>
>> tarball build-deps:  zlib, libcurl, libcrypto (openssl)
>>
>> install tarball:  unpack && make && sudo make prefix=/usr/local install
>>
>> jgarzik helper scripts, not in official git distribution:
>> http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-new-branch
>> http://www.kernel.org/pub/linux/kernel/people/jgarzik/git-changes-script
>>
>> After reading the rest of this document, come back and update your
>> copy of git to the latest:
>> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git.git
>>
>>
>> 2) download a linux kernel tree for the very first time
>>
>> $ mkdir -p linux-2.6/.git
>> $ cd linux-2.6
>> $ rsync -a --delete --verbose --stats --progress \
>> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
>> \          <- word-wrapped backslash; sigh
>>     .git/
>>
>>
>> 3) update local kernel tree to latest 2.6.x upstream ("fast-forward
>> merge")
>>
>> $ cd linux-2.6
>> $ git-pull-script \
>> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
>>
>>
>> 4) check out files from the git repository into the working directory
>>
>> $ git checkout -f
>>
>>
>> 5) check in your own modifications (e.g. do some hacking, or apply a
>> patch)
>>
>> # go to repo
>> $ cd linux-2.6
>>
>> # make some modifications
>> $ patch -sp1 < /tmp/my.patch
>> $ diffstat -p1 < /tmp/my.patch
>>
>> # NOTE: add '--add' and/or '--remove' if files were added or removed
>> $ git-update-cache <list of all files changed>
>>
>> # check in changes
>> $ git commit
>>
>>
>> 6) List all changes in working dir, in diff format.
>>
>> $ git-diff-cache -p HEAD
>>
>>
>> 7) List all changesets (i.e. show each cset's description text) in
>> local branch of local tree, that are not present in remote tree.
>>
>> $ cd my-kernel-tree-2.6
>> $ git-changes-script -L ../linux-2.6 | less
>>
>>
>> 8) List all changesets:
>>
>> $ git-whatchanged
>>
>>
>> 9) apply all patches in a Berkeley mbox-format file
>>
>> First, download and add to your PATH Linus's git tools:
>> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/git-tools.git
>>
>> $ cd my-kernel-tree-2.6
>> $ dotest /path/to/mbox  # yes, Linus has no taste in naming scripts
>>
>>
>> 10) don't forget to download tags from time to time.
>>
>> git-pull-script only downloads sha1-indexed object data, and the
>> requested remote head.  This misses updates to the .git/refs/tags/ and
>> .git/refs/heads directories.  It is advisable to update your kernel
>> .git directories periodically with a full rsync command, to make sure
>> you got everything:
>>
>> $ cd linux-2.6
>> $ rsync -a --delete --verbose --stats --progress \
>> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
>> \          <- word-wrapped backslash; sigh
>>     .git/
>>
>>
>> 11) list all branches, such as those found in my netdev-2.6 or
>> libata-dev trees.
>>
>> Download
>> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/jgarzik/netdev-2.6.git
>>     or
>> rsync://rsync.kernel.org/pub/scm/linux/kernel/git/jgarzik/libata-dev.git
>>
>>
>> $ cd netdev-2.6
>> $ ls .git/refs/heads/
>>
>> { these are the current netdev-2.6 branches }
>>
>>> 8139cp       forcedeth    master     qeth           smc91x         we18
>>> 8139too-iomap  for-linus    natsemi      r8169      smc91x-eeprom  wifi
>>> airo           hdlc         ns83820      register-netdev  starfire
>>> atmel          ieee80211    orinoco      remove-drivers   tlan
>>> chelsio        iff-running  orinoco-hch  sis900           veth
>>> dm9000         janitor      ppp          skge             viro
>>
>>
>>
>>
>> 12) make desired branch current in working directory
>>
>> $ git checkout -f $branch
>>
>>
>> 13) create a new branch, and make it current
>>
>> $ cp .git/refs/heads/master .git/refs/heads/my-new-branch-name
>> $ git checkout -f my-new-branch-name
>>
>>
>> 14) examine which branch is current
>>
>> $ ls -l .git/HEAD
>>
>>
>> 15) undo all local modifications (same as checkout):
>>
>> $ git checkout -f
>>
>>
>> 16) obtain a diff between current branch, and master branch
>>
>> In most trees WITH BRANCHES, .git/refs/heads/master contains the
>> current 'vanilla' upstream tree, for easy diffing and merging.  (in
>> trees without branches, 'master' simply contains your latest changes)
>>
>> $ git-diff-tree -p master HEAD
>>
>>
>> -
>> To unsubscribe from this list: send the line "unsubscribe git" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>