Re-done kernel archive - real one?

Re-done kernel archive - real one?

[Linus Torvalds]

Ok, nobody really objected to the notion of leaving the kernel history
behind for now, and in fact most people seemed to basically agree. So with
that decided, the old kernel testing tree was actually perfectly ok,
except it had been build up with the old-style commit date handling, which
made me not want to use it as a base for any real work.

So I re-created the dang thing (hey, it takes just a few minutes), and
pushed it out, and there's now an archive on kernel.org in my public
"personal" directory called "linux-2.6.git". I'll continue the tradition
of naming git-archive directories as "*.git", since that really ends up
being the ".git" directory for the checked-out thing.

I'm not going to announce it on linux-kernel yet, because I don't think
it's useful to anybody but a git person anyway. Besides, I don't actually
know how happy the kernel.org people are about this distribution method
and whether it ends up being a horrible disaster for the mirroring setup. 

Peter made some noises about /pub/scm, which makes sense, and would be a
better place than my public tree. Apparently there are other places that
are willing and able to host things too, so we'll see.

NOTE! The roughly 10x expansion of archive size goind from BK to git ends
up in a similar 10x bandwidth expansion, in addition to just the overhead
of reading tons of directory entries and comparing them (which is what
both a wget and rsync thing ends up doing). I'm sure we can bring that
down with smarter synchronization tools, but I also suspect that's some
way away.

So is real common usage, though, so maybe it's not that bad at all. Who 
knows. We haven't hit a single real snag so far (except it took several 
days longer than I expected, but hey, I expect lots of things ;), and I'm 
sure real usage will show lots of them.

Similarly, we don't really have real merging, which makes tracking harder, 
but I suspect actually having a tree out there will make people more 
motivated and have more of a test-case. I'm feeling good enough about the 
plumbing that I think I solved the "hard" part of it, and now it's just 
the boring 95% left - scripting around it.

I think that with the new merge model, the easiest thing to do is to just 
download all new objects, and then download the HEAD file under a new 
name.

Ie we have two phases to the merge: first get the objects, with something
like

	repo=kernel.org:/pub/kernel/people/torvalds/linux-2.6.git
	rsync --ignore-existing -acv $(repo)/ .git/

which will _not_ download the new HEAD file (since you already have one of 
your own), and then when you actually decide to merge you do

	rsync -acv $(repo)/HEAD .git/MERGE_WITH

and now you can look at your old HEAD, and the MERGE_WITH thing, look up 
the parents, and then do

	read-tree -m <parent-tree> <head-tree> <merge-with-tree>
	write-tree
	commit-tree <result-tree> -p <head-tree> -p <merge-with-tree>

(which should actually _work_, assuming that the merge had no file 
conflicts).

This seems to be a sane way to do merges, and if the scripting starts from 
there and then becomes smarter...

		Linus

Re: Re-done kernel archive - real one?

[Russell King]

On Sat, Apr 16, 2005 at 04:01:45PM -0700, Linus Torvalds wrote:
> So I re-created the dang thing (hey, it takes just a few minutes), and
> pushed it out, and there's now an archive on kernel.org in my public
> "personal" directory called "linux-2.6.git". I'll continue the tradition
> of naming git-archive directories as "*.git", since that really ends up
> being the ".git" directory for the checked-out thing.

We need to work out how we're going to manage to get our git changes to
you.  At the moment, I've very little idea how to do that.  Ideas?

At the bottom is the script itself.  There's probably some aspects of
it which aren't nice, maybe Petr can advise on this (and maybe increase
the functionality of the git shell script to fill in where necessary.)

However, I've made a start to generate the necessary emails.  How about
this format?

I'm not keen on the tree, parent, author and committer objects appearing
in this - they appear to clutter it up.  What're your thoughts?

I'd rather not have the FQDN of the machine where the commit happened
appearing in the logs.  (I've 'xxxx'd it out for the time being, because
I'd rather not have yet more email-address-like objects get into spammers
databases with which to hammer my 512kbps DSL line.)

Linus,

Please incorporate the latest ARM changes.

This will update the following files:

 arm/kernel/process.c                |   15 +++++++++++----
 arm/kernel/traps.c                  |    8 ++------
 arm/lib/changebit.S                 |   11 ++---------
 arm/lib/clearbit.S                  |   13 ++-----------
 arm/lib/setbit.S                    |   11 ++---------
 arm/lib/testchangebit.S             |   15 ++-------------
 arm/lib/testclearbit.S              |   15 ++-------------
 arm/lib/testsetbit.S                |   15 ++-------------
 arm/mach-footbridge/dc21285-timer.c |    4 ++--
 arm/mach-sa1100/h3600.c             |    2 +-
 asm-arm/ptrace.h                    |    5 +----
 asm-arm/system.h                    |    3 +++
 12 files changed, 32 insertions(+), 85 deletions(-)

through these ChangeSets:

	tree 7c4d75539c29ef7a9dde81acf84a072649f4f394
	parent d5922e9c35d21f0b6b82d1fd8b1444cfce57ca34
	author Russell King <rmk@xxxx.arm.linux.org.uk> 1113749462 +0100
	committer Russell King <rmk@xxxx.arm.linux.org.uk> 1113749462 +0100
	
	[PATCH] ARM: bitops
	
	Convert ARM bitop assembly to a macro.  All bitops follow the same
	format, so it's silly duplicating the code when only one or two
	instructions are different.
	
	Signed-off-by: Russell King <rmk@arm.linux.org.uk>
	tree fc10d3ffa6062cda10a10cb8262d8df238aea4fb
	parent 5d9a545981893629c8f95e2b8b50d15d18c6ddbc
	author Russell King <rmk@xxxx.arm.linux.org.uk> 1113749436 +0100
	committer Russell King <rmk@xxxx.arm.linux.org.uk> 1113749436 +0100
	
	[PATCH] ARM: showregs
	
	Fix show_regs() to provide a backtrace.  Provide a new __show_regs()
	function which implements the common subset of show_regs() and die().
	Add prototypes to asm-arm/system.h
	
	Signed-off-by: Russell King <rmk@arm.linux.org.uk>
	tree 5591fced9a2b5f84c6772dcbe2eb4b24e29161fc
	parent 488faba31f59c5960aabbb2a5877a0f2923937a3
	author Russell King <rmk@xxxx.arm.linux.org.uk> 1113748846 +0100
	committer Russell King <rmk@xxxx.arm.linux.org.uk> 1113748846 +0100
	
	[PATCH] ARM: h3600_irda_set_speed arguments
	
	h3600_irda_set_speed() had the wrong type for the "speed" argument.
	Fix this.
	
	Signed-off-by: Russell King <rmk@arm.linux.org.uk>
	tree 2493491da6e446e48d5443f0a549a10ed3d35b62
	parent e7905b2f22eb5d5308c9122b9c06c2d02473dd4f
	author Russell King <rmk@xxxx.arm.linux.org.uk> 1113748615 +0100
	committer Russell King <rmk@xxxx.arm.linux.org.uk> 1113748615 +0100
	
	[PATCH] ARM: footbridge rtc init
	
	The footbridge ISA RTC was being initialised before we had setup the
	kernel timer.  This caused a divide by zero error when the current
	time of day is set.  Resolve this by initialising the RTC after
	the kernel timer has been initialised.
	
	Signed-off-by: Russell King <rmk@arm.linux.org.uk>

---

#!/bin/sh
prev=$(cat .git/heads/origin)
to=$(cat .git/HEAD)
who=Linus
what=ARM
 
cat << EOT
${who},
 
Please incorporate the latest ${what} changes.
 
This will update the following files:
 
EOT
 
git diff $prev $to | diffstat -p1
 
cat << EOT
 
through these ChangeSets:
 
EOT
 
this=$to
while [ "$this" != "$prev" ]; do
  cat-file commit $this | sed 's,.*,\t&,'
  this=$(cat-file commit $this | grep ^parent | cut -d ' ' -f 2)
done


-- 
Russell King

Re: Re: Re-done kernel archive - real one?

[Petr Baudis]

Dear diary, on Sun, Apr 17, 2005 at 05:24:48PM CEST, I got a letter
where Russell King <rmk@arm.linux.org.uk> told me that...
> However, I've made a start to generate the necessary emails.  How about
> this format?
> 
> I'm not keen on the tree, parent, author and committer objects appearing
> in this - they appear to clutter it up.  What're your thoughts?

I think it would look nicer if you used git log format. Actually...

 
> #!/bin/sh
> prev=$(cat .git/heads/origin)
> to=$(cat .git/HEAD)

It is preferred to use the commit-id command instead; when git decides
to totally dig over its metametadata storage, you won't need to care.

> this=$to
> while [ "$this" != "$prev" ]; do
>   cat-file commit $this | sed 's,.*,\t&,'
>   this=$(cat-file commit $this | grep ^parent | cut -d ' ' -f 2)
> done

What about adding support to git log for not caring about side branches
(if there are multiple parents, go only over the first one; you might
add that as a switch to rev-tree, which would turn it to rev-line ;-)?

Then, you could also easily implement support for passing additional
commit ID to git log, and it would then print out the range between
these two.

That would do exactly what you want, and would be probably quite usable
for other uses too.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Sun, 17 Apr 2005, Russell King wrote:
>
> On Sat, Apr 16, 2005 at 04:01:45PM -0700, Linus Torvalds wrote:
> > So I re-created the dang thing (hey, it takes just a few minutes), and
> > pushed it out, and there's now an archive on kernel.org in my public
> > "personal" directory called "linux-2.6.git". I'll continue the tradition
> > of naming git-archive directories as "*.git", since that really ends up
> > being the ".git" directory for the checked-out thing.
> 
> We need to work out how we're going to manage to get our git changes to
> you.  At the moment, I've very little idea how to do that.  Ideas?

To me, merging is my highest priority. I suspect that once I have a tree 
from you (or anybody else) that I actually _test_ merging with, I'll be 
motivated as hell to make sure that my plumbing actually works. 

After all, it's not just you who want to have to avoid the pain of 
merging: it's definitely in my own best interests to make merging as 
easy as possible. You're _the_ most obvious initial candidate, because 
your merges almost never have any conflicts at all, even on a file level 
(much less within a file).

> However, I've made a start to generate the necessary emails.  How about
> this format?
> 
> I'm not keen on the tree, parent, author and committer objects appearing
> in this - they appear to clutter it up.  What're your thoughts?

Indeed. I'd almost drop the whole header except for the "author" line. 

Oh, and you need a separator between commits, right now your 
"Signed-off-by:" line ends up butting up with the header of the next 
commit ;)

> I'd rather not have the FQDN of the machine where the commit happened
> appearing in the logs.

That's fine. Out short-logs have always tried to have just the real name 
in them, and I do want an email-like thing for tracking the developer, but 
yes, if you remove the email, that's fine. It should be easy enough to do 
with a simple

	sed 's/<.*>//'

or similar.

And if you replace "author" with "From:" and do the date conversion, it
might look more natural.

		Linus

Re: Re-done kernel archive - real one?

[Russell King]

On Sun, Apr 17, 2005 at 09:36:09AM -0700, Linus Torvalds wrote:
> On Sun, 17 Apr 2005, Russell King wrote:
> > On Sat, Apr 16, 2005 at 04:01:45PM -0700, Linus Torvalds wrote:
> > > So I re-created the dang thing (hey, it takes just a few minutes), and
> > > pushed it out, and there's now an archive on kernel.org in my public
> > > "personal" directory called "linux-2.6.git". I'll continue the tradition
> > > of naming git-archive directories as "*.git", since that really ends up
> > > being the ".git" directory for the checked-out thing.
> > 
> > We need to work out how we're going to manage to get our git changes to
> > you.  At the moment, I've very little idea how to do that.  Ideas?
> 
> To me, merging is my highest priority. I suspect that once I have a tree 
> from you (or anybody else) that I actually _test_ merging with, I'll be 
> motivated as hell to make sure that my plumbing actually works. 

Ok, I'll throw this tree onto master.kernel.org - how about
master.kernel.org:/home/rmk/linux-2.6-rmk.git ?  I think it's in the
same format as your trees:

linux-2.6-rmk.git
|-- HEAD
`-- objects

where HEAD was copied from my .git/heads/master, and objects from
.git/objects.

> > However, I've made a start to generate the necessary emails.  How about
> > this format?
> > 
> > I'm not keen on the tree, parent, author and committer objects appearing
> > in this - they appear to clutter it up.  What're your thoughts?
> 
> Indeed. I'd almost drop the whole header except for the "author" line. 

Done.

> Oh, and you need a separator between commits, right now your 
> "Signed-off-by:" line ends up butting up with the header of the next 
> commit ;)

Done.

> > I'd rather not have the FQDN of the machine where the commit happened
> > appearing in the logs.
> 
> That's fine. Out short-logs have always tried to have just the real name 
> in them, and I do want an email-like thing for tracking the developer, but 
> yes, if you remove the email, that's fine. It should be easy enough to do 
> with a simple
> 
> 	sed 's/<.*>//'
> 
> or similar.

Done.

> And if you replace "author" with "From:" and do the date conversion, it
> might look more natural.

Also done. 8)

I still need to work out how to make my noddy script follow different
branches which may be present though.  However, for my common work
flow, it fits what I require.

Ok, how about this format:

Linus,

Please incorporate the latest ARM changes.

This will update the following files:

 arch/arm/kernel/process.c                  |   15 +++++++++++----
 arch/arm/kernel/traps.c                    |    8 ++------
 arch/arm/lib/changebit.S                   |   11 ++---------
 arch/arm/lib/clearbit.S                    |   13 ++-----------
 arch/arm/lib/setbit.S                      |   11 ++---------
 arch/arm/lib/testchangebit.S               |   15 ++-------------
 arch/arm/lib/testclearbit.S                |   15 ++-------------
 arch/arm/lib/testsetbit.S                  |   15 ++-------------
 arch/arm/mach-footbridge/dc21285-timer.c   |    4 ++--
 arch/arm/mach-sa1100/h3600.c               |    2 +-
 include/asm-arm/arch-ebsa285/debug-macro.S |    7 +++++--
 include/asm-arm/arch-rpc/debug-macro.S     |    5 ++++-
 include/asm-arm/ptrace.h                   |    5 +----
 include/asm-arm/system.h                   |    3 +++
 14 files changed, 41 insertions(+), 88 deletions(-)

through these ChangeSets:

Author: Russell King: Sun Apr 17 16:28:31 BST 2005

	[PATCH] ARM: fix debug macros
	
	Fix debug EBSA285 and RiscPC debugging macros to detect whether the
	MMU is enabled.
	
	Signed-off-by: Russell King

Author: Russell King: Sun Apr 17 15:51:02 BST 2005

	[PATCH] ARM: bitops
	
	Convert ARM bitop assembly to a macro.  All bitops follow the same
	format, so it's silly duplicating the code when only one or two
	instructions are different.
	
	Signed-off-by: Russell King

Author: Russell King: Sun Apr 17 15:50:36 BST 2005

	[PATCH] ARM: showregs
	
	Fix show_regs() to provide a backtrace.  Provide a new __show_regs()
	function which implements the common subset of show_regs() and die().
	Add prototypes to asm-arm/system.h
	
	Signed-off-by: Russell King

Author: Russell King: Sun Apr 17 15:40:46 BST 2005

	[PATCH] ARM: h3600_irda_set_speed arguments
	
	h3600_irda_set_speed() had the wrong type for the "speed" argument.
	Fix this.
	
	Signed-off-by: Russell King

Author: Russell King: Sun Apr 17 15:36:55 BST 2005

	[PATCH] ARM: footbridge rtc init
	
	The footbridge ISA RTC was being initialised before we had setup the
	kernel timer.  This caused a divide by zero error when the current
	time of day is set.  Resolve this by initialising the RTC after
	the kernel timer has been initialised.
	
	Signed-off-by: Russell King



-- 
Russell King

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Sun, 17 Apr 2005, Russell King wrote:
> 
> I still need to work out how to make my noddy script follow different
> branches which may be present though.  However, for my common work
> flow, it fits what I require.

The way to handle that is that you need to 

 - remember (or re-fetch) what the latest HEAD was that you merged with in 
   my tree.

   if you didn't remember, you can just get all my objects and do a

	merge-head $(cat .git/HEAD) $linus-current-head

   or something (using the current git archive that has a "merge-head" 
   program. That gives you the most recent common head.

 - use "rev-tree" to show reachability

	rev-tree $my-current-head $last-merge-head
		| sort -n		# sort by date rather than sha1
		| cut -d' ' -f2		# get the sha1 + "flags" mask
		| grep :1		# show the ones that are only
					# reachable from $my-current-head

and you now have a nice list of sha1's ordered by date.

Or something. I didn't test the above. Testing is for users.

		Linus

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Sun, 17 Apr 2005, Linus Torvalds wrote:
> 
>  - use "rev-tree" to show reachability

Btw, you don't even have to really remember the head you merged with. If 
you have all my objects, you can just use the current head from whatever 
my repository is, and the "rev-tree" will show all your commit objects 
that aren't in my tree.

The only reason I said "remember the last head you merged" is that that 
way you don't even have to download the objects from my tree, and you can 
make the decision totally locally without looking at what I've done since.

Of course, at some point you want to get my updated objects _anyway_, 
but..

			Linus

Re: Re-done kernel archive - real one?

[Russell King]

(Dropped HPA from the CC line - I think he was only copied for the
master.kernel.org issues.)

On Sun, Apr 17, 2005 at 12:33:22PM -0700, Linus Torvalds wrote:
> On Sun, 17 Apr 2005, Russell King wrote:
> > I still need to work out how to make my noddy script follow different
> > branches which may be present though.  However, for my common work
> > flow, it fits what I require.
> 
> The way to handle that is that you need to 
> 
>  - remember (or re-fetch) what the latest HEAD was that you merged with in 
>    my tree.
> 
>    if you didn't remember, you can just get all my objects and do a
> 
> 	merge-head $(cat .git/HEAD) $linus-current-head
> 
>    or something (using the current git archive that has a "merge-head" 
>    program. That gives you the most recent common head.

My script currently sends between two commit-ids, so...

>  - use "rev-tree" to show reachability
> 
> 	rev-tree $my-current-head $last-merge-head
> 		| sort -n		# sort by date rather than sha1
> 		| cut -d' ' -f2		# get the sha1 + "flags" mask
> 		| grep :1		# show the ones that are only
> 					# reachable from $my-current-head
> 
> and you now have a nice list of sha1's ordered by date.

This will (and does) do exactly what I want.  I'll also read into the
above a request that you want it in forward date order. 8)

-- 
Russell King

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Sun, 17 Apr 2005, Russell King wrote:
> 
> This will (and does) do exactly what I want.  I'll also read into the
> above a request that you want it in forward date order. 8)

No, I actually don't _think_ I care. In many ways I'm more used to
"reverse date order", because that's usually how you view a changelog
(with a pager, and most recent changes at the top).

Which one makes sense when asking me to merge? I don't know, and I don't
think it really even matters, but maybe we can add a "for now" to whatever 
decision you end up coming to?

		Linus

Re: Re-done kernel archive - real one?

[Russell King]

On Sun, Apr 17, 2005 at 01:08:51PM -0700, Linus Torvalds wrote:
> On Sun, 17 Apr 2005, Russell King wrote:
> > This will (and does) do exactly what I want.  I'll also read into the
> > above a request that you want it in forward date order. 8)
> 
> No, I actually don't _think_ I care. In many ways I'm more used to
> "reverse date order", because that's usually how you view a changelog
> (with a pager, and most recent changes at the top).
> 
> Which one makes sense when asking me to merge? I don't know, and I don't
> think it really even matters, but maybe we can add a "for now" to whatever 
> decision you end up coming to?

It's trivial to change - it's either sort -n or sort -nr !

I'll pick the reverse format so that we generate the changelog in the
order which people have come to expect from our previous set of tools.
Consistency with existing practises is a good thing at this point.

-- 
Russell King

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Sun, 17 Apr 2005, Russell King wrote:
> 
> It's trivial to change - it's either sort -n or sort -nr !

Yes.

Btw, it might make sense to do the "grep" before the sort, since the sort 
doesn't scale as nicely with lots of output. On the other hand, then the 
grep pattern needs to be a bit more complex (so that it doesn't pick up on 
things that have _parents_ with ":1"). I think the rev-tree output is 
obvious enough that it's pretty trivial to grep for..

		Linus

Re: Re-done kernel archive - real one?

[Russell King]

On Sun, Apr 17, 2005 at 01:26:27PM -0700, Linus Torvalds wrote:
> 
> 
> On Sun, 17 Apr 2005, Russell King wrote:
> > 
> > It's trivial to change - it's either sort -n or sort -nr !
> 
> Yes.
> 
> Btw, it might make sense to do the "grep" before the sort, since the sort 
> doesn't scale as nicely with lots of output. On the other hand, then the 
> grep pattern needs to be a bit more complex (so that it doesn't pick up on 
> things that have _parents_ with ":1"). I think the rev-tree output is 
> obvious enough that it's pretty trivial to grep for..

sed works as well - in fact, we can use it to both select lines based
upon the :1 _and_ cut out the bulk of the stuff we're not interested in.
So:

rev-tree $to $prev | \
  sort -nr | cut -d' ' -f2 | grep :1 | cut -d: -f1

becomes:

rev-tree $to $prev | \
  sed -n 's,\([[:digit:]]\+\)[[:space:]]\+\([[:xdigit:]]\+\):1.*,\1 \2,p' | \
  sort -nr | cut -d' ' -f2

(appologies to those who don't like regexps 8))

One thing to note about that sed expression though - using \+ is a GNU
extension - do we care about that?  The portable way is to use \{1,\}
but that'd clutter it some more.

-- 
Russell King

Re: Re-done kernel archive - real one?

[Russell King]

Ok, since the last one was soo successful, and I'm up for more
punishment, here's another attempt.  The diffstat is rather
interesting in this one, claiming no changes.  It should look
like this:

 arch/arm/lib/bitops.h |   33 +++++++++++++++++++++++++++++++++
 1 files changed, 33 insertions(+)

However, it seems that git diff can't handle new files appearing
yet.

The other interesting thing to note is that patches are generated
for '-p0' rather than '-p1' application, which is contary to our
historical requirements.  This is going to confuse people - can
we make it generate -p1 patches please?

Linus - assuming I un-messed-up my tree properly (it appears to
be correct and fsck-cache $(commit-id) is happy) please merge
this.  Thanks.

Linus,

Please incorporate the latest ARM changes, which can
be found at:

	master.kernel.org:/home/rmk/linux-2.6-rmk.git

This will update the following files:

 0 files changed

through these ChangeSets:

From: Russell King: Mon Apr 18 22:50:01 BST 2005
	
	[PATCH] ARM: Add missing new file for bitops patch
	
	Signed-off-by: Russell King


-- 
Russell King

Re: Re-done kernel archive - real one?

[Petr Baudis]

Dear diary, on Tue, Apr 19, 2005 at 12:16:52AM CEST, I got a letter
where Russell King <rmk@arm.linux.org.uk> told me that...
> Ok, since the last one was soo successful, and I'm up for more
> punishment, here's another attempt.  The diffstat is rather
> interesting in this one, claiming no changes.  It should look
> like this:
> 
>  arch/arm/lib/bitops.h |   33 +++++++++++++++++++++++++++++++++
>  1 files changed, 33 insertions(+)
> 
> However, it seems that git diff can't handle new files appearing
> yet.

Patches welcome. :^)

Actually it shouldn't be hard. Just for through add-queue/rm-queue and
diff it with /dev/null.

> The other interesting thing to note is that patches are generated
> for '-p0' rather than '-p1' application, which is contary to our
> historical requirements.  This is going to confuse people - can
> we make it generate -p1 patches please?

This should be fixed with latest Junio's show-diff changes. I might
change it to be identical with gitdiff-do output later though, using
tree ID and "uncommitted" as the directories instead of 'a' and 'b'.
It will stay -p1 now, though.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Mon, 18 Apr 2005, Russell King wrote:
>
> Ok, since the last one was soo successful, and I'm up for more
> punishment, here's another attempt.  The diffstat is rather
> interesting in this one, claiming no changes.  It should look
> like this:
> 
>  arch/arm/lib/bitops.h |   33 +++++++++++++++++++++++++++++++++
>  1 files changed, 33 insertions(+)
> 
> However, it seems that git diff can't handle new files appearing
> yet.

It should definitely be able to do that. 

Do a "git log | less" to look up the trees involved, and do a "git diff
<parent-tree> <child-tree>" to see the output. If you don't see your new
file, then either you have an old "git diff" that doesn't like the new
tools (and you need to add a "-z"  flag to diff-tree), or you didn't 
check in the new file successfully ;)

You can also always do "tree-diff -r old-tree new-tree" which will show
you the tree-level changes. That's the low-level plumbing stuff: it 
doesn't show you the actual file contents, just how the tree changed.

> The other interesting thing to note is that patches are generated
> for '-p0' rather than '-p1' application, which is contary to our
> historical requirements.  This is going to confuse people - can
> we make it generate -p1 patches please?

That should already be the case now after the latest diffs from Junio.

> Linus - assuming I un-messed-up my tree properly (it appears to
> be correct and fsck-cache $(commit-id) is happy) please merge
> this. 

Looks ok, which seems to mean that your scripts are buggered since they 
didn't pick up the new file.

Merge pushed out.

			Linus

Re: Re-done kernel archive - real one?

[Petr Baudis]

Dear diary, on Tue, Apr 19, 2005 at 12:16:52AM CEST, I got a letter
where Russell King <rmk@arm.linux.org.uk> told me that...
> However, it seems that git diff can't handle new files appearing
> yet.

Fixed. :-)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: Re-done kernel archive - real one?

[Russell King]

On Sat, Apr 16, 2005 at 04:01:45PM -0700, Linus Torvalds wrote:
> So I re-created the dang thing (hey, it takes just a few minutes), and
> pushed it out, and there's now an archive on kernel.org in my public
> "personal" directory called "linux-2.6.git". I'll continue the tradition
> of naming git-archive directories as "*.git", since that really ends up
> being the ".git" directory for the checked-out thing.

BTW, there appears to be "errors" in the history committed thus far.
I'm not sure where this came from though.  Some of them could be
UTF8 vs ASCII issues, but there's a number which seem to have extra
random crap in them ("^M)" and lots of blank lines).

One thing which definitely needs to be considered is - what character
encoding are the comments to be stored as?  Currently, it's whatever
the committers character encoding is, which will be completely random.
For instance, dwmw2 will definitely be using UTF, whereas I'll definitely
be using ISO-8859-1  (UTF is far too much of a pain in the ass to use,
unless _all_ your systems are running UTF, which mine don't yet.)

ID: 75f86bac962b7609b0f3c21d25e10647ff8ed280
[PATCH] intel8x0: AC'97 audio patch for Intel ESB2
         
        This patch adds the Intel ESB2 DID's to the intel8x0.c file for AC'97 audio
        support.
         
        Signed-off-by: <A0>Jason Gaston <Jason.d.gaston@intel.com>
        Signed-off-by: Andrew Morton <akpm@osdl.org>
        Signed-off-by: Linus Torvalds <torvalds@osdl.org>

ID: baab5c52135dfa2a02c5dcc8422fc5d048acc682
[PATCH] ppc32: fix compilation error in arch/ppc/kernel/time.c
         
        make defconfig give the following error on ppc (gcc-4):
         
        arch/ppc/kernel/time.c:92: error: static declaration of <E2><80><98>time_offset<E2><80><99>
        follows non-static declaration
        include/linux/timex.h:236: error: previous declaration of <E2><80><98>time_offset<E2><80><99>
        was here
         
        The following patch solves it (time_offset is declared in timer.c).
         
        Signed-Off-By: Benoit Boissinot <benoit.boissinot@ens-lyon.org>
        Signed-off-by: Andrew Morton <akpm@osdl.org>
        Signed-off-by: Linus Torvalds <torvalds@osdl.org>

ID: fa357627062fa6f4727638bc4f302b23f5a3acb8
[PATCH] vmscan: pageout(): remove unneeded test
         
        ^M)
         
         
         
        We only call pageout() for dirty pages, so this test is redundant.
         
        Signed-off-by: Andrew Morton <akpm@osdl.org>
        Signed-off-by: Linus Torvalds <torvalds@osdl.org>

ID: e5fc9026c6a372a67e652c6ba1648e5f8e543600
[PATCH] arm: fix SIGBUS handling
         
        ^M)
         
         
        From: Russell King <rmk+lkml@arm.linux.org.uk>
         
        ARM wasn't raising a SIGBUS with a siginfo structure.  Fix
        __do_user_fault() to allow us to use it for SIGBUS conditions, and arrange
        for the sigbus path to use this.
         
        We need to prevent the siginfo code being called if we do not have a user
        space context to call it, so consolidate the "user_mode()" tests.
         
        Thanks to Ian Campbell who spotted this oversight.
         
        Signed-off-by: Russell King <rmk@arm.linux.org.uk>
        Signed-off-by: Andrew Morton <akpm@osdl.org>
        Signed-off-by: Linus Torvalds <torvalds@osdl.org>


-- 
Russell King

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Sun, 17 Apr 2005, Russell King wrote:
> 
> BTW, there appears to be "errors" in the history committed thus far.
> I'm not sure where this came from though.  Some of them could be
> UTF8 vs ASCII issues, but there's a number which seem to have extra
> random crap in them ("^M)" and lots of blank lines).

Ah, yes. That is actually from the original emails from Andrew. I do not 
know why, but I see them there. It's his script that does something 
strange.

(Andrew: in case you care, the first one is

	[patch 003/198] arm: fix SIGBUS handling

which has the email looking like

	...
	From: akpm@osdl.org
	Date: Tue, 12 Apr 2005 03:30:35 -0700
	Status: 
	X-Status: 
	X-Keywords:                   
	
	^M)
	
	
	From: Russell King <rmk+lkml@arm.linux.org.uk>
	
	ARM wasn't raising a SIGBUS with a siginfo structure.  Fix
	__do_user_fault() to allow us to use it for SIGBUS conditions, and arrange
	for the sigbus path to use this.
	...

> One thing which definitely needs to be considered is - what character
> encoding are the comments to be stored as?

To git, it's just a byte stream, and you can have binary comments if you
want to. I personally would prefer to move towards UTF eventually, but I
really don't think it matters a whole lot as long as 99.9% of everything
we'd see there is still 7-bit ascii.

> ID: 75f86bac962b7609b0f3c21d25e10647ff8ed280
> [PATCH] intel8x0: AC'97 audio patch for Intel ESB2
>          
>         This patch adds the Intel ESB2 DID's to the intel8x0.c file for AC'97 audio
>         support.
>          
>         Signed-off-by: <A0>Jason Gaston <Jason.d.gaston@intel.com>

That <A0> is also there in Andrew's original email. It's "space with the
high bit set", and I have no idea why.

		Linus

Re: Re-done kernel archive - real one?

[David A. Wheeler]

On Sun, 17 Apr 2005, Russell King wrote:
>>BTW, there appears to be "errors" in the history committed thus far.
>>I'm not sure where this came from though.  Some of them could be
>>UTF8 vs ASCII issues, ....> 
...
>>One thing which definitely needs to be considered is - what character
>>encoding are the comments to be stored as?

Linus Torvalds replied:
> To git, it's just a byte stream, and you can have binary comments if you
> want to. I personally would prefer to move towards UTF eventually, but I
> really don't think it matters a whole lot as long as 99.9% of everything
> we'd see there is still 7-bit ascii.

I would _heartily_ recommend moving towards UTF-8 as the
internal charset for all comments.  Alternatives are possible
(e.g., recording the charset in the header), but they're
incredibly messy.  Even if you don't normally work in UTF-8,
it's pretty easy to set most editors up to read & write UTF-8.
Having the data stored as a constant charset eliminates
a raft of error-prone code.

--- David A. Wheeler

Re: Re-done kernel archive - real one?

[Petr Baudis]

Dear diary, on Sun, Apr 17, 2005 at 08:13:59PM CEST, I got a letter
where "David A. Wheeler" <dwheeler@dwheeler.com> told me that...
> On Sun, 17 Apr 2005, Russell King wrote:
> >>BTW, there appears to be "errors" in the history committed thus far.
> >>I'm not sure where this came from though.  Some of them could be
> >>UTF8 vs ASCII issues, ....> 
> ...
> >>One thing which definitely needs to be considered is - what character
> >>encoding are the comments to be stored as?
> 
> Linus Torvalds replied:
> >To git, it's just a byte stream, and you can have binary comments if you
> >want to. I personally would prefer to move towards UTF eventually, but I
> >really don't think it matters a whole lot as long as 99.9% of everything
> >we'd see there is still 7-bit ascii.
> 
> I would _heartily_ recommend moving towards UTF-8 as the
> internal charset for all comments.  Alternatives are possible
> (e.g., recording the charset in the header), but they're
> incredibly messy.  Even if you don't normally work in UTF-8,
> it's pretty easy to set most editors up to read & write UTF-8.
> Having the data stored as a constant charset eliminates
> a raft of error-prone code.

I tend to agree here. My toilet stuff is what can handle various
locale-based conversions at the commit-tree / cat-file tree sides etc,
but UTF-8 should be certainly the way to go internally.

Not that the plumbing should actually _care_ at all; anyone who uses it
should take the care, so this is more of a "social" thing.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: Re-done kernel archive - real one?

[Russell King]

On Sun, Apr 17, 2005 at 02:13:59PM -0400, David A. Wheeler wrote:
> On Sun, 17 Apr 2005, Russell King wrote:
> >>BTW, there appears to be "errors" in the history committed thus far.
> >>I'm not sure where this came from though.  Some of them could be
> >>UTF8 vs ASCII issues, ....> 
> ...
> >>One thing which definitely needs to be considered is - what character
> >>encoding are the comments to be stored as?
> 
> Linus Torvalds replied:
> > To git, it's just a byte stream, and you can have binary comments if you
> > want to. I personally would prefer to move towards UTF eventually, but I
> > really don't think it matters a whole lot as long as 99.9% of everything
> > we'd see there is still 7-bit ascii.
> 
> I would _heartily_ recommend moving towards UTF-8 as the
> internal charset for all comments.  Alternatives are possible
> (e.g., recording the charset in the header), but they're
> incredibly messy.  Even if you don't normally work in UTF-8,
> it's pretty easy to set most editors up to read & write UTF-8.
> Having the data stored as a constant charset eliminates
> a raft of error-prone code.

Except, I believe, MicroEMACS, which both Linus and myself use.  As
far as I know, there aren't any patches to make it UTF-8 compliant.

The alternative is, I suppose, iconv.  However, iconv in _my_ glibc
seems buggy (segfaults) and my efforts for building glibc 2.3.2 for
ARM have failed.  Effectively that means iconv is inaccessible to
me.

-- 
Russell King

Re: Re-done kernel archive - real one?

[David A. Wheeler]

>>On Sun, 17 Apr 2005, Russell King wrote:
>>>>One thing which definitely needs to be considered is - what character
>>>>encoding are the comments to be stored as?
...

I replied:
>>I would _heartily_ recommend moving towards UTF-8 as the
>>internal charset for all comments.

Petr said:
 >Not that the plumbing should actually _care_ at all; anyone who uses it
 >should take the care, so this is more of a "social" thing.

The _plumbing_ shouldn't care, but the stuff above needs to know
how to interpret the stuff that the plumbing produces.

Russell King said:
> Except, I believe, MicroEMACS, which both Linus and myself use.  As
> far as I know, there aren't any patches to make it UTF-8 compliant.

Since plain ASCII is a subset of UTF-8,
as long as MicroEMACS users only create ASCII comments,
then the comments you create in MicroEMACS will still be UTF-8.
No big deal.

For reading comments, if the text is almost entirely
plain ASCII, you could just ignore the problem and have the
occasional character scramble.  If you need more, you'll
need a tool that's more internationalized or a working iconv,
but if that's important you'd be motivated.

Again, I'm looking for more generalized solutions, where
non-English comments are more common than in Linux kernel code.

--- David A. Wheeler

Re: Re-done kernel archive - real one?

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 1698 bytes --]

On Sun, 2005-04-17 at 19:20 +0100, Russell King wrote:
> On Sun, Apr 17, 2005 at 02:13:59PM -0400, David A. Wheeler wrote:
> > On Sun, 17 Apr 2005, Russell King wrote:
> > >>BTW, there appears to be "errors" in the history committed thus far.
> > >>I'm not sure where this came from though.  Some of them could be
> > >>UTF8 vs ASCII issues, ....> 
> > ...
> > >>One thing which definitely needs to be considered is - what character
> > >>encoding are the comments to be stored as?
> > 
> > Linus Torvalds replied:
> > > To git, it's just a byte stream, and you can have binary comments if you
> > > want to. I personally would prefer to move towards UTF eventually, but I
> > > really don't think it matters a whole lot as long as 99.9% of everything
> > > we'd see there is still 7-bit ascii.
> > 
> > I would _heartily_ recommend moving towards UTF-8 as the
> > internal charset for all comments.  Alternatives are possible
> > (e.g., recording the charset in the header), but they're
> > incredibly messy.  Even if you don't normally work in UTF-8,
> > it's pretty easy to set most editors up to read & write UTF-8.
> > Having the data stored as a constant charset eliminates
> > a raft of error-prone code.
> 
> Except, I believe, MicroEMACS, which both Linus and myself use.  As
> far as I know, there aren't any patches to make it UTF-8 compliant.
> 
> The alternative is, I suppose, iconv.  However, iconv in _my_ glibc
> seems buggy (segfaults) and my efforts for building glibc 2.3.2 for
> ARM have failed.  Effectively that means iconv is inaccessible to
> me.
> 

OT, and probably not much help, but glibc-2.3.5 is out ...


-- 
Martin Schlemmer


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Re-done kernel archive - real one?

[H. Peter Anvin]

Linus Torvalds wrote:
> Ok, nobody really objected to the notion of leaving the kernel history
> behind for now, and in fact most people seemed to basically agree. So with
> that decided, the old kernel testing tree was actually perfectly ok,
> except it had been build up with the old-style commit date handling, which
> made me not want to use it as a base for any real work.
> 
> So I re-created the dang thing (hey, it takes just a few minutes), and
> pushed it out, and there's now an archive on kernel.org in my public
> "personal" directory called "linux-2.6.git". I'll continue the tradition
> of naming git-archive directories as "*.git", since that really ends up
> being the ".git" directory for the checked-out thing.
> 
> I'm not going to announce it on linux-kernel yet, because I don't think
> it's useful to anybody but a git person anyway. Besides, I don't actually
> know how happy the kernel.org people are about this distribution method
> and whether it ends up being a horrible disaster for the mirroring setup. 
> 
> Peter made some noises about /pub/scm, which makes sense, and would be a
> better place than my public tree. Apparently there are other places that
> are willing and able to host things too, so we'll see.
> 

I would suggest something like /pub/scm/kernel/git/<people> on 
kernel.org.  This is easy to do, and being outside the "mirrored 
worldwide" set shouldn't cause anyone any issues.

/pub/linux/... is mirrored worldwide and that might cause some 
consternation.

If this is OK with everyone I'll try to set them up (I'm at LCA at the 
moment, so I might have limited connectivity at any one particular 
moment in time.)

	-hpa

Re: Re-done kernel archive - real one?

[Jochen Roemling]

Linus Torvalds wrote:

>Ie we have two phases to the merge: first get the objects, with something
>like
>
>	repo=kernel.org:/pub/kernel/people/torvalds/linux-2.6.git
>	rsync --ignore-existing -acv $(repo)/ .git/
>  
>
Could you place a tarball there for people like me who are no "real" 
kernel hackers and don't have a kernel.org account? Or is there an 
"anonymous" account that I'm just to ignorant to know of?

I'm just somebody who is very interested in the new things happening 
here, but I guess there will be some month to go and a lot of things to 
learn until you see the first kernel patch that is signed off by me. :-)

By the way: Does the git repository include all 60.000 changes of just 
the latest version of the 17.000 file in the kernel?

Best regards
Jochen Römling

Re: Re-done kernel archive - real one?

[Randy.Dunlap]

On Sun, 17 Apr 2005 23:50:40 +0200 Jochen Roemling wrote:

| Linus Torvalds wrote:
| 
| >Ie we have two phases to the merge: first get the objects, with something
| >like
| >
| >	repo=kernel.org:/pub/kernel/people/torvalds/linux-2.6.git
| >	rsync --ignore-existing -acv $(repo)/ .git/
| >  
| >
| Could you place a tarball there for people like me who are no "real" 
| kernel hackers and don't have a kernel.org account? Or is there an 
| "anonymous" account that I'm just to ignorant to know of?

You don't need a kernel.org account to rsync it... this works too:

rsync -avz -e ssh --progress --ignore-existing  rsync://rsync.kernel.org/pub/linux/kernel/people/torvalds/linux-2.6.git/ .git/

| I'm just somebody who is very interested in the new things happening 
| here, but I guess there will be some month to go and a lot of things to 
| learn until you see the first kernel patch that is signed off by me. :-)
| 
| By the way: Does the git repository include all 60.000 changes of just 
| the latest version of the 17.000 file in the kernel?

It's currently just 2.6.12-rc2 (17,000 files) plus changes...

Two people have announced full history gits if you are interested
in that...  Check the git email archives for mail from Ingo Molnar:
Subject: full kernel history, in patchset format

and Thomas Gleixner <tglx@linutronix.de>:
Subject: BK -> git export done

---
~Randy

Re: Re-done kernel archive - real one?

[Petr Baudis]

Dear diary, on Mon, Apr 18, 2005 at 12:09:21AM CEST, I got a letter
where "Randy.Dunlap" <rddunlap@osdl.org> told me that...
> On Sun, 17 Apr 2005 23:50:40 +0200 Jochen Roemling wrote:
> 
> | Linus Torvalds wrote:
> | 
> | >Ie we have two phases to the merge: first get the objects, with something
> | >like
> | >
> | >	repo=kernel.org:/pub/kernel/people/torvalds/linux-2.6.git
> | >	rsync --ignore-existing -acv $(repo)/ .git/
> | >  
> | >
> | Could you place a tarball there for people like me who are no "real" 
> | kernel hackers and don't have a kernel.org account? Or is there an 
> | "anonymous" account that I'm just to ignorant to know of?
> 
> You don't need a kernel.org account to rsync it... this works too:
> 
> rsync -avz -e ssh --progress --ignore-existing  rsync://rsync.kernel.org/pub/linux/kernel/people/torvalds/linux-2.6.git/ .git/

Or

	mkdir linux-2.6.git
	cd linux-2.6.git
	RSYNC_FLAGS="--progress --stats" git init rsync://rsync.kernel.org/pub/linux/kernel/people/torvalds/linux-2.6.git/

which also does the initial checkout for you. ;-) (Requires latest
git-pasky, though.) Then you can bring the latest and greatest to your
tree by a mere git pull.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: Re-done kernel archive - real one?

[David Woodhouse]

On Sat, 2005-04-16 at 16:01 -0700, Linus Torvalds wrote:
> So I re-created the dang thing (hey, it takes just a few minutes), and
> pushed it out, and there's now an archive on kernel.org in my public
> "personal" directory called "linux-2.6.git". I'll continue the tradition
> of naming git-archive directories as "*.git", since that really ends up
> being the ".git" directory for the checked-out thing.

Do you want the commits list running for it yet? Do you want the
changesets which are already in it re-mailed without a 'TESTING' tag?

-- 
dwmw2

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Mon, 18 Apr 2005, David Woodhouse wrote:
> 
> Do you want the commits list running for it yet? Do you want the
> changesets which are already in it re-mailed without a 'TESTING' tag?

I really don't know. I'm actually very happy where this thing is right
now, and completing that first merge successfully was a big milestone to
me personally. That said, actually _using_ this thing is not for the
faint-of-heart, and while I think "git" already is showing itself to be
useful, I'm very very biased.

In other words, I really wonder what an outsider that doesn't have the
same kind of mental bias thinks of the current git tree. Is it useful, or
is it still just a toy for Linus to test out his crazy SCM-wannabe.

Can people usefully track my current kernel git repository, or do you have
to be crazy to do so? That's really the question. You be the judge. Me,
I'm just giddy from a merge that was clearly done using interfaces that
aren't actually really usable for anybody but me, and barely me at that ;)

		Linus

Btw, I also do want this to show up in the BK trees for people who use 
BitKeeper - the same way we always supported tar-ball + patch users 
before. So I'll have to try to come up with some sane way to do that too. 
Any ideas? The first series of 198 patches is obvious enough and can be 
just done that way direcly, but the merge..

Re: Re-done kernel archive - real one?

[Russell King]

On Sun, Apr 17, 2005 at 03:17:50PM -0700, Linus Torvalds wrote:
> Can people usefully track my current kernel git repository, or do you have
> to be crazy to do so? That's really the question. You be the judge. Me,
> I'm just giddy from a merge that was clearly done using interfaces that
> aren't actually really usable for anybody but me, and barely me at that ;)

I guess I'll have the pleasure to find that out when I update my tree
with your latest changes... which I think is a project for tomorrow.

-- 
Russell King

Re: Re-done kernel archive - real one?

[Russell King]

On Sun, Apr 17, 2005 at 11:19:59PM +0100, Russell King wrote:
> On Sun, Apr 17, 2005 at 03:17:50PM -0700, Linus Torvalds wrote:
> > Can people usefully track my current kernel git repository, or do you have
> > to be crazy to do so? That's really the question. You be the judge. Me,
> > I'm just giddy from a merge that was clearly done using interfaces that
> > aren't actually really usable for anybody but me, and barely me at that ;)
> 
> I guess I'll have the pleasure to find that out when I update my tree
> with your latest changes... which I think is a project for tomorrow.

I pulled it tonight into a pristine tree (which of course worked.)

In doing so, I noticed that I'd messed up one of the commits - there's
a missing new file.  Grr.  I'll put that down to being a newbie git.

-- 
Russell "newbie git" King

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Sun, 17 Apr 2005, Russell King wrote:
> 
> I pulled it tonight into a pristine tree (which of course worked.)

Goodie.

> In doing so, I noticed that I'd messed up one of the commits - there's
> a missing new file.  Grr.  I'll put that down to being a newbie git.

Actually, you should put that down to horribly bad interface tools.  With
BK, we had these nice tools that pointed out that there were files that
you might want to commit (ie "bk citool"), and made this very obvious.

Tools absolutely matter. And it will take time for us to build up that 
kind of helper infrastructure. So being newbie might be part of it, but 
it's the smaller part, I say. Rough interfaces is a big issue.

		Linus

Re: Re-done kernel archive - real one?

[Russell King]

On Sun, Apr 17, 2005 at 04:24:24PM -0700, Linus Torvalds wrote:
> On Sun, 17 Apr 2005, Russell King wrote:
> > I pulled it tonight into a pristine tree (which of course worked.)
> 
> Goodie.

Note the "pristine".  Now comes the real test...

> > In doing so, I noticed that I'd messed up one of the commits - there's
> > a missing new file.  Grr.  I'll put that down to being a newbie git.
> 
> Actually, you should put that down to horribly bad interface tools.  With
> BK, we had these nice tools that pointed out that there were files that
> you might want to commit (ie "bk citool"), and made this very obvious.
> 
> Tools absolutely matter. And it will take time for us to build up that 
> kind of helper infrastructure. So being newbie might be part of it, but 
> it's the smaller part, I say. Rough interfaces is a big issue.

Ok, I just tried pulling your tree into the tree you pulled from, and
got this:

Tree change: e7905b2f22eb5d5308c9122b9c06c2d02473dd4f ee423ea56280512778a5961ee58a785a73acb7d1
...
*100644->100644 blob    46f0a3caae02b4bb8f903d7ac86456aa0c37954b->ba4afd7956173b6f89eb6b0b9ad23b392d5c0aee      arch/arm/kernel/process.c
*100644->100644 blob    4a36fa7192e11df36f5e0928b064239dabe1e305->ec0bc8f315ab5d78a4220e176e7aee76d52d1c74      arch/arm/kernel/traps.c
*100644->100644 blob    311d19ee00208faf02359f9e7c5394577a40f253->bf923a953703c6ca0c88eac3b2850cf07b838996      arch/arm/lib/changebit.S
*100644->100644 blob    c07afa31695654e6489ec59c3f837183b325e9da->41f89b3a393d5af939f04f63c5bf4991b2bf6599      arch/arm/lib/clearbit.S
...
Tracked branch, applying changes...
Merging e7905b2f22eb5d5308c9122b9c06c2d02473dd4f -> ee423ea56280512778a5961ee58a785a73acb7d1
        to df4449813c900973841d0fa5a9e9bc7186956e1e...
COPYING: needs update
CREDITS: needs update
Documentation/00-INDEX: needs update
Documentation/BK-usage/00-INDEX: needs update
...
patching file arch/arm/kernel/process.c
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file arch/arm/kernel/process.c.rejpatching file arch/arm/kernel/traps.c
Reversed (or previously applied) patch detected!  Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file arch/arm/kernel/traps.c.rej
patching file arch/arm/lib/changebit.S
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file arch/arm/lib/changebit.S.rej
patching file arch/arm/lib/clearbit.S
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file arch/arm/lib/clearbit.S.rej

so obviously git pull isn't able to indentify what's already in the
local repository.

Interestingly, the files listed above as having rejects are excluded
from the list of "needs update".  And I don't know why git is staying
that these files need updating, because they haven't changed since
they were initially checked out.

This was with some random version of git-pasky-0.04.  Unfortunately,
this version doesn't have the sha1 ID appended, so I couldn't say
definitively that it's the latest and greatest.  It might be a day
old.

-- 
Russell King

Re: Re-done kernel archive - real one?

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 3598 bytes --]

On Mon, 2005-04-18 at 10:23 +0100, Russell King wrote:
> On Sun, Apr 17, 2005 at 04:24:24PM -0700, Linus Torvalds wrote:
> > On Sun, 17 Apr 2005, Russell King wrote:
> > > I pulled it tonight into a pristine tree (which of course worked.)
> > 
> > Goodie.
> 
> Note the "pristine".  Now comes the real test...
> 
> > > In doing so, I noticed that I'd messed up one of the commits - there's
> > > a missing new file.  Grr.  I'll put that down to being a newbie git.
> > 
> > Actually, you should put that down to horribly bad interface tools.  With
> > BK, we had these nice tools that pointed out that there were files that
> > you might want to commit (ie "bk citool"), and made this very obvious.
> > 
> > Tools absolutely matter. And it will take time for us to build up that 
> > kind of helper infrastructure. So being newbie might be part of it, but 
> > it's the smaller part, I say. Rough interfaces is a big issue.
> 
> Ok, I just tried pulling your tree into the tree you pulled from, and
> got this:
> 
> Tree change: e7905b2f22eb5d5308c9122b9c06c2d02473dd4f ee423ea56280512778a5961ee58a785a73acb7d1
> ...
> *100644->100644 blob    46f0a3caae02b4bb8f903d7ac86456aa0c37954b->ba4afd7956173b6f89eb6b0b9ad23b392d5c0aee      arch/arm/kernel/process.c
> *100644->100644 blob    4a36fa7192e11df36f5e0928b064239dabe1e305->ec0bc8f315ab5d78a4220e176e7aee76d52d1c74      arch/arm/kernel/traps.c
> *100644->100644 blob    311d19ee00208faf02359f9e7c5394577a40f253->bf923a953703c6ca0c88eac3b2850cf07b838996      arch/arm/lib/changebit.S
> *100644->100644 blob    c07afa31695654e6489ec59c3f837183b325e9da->41f89b3a393d5af939f04f63c5bf4991b2bf6599      arch/arm/lib/clearbit.S
> ...
> Tracked branch, applying changes...
> Merging e7905b2f22eb5d5308c9122b9c06c2d02473dd4f -> ee423ea56280512778a5961ee58a785a73acb7d1
>         to df4449813c900973841d0fa5a9e9bc7186956e1e...
> COPYING: needs update
> CREDITS: needs update
> Documentation/00-INDEX: needs update
> Documentation/BK-usage/00-INDEX: needs update
> ...
> patching file arch/arm/kernel/process.c
> Reversed (or previously applied) patch detected!  Skipping patch.
> 2 out of 2 hunks ignored -- saving rejects to file arch/arm/kernel/process.c.rejpatching file arch/arm/kernel/traps.c
> Reversed (or previously applied) patch detected!  Skipping patch.
> 3 out of 3 hunks ignored -- saving rejects to file arch/arm/kernel/traps.c.rej
> patching file arch/arm/lib/changebit.S
> Reversed (or previously applied) patch detected!  Skipping patch.
> 2 out of 2 hunks ignored -- saving rejects to file arch/arm/lib/changebit.S.rej
> patching file arch/arm/lib/clearbit.S
> Reversed (or previously applied) patch detected!  Skipping patch.
> 2 out of 2 hunks ignored -- saving rejects to file arch/arm/lib/clearbit.S.rej
> 
> so obviously git pull isn't able to indentify what's already in the
> local repository.
> 
> Interestingly, the files listed above as having rejects are excluded
> from the list of "needs update".  And I don't know why git is staying
> that these files need updating, because they haven't changed since
> they were initially checked out.
> 
> This was with some random version of git-pasky-0.04.  Unfortunately,
> this version doesn't have the sha1 ID appended, so I couldn't say
> definitively that it's the latest and greatest.  It might be a day
> old.
> 

gitmerge.sh does not yet have support for the new merge stuff as far as
I know, and if it does, then its a very recent version (ie, one that
have the sha1 ID appended).


-- 
Martin Schlemmer


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Re-done kernel archive - real one?

[Petr Baudis]

Dear diary, on Mon, Apr 18, 2005 at 11:23:32AM CEST, I got a letter
where Russell King <rmk@arm.linux.org.uk> told me that...
> Ok, I just tried pulling your tree into the tree you pulled from, and
> got this:
> 
> Tree change: e7905b2f22eb5d5308c9122b9c06c2d02473dd4f ee423ea56280512778a5961ee58a785a73acb7d1
> ...
> *100644->100644 blob    46f0a3caae02b4bb8f903d7ac86456aa0c37954b->ba4afd7956173b6f89eb6b0b9ad23b392d5c0aee      arch/arm/kernel/process.c
> *100644->100644 blob    4a36fa7192e11df36f5e0928b064239dabe1e305->ec0bc8f315ab5d78a4220e176e7aee76d52d1c74      arch/arm/kernel/traps.c
> *100644->100644 blob    311d19ee00208faf02359f9e7c5394577a40f253->bf923a953703c6ca0c88eac3b2850cf07b838996      arch/arm/lib/changebit.S
> *100644->100644 blob    c07afa31695654e6489ec59c3f837183b325e9da->41f89b3a393d5af939f04f63c5bf4991b2bf6599      arch/arm/lib/clearbit.S
> ...
> Tracked branch, applying changes...
> Merging e7905b2f22eb5d5308c9122b9c06c2d02473dd4f -> ee423ea56280512778a5961ee58a785a73acb7d1
>         to df4449813c900973841d0fa5a9e9bc7186956e1e...
> COPYING: needs update
> CREDITS: needs update
> Documentation/00-INDEX: needs update
> Documentation/BK-usage/00-INDEX: needs update
> ...
> patching file arch/arm/kernel/process.c
> Reversed (or previously applied) patch detected!  Skipping patch.
> 2 out of 2 hunks ignored -- saving rejects to file arch/arm/kernel/process.c.rejpatching file arch/arm/kernel/traps.c
> Reversed (or previously applied) patch detected!  Skipping patch.
> 3 out of 3 hunks ignored -- saving rejects to file arch/arm/kernel/traps.c.rej
> patching file arch/arm/lib/changebit.S
> Reversed (or previously applied) patch detected!  Skipping patch.
> 2 out of 2 hunks ignored -- saving rejects to file arch/arm/lib/changebit.S.rej
> patching file arch/arm/lib/clearbit.S
> Reversed (or previously applied) patch detected!  Skipping patch.
> 2 out of 2 hunks ignored -- saving rejects to file arch/arm/lib/clearbit.S.rej
> 
> so obviously git pull isn't able to indentify what's already in the
> local repository.

For unknown reason, patch sucks at this, even if I pass it

-N  or  --forward
  Ignore patches that seem to be reversed or already applied.  See also -R.

:-(

git merge does not use the in-GIT merging capabilities yet.

Could you do

	merge-base $(tree-id ee423ea56280512778a5961ee58a785a73acb7d1) \
		$(tree-id df4449813c900973841d0fa5a9e9bc7186956e1e)

and check if it returns e7905b2f22eb5d5308c9122b9c06c2d02473dd4f, please?
I guess it won't. So you could try to wipe the -b "$orig_head" from
gitmerge.sh invocation in gitpull.sh.

> Interestingly, the files listed above as having rejects are excluded
> from the list of "needs update".  And I don't know why git is staying
> that these files need updating, because they haven't changed since
> they were initially checked out.

Because it is checking out only the files which changed (pass -a to git
merge to check out everything; I will probably make git pull take a -m
argument which will take merge arguments ;). And then it does
update-cache --refresh which complains about missing files. :/

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Mon, 18 Apr 2005, Russell King wrote:
> 
> Ok, I just tried pulling your tree into the tree you pulled from, and
> got this:

No, that can't work. The pesky tools are helpful, but they really don't do 
merges worth cr*p right now, excuse my french. 

The _real_ way to pull is to do the (horribly complex) thing I described
by the merge, but noticing that one of the commits you are merging is a
proper subset of the other one, and just updating the head instead of
actually doing a real merge (ie skipping the "read-tree -m" and
"write-tree" phases).

> This was with some random version of git-pasky-0.04.  Unfortunately,
> this version doesn't have the sha1 ID appended, so I couldn't say
> definitively that it's the latest and greatest.  It might be a day
> old.

I'm afraid that until Pasky's tools script this properly, a "pull" really 
ends up being something like this (which _can_ be scripted, never fear):

NOTE NOTE NOTE! This is untested! I'm writing this within the email 
editor, so do _not_ do this on a tree that you care about.

	#!/bin/sh
	#
	# use "$1" or something in a real script, this 
	# just hard-codes it.
	#
	merge_repo=master.kernel.org:/pub/linux/kernel/people/torvalds/linux-2.6.git

	echo "Getting object database"
	rsync -avz --ignore-existing $merge_repo/ .git/

	echo "Getting remote head"
	rsync -avz $merge_repo/HEAD .git/MERGE_HEAD

	head=$(cat .git/HEAD)
	merge_head=$(cat .git/MERGE-HEAD)
	common=$(merge-base $head $merge_head)
	if [ -z "$common" ]; then
		echo "Unable to find common commit between" $merge_head $head
		exit 1
	fi

	# Get the trees associated with those commits
	common_tree=tree=$(cat-file commit $common | sed 's/tree //;q')
	head_tree=tree=$(cat-file commit $head | sed 's/tree //;q')
	merge_tree=tree=$(cat-file commit $merge | sed 's/tree //;q')

	if [ "$common" == "$merge_head" ]; then
		echo "Already up-to-date. Yeeah!"
		exit 0
	fi
	if [ "$common" == "$head" ]; then
		echo "Updating from $head to $merge_head."
		echo "Destroying all noncommitted data!"
		echo "Kill me within 3 seconds.."
		sleep 3
		read-tree $merge_tree && checkout-cache -f -a
		echo $merge_head > .git/HEAD
		exit 0
	fi
	echo "Trying to merge $merge_head into $head"
	read-tree -m $common_tree $head_tree $merge_tree
	result_tree=$(write-tree) || exit 1
	result_commit=$(echo "Merge $merge_repo" | commit-tree $result_tree -p $head -p $merge_head)
	echo "Committed merge $result_commit"
	echo $result_commit > .git/HEAD
	read-tree $result_tree && checkout-cache -f -a

The above looks like it might work, but I also warn you: it's not only
untested, but it's pretty fragile in that if something breaks, you are
probably left with a mess. I _tried_ to do the right thing, but... So it
obviously will need testing, tweaking and just general tender loving care.

And if the merge isn't clean, it will exit early thanks to the

	write-tree || exit 1

and now you have to resolve the merge yourself. There are tools to help
you do so automatically, but that's really a separate script.

You shouldn't hit the "merge" case at all right now, you should hit the 
"Updating from $head to $merge_head" thing.

If Pesky wants to take the above script, test it, and see if it works,
that would be good. It's definitely a much better "pull" than trying to
apply the patches forward..

		Linus

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Mon, 18 Apr 2005, Linus Torvalds wrote:
> 
> No, that can't work. The pesky tools are helpful [...]
> I'm afraid that until Pasky's tools script this properly, [... ]
> If Pesky wants to take the above script, test it, [...]

Ok, one out of three isn't too bad, is it? Pesky/Pasky, so close yet so 
far. Sorry,

		Linus

Re: Re-done kernel archive - real one?

[Petr Baudis]

Dear diary, on Mon, Apr 18, 2005 at 07:05:19PM CEST, I got a letter
where Linus Torvalds <torvalds@osdl.org> told me that...
> 
> 
> On Mon, 18 Apr 2005, Linus Torvalds wrote:
> > 
> > No, that can't work. The pesky tools are helpful [...]
> > I'm afraid that until Pasky's tools script this properly, [... ]
> > If Pesky wants to take the above script, test it, [...]
> 
> Ok, one out of three isn't too bad, is it? Pesky/Pasky, so close yet so 
> far. Sorry,

No problem. :-) Or you can just call me Petr if you want. ;-)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: Re-done kernel archive - real one?

[Russell King]

On Mon, Apr 18, 2005 at 08:23:16AM -0700, Linus Torvalds wrote:
> NOTE NOTE NOTE! This is untested! I'm writing this within the email 
> editor, so do _not_ do this on a tree that you care about.

It did the right thing for me at least!

> 	#!/bin/sh
> 	#
> 	# use "$1" or something in a real script, this 
> 	# just hard-codes it.
> 	#
> 	merge_repo=master.kernel.org:/pub/linux/kernel/people/torvalds/linux-2.6.git
> 
> 	echo "Getting object database"
> 	rsync -avz --ignore-existing $merge_repo/ .git/
> 
> 	echo "Getting remote head"
> 	rsync -avz $merge_repo/HEAD .git/MERGE_HEAD
> 
> 	head=$(cat .git/HEAD)
> 	merge_head=$(cat .git/MERGE-HEAD)
> 	common=$(merge-base $head $merge_head)
> 	if [ -z "$common" ]; then
> 		echo "Unable to find common commit between" $merge_head $head
> 		exit 1
> 	fi
> 
> 	# Get the trees associated with those commits
> 	common_tree=tree=$(cat-file commit $common | sed 's/tree //;q')
> 	head_tree=tree=$(cat-file commit $head | sed 's/tree //;q')
> 	merge_tree=tree=$(cat-file commit $merge | sed 's/tree //;q')

This wants to be:

	common_tree=$(cat-file commit $common | sed 's/tree //;q')
	head_tree=$(cat-file commit $head | sed 's/tree //;q')
	merge_tree=$(cat-file commit $merge_head | sed 's/tree //;q')

> 	if [ "$common" == "$merge_head" ]; then
> 		echo "Already up-to-date. Yeeah!"
> 		exit 0
> 	fi
> 	if [ "$common" == "$head" ]; then
> 		echo "Updating from $head to $merge_head."
> 		echo "Destroying all noncommitted data!"
> 		echo "Kill me within 3 seconds.."
> 		sleep 3
> 		read-tree $merge_tree && checkout-cache -f -a

Don't we want to do an update-cache --refresh here?

> 		echo $merge_head > .git/HEAD
> 		exit 0
> 	fi
> 	echo "Trying to merge $merge_head into $head"
> 	read-tree -m $common_tree $head_tree $merge_tree
> 	result_tree=$(write-tree) || exit 1
> 	result_commit=$(echo "Merge $merge_repo" | commit-tree $result_tree -p $head -p $merge_head)
> 	echo "Committed merge $result_commit"
> 	echo $result_commit > .git/HEAD
> 	read-tree $result_tree && checkout-cache -f -a
> 
> The above looks like it might work, but I also warn you: it's not only
> untested, but it's pretty fragile in that if something breaks, you are
> probably left with a mess. I _tried_ to do the right thing, but... So it
> obviously will need testing, tweaking and just general tender loving care.

Maybe Petr can improve the error handling, and incorporate it (or at
least some of it) into git-pasky

> You shouldn't hit the "merge" case at all right now, you should hit the 
> "Updating from $head to $merge_head" thing.

Exactly what happened.  Thanks.

-- 
Russell King

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Mon, 18 Apr 2005, Russell King wrote:
> > 
> > 	# Get the trees associated with those commits
> > 	common_tree=tree=$(cat-file commit $common | sed 's/tree //;q')
> > 	head_tree=tree=$(cat-file commit $head | sed 's/tree //;q')
> > 	merge_tree=tree=$(cat-file commit $merge | sed 's/tree //;q')
> 
> This wants to be:
> 
> 	common_tree=$(cat-file commit $common | sed 's/tree //;q')
> 	head_tree=$(cat-file commit $head | sed 's/tree //;q')
> 	merge_tree=$(cat-file commit $merge_head | sed 's/tree //;q')

Yup, I already fixed that in the current git version.

> > 		read-tree $merge_tree && checkout-cache -f -a
> 
> Don't we want to do an update-cache --refresh here?

Yes. Thanks, added.

		Linus

Re: Re-done kernel archive - real one?

[Petr Baudis]

Dear diary, on Mon, Apr 18, 2005 at 11:53:57PM CEST, I got a letter
where Russell King <rmk@arm.linux.org.uk> told me that...
> Maybe Petr can improve the error handling, and incorporate it (or at
> least some of it) into git-pasky

This does not need to touch git pull at all now; all the relevant logic
can change in git merge (and git commit), and I'm hacking that now. It
should be rather easy, I think.

I think I won't make git merge commit automatically - I think the user
should get a chance to do a git diff on what is getting merged to check
if everything is all right.

What is actually a little annoying is having to cd ,,merge and then
back, though. I don't know, but the current pull-merge script does not
bother with the temporary merge directory neither, even though Linus
wanted it. Linus, do you still do? ;-)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: Re-done kernel archive - real one?

[Russell King]

On Tue, Apr 19, 2005 at 12:48:52AM +0200, Petr Baudis wrote:
> Dear diary, on Mon, Apr 18, 2005 at 11:53:57PM CEST, I got a letter
> where Russell King <rmk@arm.linux.org.uk> told me that...
> > Maybe Petr can improve the error handling, and incorporate it (or at
> > least some of it) into git-pasky
> 
> This does not need to touch git pull at all now; all the relevant logic
> can change in git merge (and git commit), and I'm hacking that now. It
> should be rather easy, I think.
> 
> I think I won't make git merge commit automatically - I think the user
> should get a chance to do a git diff on what is getting merged to check
> if everything is all right.
> 
> What is actually a little annoying is having to cd ,,merge and then
> back, though. I don't know, but the current pull-merge script does not
> bother with the temporary merge directory neither, even though Linus
> wanted it. Linus, do you still do? ;-)

In the case I highlighted, we don't want to end up having to require
user intervention.  This is a common case here, and was one which was
entirely scripted with BK.

Essentially, with BK, at 7am localtime each morning, I'd:

- update my baseline linux 2.6 tree
- for each working tree which may be pulled from
  - if the baseline is a superset
    - update working tree from baseline

The net result is that my workflow consisted entirely of:

1. commit whatever into working tree
2. test
3. send linus a pull request
4. repeat next day

The tree resynchronisation happened completely and entirely in the
background with no user intervention required at all.

With your suggested requirement for user intervention whenever there's
a merge, it means that this just isn't possible - you could automate
the pulls, but you need to ensure that you'd visited each and every
unmerged tree before the next day, or you don't script it at all and
do the whole thing manually.

Hey, I'm lazy, and that means that just won't get done, and my trees
will end up being horrendously out of date all the time.  But isn't
this precisely what we have computers and scripts for?

-- 
Russell King

Re: Re-done kernel archive - real one?

[Petr Baudis]

Dear diary, on Tue, Apr 19, 2005 at 12:59:52AM CEST, I got a letter
where Russell King <rmk@arm.linux.org.uk> told me that...
> In the case I highlighted, we don't want to end up having to require
> user intervention.  This is a common case here, and was one which was
> entirely scripted with BK.

Well, you can script that

	cd ,,merge && echo 'Merge with Linus' | git ci && cd ..

too. ;-)

But it seems like a good idea to just proceed with commit in case of no
conflicts (and possibly have a switch which will tell git merge to just
merge, not commit).

> Essentially, with BK, at 7am localtime each morning, I'd:
> 
> - update my baseline linux 2.6 tree
> - for each working tree which may be pulled from
>   - if the baseline is a superset
>     - update working tree from baseline
> 
> The net result is that my workflow consisted entirely of:
> 
> 1. commit whatever into working tree
> 2. test
> 3. send linus a pull request
> 4. repeat next day
> 
> The tree resynchronisation happened completely and entirely in the
> background with no user intervention required at all.

And in the case of conflicts...?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: Re-done kernel archive - real one?

[Russell King]

On Tue, Apr 19, 2005 at 01:09:42AM +0200, Petr Baudis wrote:
> > Essentially, with BK, at 7am localtime each morning, I'd:
> > 
> > - update my baseline linux 2.6 tree
> > - for each working tree which may be pulled from
> >   - if the baseline is a superset
> >     - update working tree from baseline
> > 
> > The net result is that my workflow consisted entirely of:
> > 
> > 1. commit whatever into working tree
> > 2. test
> > 3. send linus a pull request
> > 4. repeat next day
> > 
> > The tree resynchronisation happened completely and entirely in the
> > background with no user intervention required at all.
> 
> And in the case of conflicts...?

If the baseline is a superset of the working tree, there will never be
any conflicts.  Note that as I said above, this is a condition on doing
the pull in the first place.

How we determine that with git is another matter though. 8)

-- 
Russell King

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Tue, 19 Apr 2005, Petr Baudis wrote:
> 
> What is actually a little annoying is having to cd ,,merge and then
> back, though. I don't know, but the current pull-merge script does not
> bother with the temporary merge directory neither, even though Linus
> wanted it. Linus, do you still do? ;-)

No, now that the merge is done entirely in the index file, I don't care 
any more. The index file _is_ the temporary directory as far as I'm 
concerned.

		Linus

Re: Re-done kernel archive - real one?

[Russell King]

On Mon, Apr 18, 2005 at 10:23:32AM +0100, Russell King wrote:
> On Sun, Apr 17, 2005 at 04:24:24PM -0700, Linus Torvalds wrote:
> > On Sun, 17 Apr 2005, Russell King wrote:
> > > I pulled it tonight into a pristine tree (which of course worked.)
> > 
> > Goodie.
> 
> Note the "pristine".  Now comes the real test...
> 
> > > In doing so, I noticed that I'd messed up one of the commits - there's
> > > a missing new file.  Grr.  I'll put that down to being a newbie git.
> > 
> > Actually, you should put that down to horribly bad interface tools.  With
> > BK, we had these nice tools that pointed out that there were files that
> > you might want to commit (ie "bk citool"), and made this very obvious.
> > 
> > Tools absolutely matter. And it will take time for us to build up that 
> > kind of helper infrastructure. So being newbie might be part of it, but 
> > it's the smaller part, I say. Rough interfaces is a big issue.
> 
> Ok, I just tried pulling your tree into the tree you pulled from, and
> got this:
>...

Since this happened, I've been working out what state my tree is in,
and I restored it back to a state where I had one dangling commit head,
which was _my_ head.

I then checked whether my objects matched the objects which I uploaded
to master.kernel.org, and discovered I'd removed some extra ones.  With
them restored, I have an additional dangling commit.

Now, I'm pretty sure that I had an up to date tree when I did the
original commits, so I'm a little confused.

What I'm seeing is:

b4a9a5114b3c6da131a832a8e2cd1941161eb348
+- e7905b2f22eb5d5308c9122b9c06c2d02473dd4f
   +- dc90c0db0dd5214aca5304fd17ccd741031e5493 <-- extra dangling head
   +- 488faba31f59c5960aabbb2a5877a0f2923937a3
      +- 5d9a545981893629c8f95e2b8b50d15d18c6ddbc
         +- d5922e9c35d21f0b6b82d1fd8b1444cfce57ca34
            +- ff219d69be01af1fd04ada305b5fe7cd4c563cc6
               +- df4449813c900973841d0fa5a9e9bc7186956e1e <-- my head

It's very much like I somehow committed against the _parent_ of the
head, rather than the head itself.

However, I've lost the state that this tree was in when I did the initial
commit, so who knows why this happened...  I think it's something to
keep an eye out for though.

-- 
Russell King

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Mon, 18 Apr 2005, Russell King wrote:
> 
> Since this happened, I've been working out what state my tree is in,
> and I restored it back to a state where I had one dangling commit head,
> which was _my_ head.

For the future, if your tree gets messed up to the point where you say 
"screw it" and just want to go back in time, you can do this (it's 
equivalent to "undo" in BK speak):

	git log | less -S

	.. find which HEAD it was that you trusted..

In this case your HEAD before I merged with it was this one:

	df4449813c900973841d0fa5a9e9bc7186956e1e

So to get back to that one, you can do

	echo df4449813c900973841d0fa5a9e9bc7186956e1e > .git/HEAD

and now

	cat-file commit $(cat .git/HEAD) | head -1

gives you

	tree a43c4447b2edc9fb01a6369f10c1165de4494c88

so you can restore your checked-out state with

	read-tree a43c4447b2edc9fb01a6369f10c1165de4494c88
	checkout-cache -f -a
	update-cache --refresh

and your tree should be valid again.

Now, to remove any bogus objects, you can then run my "git-prune-script"
(look at it carefully first to make sure you realize what you are doing).

NOTE NOTE NOTE! This will _revert_ everything you had done after the 
"trusted" point. So you may not actually want to do this. Instead:

> It's very much like I somehow committed against the _parent_ of the
> head, rather than the head itself.

That's very common if you just forget to update your new ".git/HEAD" when 
you do a commit.

Again, it's the tools that make it a bit too easy to mess up. The 
"commit-tree" thing is supposed to really only be used from scripts (which 
would do something like

	result=$(commit-tree ...) && echo $result > .git/HEAD

but when doing things by hand, if you forget to update your HEAD, your 
next commit will be done against the wrong head, and you get dangling 
commits.

The good news is that this is not that hard to fix up. The _trees_ are all
correct, and the objects are all correct, so what you can do is just
generate a few new (proper) commit objects, with the right parents. Then
you can do the "git-prune-script" thing that will throw away the old
broken commits, since they won't be reachable from your new commits (even
though their _trees_ will be there and be the same).

So in this case:

	b4a9a5114b3c6da131a832a8e2cd1941161eb348
	+- e7905b2f22eb5d5308c9122b9c06c2d02473dd4f
	   +- dc90c0db0dd5214aca5304fd17ccd741031e5493 <-- extra dangling head
	   +- 488faba31f59c5960aabbb2a5877a0f2923937a3

you can do

	cat-file commit dc90c0db0dd5214aca5304fd17ccd741031e5493

to remind you what your old tree and commit message was, and then just 
re-commit that tree with the same message but with the proper parent:

	commit-tree xxxx -p 488faba31f59c5960aabbb2a5877a0f2923937a3

and then you need to do the same thing for the other commits (which will 
now need to be re-based to have the new commit-chain as their parents).

Then, when you fixed up the final one, remember to update .git/HEAD with 
its commit ID, and now the prune-thing will get rid of the old dangling 
commits that you just created new duplicates of.

		Linus

Re: Re-done kernel archive - real one?

[Petr Baudis]

Dear diary, on Mon, Apr 18, 2005 at 01:24:24AM CEST, I got a letter
where Linus Torvalds <torvalds@osdl.org> told me that...
> 
> 
> On Sun, 17 Apr 2005, Russell King wrote:
> > 
> > I pulled it tonight into a pristine tree (which of course worked.)
> 
> Goodie.
> 
> > In doing so, I noticed that I'd messed up one of the commits - there's
> > a missing new file.  Grr.  I'll put that down to being a newbie git.
> 
> Actually, you should put that down to horribly bad interface tools.  With
> BK, we had these nice tools that pointed out that there were files that
> you might want to commit (ie "bk citool"), and made this very obvious.
> 
> Tools absolutely matter. And it will take time for us to build up that 
> kind of helper infrastructure. So being newbie might be part of it, but 
> it's the smaller part, I say. Rough interfaces is a big issue.

I just committed some simple git status, which is equivalent to svn
status or cvs update (except it does no update). So it shows all the
files not tracked by git with a question mark in front of them.

This will need some ignore rules, though (currently it just ignores *.o
and the tags file). Now it turns out that it is rather unfortunate that
git ignores hidden files, since this would be a perfect object for that
- I think it is useful to have the ignore list tracked by git. I think I
will just name it git-ignores to be found in the working directory for
now.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: Re-done kernel archive - real one?

[Greg KH]

On Sun, Apr 17, 2005 at 04:24:24PM -0700, Linus Torvalds wrote:
> 
> Tools absolutely matter. And it will take time for us to build up that 
> kind of helper infrastructure. So being newbie might be part of it, but 
> it's the smaller part, I say. Rough interfaces is a big issue.

Speaking of tools, you had a "dotest" program to apply patches in email
form to a bk tree.  And from what I can gather, you've changed that to
handle git archives, right?  Any pointers to where I can find this so I
could try to build up some git trees for you to merge with?  I think I
can even make a tree with a merge issue if you want to test that out :)

thanks,

greg k-h

Re: Re-done kernel archive - real one?

[Randy.Dunlap]

On Mon, 18 Apr 2005 08:04:57 -0700 Greg KH wrote:

| On Sun, Apr 17, 2005 at 04:24:24PM -0700, Linus Torvalds wrote:
| > 
| > Tools absolutely matter. And it will take time for us to build up that 
| > kind of helper infrastructure. So being newbie might be part of it, but 
| > it's the smaller part, I say. Rough interfaces is a big issue.
| 
| Speaking of tools, you had a "dotest" program to apply patches in email
| form to a bk tree.  And from what I can gather, you've changed that to
| handle git archives, right?  Any pointers to where I can find this so I
| could try to build up some git trees for you to merge with?  I think I
| can even make a tree with a merge issue if you want to test that out :)

it's at
http://www.kernel.org/pub/linux/kernel/people/torvalds/git-tools.git/

---
~Randy

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Mon, 18 Apr 2005, Greg KH wrote:
>
> On Sun, Apr 17, 2005 at 04:24:24PM -0700, Linus Torvalds wrote:
> > 
> > Tools absolutely matter. And it will take time for us to build up that 
> > kind of helper infrastructure. So being newbie might be part of it, but 
> > it's the smaller part, I say. Rough interfaces is a big issue.
> 
> Speaking of tools, you had a "dotest" program to apply patches in email
> form to a bk tree.  And from what I can gather, you've changed that to
> handle git archives, right?

Yup.

It's a git archive at 

	kernel.org:/pub/linux/kernel/people/torvalds/git-tools.git

and it seems to work. It's what I've used for all the kernel patches 
(except for the merge), and it's what I use for the git stuff that shows 
up as authored by others.

		Linus

Re: Re-done kernel archive - real one?

[Greg KH]

On Mon, Apr 18, 2005 at 08:42:14AM -0700, Linus Torvalds wrote:
> 
> 
> On Mon, 18 Apr 2005, Greg KH wrote:
> >
> > On Sun, Apr 17, 2005 at 04:24:24PM -0700, Linus Torvalds wrote:
> > > 
> > > Tools absolutely matter. And it will take time for us to build up that 
> > > kind of helper infrastructure. So being newbie might be part of it, but 
> > > it's the smaller part, I say. Rough interfaces is a big issue.
> > 
> > Speaking of tools, you had a "dotest" program to apply patches in email
> > form to a bk tree.  And from what I can gather, you've changed that to
> > handle git archives, right?
> 
> Yup.
> 
> It's a git archive at 
> 
> 	kernel.org:/pub/linux/kernel/people/torvalds/git-tools.git
> 
> and it seems to work. It's what I've used for all the kernel patches 
> (except for the merge), and it's what I use for the git stuff that shows 
> up as authored by others.

Hm, have you pushed all of the recent changes public?  I get the
following when trying to apply a patch:
$ ../git-tools/dotest < ~/linux/patches/usb/usb-visor-tapwave_zodiac.patch 
 mailsplit <mbox> <directory>
$ 

I set my PATH to point to the git-tools/ directory:
$ which mailsplit
/home/greg/linux/git/git-tools/mailsplit

Any thoughts?

thanks,

greg k-h

Re: Re-done kernel archive - real one?

[Greg KH]

On Mon, Apr 18, 2005 at 03:05:41PM -0700, Greg KH wrote:
> On Mon, Apr 18, 2005 at 08:42:14AM -0700, Linus Torvalds wrote:
> > 
> > 
> > On Mon, 18 Apr 2005, Greg KH wrote:
> > >
> > > On Sun, Apr 17, 2005 at 04:24:24PM -0700, Linus Torvalds wrote:
> > > > 
> > > > Tools absolutely matter. And it will take time for us to build up that 
> > > > kind of helper infrastructure. So being newbie might be part of it, but 
> > > > it's the smaller part, I say. Rough interfaces is a big issue.
> > > 
> > > Speaking of tools, you had a "dotest" program to apply patches in email
> > > form to a bk tree.  And from what I can gather, you've changed that to
> > > handle git archives, right?
> > 
> > Yup.
> > 
> > It's a git archive at 
> > 
> > 	kernel.org:/pub/linux/kernel/people/torvalds/git-tools.git
> > 
> > and it seems to work. It's what I've used for all the kernel patches 
> > (except for the merge), and it's what I use for the git stuff that shows 
> > up as authored by others.
> 
> Hm, have you pushed all of the recent changes public?  I get the
> following when trying to apply a patch:
> $ ../git-tools/dotest < ~/linux/patches/usb/usb-visor-tapwave_zodiac.patch 
>  mailsplit <mbox> <directory>

Doh, you want it as a command line argument now.  Stupid me...

Anyway, I try it this way and get:

$ dotest ~/linux/patches/usb/usb-visor-tapwave_zodiac.patch             

Applying USB: visor Tapwave Zodiac support patch

fatal: preparing to update file 'drivers/usb/serial/visor.c' not uptodate in cache


What did I forget to do?

thanks,

greg k-h

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Mon, 18 Apr 2005, Greg KH wrote:
> 
> Anyway, I try it this way and get:

You should update to the newest version anyway..

> $ dotest ~/linux/patches/usb/usb-visor-tapwave_zodiac.patch             
> 
> Applying USB: visor Tapwave Zodiac support patch
> 
> fatal: preparing to update file 'drivers/usb/serial/visor.c' not uptodate in cache
> 
> What did I forget to do?

The most common reason is that the scripts _really_ want the index to 
match your current tree exactly. Run "update-cache --refresh". And if you 
have any uncommitted information, make sure to commit it first.

(Not _strictly_ true - you can leave edited files in your directory, and 
just hope the patch never touches them. The thing you should _not_ do is 
to do an "update-cache xxxx.c" to commit any changes to the 'index', 
because then the patch applicator will actually commit that one too).

		Linus

Re: Re-done kernel archive - real one?

[Greg KH]

On Mon, Apr 18, 2005 at 04:16:45PM -0700, Linus Torvalds wrote:
> 
> 
> On Mon, 18 Apr 2005, Greg KH wrote:
> > 
> > Anyway, I try it this way and get:
> 
> You should update to the newest version anyway..
> 
> > $ dotest ~/linux/patches/usb/usb-visor-tapwave_zodiac.patch             
> > 
> > Applying USB: visor Tapwave Zodiac support patch
> > 
> > fatal: preparing to update file 'drivers/usb/serial/visor.c' not uptodate in cache
> > 
> > What did I forget to do?
> 
> The most common reason is that the scripts _really_ want the index to 
> match your current tree exactly. Run "update-cache --refresh". And if you 
> have any uncommitted information, make sure to commit it first.

Ah, that was the step I was missing, thanks, it's working now.

greg k-h

Re: Re-done kernel archive - real one?

[Linus Torvalds]

On Mon, 18 Apr 2005, Greg KH wrote:
> 
> Hm, have you pushed all of the recent changes public?

Oops. Obviously not. Will fix.

		Linus

Re: Re-done kernel archive - real one?

[H. Peter Anvin]

I have set up /pub/scm/linux/kernel/git on kernel.org.  Everyone who had 
directories in /pub/linux/kernel/people now have directories in 
/pub/scm/linux/kernel/git.  For non-kernel trees it would probably be 
better to have different trees, however; I also would like to request 
that git itself is moved to /pub/software/scm/git; I have created that 
directory and made it owned by Linus.

	-hpa

Re: Re-done kernel archive - real one?

[randy_dunlap]

On Sun, 17 Apr 2005 15:17:50 -0700 (PDT) Linus Torvalds wrote:

| 
| 
| On Mon, 18 Apr 2005, David Woodhouse wrote:
| > 
| > Do you want the commits list running for it yet? Do you want the
| > changesets which are already in it re-mailed without a 'TESTING' tag?
| 
| I really don't know. I'm actually very happy where this thing is right
| now, and completing that first merge successfully was a big milestone to
| me personally. That said, actually _using_ this thing is not for the
| faint-of-heart, and while I think "git" already is showing itself to be
| useful, I'm very very biased.
| 
| In other words, I really wonder what an outsider that doesn't have the
| same kind of mental bias thinks of the current git tree. Is it useful, or
| is it still just a toy for Linus to test out his crazy SCM-wannabe.
| 
| Can people usefully track my current kernel git repository, or do you have
| to be crazy to do so? That's really the question. You be the judge. Me,
| I'm just giddy from a merge that was clearly done using interfaces that
| aren't actually really usable for anybody but me, and barely me at that ;)

TBD...

| 		Linus
| 
| Btw, I also do want this to show up in the BK trees for people who use 
| BitKeeper - the same way we always supported tar-ball + patch users 
| before. So I'll have to try to come up with some sane way to do that too. 
| Any ideas? The first series of 198 patches is obvious enough and can be 
| just done that way direcly, but the merge..

David did the commits-mailing-list script and I'm working on a
commits web-page like what was formerly seen at:
http://www.kernel.org/pub/linux/kernel/v2.6/testing/cset/
(with daily tarball)

based on some older scripts from David, however I'm wondering if
a variant of the gitlog.sh script wouldn't be a better starting
point for it.

---
~Randy

Re: Re-done kernel archive - real one?

[David Woodhouse]

On Sun, 2005-04-17 at 15:22 -0700, randy_dunlap wrote:
> David did the commits-mailing-list script and I'm working on a
> commits web-page like what was formerly seen at:
> http://www.kernel.org/pub/linux/kernel/v2.6/testing/cset/
> (with daily tarball)
> 
> based on some older scripts from David, however I'm wondering if
> a variant of the gitlog.sh script wouldn't be a better starting
> point for it.

My commits-list script is in fact based on gitlog.sh. You'll probably
find useful things to crib from in both that and the original
bkexport.sh script.

The commits script also wants updating to print the date properly now
that we've changed how it's stored -- I'll try to find some time this
week to update it and set it running on master.kernel.org again, but it
may end up waiting till after LCA.

-- 
dwmw2

Re: Re-done kernel archive - real one?

[randy_dunlap]

[-- Attachment #1: Type: text/plain, Size: 1113 bytes --]

On Mon, 18 Apr 2005 09:21:45 +1000 David Woodhouse wrote:

| On Sun, 2005-04-17 at 15:22 -0700, randy_dunlap wrote:
| > David did the commits-mailing-list script and I'm working on a
| > commits web-page like what was formerly seen at:
| > http://www.kernel.org/pub/linux/kernel/v2.6/testing/cset/
| > (with daily tarball)
| > 
| > based on some older scripts from David, however I'm wondering if
| > a variant of the gitlog.sh script wouldn't be a better starting
| > point for it.
| 
| My commits-list script is in fact based on gitlog.sh. You'll probably
| find useful things to crib from in both that and the original
| bkexport.sh script.

Yep, already had all of that in progress.

| The commits script also wants updating to print the date properly now
| that we've changed how it's stored -- I'll try to find some time this
| week to update it and set it running on master.kernel.org again, but it
| may end up waiting till after LCA.

Here's a working gitgenwebpage.sh script, although it may be possible
to do it better/faster, and merge it with the commits mail script
(as you mentioned).


---
~Randy

[-- Attachment #2: gitgenwebpage.sh --]
[-- Type: application/x-sh, Size: 4282 bytes --]