[ANNOUNCE] pg - A patch porcelain for GIT

[ANNOUNCE] pg - A patch porcelain for GIT

[Shawn Pearce]

I just posted the first public version of pg, a GIT porcelain for
managing patches.  Think StGIT, but better in some ways:

Feature Summary:

- Maximum compatibility with other GIT porcelains.

    pg was designed to interoperate with core GIT and the other
    GIT porcelains as much as possible.  GIT favorites like git-am
    can be used to modify a pg managed patch, and vice-versa,
    and without requiring changes to the other GIT tools.

- Simplified command line user interface.

    pg tries to simplify GIT by 'hiding' the index and behaving like
    more traditional SCMs which only look at `HEAD` (last commit)
    and the working directory (files).

- Preserves change history of patches.

    The complete change history associated with each patch is
    maintained directly within GIT.  By storing the evolution of a
    patch as a sequence of GIT commits standard GIT history tools
    such as gitk can be used.

- Its prune proof.

    The metadata structure is stored entirely within the refs
    directory and the object database, which means you can safely use
    git-prune without damaging your work, even for unapplied patches.

- Preserves patch series during clone.

    The metadata structure used by pg allows git-clone to preserve
    the patch series information, without changes required to
    git-clone.  (Patch series information is not preserved during
    git-pull/git-push however.)

- Mix and matching of changes (bug fixes/features).

    By maintaining changes as individual patches it is possible to
    apply individual changes to the current working directory and
    to unapply them just as easily.

- Automatic detection (and cancellation) of returning patches.

    pg automatically detects when a patch is received from
    the upstream GIT repository during a pg-rebase and deletes
    (cancels) the local version of the patch from the patch series.
    The automatic cancelling makes it easy to use pg to track and
    develop changes on top of a GIT project.

- Fast

    pg operations generally perform faster than StGIT operations,
    at least on my large (~7000 file) repositories.


And for those so inclined:

  Homepage:       http://www.spearce.org/projects/scm/pg/
  GIT Repository: http://www.spearce.org/projects/scm/pg.git


-- 
Shawn.

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Greg KH]

On Fri, Feb 10, 2006 at 02:59:14PM -0500, Shawn Pearce wrote:
> I just posted the first public version of pg, a GIT porcelain for
> managing patches.  Think StGIT, but better in some ways:
> 
> Feature Summary:

Hm, is there any way to import an existing patch into pg?

thanks,

greg k-h

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Shawn Pearce]

Greg KH <greg@kroah.com> wrote:
> On Fri, Feb 10, 2006 at 02:59:14PM -0500, Shawn Pearce wrote:
> > I just posted the first public version of pg, a GIT porcelain for
> > managing patches.  Think StGIT, but better in some ways:
> > 
> > Feature Summary:
> 
> Hm, is there any way to import an existing patch into pg?

Doh!  I haven't needed to do that yet.  I'll code up a pg-import
later tonight.  But since git and pg play nice together you can
do this:

	pg-new Patch-Name
	git-apply the-patch-file.patch
	pg-ci -m"Importing the-patch-file.patch..."

or even:

	pg-new Patch-Name
	git-am mbox

and keep the 'history' stored in the mailbox.

So pg-import won't amount to a very long script.  :-|

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Greg KH]

On Fri, Feb 10, 2006 at 04:04:01PM -0500, Shawn Pearce wrote:
> Greg KH <greg@kroah.com> wrote:
> > On Fri, Feb 10, 2006 at 02:59:14PM -0500, Shawn Pearce wrote:
> > > I just posted the first public version of pg, a GIT porcelain for
> > > managing patches.  Think StGIT, but better in some ways:
> > > 
> > > Feature Summary:
> > 
> > Hm, is there any way to import an existing patch into pg?
> 
> Doh!  I haven't needed to do that yet.  I'll code up a pg-import
> later tonight.  But since git and pg play nice together you can
> do this:
> 
> 	pg-new Patch-Name
> 	git-apply the-patch-file.patch
> 	pg-ci -m"Importing the-patch-file.patch..."
> 
> or even:
> 
> 	pg-new Patch-Name
> 	git-am mbox

well, as my quilt tree is around 200 patches right now, that would be
annoying to have to do by hand :)

thanks,

greg k-h

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Petr Baudis]

  Hi,

Dear diary, on Fri, Feb 10, 2006 at 08:59:14PM CET, I got a letter
where Shawn Pearce <spearce@spearce.org> said that...
> I just posted the first public version of pg, a GIT porcelain for
> managing patches.  Think StGIT, but better in some ways:

  it sounds interesting. I've been thinking about wrapping some patch
queue tool in Cogito (post-1.0) and pg might be a better choice than
StGIT.

  One thing I dislike on both StGIT and pg is that they both try to
build a full-fledged porcelain on top of GIT, instead of just focusing
on the patch management, doing it well and providing a convenient user
interface (well, can't say about pg's interface, didn't try it yet).
Instead of having pg-add, pg-log, or pg-status it might be more fruitful
to contribute the features you are missing to git-core or Cogito.

> And for those so inclined:
> 
>   Homepage:       http://www.spearce.org/projects/scm/pg/
>   GIT Repository: http://www.spearce.org/projects/scm/pg.git

But while it claims to be compatible with all the porcelains, it at
least cannot be clone by them. ;) The GIT repository is not quite a
valid GIT repository since it is missing the HEAD and Cogito clones
based on this file instead of just assuming that your head is on the
master branch.


Also, when cloning it gives me a little unnerving errors like

error: File 6427c0154400f578d9cdff178e01e946db6f714f
(http://www.spearce.org/projects/scm/pg.git/objects/64/27c0154400f578d9cdff178e01e946db6f714f)
corrupt

(but strangely, fsck-objects later does not complain).

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Of the 3 great composers Mozart tells us what it's like to be human,
Beethoven tells us what it's like to be Beethoven and Bach tells us
what it's like to be the universe.  -- Douglas Adams

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Shawn Pearce]

Petr Baudis <pasky@suse.cz> wrote:
>   One thing I dislike on both StGIT and pg is that they both try to
> build a full-fledged porcelain on top of GIT, instead of just focusing
> on the patch management, doing it well and providing a convenient user
> interface (well, can't say about pg's interface, didn't try it yet).
> Instead of having pg-add, pg-log, or pg-status it might be more fruitful
> to contribute the features you are missing to git-core or Cogito.

Very valid points.  Before writing pg I used Cogito exclusively
and found git-core too cumbersome to use directly.  When I started
writing pg I didn't intend on replacing everything GIT and Cogito
offers; I was trying to only create the patch stack but still use
the Cogito tools for everything else.

But two things happened:

  1) Cogito didn't run well on a Solaris box I wanted to try and
     use it in; apparently we don't have enough GNU shell commands
     available and Cogito fell over.  (But right now I'd bet pg
     will behave the same if not worse. I haven't had time to try
     it. *sigh*)

  2) I found myself suddenly typing 'pg-log' and 'pg-diff' rather
     than 'git-log' and 'git-diff'.  Call it future muscle memory?
     I hadn't written either of these scripts so I was getting a lot
     of '-bash: pg-log: command not found' errors from my shell.
     So they both became 1 line wrappers around the git-core
     versions, just to save my sanity.

I would agree with trying to integrate some of the workflow idealogy
presented by StGIT and pg into something more mainstream such as
git-core or Cogito.  Right now I'm using pg as a proving ground to
feel out how some of that might work in one particular environment:

  A development team I work with is stuck using PVCS Version
  Manager 6.  Moving source code from a developer to a tester is a
  huge nightmare; not only must the developer check the code into
  the version control system but he/she must also write a bug report
  in a bug database to tell someone else to get the source file
  and give it to the tester.  Its a horrible workflow.  GIT + pg +
  additional custom scripts seems to be easing the pain somewhat;
  but sadly we can't just rip out PVCS Version Manager and use GIT.


> > And for those so inclined:
> > 
> >   Homepage:       http://www.spearce.org/projects/scm/pg/
> >   GIT Repository: http://www.spearce.org/projects/scm/pg.git
> 
> But while it claims to be compatible with all the porcelains, it at
> least cannot be clone by them. ;) The GIT repository is not quite a
> valid GIT repository since it is missing the HEAD and Cogito clones
> based on this file instead of just assuming that your head is on the
> master branch.

Fixed.  That's my fault - my hosting provider doesn't have GIT
installed and thus I had to publish my repository over rsync+ssh.
But git-push doesn't support that protocol type anymore.  :-| So
I packed everything into pack files, pruned the object directory,
and rsync'd it up.  I guess my rsync script didn't copy HEAD.

> Also, when cloning it gives me a little unnerving errors like
> 
> error: File 6427c0154400f578d9cdff178e01e946db6f714f
> (http://www.spearce.org/projects/scm/pg.git/objects/64/27c0154400f578d9cdff178e01e946db6f714f)
> corrupt

I've seen the same.  I think it is either a bug in my rsync script
or a bug in the GIT http clone code; because that is the current
tip commit of the master branch.  And I've only seen that error for
the tip commit, and only if the object doesn't exist in the object
directory because I've done git-pack && git-prune-packed.

-- 
Shawn.

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Petr Baudis]

Dear diary, on Fri, Feb 10, 2006 at 10:38:18PM CET, I got a letter
where Shawn Pearce <spearce@spearce.org> said that...
> But two things happened:
> 
>   1) Cogito didn't run well on a Solaris box I wanted to try and
>      use it in; apparently we don't have enough GNU shell commands
>      available and Cogito fell over.  (But right now I'd bet pg
>      will behave the same if not worse. I haven't had time to try
>      it. *sigh*)

I'm always listening for bugreports. Besides requiring bash, Cogito _is_
expected to run on POSIX stuff!

>   2) I found myself suddenly typing 'pg-log' and 'pg-diff' rather
>      than 'git-log' and 'git-diff'.  Call it future muscle memory?
>      I hadn't written either of these scripts so I was getting a lot
>      of '-bash: pg-log: command not found' errors from my shell.
>      So they both became 1 line wrappers around the git-core
>      versions, just to save my sanity.

I see. IIRC Catalin gave the similar reasoning. (Obviously, my
egoistical me might be just hurt by it not wrapping Cogito. ;))

> > But while it claims to be compatible with all the porcelains, it at
> > least cannot be clone by them. ;) The GIT repository is not quite a
> > valid GIT repository since it is missing the HEAD and Cogito clones
> > based on this file instead of just assuming that your head is on the
> > master branch.
> 
> Fixed.

Thanks.

> > Also, when cloning it gives me a little unnerving errors like
> > 
> > error: File 6427c0154400f578d9cdff178e01e946db6f714f
> > (http://www.spearce.org/projects/scm/pg.git/objects/64/27c0154400f578d9cdff178e01e946db6f714f)
> > corrupt
> 
> I've seen the same.  I think it is either a bug in my rsync script
> or a bug in the GIT http clone code; because that is the current
> tip commit of the master branch.  And I've only seen that error for
> the tip commit, and only if the object doesn't exist in the object
> directory because I've done git-pack && git-prune-packed.

On a second thought, this is probably simply caused by the web server
not reporting 404 on missing files.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Of the 3 great composers Mozart tells us what it's like to be human,
Beethoven tells us what it's like to be Beethoven and Bach tells us
what it's like to be the universe.  -- Douglas Adams

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Junio C Hamano]

Petr Baudis <pasky@suse.cz> writes:

> On a second thought, this is probably simply caused by the web server
> not reporting 404 on missing files.

I suspect that is it.  The webserver is broken^W not being a
good network citizen.

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Petr Baudis]

  Hi,

Dear diary, on Fri, Feb 10, 2006 at 10:17:40PM CET, I got a letter
where Petr Baudis <pasky@suse.cz> said that...
> > I just posted the first public version of pg, a GIT porcelain for
> > managing patches.  Think StGIT, but better in some ways:
> 
>   it sounds interesting. I've been thinking about wrapping some patch
> queue tool in Cogito (post-1.0) and pg might be a better choice than
> StGIT.

  so I've used it a bit and I'm going back to StGIT, at least for now.
It is not really usable for me so far, since it is missing two crucial
things:

	* Patch description tracking. Patch description is almost as
	important as patch contents for me, and pg just doesn't track it
	for now.  It would be best if it just seeded the patch
	description by the first commit message and then allow you edit
	it at the refresh time.

	* Mail interface. StGIT can pre-fill the patch description with
	my signoff line, but more importantly when I write

		stg mail patchname

	it will mail the patch to the addresses I configured it to,
	prepend [PATCH] to the subject line and stuff.

  So, my patchqueue workflow is "I do some random third-party patches
for some software and want to manage, update, and submit them easily."
PG does not make it much easier now, unfortunately.

  Some common gripes for both StGIT and pg (well, I'm using some
ridiculously old StGIT version, so this may not apply anymore there):

	* stg new --force - seriously, what's the point?! I always to
	the change first and when it's any good, I want to create a
	patch for it.

	* I can't just get the patch in its "canonical ready-to-mail
	form" on stdout so that I could easily review it. Why is
	pg-export insisting to dump it to a file?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Of the 3 great composers Mozart tells us what it's like to be human,
Beethoven tells us what it's like to be Beethoven and Bach tells us
what it's like to be the universe.  -- Douglas Adams

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Catalin Marinas]

Petr Baudis <pasky@suse.cz> wrote:
>   Some common gripes for both StGIT and pg (well, I'm using some
> ridiculously old StGIT version, so this may not apply anymore there):
>
> 	* stg new --force - seriously, what's the point?! I always to
> 	the change first and when it's any good, I want to create a
> 	patch for it.

This was fixed couple of weeks ago in the main branch. No need to pass
--force anymore.

> 	* I can't just get the patch in its "canonical ready-to-mail
> 	form" on stdout so that I could easily review it. Why is
> 	pg-export insisting to dump it to a file?

To view the patch you can use 'stg diff -r <patch>/' but it doesn't
show the description. Dumping the full patch on stdout would be
useful, indeed. The export and mail commands use different templates
and the latter even adds the standard mail headers. Which of these two
commands would you prefer to dump the patch on stdout (both is fine as
well)?

Another thing that's missing in StGIT is the import of a series of
patches. At the moment I run a small shell script to import individual
patches.

-- 
Catalin

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Karl Hasselström]

On 2006-02-14 09:26:41 +0000, Catalin Marinas wrote:

> Another thing that's missing in StGIT is the import of a series of
> patches. At the moment I run a small shell script to import
> individual patches.

One thing I would like to see in stgit is the opposite of "stg
commit"; instead of converting patches to regular commits, take the
topmost regular commits and convert them to patches.

For example, "stg uncommit foo bar baz" would -- regardless of any
existing patches, applied or not -- convert the top three regular
commits, with comments and all, to stgit patches called foo, bar, and
baz. These would be already applied, at the bottom of the stack. I
imagine all one would have to do is to modify some stgit metadata, so
the operation could be really cheap.

Of course, "stg uncommit" is allowed to reject any commit with more
than one parent, since those can't be represented as stgit patches.

This would perhaps not add much power to an all-stgit workflow, but it
would be a really convenient way to edit recent git history. Sort of
like a more convenient rebase. And a great way to lure new users. :-)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Chuck Lever]

[-- Attachment #1: Type: text/plain, Size: 1257 bytes --]

Karl Hasselström wrote:
> On 2006-02-14 09:26:41 +0000, Catalin Marinas wrote:
> 
> 
>>Another thing that's missing in StGIT is the import of a series of
>>patches. At the moment I run a small shell script to import
>>individual patches.
> 
> 
> One thing I would like to see in stgit is the opposite of "stg
> commit"; instead of converting patches to regular commits, take the
> topmost regular commits and convert them to patches.
> 
> For example, "stg uncommit foo bar baz" would -- regardless of any
> existing patches, applied or not -- convert the top three regular
> commits, with comments and all, to stgit patches called foo, bar, and
> baz. These would be already applied, at the bottom of the stack. I
> imagine all one would have to do is to modify some stgit metadata, so
> the operation could be really cheap.
> 
> Of course, "stg uncommit" is allowed to reject any commit with more
> than one parent, since those can't be represented as stgit patches.
> 
> This would perhaps not add much power to an all-stgit workflow, but it
> would be a really convenient way to edit recent git history. Sort of
> like a more convenient rebase. And a great way to lure new users. :-)

i think you want "stg pick --reverse" ?

[-- Attachment #2: cel.vcf --]
[-- Type: text/x-vcard, Size: 451 bytes --]

begin:vcard
fn:Chuck Lever
n:Lever;Charles
org:Network Appliance, Incorporated;Open Source NFS Client Development
adr:535 West William Street, Suite 3100;;Center for Information Technology Integration;Ann Arbor;MI;48103-4943;USA
email;internet:cel@citi.umich.edu
title:Member of Technical Staff
tel;work:+1 734 763-4415
tel;fax:+1 734 763 4434
tel;home:+1 734 668-1089
x-mozilla-html:FALSE
url:http://troy.citi.umich.edu/u/cel/
version:2.1
end:vcard

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Karl Hasselström]

On 2006-02-14 10:22:51 -0500, Chuck Lever wrote:

> Karl Hasselström wrote:
>
> > One thing I would like to see in stgit is the opposite of "stg
> > commit"; instead of converting patches to regular commits, take
> > the topmost regular commits and convert them to patches.
> >
> > For example, "stg uncommit foo bar baz" would -- regardless of any
> > existing patches, applied or not -- convert the top three regular
> > commits, with comments and all, to stgit patches called foo, bar,
> > and baz. These would be already applied, at the bottom of the
> > stack. I imagine all one would have to do is to modify some stgit
> > metadata, so the operation could be really cheap.
> >
> > Of course, "stg uncommit" is allowed to reject any commit with
> > more than one parent, since those can't be represented as stgit
> > patches.
> >
> > This would perhaps not add much power to an all-stgit workflow,
> > but it would be a really convenient way to edit recent git
> > history. Sort of like a more convenient rebase. And a great way to
> > lure new users. :-)
>
> i think you want "stg pick --reverse" ?

No, I literally want the opposite of "stg commit", so that the
sequence "stg commit; stg uncommit" has zero net effect.

Say we have the following situation (stack growing downward, of
course):

          :
          |
          a
          |
          b
          |
          c <- bases/master
          |
          d <- applied patch "foo"
          |
          e <- applied patch "bar"; HEAD
          |
          f <- unapplied patch "baz"
          |
          :

In this situation, the hypothetical "stg uncommit" command would have
the following effect:

  $ stg uncommit goo baa

          :
          |
          a <- bases/master
          |
          b <- applied patch "baa"
          |
          c <- applied patch "goo"
          |
          d <- applied patch "foo"
          |
          e <- applied patch "bar"; HEAD
          |
          f <- unapplied patch "baz"
          |
          :

Note that HEAD is unchanged; the only thing that has happend is that
stgit has taken over the topmost two commits, and turned them into
patches. No git operations whatsoever have taken place; all stgit had
to do was change the value of bases/master and add bookkeeping
information for the two new patches.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Chuck Lever]

[-- Attachment #1: Type: text/plain, Size: 1717 bytes --]

Karl Hasselström wrote:
> On 2006-02-14 10:22:51 -0500, Chuck Lever wrote:
> 
> 
>>Karl Hasselström wrote:
>>
>>
>>>One thing I would like to see in stgit is the opposite of "stg
>>>commit"; instead of converting patches to regular commits, take
>>>the topmost regular commits and convert them to patches.
>>>
>>>For example, "stg uncommit foo bar baz" would -- regardless of any
>>>existing patches, applied or not -- convert the top three regular
>>>commits, with comments and all, to stgit patches called foo, bar,
>>>and baz. These would be already applied, at the bottom of the
>>>stack. I imagine all one would have to do is to modify some stgit
>>>metadata, so the operation could be really cheap.
>>>
>>>Of course, "stg uncommit" is allowed to reject any commit with
>>>more than one parent, since those can't be represented as stgit
>>>patches.
>>>
>>>This would perhaps not add much power to an all-stgit workflow,
>>>but it would be a really convenient way to edit recent git
>>>history. Sort of like a more convenient rebase. And a great way to
>>>lure new users. :-)
>>
>>i think you want "stg pick --reverse" ?
> 
> 
> No, I literally want the opposite of "stg commit", so that the
> sequence "stg commit; stg uncommit" has zero net effect.

gotcha.

well, that would work OK for maintainers, but would be kind of strange 
for folks who are pulling from such a repository.  how would that work?

my impression of git is that you don't change stuff that's already 
committed.  you revert changes by applying a new commit that backs out 
the original changes.  i'm speculating, but i suspect that's why there's 
a "stg pick --reverse" and not a "stg uncommit."


[-- Attachment #2: cel.vcf --]
[-- Type: text/x-vcard, Size: 451 bytes --]

begin:vcard
fn:Chuck Lever
n:Lever;Charles
org:Network Appliance, Incorporated;Open Source NFS Client Development
adr:535 West William Street, Suite 3100;;Center for Information Technology Integration;Ann Arbor;MI;48103-4943;USA
email;internet:cel@citi.umich.edu
title:Member of Technical Staff
tel;work:+1 734 763 4415
tel;fax:+1 734 763 4434
tel;home:+1 734 668 1089
x-mozilla-html:FALSE
url:http://troy.citi.umich.edu/u/cel/
version:2.1
end:vcard

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Petr Baudis]

Dear diary, on Tue, Feb 14, 2006 at 09:58:02PM CET, I got a letter
where Chuck Lever <cel@citi.umich.edu> said that...
> my impression of git is that you don't change stuff that's already 
> committed.  you revert changes by applying a new commit that backs out 
> the original changes.  i'm speculating, but i suspect that's why there's 
> a "stg pick --reverse" and not a "stg uncommit."

It is ok as long as you know what are you doing - if you don't push out
the commits you've just "undid" (or work on a public accessible
repository in the first place, but I think that's kind of rare these
days; quick survey - does anyone reading these lines do that?), there's
nothing wrong on it, and it gives you nice flexibility.

For example, to import bunch of patches (I guess that's the original
intention behind this) you just run git-am on them and then stg uncommit
all of the newly added commits.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Of the 3 great composers Mozart tells us what it's like to be human,
Beethoven tells us what it's like to be Beethoven and Bach tells us
what it's like to be the universe.  -- Douglas Adams

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Sam Vilain]

Petr Baudis wrote:
>>my impression of git is that you don't change stuff that's already 
>>committed.  you revert changes by applying a new commit that backs out 
>>the original changes.  i'm speculating, but i suspect that's why there's 
>>a "stg pick --reverse" and not a "stg uncommit."
> It is ok as long as you know what are you doing - if you don't push out
> the commits you've just "undid" (or work on a public accessible
> repository in the first place, but I think that's kind of rare these
> days; quick survey - does anyone reading these lines do that?), there's
> nothing wrong on it, and it gives you nice flexibility.

Yes, and this is one problem I envision with publishing a git repository
with an stgit stack applied - somebody later doing a pull of it will not
find the head revision they had.  I'm not sure what the net effect of
this will be, though.

Sam.

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Shawn Pearce]

Sam Vilain <sam@vilain.net> wrote:
> Petr Baudis wrote:
> >>my impression of git is that you don't change stuff that's already 
> >>committed.  you revert changes by applying a new commit that backs out 
> >>the original changes.  i'm speculating, but i suspect that's why there's 
> >>a "stg pick --reverse" and not a "stg uncommit."
> >It is ok as long as you know what are you doing - if you don't push out
> >the commits you've just "undid" (or work on a public accessible
> >repository in the first place, but I think that's kind of rare these
> >days; quick survey - does anyone reading these lines do that?), there's
> >nothing wrong on it, and it gives you nice flexibility.
> 
> Yes, and this is one problem I envision with publishing a git repository
> with an stgit stack applied - somebody later doing a pull of it will not
> find the head revision they had.  I'm not sure what the net effect of
> this will be, though.

It would cause some pain for anyone pulling from it with git-pull, as
git-pull won't happily go backwards from what I've seen. But I think
you can force it to do so even if it won't make sense during the
resulting merge, which then leaves the user in an interesting state.

This is actually why pg-rebase doesn't care what you move to
when you grab the remote's commit; it just jumps to that commit
and pushes your patch stack back down onto it.  So if the remote
rebuilds itself through a new commit lineage which you have never
seen before the next pg-rebase will still update to it.  But on
the other hand if you have a commit that isn't in your local patch
stack its gone into the bit bucket.

Publishing a repository with a stg (or pg) patch series isn't
a problem; the problem is that no clients currently know how to
follow along with the remote repository's patch series.  And I can't
think of a sensible behavior for doing so that isn't what git-core is
already doing today for non patch series type clients (as in don't go
backwards by popping but instead by pushing a negative delta).  :-)

-- 
Shawn.

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Petr Baudis]

Dear diary, on Wed, Feb 15, 2006 at 01:35:10AM CET, I got a letter
where Shawn Pearce <spearce@spearce.org> said that...
> Publishing a repository with a stg (or pg) patch series isn't
> a problem; the problem is that no clients currently know how to
> follow along with the remote repository's patch series.  And I can't
> think of a sensible behavior for doing so that isn't what git-core is
> already doing today for non patch series type clients (as in don't go
> backwards by popping but instead by pushing a negative delta).  :-)

New Cogito will automagically do the right thing if you are just
fast-forwarding and you are using cg-update - if the branch rebased, it
will happily follow (but cg-fetch + cg-merge will NOT and it will fall
back to the tree merge).

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Of the 3 great composers Mozart tells us what it's like to be human,
Beethoven tells us what it's like to be Beethoven and Bach tells us
what it's like to be the universe.  -- Douglas Adams

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[J. Bruce Fields]

On Tue, Feb 14, 2006 at 07:35:10PM -0500, Shawn Pearce wrote:
> Publishing a repository with a stg (or pg) patch series isn't
> a problem; the problem is that no clients currently know how to
> follow along with the remote repository's patch series.  And I can't
> think of a sensible behavior for doing so that isn't what git-core is
> already doing today for non patch series type clients (as in don't go
> backwards by popping but instead by pushing a negative delta).  :-)

If you represent each patch as a branch, with each modification to the
patch a commit on the corresponding branch, and each "push" operation a
merge from the branch corresponding to the previous patch to a branch
corresponding to the new patch (isn't that what pg's trying to do?),
then it should be possible just to track the branch corresponding to the
top patch.

In theory I guess it should also be possible to merge patch series that
have followed two lines of development, by merging each corresponding
branch.

The history would be really complicated.  You'd need to figure out how
to track the patch comments too, and you'd need scripts to convert to
just a simple series of commits for submitting upstream.  Probably not
worth the trouble, but I don't know.

If you really want revision control on patches the simplest thing might
be just to run quilt or Andrew Morton's scripts on top of a git
repository--the documentation with Andrew's scripts recommends doing
that with CVS.

--b,

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Shawn Pearce]

"J. Bruce Fields" <bfields@fieldses.org> wrote:
> On Tue, Feb 14, 2006 at 07:35:10PM -0500, Shawn Pearce wrote:
> > Publishing a repository with a stg (or pg) patch series isn't
> > a problem; the problem is that no clients currently know how to
> > follow along with the remote repository's patch series.  And I can't
> > think of a sensible behavior for doing so that isn't what git-core is
> > already doing today for non patch series type clients (as in don't go
> > backwards by popping but instead by pushing a negative delta).  :-)
> 
> If you represent each patch as a branch, with each modification to the
> patch a commit on the corresponding branch, and each "push" operation a
> merge from the branch corresponding to the previous patch to a branch
> corresponding to the new patch (isn't that what pg's trying to do?),
> then it should be possible just to track the branch corresponding to the
> top patch.

Yes that's pg in a nutshell.

But what happens when I pop back two patches (of three) and then push
down a different (fourth) patch?  The tree just rewound backwards
and then forwards again in a different direction.  (I apologize for
not being able to draw a nice ASCII art diagram of this, that's a
skill I'll have to learn to keep up with you guys.)  This is the
issue with Junio's pu branch in git.git and is why some people
apparently don't follow it.

StGIT and pg aren't the only ones who suffer from this wonderful
little feature of GIT.

> In theory I guess it should also be possible to merge patch series that
> have followed two lines of development, by merging each corresponding
> branch.

Of course.  If I delete all of the refs used by pg to mark the patch
boundaries its just another GIT branch.  Ditto for StGIT.  So clearly
you can merge them together just like any other GIT branch.

The open question is could you preserve the patch boundaries
while doing the merge.  Probably not.  It would become way to
complicated as you would want to merge the entire branch and not each
individual patch as the individual patch merges may not work but the
larger branch merge might go through without human intervention.
Of course you can try to keep the patch boundaries by exporting
all of the patches from the one branch and push them on top of
the current branch.  But isn't that what a 3 way merge is anyway?
And again that might not work as well as taking the larger patch
and pushing that down.  :-)

> The history would be really complicated.  You'd need to figure out how
> to track the patch comments too, and you'd need scripts to convert to
> just a simple series of commits for submitting upstream.  Probably not
> worth the trouble, but I don't know.

I think I'm almost there with pg.  One of my next tasks is the
patch log ripping code.  This is really only complicated because GIT
won't let me store the base of a 3 way merge as part of a commit;
all I can store is the set of parents.  If I had the base in the
commit (and specifically marked as such so I can tell it from the
end points) then I could easily walk through the log to extract all
commits relevant to a patch and seek forward and backward over it.

Perhaps I could cheat and record 3 parents: (HEAD, base, last).
I wonder what gitk would make of that mess.  I doubt it would display
any better than the current (HEAD, last) format I'm using now.

> If you really want revision control on patches the simplest thing might
> be just to run quilt or Andrew Morton's scripts on top of a git
> repository--the documentation with Andrew's scripts recommends doing
> that with CVS.

True but you also then run into problems about needing to know which
base each patch revision was applied against so you can reproduce
a source tree plus patch at a specific point in time.

When I started pg I first thought about recording a patch in a
secondary index/working directory where each patch was its own file
and use git-diff-tree to extract the patch, commit it as a blob in
the secondary index/directory, and push it onto the series by reading
the blob in and running git-apply on the patch stored within it.

I ruled this strategy out as it just felt like it would be too
slow and rather unnatural to work on with existing GIT tools.
I didn't want to write a full porcelain; I was really hoping to just
extend Cogito to get a patch stack that worked the way *I* wanted a
patch stack to work, which was close to StGIT but not quite StGIT.
(Of course it didn't work out this way when I picked a prefix of
'pg' instead of 'cg' for my new commands.)  :-|

-- 
Shawn.

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[J. Bruce Fields]

On Wed, Feb 15, 2006 at 01:54:11AM -0500, Shawn Pearce wrote:
> "J. Bruce Fields" <bfields@fieldses.org> wrote:
> > On Tue, Feb 14, 2006 at 07:35:10PM -0500, Shawn Pearce wrote:
> > > Publishing a repository with a stg (or pg) patch series isn't
> > > a problem; the problem is that no clients currently know how to
> > > follow along with the remote repository's patch series.  And I can't
> > > think of a sensible behavior for doing so that isn't what git-core is
> > > already doing today for non patch series type clients (as in don't go
> > > backwards by popping but instead by pushing a negative delta).  :-)
> > 
> > If you represent each patch as a branch, with each modification to the
> > patch a commit on the corresponding branch, and each "push" operation a
> > merge from the branch corresponding to the previous patch to a branch
> > corresponding to the new patch (isn't that what pg's trying to do?),
> > then it should be possible just to track the branch corresponding to the
> > top patch.
> 
> Yes that's pg in a nutshell.
> 
> But what happens when I pop back two patches (of three) and then push
> down a different (fourth) patch?  The tree just rewound backwards
> and then forwards again in a different direction.

So you've got p1, p2, and p3 applied, each with its corresponding
branch--respectively, b1, b2, and b3.  Popping two patches just checks
out b1, and doesn't affect the repository at all.  If you push a new
patch, p4, you've just created a new branch, b4--you haven't touched the
existing branches.  If you push p2 and p3 back on, you're just merging
the new changes from b4 into b2 and then merging the newly merged b2
into b3.

>From the point of view of someone tracking b3, this is all fine.  OK,
maybe it's excessively complicated, but pulls should work, because it
never sees history diseappear as it does when you represent each patch
with a commit on a single branch.

> > If you really want revision control on patches the simplest thing might
> > be just to run quilt or Andrew Morton's scripts on top of a git
> > repository--the documentation with Andrew's scripts recommends doing
> > that with CVS.
> 
> True but you also then run into problems about needing to know which
> base each patch revision was applied against so you can reproduce
> a source tree plus patch at a specific point in time.

Right, so you keep the tree under revision control as well as the
patches.

--b.

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Junio C Hamano]

Shawn Pearce <spearce@spearce.org> writes:

> I think I'm almost there with pg.  One of my next tasks is the
> patch log ripping code.  This is really only complicated because GIT
> won't let me store the base of a 3 way merge as part of a commit;
> all I can store is the set of parents.  If I had the base in the
> commit (and specifically marked as such so I can tell it from the
> end points) then I could easily walk through the log to extract all
> commits relevant to a patch and seek forward and backward over it.

I think I know what you are talking about.  Maybe you might be
interested to take a look at TO script in my todo branch [*1*]?

Also there is a sample hook script templates/hooks--pre-rebase 
that uses the same idea used in the said script.  These are what
I use to manage the "next" branch (an aggregation of topic
branches being cooked).

By the way, please do *not* do this:

    Mail-Followup-To: "J. Bruce Fields" <bfields@fieldses.org>,
            Sam Vilain <sam@vilain.net>, Petr Baudis <pasky@suse.cz>,...
	    ...

I wanted to reply to *you*, but by having the header you robbed
about 30 seconds from me, forcing me to edit the "To:"
addressee.


[Footnote]

*1* todo branch does not have *any* ancestry relationship with
my primary branches, so please do not try to merge it in your
primary repository (unless you know what you are doing).  Make a
clone into a separate directory and running "git checkout todo"
there is the cleanest way to peek into it.

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Catalin Marinas]

On 16/02/06, Junio C Hamano <junkio@cox.net> wrote:
> By the way, please do *not* do this:
>
>     Mail-Followup-To: "J. Bruce Fields" <bfields@fieldses.org>,
>             Sam Vilain <sam@vilain.net>, Petr Baudis <pasky@suse.cz>,...
>             ...

I think that's a "feature" of mutt that I couldn't understand. Every
time I got this header and looked at the mail client, it was... mutt
:-).

--
Catalin

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Fernando J. Pereda]

[-- Attachment #1: Type: text/plain, Size: 945 bytes --]

On Thu, Feb 16, 2006 at 10:33:12AM +0000, Catalin Marinas wrote:
> On 16/02/06, Junio C Hamano <junkio@cox.net> wrote:
> > By the way, please do *not* do this:
> >
> >     Mail-Followup-To: "J. Bruce Fields" <bfields@fieldses.org>,
> >             Sam Vilain <sam@vilain.net>, Petr Baudis <pasky@suse.cz>,...
> >             ...
> 
> I think that's a "feature" of mutt that I couldn't understand. Every
> time I got this header and looked at the mail client, it was... mutt
> :-).

That's because you told mutt you are subscribed to that list, so mutt
won't add you to Mail-Followup-To:, so you don't get duplicates. If you
don't tell mutt you are subscribed to the list, It will add your own
addres there.

I think it is a nice feature, although it seems to annoy Junio :)

Cheers,
Ferdy

-- 
Fernando J. Pereda Garcimartín
Gentoo Developer (Alpha,net-mail,mutt,git)
20BB BDC3 761A 4781 E6ED  ED0B 0A48 5B0C 60BD 28D4

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Junio C Hamano]

"Fernando J. Pereda" <ferdy@ferdyx.org> writes:

> That's because you told mutt you are subscribed to that list, so mutt
> won't add you to Mail-Followup-To:, so you don't get duplicates. If you
> don't tell mutt you are subscribed to the list, It will add your own
> addres there.
>
> I think it is a nice feature, although it seems to annoy Junio :)

Rightfully so.

Last I heard that was a feature mutt people regret.  Go back to
the mail archive for details -- you robbed me 30 seconds so I
won't do a research for you this time as I usually do ;-).

The "feature" is to allow you be lazy and not filter on your own
end and force everybody who wants to respond to you to fix up
the addressee header.  In other words, it is not a "feature" for
people who receives your mail at all.

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Catalin Marinas]

Junio C Hamano <junkio@cox.net> wrote:
> "Fernando J. Pereda" <ferdy@ferdyx.org> writes:
>
>> That's because you told mutt you are subscribed to that list, so mutt
>> won't add you to Mail-Followup-To:, so you don't get duplicates. If you
>> don't tell mutt you are subscribed to the list, It will add your own
>> addres there.
>>
>> I think it is a nice feature, although it seems to annoy Junio :)
>
> Rightfully so.
>
> Last I heard that was a feature mutt people regret.  Go back to
> the mail archive for details -- you robbed me 30 seconds so I
> won't do a research for you this time as I usually do ;-).

For Gnus users:

(setq message-use-mail-followup-to nil)

-- 
Catalin

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Catalin Marinas]

On 14/02/06, Petr Baudis <pasky@suse.cz> wrote:
> Dear diary, on Tue, Feb 14, 2006 at 09:58:02PM CET, I got a letter
> where Chuck Lever <cel@citi.umich.edu> said that...
> > my impression of git is that you don't change stuff that's already
> > committed.  you revert changes by applying a new commit that backs out
> > the original changes.  i'm speculating, but i suspect that's why there's
> > a "stg pick --reverse" and not a "stg uncommit."
>
> It is ok as long as you know what are you doing - if you don't push out
> the commits you've just "undid" (or work on a public accessible
> repository in the first place, but I think that's kind of rare these
> days; quick survey - does anyone reading these lines do that?), there's
> nothing wrong on it, and it gives you nice flexibility.
>
> For example, to import bunch of patches (I guess that's the original
> intention behind this) you just run git-am on them and then stg uncommit
> all of the newly added commits.

This is a sensible way of using an uncommit command but I initially
thought it would be better to make things harder for people wanting to
re-write the history. Anyway, I'll keep this command on my todo list.

--
Catalin

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Karl Hasselström]

On 2006-02-15 17:25:30 +0000, Catalin Marinas wrote:

> On 14/02/06, Petr Baudis <pasky@suse.cz> wrote:
>
> > It is ok as long as you know what are you doing - if you don't
> > push out the commits you've just "undid" (or work on a public
> > accessible repository in the first place, but I think that's kind
> > of rare these days; quick survey - does anyone reading these lines
> > do that?), there's nothing wrong on it, and it gives you nice
> > flexibility.
> >
> > For example, to import bunch of patches (I guess that's the
> > original intention behind this) you just run git-am on them and
> > then stg uncommit all of the newly added commits.
>
> This is a sensible way of using an uncommit command but I initially
> thought it would be better to make things harder for people wanting
> to re-write the history. Anyway, I'll keep this command on my todo
> list.

stgit rewrites history all the time anyway. And as far as I recall,
there's nothing in the documentation that warns the user not to
publish stgit-managed branches. :-)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

[PATCH 0/2] stg uncommit

[Karl Hasselström]

Here is that uncommit command I was going on and on and on about.
There's also some stricter checking that refs/heads/bases is reset to
HEAD whenever we reach zero applied patches, since otherwise you can't
uncommit patches on an empty stomach.

Note the extremely cool feature that you can uncommit regardless of
how dirty your working tree is!

-- 
Karl Hasselström

[PATCH 1/2] Update .git/refs/heads/base after patch deletion

[Karl Hasselström]

Save the current HEAD into refs/heads/base if the stack is empty after
a patch has been deleted. This was not done before, which caused
refs/heads/base to not be updated after 'stg commit'. To guard against
existing repositories with no applied patches and HEAD !=
refs/heads/base, also do the update every time someone asks for the
name of refs/heads/base.

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

 stgit/stack.py |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/stgit/stack.py b/stgit/stack.py
index 68a2936..bc39d14 100644
--- a/stgit/stack.py
+++ b/stgit/stack.py
@@ -366,6 +366,7 @@ class Series:
         return names
 
     def get_base_file(self):
+        self.__begin_stack_check()
         return self.__base_file
 
     def get_protected(self):
@@ -686,6 +687,7 @@ class Series:
         f = file(self.__unapplied_file, 'w+')
         f.writelines([line + '\n' for line in unapplied])
         f.close()
+        self.__begin_stack_check()
 
     def forward_patches(self, names):
         """Try to fast-forward an array of patches.

[PATCH 2/2] Add 'stg uncommit' command

[Karl Hasselström]

Add an uncommit command, which is exactly the opposite of 'stg
commit'.

Signed-off-by: Karl Hasselström <kha@treskal.com>

---

 stgit/commands/commit.py   |    5 ++-
 stgit/commands/uncommit.py |   80 ++++++++++++++++++++++++++++++++++++++++++++
 stgit/main.py              |    2 +
 stgit/stack.py             |   12 +++++--
 4 files changed, 94 insertions(+), 5 deletions(-)

diff --git a/stgit/commands/commit.py b/stgit/commands/commit.py
index a3b7277..ed9a0b3 100644
--- a/stgit/commands/commit.py
+++ b/stgit/commands/commit.py
@@ -28,8 +28,9 @@ usage = """%prog [options]
 Merge the applied patches into the base of the current stack and
 remove them from the series while advancing the base.
 
-Use this command only if you want to permanently store the applied
-patches and no longer manage them with StGIT."""
+Use this command if you want to permanently store the applied patches
+and no longer manage them with StGIT. If you should change your mind
+later, use 'stg uncommit'."""
 
 options = []
 
diff --git a/stgit/commands/uncommit.py b/stgit/commands/uncommit.py
new file mode 100644
index 0000000..4ac0dfb
--- /dev/null
+++ b/stgit/commands/uncommit.py
@@ -0,0 +1,80 @@
+__copyright__ = """
+Copyright (C) 2006, Catalin Marinas <catalin.marinas@gmail.com>
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 2 as
+published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+"""
+
+import sys, os
+from optparse import OptionParser, make_option
+
+from stgit.commands.common import *
+from stgit.utils import *
+from stgit import stack, git
+
+help = 'turn regular git commits into StGIT patches'
+usage = """%prog [options] <patchname1> [<patchname2> ... ]
+
+Takes one or more git commits at the base of the current stack, and
+turns them into StGIT patches. These new patches are alreay applied,
+at the bottom of the stack. This is the exact opposite of 'stg
+commit'.
+
+You can either give one patch name for each commit you wish to
+uncommit, or use the --number option and exactly one patch name; StGIT
+will then create numbered patches with the given patch name as prefix.
+
+Only commits with exactly one parent can be uncommitted; in other
+words, you can't uncommmit a merge."""
+
+options = [make_option('-n', '--number', type = 'int',
+                       help = 'uncommit the specified number of commits')]
+
+def func(parser, options, args):
+    if len(args) == 0:
+        parser.error('you must specify at least one patch name')
+    if options.number:
+        if len(args) != 1:
+            parser.error('when using --number, specify exactly one patch name')
+        patchnames = ['%s%d' % (args[0], i)
+                      for i in xrange(options.number - 1, -1, -1)]
+    else:
+        patchnames = args
+
+    if crt_series.get_protected():
+        raise CmdException, 'This branch is protected. Uncommit is not permitted'
+
+    print 'Uncommitting %d patches...' % len(patchnames),
+    sys.stdout.flush()
+
+    for patchname in patchnames:
+        base_file = crt_series.get_base_file()
+        commit_id = read_string(base_file)
+        commit = git.Commit(commit_id)
+        try:
+            parent, = commit.get_parents()
+        except ValueError:
+            raise CmdException, ('Commit %s does not have exactly one parent'
+                                 % commit_id)
+        author_name, author_email, author_date = name_email_date(
+            commit.get_author())
+        crt_series.new_patch(patchname,
+                             can_edit = False, before_existing = True,
+                             top = commit_id, bottom = parent,
+                             message = commit.get_log(),
+                             author_name = author_name,
+                             author_email = author_email,
+                             author_date = author_date)
+        write_string(base_file, parent)
+
+    print 'done'
diff --git a/stgit/main.py b/stgit/main.py
index 6d86ee4..4a48668 100644
--- a/stgit/main.py
+++ b/stgit/main.py
@@ -57,6 +57,7 @@ import stgit.commands.series
 import stgit.commands.status
 import stgit.commands.top
 import stgit.commands.unapplied
+import stgit.commands.uncommit
 
 
 #
@@ -92,6 +93,7 @@ commands = {
     'status':   stgit.commands.status,
     'top':      stgit.commands.top,
     'unapplied':stgit.commands.unapplied,
+    'uncommit': stgit.commands.uncommit,
     }
 
 def print_help():
diff --git a/stgit/stack.py b/stgit/stack.py
index bc39d14..05389bb 100644
--- a/stgit/stack.py
+++ b/stgit/stack.py
@@ -621,7 +621,8 @@ class Series:
                   unapplied = False, show_patch = False,
                   top = None, bottom = None,
                   author_name = None, author_email = None, author_date = None,
-                  committer_name = None, committer_email = None):
+                  committer_name = None, committer_email = None,
+                  before_existing = False):
         """Creates a new patch
         """
         if self.__patch_applied(name) or self.__patch_unapplied(name):
@@ -664,8 +665,13 @@ class Series:
             f.writelines([line + '\n' for line in patches])
             f.close()
         else:
-            append_string(self.__applied_file, patch.get_name())
-            self.__set_current(name)
+            if before_existing:
+                insert_string(self.__applied_file, patch.get_name())
+                if not self.get_current():
+                    self.__set_current(name)
+            else:
+                append_string(self.__applied_file, patch.get_name())
+                self.__set_current(name)
 
     def delete_patch(self, name):
         """Deletes a patch

Re: [PATCH 2/2] Add 'stg uncommit' command

[Catalin Marinas]

Karl Hasselström wrote:
> Add an uncommit command, which is exactly the opposite of 'stg
> commit'.

Applied with two minor modifications. See below:

> --- a/stgit/commands/commit.py
> +++ b/stgit/commands/commit.py
> @@ -28,8 +28,9 @@ usage = """%prog [options]
>  Merge the applied patches into the base of the current stack and
>  remove them from the series while advancing the base.
>  
> -Use this command only if you want to permanently store the applied
> -patches and no longer manage them with StGIT."""
> +Use this command if you want to permanently store the applied patches
> +and no longer manage them with StGIT. If you should change your mind
> +later, use 'stg uncommit'."""

I removed this change because, even if uncommit does the opposite of
commit does, the intended use is not to use commit/uncommit in pairs.

> diff --git a/stgit/commands/uncommit.py b/stgit/commands/uncommit.py
> new file mode 100644
> index 0000000..4ac0dfb
> --- /dev/null
> +++ b/stgit/commands/uncommit.py
> @@ -0,0 +1,80 @@
> +__copyright__ = """
> +Copyright (C) 2006, Catalin Marinas <catalin.marinas@gmail.com>

I added your name on the copyright since this is a new file.

Thanks,

Catalin

Re: [PATCH 2/2] Add 'stg uncommit' command

[Karl Hasselström]

On 2006-02-19 10:51:06 +0000, Catalin Marinas wrote:

> Karl Hasselström wrote:
>
> > diff --git a/stgit/commands/uncommit.py b/stgit/commands/uncommit.py
> > new file mode 100644
> > index 0000000..4ac0dfb
> > --- /dev/null
> > +++ b/stgit/commands/uncommit.py
> > @@ -0,0 +1,80 @@
> > +__copyright__ = """
> > +Copyright (C) 2006, Catalin Marinas <catalin.marinas@gmail.com>
>
> I added your name on the copyright since this is a new file.

I did that too at first, but then I changed it back since I reckoned
more than 50% of the file was copy-pasted from elsewhere. But thanks.
:-)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: [PATCH 2/2] Add 'stg uncommit' command

[Karl Hasselström]

On 2006-02-19 14:45:58 +0100, Karl Hasselström wrote:

> On 2006-02-19 10:51:06 +0000, Catalin Marinas wrote:
>
> > I added your name on the copyright since this is a new file.
>
> I did that too at first, but then I changed it back since I reckoned
> more than 50% of the file was copy-pasted from elsewhere. But
> thanks. :-)

By the way, it seems like my name got munged when you edited the
commit.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: [PATCH 2/2] Add 'stg uncommit' command

[Sam Vilain]

Karl Hasselström wrote:
>>>I added your name on the copyright since this is a new file.
>>I did that too at first, but then I changed it back since I reckoned
>>more than 50% of the file was copy-pasted from elsewhere. But
>>thanks. :-)
> By the way, it seems like my name got munged when you edited the
> commit.

I have noticed this munging happening, when I am using a UTF-8 locale.

While we are talking about non-linear messing around with the commit
history, can I request a feature?

Currently, I have a patch stack where I want the top of one of the
patches to always be a particular revision.  Consider the last revision
to be like a "difference" revision (ie "what's left" when re-organising
a patch set).

How about a "stg push --fudge-to c033171d..." command for this?

Sam.

Re: [PATCH 2/2] Add 'stg uncommit' command

[Catalin Marinas]

On 19/02/06, Karl Hasselström <kha@treskal.com> wrote:
> By the way, it seems like my name got munged when you edited the
> commit.

I fixed the escaping in the name_email* functions (I'll push it
tonight). It was adding a \ for every character it didn't know. It now
only escapes the quotes and back-slashes. This is needed when passing
the strings via the GIT_AUTHOR_* variables.

--
Catalin

Re: [PATCH 2/2] Add 'stg uncommit' command

[Karl Hasselström]

On 2006-02-20 17:20:47 +0000, Catalin Marinas wrote:

> On 19/02/06, Karl Hasselström <kha@treskal.com> wrote:
>
> > By the way, it seems like my name got munged when you edited the
> > commit.
>
> I fixed the escaping in the name_email* functions (I'll push it
> tonight). It was adding a \ for every character it didn't know. It
> now only escapes the quotes and back-slashes. This is needed when
> passing the strings via the GIT_AUTHOR_* variables.

It put curly braces around the name as well.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: [PATCH 2/2] Add 'stg uncommit' command

[Catalin Marinas]

On 20/02/06, Karl Hasselström <kha@treskal.com> wrote:
> On 2006-02-20 17:20:47 +0000, Catalin Marinas wrote:
> > I fixed the escaping in the name_email* functions (I'll push it
> > tonight). It was adding a \ for every character it didn't know. It
> > now only escapes the quotes and back-slashes. This is needed when
> > passing the strings via the GIT_AUTHOR_* variables.
>
> It put curly braces around the name as well.

It wasn't StGIT. Running "git log" on my machine only shows \'s and
some weird characters. Maybe it's your terminal showing braces.

--
Catalin

Re: [PATCH 2/2] Add 'stg uncommit' command

[Karl Hasselström]

On 2006-02-20 22:49:22 +0000, Catalin Marinas wrote:

> On 20/02/06, Karl Hasselström <kha@treskal.com> wrote:
>
> > It put curly braces around the name as well.
>
> It wasn't StGIT. Running "git log" on my machine only shows \'s and
> some weird characters.

Those weird characters are the two bytes that make up the character
"ö" (o with two dots on top of it) in utf8. That's what the utf8
variant of my name looks like when displayed in latin1. :-/

> Maybe it's your terminal showing braces.

You're right, they don't show when I use git-log, so I guess they
aren't really there after all. But they do show up in gitk.

Hopefully all this weirdness will go away with the backslashes. :-)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Karl Hasselström]

On 2006-02-14 15:58:02 -0500, Chuck Lever wrote:

> Karl Hasselström wrote:
>
> > No, I literally want the opposite of "stg commit", so that the
> > sequence "stg commit; stg uncommit" has zero net effect.
>
> well, that would work OK for maintainers, but would be kind of
> strange for folks who are pulling from such a repository. how would
> that work?

I didn't plan to publish branches where this kind of history munging
was being done. It's precisely like "git rebase" in that regard --
it's a tool for cleaning up history before it is published.

> my impression of git is that you don't change stuff that's already
> committed. you revert changes by applying a new commit that backs
> out the original changes.

You don't change stuff that's already committed _and published_ (well,
except for pu branches :-). Rewriting history is perfectly OK up until
the moment someone has pulled your branch.

> i'm speculating, but i suspect that's why there's a "stg pick
> --reverse" and not a "stg uncommit."

I don't think I've been very successful in communicating exactly what
I want "stg uncommit" for. It's not that I want to undo a committed
change -- what I want is to transform it into an stgit patch so that I
can edit it with a minimum of effort.

  $ edit edit edit
  $ git-commit -a -m "create foo"
  $ edit edit edit
  $ git-commit -a -m "improve foo"
  $ edit edit edit
  $ git-commit -a -m "improve bar"

  # Oops, I realize that the "create foo" changeset had a debug
  # printout left in it, and I wasn't already using stgit.

  $ stg init
  $ stg uncommit improve-bar improve-foo create-foo
  $ stg stg pop --to=create-foo
  $ edit --remove=debug-printout
  $ stg refresh
  $ stg push --all

Similar use-cases for e.g. reordering commits, merging commits,
deleting one commit in the middle of a chain of good ones, etc. are
easy to come up with. The point is that stgit alreay handles all this,
_but only if you have been using stgit from the start_. What "stg
uncommit" does is basically to import (linear) git history into stgit,
where a powerful toolset exists to edit it.

You can actually do this today; just create a new branch where you
want your new stgit stack to be based, and "stg pick" the
commits/patches from the old branch:

  $ git-checkout -b new-branch HEAD^^^
  $ stg init
  $ stg pick old-branch^^^ -n create-foo
  $ stg pick old-branch^^ -n improve-foo
  $ stg pick old-branch^ -n improve-bar
  $ git-branch -D old-branch
  $ git-checkout -b old-branch
  $ git-branch -d new-branch

This series of commands also converts the top three commits to stgit
patches, and leaves the user on the same branch where she started (it
does _exactly_ the same job as "stg uncommit improve-bar improve-foo
create-foo"), but it's a lot of work, and a typo could lose commits.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Andreas Ericsson]

Karl Hasselström wrote:
> On 2006-02-14 15:58:02 -0500, Chuck Lever wrote:
> 
> 
>>Karl Hasselström wrote:
>>
>>
>>>No, I literally want the opposite of "stg commit", so that the
>>>sequence "stg commit; stg uncommit" has zero net effect.
>>
>>well, that would work OK for maintainers, but would be kind of
>>strange for folks who are pulling from such a repository. how would
>>that work?
> 
> 
> I didn't plan to publish branches where this kind of history munging
> was being done. It's precisely like "git rebase" in that regard --
> it's a tool for cleaning up history before it is published.
> 
> 
>>my impression of git is that you don't change stuff that's already
>>committed. you revert changes by applying a new commit that backs
>>out the original changes.
> 
> 
> You don't change stuff that's already committed _and published_ (well,
> except for pu branches :-). Rewriting history is perfectly OK up until
> the moment someone has pulled your branch.
> 
> 
>>i'm speculating, but i suspect that's why there's a "stg pick
>>--reverse" and not a "stg uncommit."
> 
> 
> I don't think I've been very successful in communicating exactly what
> I want "stg uncommit" for. It's not that I want to undo a committed
> change -- what I want is to transform it into an stgit patch so that I
> can edit it with a minimum of effort.
> 
>   $ edit edit edit
>   $ git-commit -a -m "create foo"
>   $ edit edit edit
>   $ git-commit -a -m "improve foo"
>   $ edit edit edit
>   $ git-commit -a -m "improve bar"
> 
>   # Oops, I realize that the "create foo" changeset had a debug
>   # printout left in it, and I wasn't already using stgit.
> 
>   $ stg init
>   $ stg uncommit improve-bar improve-foo create-foo
>   $ stg stg pop --to=create-foo
>   $ edit --remove=debug-printout
>   $ stg refresh
>   $ stg push --all
> 

The same workflow, with less hassle (and already implemented)

$ git format-patch -k HEAD~3
$ edit 0001-*
$ git am -k 000*


> Similar use-cases for e.g. reordering commits, merging commits,
> deleting one commit in the middle of a chain of good ones, etc. are
> easy to come up with. The point is that stgit alreay handles all this,
> _but only if you have been using stgit from the start_. What "stg
> uncommit" does is basically to import (linear) git history into stgit,
> where a powerful toolset exists to edit it.
> 
> You can actually do this today; just create a new branch where you
> want your new stgit stack to be based, and "stg pick" the
> commits/patches from the old branch:
> 
>   $ git-checkout -b new-branch HEAD^^^
>   $ stg init
>   $ stg pick old-branch^^^ -n create-foo
>   $ stg pick old-branch^^ -n improve-foo
>   $ stg pick old-branch^ -n improve-bar
>   $ git-branch -D old-branch
>   $ git-checkout -b old-branch
>   $ git-branch -d new-branch
> 
> This series of commands also converts the top three commits to stgit
> patches, and leaves the user on the same branch where she started (it
> does _exactly_ the same job as "stg uncommit improve-bar improve-foo
> create-foo"), but it's a lot of work, and a typo could lose commits.
> 

Isn't this akin to what "git cherry-pick" does, except for the "convert 
to stgit patches" thing?

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Karl Hasselström]

On 2006-02-15 11:42:55 +0100, Andreas Ericsson wrote:

> Karl Hasselström wrote:
>
> > You can actually do this today; just create a new branch where you
> > want your new stgit stack to be based, and "stg pick" the
> > commits/patches from the old branch:
> >
> >   $ git-checkout -b new-branch HEAD^^^
> >   $ stg init
> >   $ stg pick old-branch^^^ -n create-foo
> >   $ stg pick old-branch^^ -n improve-foo
> >   $ stg pick old-branch^ -n improve-bar
> >   $ git-branch -D old-branch
> >   $ git-checkout -b old-branch
> >   $ git-branch -d new-branch
> >
> > This series of commands also converts the top three commits to
> > stgit patches, and leaves the user on the same branch where she
> > started (it does _exactly_ the same job as "stg uncommit
> > improve-bar improve-foo create-foo"), but it's a lot of work, and
> > a typo could lose commits.
>
> Isn't this akin to what "git cherry-pick" does, except for the
> "convert to stgit patches" thing?

Yes, "stg pick" and git-cherry-pick are very similar as far as I know,
the only difference being that "stg pick" creates an stgit patch while
git-cherry-pick creates a regular commit. (And an applied stgit patch
is just a regular commit which stgit maintains some metadata about.)

However, using git-cherry-pick in this scenario would just recreate
the initial state exactly, since converting the commits to stgit
patches was what it was all about.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Karl Hasselström]

On 2006-02-15 12:25:02 +0100, Karl Hasselström wrote:

> However, using git-cherry-pick in this scenario would just recreate
> the initial state exactly, since converting the commits to stgit
> patches was what it was all about.

"since" -> "and"

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Catalin Marinas]

Petr Baudis wrote:
> 	* I can't just get the patch in its "canonical ready-to-mail
> 	form" on stdout so that I could easily review it. Why is
> 	pg-export insisting to dump it to a file?

I pushed tonight 2 patches for this. One of them adds a --stdout option
to 'export' so that you can see the patches. The other patch adds a
--mbox option to 'mail' that generates an mbox file on the stdout. This
is useful not only for reviewing patches.

-- 
Catalin

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Sam Vilain]

Shawn Pearce wrote:
> I just posted the first public version of pg, a GIT porcelain for
> managing patches.  Think StGIT, but better in some ways:
> 
> Feature Summary:
> - Maximum compatibility with other GIT porcelains.
> - Simplified command line user interface.

How do I edit the description of an existing patch using pg?  Perhaps an
option to pg-push ?

Sam.

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Shawn Pearce]

Sam Vilain <sam@vilain.net> wrote:
> Shawn Pearce wrote:
> >I just posted the first public version of pg, a GIT porcelain for
> >managing patches.  Think StGIT, but better in some ways:
> >
> >Feature Summary:
> >- Maximum compatibility with other GIT porcelains.
> >- Simplified command line user interface.
> 
> How do I edit the description of an existing patch using pg?  Perhaps an
> option to pg-push ?

There isn't any description associated with a patch beyond its name
(which can be changed with pg-rename).  Unlike StGIT pg currently
doesn't store a description with each patch.

This is partly because I want pg to extract the comments given to
pg-ci to make the description of the patch during an export with
pg-export - but I haven't written the code to walk back along the
related commits and extract each comment.  On the other hand this
might not be the best description for a patch.  :-)

-- 
Shawn.

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Sam Vilain]

Shawn Pearce wrote:
>>>I just posted the first public version of pg, a GIT porcelain for
>>>managing patches.  Think StGIT, but better in some ways:
>>>Feature Summary:
>>>- Maximum compatibility with other GIT porcelains.
>>>- Simplified command line user interface.
>>How do I edit the description of an existing patch using pg?  Perhaps an
>>option to pg-push ?
> There isn't any description associated with a patch beyond its name
> (which can be changed with pg-rename).  Unlike StGIT pg currently
> doesn't store a description with each patch.
> This is partly because I want pg to extract the comments given to
> pg-ci to make the description of the patch during an export with
> pg-export - but I haven't written the code to walk back along the
> related commits and extract each comment.  On the other hand this
> might not be the best description for a patch.  :-)

ok.  Well, perhaps a nice solution might be just to aggregate the
comments as each new commit is made.  ie, the previous comment is
prepended to the new comment unless you use the editor or a special
-M (or whatever) option that replaces the running comment.

I tried importing a patchset into pg, and made some changes to it to see
the patch revisioning going on.  However, I can't see this happening.
Can you perhaps include this information in your tutorial?

As far as other, more general critiques of the software goes:  What
about merging?  stgit has a very nice way of merging; I specify how to
merge using a config file, and when I rebase my patches with "stg pull",
it fires up my custom editor.  All I really want is a way to specify how
to handle merges, with the ancestor/left/right files on hand.  I want to
use something as simple as this script:

#!/bin/sh

branch1="$1"
branch2="$2"
ancestor="$3"
output="$4"

echo "Merging:"
echo
echo "   $branch1"
echo " - $ancestor"
echo
echo " with:"
echo
echo "   $branch2"
echo " - $ancestor"
echo
echo " to: $output"
echo ""
echo -n "Trying diff3..."

if diff3 -L local -L older -L remote -m -E "$branch1" "$ancestor" \
    "$branch2" > "$output"
then
     echo "OK"
else
     echo "failed"
     echo "falling back to ediff-merge"
     emacs --eval "(ediff-merge-files-with-ancestor \"${branch1}\"
                    \"${branch2}\" \"${ancestor}\" nil \"${output}\")"
fi

Those commands I got from the default .stgitrc config.

That's all the features I'm really after.

Sam.

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Shawn Pearce]

Sam Vilain <sam@vilain.net> wrote:
> ok.  Well, perhaps a nice solution might be just to aggregate the
> comments as each new commit is made.  ie, the previous comment is
> prepended to the new comment unless you use the editor or a special
> -M (or whatever) option that replaces the running comment.

Yea, that's not a bad idea.  If you are creating a new commit you
probably would want to edit the running description for the patch;
or at least be reminded of what it is.
 
> I tried importing a patchset into pg, and made some changes to it to see
> the patch revisioning going on.  However, I can't see this happening.
> Can you perhaps include this information in your tutorial?

Revisioning doesn't happen for the series, just the individual
patches.  But I've thought about series revisoning and keeping a
secondary GIT index/commit chain external to the main repository
for exactly this purpose.

Each change to a patch (pg-ci) is a new commit object in GIT with
the prior commit object as its parent; if you use pg-ci a few times
with the same patch on the stack then look at the log with git-log
or gitk you'll see the commits are chained together.

When you pop patches and reorder them in the series the resulting
merges are stored as commits with two parents: one for the HEAD
at the time of the merge and one for the commit which was the last
commit in the patch being pushed (HEAD^1 and HEAD^2 respectively).
For example:

	pg-new A
	echo a >>somefile
	pg-ci -m"This is a"
	pg-new B
	echo b >>somefile
	pg-ci -m"This is b"

	pg-pop -a
	pg-push B  # base used to be HEAD+A, now its HEAD
	pg-push A  # base used to be HEAD, now its HEAD+B

The challenge then becomes walking through the merge history.
If you look at pg's own history you'll see an interesting knot
in gitk at a7e73545e511c5c2daea1f6c7bf06cf3179e7f0da (Refreshed
patch Create-Rebase-Tool).  This was produced because I reorded
the patches in the stack and thus had to merge them.  It was an
automatic merge, but it still generated merge commit objects.

Good suggestion about including some details about it in the
tutorial.

> As far as other, more general critiques of the software goes:  What
> about merging?  stgit has a very nice way of merging; I specify how to
> merge using a config file, and when I rebase my patches with "stg pull",
> it fires up my custom editor.  All I really want is a way to specify how
> to handle merges, with the ancestor/left/right files on hand.  I want to
> use something as simple as this script:
> 
>     echo "falling back to ediff-merge"
>     emacs --eval "(ediff-merge-files-with-ancestor \"${branch1}\"
>                    \"${branch2}\" \"${ancestor}\" nil \"${output}\")"

pg doesn't currently invoke any user code when an automatic merge
fails during pg-push or pg-rebase.  It does attempt to produce
a 3 way merge and leaves the resulting portions for you in the
filesystem.  If you look at MERGING.txt you'll see that up to 5
files can come out of a merge (here I'm using the tracked file X.c):

	X.c
	X.c-head
	X.c-last
	X.c-pbase
	X.c-rej

These just get left in the filesystem for you to use as you want;
in your case it sounds like you'd want to invoke:

	emacs --eval "(ediff-merge-files-with-ancestor
		\"X.c-head\"
		\"X.c-last\"
		\"X.c-pbase\"
		nil
		\"X.c\"
		)"

X.c already contains the result of performing:

	diff X.c-pbase X.c-last | patch X.c

so it already has any hunks which were part of your patch and
which applied cleanly to X.c-head (which is the file coming in as
the new base).  Thus you are left only with the rejecting hunks,
which are in X.c-rej.

Personally I've always preferred being given the rejects from
patch to work out a merge problem then to be given the mess that
RCS merge leaves you with.  (I've _never_ been able to decipher
what I want from an RCS merge conflict.)

What is the desired behavior when multiple files have conflicts?
Stop and let the user work on one file before moving to the next?
Open all merge editors in parallel?  Neither seems right to me in
all situations, which is why I just left the `mess' in the filesystem
for the user to resolve at their own pace.

> That's all the features I'm really after.

I like what you are suggesting and will try to incorporate these
improvements this week.

-- 
Shawn.

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Catalin Marinas]

Shawn Pearce <spearce@spearce.org> wrote:
> I just posted the first public version of pg, a GIT porcelain for
> managing patches.  Think StGIT, but better in some ways:

Couldn't help replying to such a topic :-) (only that the ":" ending
of the above phrase might make people think that some features you
listed are not available in StGIT).

Without much testing, I think pg is a good tool but it is different
from StGIT in many ways. It mainly resembles the topic branches way of
working with the advantage of having them stacked on each-other. Each
patch seems to be equivalent to a topic branch where you can commit
changes. Rebasing a patch is equivalent to a merge in a branch with
the merge commit having a general description like "Refreshed patch
..." and two parents - the new base and the old top.

While I don't say the above is a bad thing, it is pretty different
from StGIT. With StGIT, the history of the tree only shows one commit
per patch with the patch description chosen by the user. If you edit
the description or modify the patch, the old patch or description is
dropped from the main branch (visible via HEAD) and you only get the
latest one. This clean history has many advantages when sending
patches upstream either via e-mail or by asking for a pull.

> - Simplified command line user interface.
>
>     pg tries to simplify GIT by 'hiding' the index and behaving like
>     more traditional SCMs which only look at `HEAD` (last commit)
>     and the working directory (files).

This is the case with StGIT as well. It doesn't usually require the
use of GIT commands directly.

> - Preserves change history of patches.
>
>     The complete change history associated with each patch is
>     maintained directly within GIT.  By storing the evolution of a
>     patch as a sequence of GIT commits standard GIT history tools
>     such as gitk can be used.

There have been discussions to adding this to StGIT as well (and there
is a patch already from Chuck). It is a good thing to have but I'm
opposed to the idea of having the history accessible from the top of
the patch. Since the patch can be refreshed indefinitely, it would
make the main history (visible from HEAD) really ugly and also cause
problems with people pulling from a tree. I prefer to have a separate
command (like 'stg id patch/log') that gives access to the history.

> - Its prune proof.
>
>     The metadata structure is stored entirely within the refs
>     directory and the object database, which means you can safely use
>     git-prune without damaging your work, even for unapplied
>     patches.

That's missing indeed in StGIT but it will be available in the next
release. I didn't push this yet because I wasn't sure what to do with
the refresh history of a patch.

> - Automatic detection (and cancellation) of returning patches.
>
>     pg automatically detects when a patch is received from
>     the upstream GIT repository during a pg-rebase and deletes
>     (cancels) the local version of the patch from the patch series.
>     The automatic cancelling makes it easy to use pg to track and
>     develop changes on top of a GIT project.

StGIT has been doing this from the beginning. You would need to run a
'stg clean' after a rebase (or push). I prefer to run this command
manually so that 'stg series -e' would show the empty patches and let
me decided what to do with them.

> - Fast
>
>     pg operations generally perform faster than StGIT operations,
>     at least on my large (~7000 file) repositories.

Might be possible but I haven't done any tests. There are some
optimisations in StGIT that make it pretty fast: (1) if the base of
the patch has not changed, it can fast-forward the pushed patches
which is O(1) and (2) StGIT first tries to use git-apply when pushing
a patch and use a three-way merge only if this fails (the operation
usually succeeds for most of the patches). There are some speed
problems with three-way merging if there are many file
removals/additions because the external merge tool is called for each
of them but the same problem exists for any other tool.

-- 
Catalin

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Shawn Pearce]

Catalin Marinas <catalin.marinas@arm.com> wrote:
> Without much testing, I think pg is a good tool but it is different
> from StGIT in many ways. It mainly resembles the topic branches way of
> working with the advantage of having them stacked on each-other. Each
> patch seems to be equivalent to a topic branch where you can commit
> changes. Rebasing a patch is equivalent to a merge in a branch with
> the merge commit having a general description like "Refreshed patch
> ..." and two parents - the new base and the old top.

Yes, exactly.
 
> While I don't say the above is a bad thing, it is pretty different
> from StGIT. With StGIT, the history of the tree only shows one commit
> per patch with the patch description chosen by the user. If you edit
> the description or modify the patch, the old patch or description is
> dropped from the main branch (visible via HEAD) and you only get the
> latest one. This clean history has many advantages when sending
> patches upstream either via e-mail or by asking for a pull.

Yes.  I didn't intend on exporting the entire patch history for
delivery upstream; I only intended on exporting the batch between
its base and last markers, which amounts to giving a single diff
such as what StGIT would generate.  But I had planned on pulling
the commit comments from all history into the header of the patch
during export, I just haven't gotten there yet.
 
> > - Preserves change history of patches.
> >
> >     The complete change history associated with each patch is
> >     maintained directly within GIT.  By storing the evolution of a
> >     patch as a sequence of GIT commits standard GIT history tools
> >     such as gitk can be used.
> 
> There have been discussions to adding this to StGIT as well (and there
> is a patch already from Chuck). It is a good thing to have but I'm
> opposed to the idea of having the history accessible from the top of
> the patch. Since the patch can be refreshed indefinitely, it would
> make the main history (visible from HEAD) really ugly and also cause
> problems with people pulling from a tree. I prefer to have a separate
> command (like 'stg id patch/log') that gives access to the history.

I definately agree.  I have been rather unhappy with the log
structure that pg is giving me when I flip patches around on the
stack.  So I'm certainly considering keeping the history of the
patch in a parallel tree stored within the same object and refs
database but I haven't really figured out how this could work.
 
> > - Its prune proof.
> >
> >     The metadata structure is stored entirely within the refs
> >     directory and the object database, which means you can safely use
> >     git-prune without damaging your work, even for unapplied
> >     patches.
> 
> That's missing indeed in StGIT but it will be available in the next
> release. I didn't push this yet because I wasn't sure what to do with
> the refresh history of a patch.

I see you actually already pushed out a change for this for StGIT.
That's good news.  :-) I noticed the solution StGIT used is close to
pg's, except that StGIT has the simplified single-commit-per-patch
model so its less refs than pg.

> > - Automatic detection (and cancellation) of returning patches.
> >
> >     pg automatically detects when a patch is received from
> >     the upstream GIT repository during a pg-rebase and deletes
> >     (cancels) the local version of the patch from the patch series.
> >     The automatic cancelling makes it easy to use pg to track and
> >     develop changes on top of a GIT project.
> 
> StGIT has been doing this from the beginning. You would need to run a
> 'stg clean' after a rebase (or push). I prefer to run this command
> manually so that 'stg series -e' would show the empty patches and let
> me decided what to do with them.

Actually StGIT didn't do this correctly for one of my use cases
and that's one of the things that drove me to trying to write pg
(because I wondered if there was a way to resolve it automatically).
Try building a patch series such as:

	... start with an empty stack ...

	... create patch A ...
	... edit file hello.c ...
	... refresh patch A ...

	... create patch B ...
	... edit file hello.c (same line region as patch A) ...
	... refresh patch B ...

	... generate patch A+B (as one patch!) ...
	... send A+B upstream ...

	... pull upstream down ...

StGIT seemed to not handle this when it tried to reapply the two
already applied patches.  A won't apply because the file coming
down is actually A+B, not A's predecessor and not A.  B won't apply
because the file also isn't A (B's predecessor).

pg resolves this by attempting to automatically fold patches during
a pg-rebase (equiv. of stg pull).  If a patch fails to push cleanly
and there's another patch immediately behind it which also should
be reapplied pg aborts and retries pushing the combination of the
patches.  This fixes my A+B case quite nicely during a rebase.  :-)

Of course it doesn't deal with the upstream giving me A+B+C and I
have only A+B locally in my patches.  But I can't have everything
now can I.  :-)
 
> > - Fast
> >
> >     pg operations generally perform faster than StGIT operations,
> >     at least on my large (~7000 file) repositories.
> 
> Might be possible but I haven't done any tests. There are some
> optimisations in StGIT that make it pretty fast: (1) if the base of
> the patch has not changed, it can fast-forward the pushed patches
> which is O(1) and (2) StGIT first tries to use git-apply when pushing
> a patch and use a three-way merge only if this fails (the operation
> usually succeeds for most of the patches). There are some speed
> problems with three-way merging if there are many file
> removals/additions because the external merge tool is called for each
> of them but the same problem exists for any other tool.

pg uses the same optimization for pushing and popping patches. It
also has a special case for the trivially empty patch which StGIT
doesn't seem to have (as StGIT must have a commit for every patch,
pg doesn't require a commit in an empty patch).

However one thing I'm playing around with is using git-read-tree -u
-m to rebase a patch rather than git-diff-tree piped to git-apply
(at least when its not a trivial forward or rewind).  I found that
most of the time to push a patch was spent in git-diff-tree and
hardly anytime was in git-apply.  Using git-read-tree to merge in the
change works nicely in the common case of different patches changing
different files with it falling back to the external merge strategy
when there's unmerged stages in the index.  The open question is
what percentage is this one way or the other?

So I think StGIT is causing a bit more CPU and disk IO than pg is,
but some of these `optimizations' were only put into pg today (and
pushed to my website around 5 pm EST).  I'm actually considering
benchmarking StGIT and pg against the same set of changes to see
how pg compares to StGIT - because I'm now rather curious if pg is
better or worse.


Another difference is the fast-forward when the base of the patch
isn't changed.  In pg this is just:

	git-update-ref HEAD $last $head &&
	git-read-tree -u -m $head $last

which should be slightly faster than StGIT as pg is skipping the
update-index step:

	git-update-index -q --unmerged --refresh
	git-read-tree -u -m head patch
	git-update-ref HEAD patch head

because like StGIT I check for a clean tree before starting the
push; a tree is only clean if the index doesn't need to be refreshed
(plus all the other normal considerations like no unmerged files).
This drops a working directory scan before the read-tree.  :-)

-- 
Shawn.

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Shawn Pearce]

Shawn Pearce <spearce@spearce.org> wrote:
> > >     pg operations generally perform faster than StGIT operations,
> > >     at least on my large (~7000 file) repositories.
> > 
> > Might be possible but I haven't done any tests. There are some
> > optimisations in StGIT that make it pretty fast: (1) if the base of
> > the patch has not changed, it can fast-forward the pushed patches
> > which is O(1) and (2) StGIT first tries to use git-apply when pushing
> > a patch and use a three-way merge only if this fails (the operation
> > usually succeeds for most of the patches). There are some speed
> > problems with three-way merging if there are many file
> > removals/additions because the external merge tool is called for each
> > of them but the same problem exists for any other tool.

I did a quick benchmark of the latest StGIT and pg versions available
by cloning a linux repository twice (pg and stg) and constructing
an identical patch in in each which modified the same set of files
(3 files in 3 different subdirectories).

Popping and pushing this patch of 3 files is a fast forward/rewind
case to both implementations, but pg was faster for me:

stg pop; stg push (fast-forward)
  pop  real 0m13.824s user 0m1.904s sys 0m3.934s
  pop  real 0m13.670s user 0m1.882s sys 0m3.973s
  pop  real 0m21.070s user 0m2.005s sys 0m4.069s
  pop  real 0m15.346s user 0m1.757s sys 0m3.846s
  pop  real 0m16.960s user 0m1.888s sys 0m3.866s
          
  push real 0m20.650s user 0m2.027s sys 0m4.015s
  push real 0m15.624s user 0m1.958s sys 0m3.966s
  push real 0m13.277s user 0m1.746s sys 0m3.796s
  push real 0m12.739s user 0m1.764s sys 0m3.822s
  push real 0m15.161s user 0m1.973s sys 0m3.939s
        
pg-pop; pg-push (fast-forward)
  pop  real 0m10.009s user 0m1.919s sys 0m2.265s
  pop  real 0m 4.710s user 0m1.692s sys 0m1.560s
  pop  real 0m 4.333s user 0m1.664s sys 0m1.554s
  pop  real 0m 5.480s user 0m1.848s sys 0m1.638s
  pop  real 0m 4.412s user 0m1.680s sys 0m1.604s
          
  push real 0m5.813s user 0m1.750s sys 0m1.733s
  push real 0m4.345s user 0m1.686s sys 0m1.632s
  push real 0m5.326s user 0m1.721s sys 0m1.658s
  push real 0m4.740s user 0m1.691s sys 0m1.647s
  push real 0m4.487s user 0m1.702s sys 0m1.637s


I tried to do the merge case which requires reconstructing the
patch onto a new base revision.  This was easy to test over and over
again on pg (pg-rebase; pg-rebase --undo) but I don't know how I can
safely undo the stg pull so I can repeat it on the stg repository.
Instead I tested git-diff-tree|git-apply as that's what StGIT uses
internally.  StGIT does a check for a clean tree before starting
its merge so I cheated here and used the pg version of that check
as part of the diff/apply cost to try and make it slightly more
fair to pg-rebase.

git-diff-tree -p %s %s | git-apply --index
  diff/apply real 0m6.246s user 0m1.377s sys 0m1.634s
  diff/apply real 0m5.293s user 0m1.342s sys 0m1.591s
  diff/apply real 0m5.929s user 0m1.445s sys 0m1.614s
  diff/apply real 0m5.258s user 0m1.333s sys 0m1.567s
  diff/apply real 0m5.255s user 0m1.339s sys 0m1.581s

pg-rebase; pg-rebase --undo (merge)
  rebase real 0m9.505s user 0m4.028s sys 0m3.366s
  rebase real 0m9.415s user 0m4.020s sys 0m3.334s
  rebase real 0m9.387s user 0m4.028s sys 0m3.364s
  rebase real 0m9.159s user 0m4.024s sys 0m3.353s
  rebase real 0m9.008s user 0m4.025s sys 0m3.326s

  undo   real 0m6.531s user 0m1.852s sys 0m2.403s
  undo   real 0m6.383s user 0m1.815s sys 0m2.405s
  undo   real 0m6.510s user 0m1.849s sys 0m2.406s
  undo   real 0m6.519s user 0m1.846s sys 0m2.408s
  undo   real 0m6.496s user 0m1.878s sys 0m2.413s

I guess you could say I didn't entirely expect this result.
The diff-tree/apply approach is faster for a single commit then
read-tree -u -m is; even if totally different files are being
impacted and thus all stages collapse neatly to stage 0 in the index.
No wonder StGIT uses diff/apply!

-- 
Shawn.

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Catalin Marinas]

On 14/02/06, Shawn Pearce <spearce@spearce.org> wrote:
> The diff-tree/apply approach is faster for a single commit then
> read-tree -u -m is; even if totally different files are being
> impacted and thus all stages collapse neatly to stage 0 in the index.
> No wonder StGIT uses diff/apply!

For the simple tests you did the difference is not that big. It
becomes a real problem when there are many file deletions/additions in
the upstream tree since git-read-tree doesn't handle them and
git-merge-index would need to call the external tool for each of them.

To test the above, clone the 2.6.12 kernel version, create some
trivial patches and rebase to 2.6.16-rc3. StGIT was running even for 5
minutes per patch before implementing the diff-tree/apply method.

--
Catalin

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Catalin Marinas]

On 14/02/06, Shawn Pearce <spearce@spearce.org> wrote:
> Catalin Marinas <catalin.marinas@arm.com> wrote:
> > > - Automatic detection (and cancellation) of returning patches.
[...]
> > StGIT has been doing this from the beginning. You would need to run a
> > 'stg clean' after a rebase (or push). I prefer to run this command
> > manually so that 'stg series -e' would show the empty patches and let
> > me decided what to do with them.
>
> Actually StGIT didn't do this correctly for one of my use cases
> and that's one of the things that drove me to trying to write pg
> (because I wondered if there was a way to resolve it automatically).
> Try building a patch series such as:
[...]
> StGIT seemed to not handle this when it tried to reapply the two
> already applied patches.  A won't apply because the file coming
> down is actually A+B, not A's predecessor and not A.  B won't apply
> because the file also isn't A (B's predecessor).

You are right, if two patches modify the same line and both were
merged upstream, the three-way merging would report a conflict for the
first patch and maybe the second (depending on how the first conflict
was resolved).

> pg resolves this by attempting to automatically fold patches during
> a pg-rebase (equiv. of stg pull).  If a patch fails to push cleanly
> and there's another patch immediately behind it which also should
> be reapplied pg aborts and retries pushing the combination of the
> patches.  This fixes my A+B case quite nicely during a rebase.  :-)

But what would happen if there was a third-party patch that's
modifying the same line? A+B application would fail in this case. Does
pg go back to only apply A and report a conflict?

There is another problem with this approach if you have tens of
patches. Would pg try to fold all of them?

Some time ago I had a look at Darcs and its patch theory (patch
commuting). Their approach to conflicts was to include the conflicts
in patch A and propagate them to the last patch to be merged. It's
like creating two versions of the conflicting hunk, one of them
corresponding to the local tree (that in patch A) and the other to the
upstream tree. Merging patch B is only done in the local hunk in the
end both conflicting hunks would be identical and one of them removed.

While the above algrithm seems to work OK in Darcs (but quite resource
intensive), it's pretty hard to implement and I don't think it's worth
for a small number of cases this could occur.

--
Catalin

Re: [ANNOUNCE] pg - A patch porcelain for GIT

[Shawn Pearce]

Catalin Marinas <catalin.marinas@gmail.com> wrote:
> > pg resolves this by attempting to automatically fold patches during
> > a pg-rebase (equiv. of stg pull).  If a patch fails to push cleanly
> > and there's another patch immediately behind it which also should
> > be reapplied pg aborts and retries pushing the combination of the
> > patches.  This fixes my A+B case quite nicely during a rebase.  :-)
> 
> But what would happen if there was a third-party patch that's
> modifying the same line? A+B application would fail in this case. Does
> pg go back to only apply A and report a conflict?

When this occurs pg just gives up and leaves both patches A and
B unapplied and gives you the list of patches which it couldn't
apply but wanted to.  The working directory is left clean; its the
new base plus whatever patches before A that did apply cleanly.
I could have pg go back and try pushing A again and leave the
conflict ready for you to resolve but I don't always want that.
Since the user can have that happen with a quick no-arg `pg-push`
I leave it to the user to retry pushing A if they really think
that's worth trying.

However if the last patch fails to push during a pg-rebase then pg
leaves it alone and your working directory is dirty and you are left
with that last patch partially applied.  At which point you can back
it out by popping it off the stack or finish the conflict resolution.
 
> There is another problem with this approach if you have tens of
> patches. Would pg try to fold all of them?

Yea.  Which might not be pretty.  10 patches would cause pg to
attempt applying 11 patches before giving up, but each time the patch
is increased in size to include its predecessors who also didn't
apply cleanly.  As soon as a larger cluster applies pg goes back to
trying single patch application.  Obviously this could take a while
as the patch size is growing on each attempt and we are duplicating
work every time as pg always starts from a clean working directory.

Example: Say I have A, B, C, D, E, F on the stack.  A wasn't provided
by the upstream and pushes down cleanly.  B+C+D was given to me
by the upstream so pg first tries B, fails, then B+C, fails, then
B+C+D, succeeds, so it folds B+C+D into D and finishes pushing D.
Then it tries E, if E succeeds it tries F on its own.  If E fails it
tries E+F.  What's left in the working directory depends on if the
last operation was an auto-fold attempt or not and if it applied
cleanly (or not).

> Some time ago I had a look at Darcs and its patch theory (patch
> commuting). Their approach to conflicts was to include the conflicts
> in patch A and propagate them to the last patch to be merged. It's
> like creating two versions of the conflicting hunk, one of them
> corresponding to the local tree (that in patch A) and the other to the
> upstream tree. Merging patch B is only done in the local hunk in the
> end both conflicting hunks would be identical and one of them removed.
> 
> While the above algrithm seems to work OK in Darcs (but quite resource
> intensive), it's pretty hard to implement and I don't think it's worth
> for a small number of cases this could occur.

Hmm.  I had looked at Darcs over a year ago and found it to be a
rather interesting idea but at the time it couldn't handle my ~7000
file tree (and GIT wasn't even getting started yet).  I was actually
thinking about trying to drag the rejecting hunks forward somehow
when doing the auto-folding but I hadn't quite found a way to do
that easily.  I have a gut feeling that most of the time when this
problem occurs its on a subset of the files involved in any given
patch and that if I can push down a patch cleanly for 90+% of the
files while delaying the conflicts forward that might actually be
somewhat reasonable.  But maybe not.  :-)

-- 
Shawn.