* [StGIT PATCH 0/4] Clean up subprocess calling
@ 2007-08-26 20:33 Karl Hasselström
2007-08-26 20:33 ` [StGIT PATCH 1/4] Refactor output handling to break circular dependency Karl Hasselström
` (3 more replies)
0 siblings, 4 replies; 14+ messages in thread
From: Karl Hasselström @ 2007-08-26 20:33 UTC (permalink / raw)
To: Catalin Marinas; +Cc: git
We had nearly ten separate functions implementing about five different
ways of calling a subprocess. This series cleans it up. The aggregate
diffstat looks scary, but that's mostly because these functions were
called in a bajillion different places.
If you choose to pull from my repository (which I highly recommend!),
note that this series is built on top of the misc fixes, so you get
those as well.
The following changes since commit 216a1524c4acbd9952ffaeec054e30cf14dde5fc:
Karl Hasselström (1):
Compile regexp just once
are available in the git repository at:
git://repo.or.cz/stgit/kha.git run
Karl Hasselström (4):
Refactor output handling to break circular dependency
Refactor subprocess creation
Assert that the argument to Run is a sequence of strings
Add optional logging of subprocess execution
stgit/basedir.py | 13 +-
stgit/commands/applied.py | 1 +
stgit/commands/assimilate.py | 1 +
stgit/commands/branch.py | 1 +
stgit/commands/clean.py | 1 +
stgit/commands/commit.py | 1 +
stgit/commands/common.py | 1 +
stgit/commands/delete.py | 1 +
stgit/commands/diff.py | 1 +
stgit/commands/export.py | 10 +-
stgit/commands/files.py | 1 +
stgit/commands/fold.py | 1 +
stgit/commands/hide.py | 1 +
stgit/commands/id.py | 1 +
stgit/commands/imprt.py | 1 +
stgit/commands/log.py | 1 +
stgit/commands/mail.py | 1 +
stgit/commands/patches.py | 1 +
stgit/commands/pick.py | 1 +
stgit/commands/pull.py | 1 +
stgit/commands/push.py | 1 +
stgit/commands/refresh.py | 1 +
stgit/commands/rename.py | 1 +
stgit/commands/series.py | 1 +
stgit/commands/sync.py | 1 +
stgit/commands/top.py | 1 +
stgit/commands/unapplied.py | 1 +
stgit/commands/uncommit.py | 1 +
stgit/commands/unhide.py | 1 +
stgit/config.py | 56 ++-----
stgit/git.py | 331 +++++++++++++++++-------------------------
stgit/gitmergeonefile.py | 21 +--
stgit/main.py | 2 +-
stgit/out.py | 100 +++++++++++++
stgit/run.py | 141 ++++++++++++++++++
stgit/stack.py | 1 +
stgit/utils.py | 81 +----------
37 files changed, 436 insertions(+), 347 deletions(-)
create mode 100644 stgit/out.py
create mode 100644 stgit/run.py
--
Karl Hasselström, kha@treskal.com
www.treskal.com/kalle
^ permalink raw reply [flat|nested] 14+ messages in thread
* [StGIT PATCH 1/4] Refactor output handling to break circular dependency
2007-08-26 20:33 [StGIT PATCH 0/4] Clean up subprocess calling Karl Hasselström
@ 2007-08-26 20:33 ` Karl Hasselström
2007-08-26 20:33 ` [StGIT PATCH 2/4] Refactor subprocess creation Karl Hasselström
` (2 subsequent siblings)
3 siblings, 0 replies; 14+ messages in thread
From: Karl Hasselström @ 2007-08-26 20:33 UTC (permalink / raw)
To: Catalin Marinas; +Cc: git
I ran into problems when trying to use the fancy output handling in a
new module that was (recursively) imported by stgit.util -- I couldn't
use the out object because it wasn't defined yet.
Break out the output handler to its own module to break the circular
dependency.
Signed-off-by: Karl Hasselström <kha@treskal.com>
---
stgit/commands/applied.py | 1
stgit/commands/assimilate.py | 1
stgit/commands/branch.py | 1
stgit/commands/clean.py | 1
stgit/commands/commit.py | 1
stgit/commands/common.py | 1
stgit/commands/delete.py | 1
stgit/commands/diff.py | 1
stgit/commands/export.py | 1
stgit/commands/files.py | 1
stgit/commands/fold.py | 1
stgit/commands/hide.py | 1
stgit/commands/id.py | 1
stgit/commands/imprt.py | 1
stgit/commands/log.py | 1
stgit/commands/mail.py | 1
stgit/commands/patches.py | 1
stgit/commands/pick.py | 1
stgit/commands/pull.py | 1
stgit/commands/push.py | 1
stgit/commands/refresh.py | 1
stgit/commands/rename.py | 1
stgit/commands/series.py | 1
stgit/commands/sync.py | 1
stgit/commands/top.py | 1
stgit/commands/unapplied.py | 1
stgit/commands/uncommit.py | 1
stgit/commands/unhide.py | 1
stgit/git.py | 1
stgit/gitmergeonefile.py | 3 +
stgit/main.py | 2 -
stgit/out.py | 100 ++++++++++++++++++++++++++++++++++++++++++
stgit/stack.py | 1
stgit/utils.py | 81 ----------------------------------
34 files changed, 134 insertions(+), 82 deletions(-)
diff --git a/stgit/commands/applied.py b/stgit/commands/applied.py
index 0925de0..b9bb716 100644
--- a/stgit/commands/applied.py
+++ b/stgit/commands/applied.py
@@ -21,6 +21,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/commands/assimilate.py b/stgit/commands/assimilate.py
index 5134e34..c5f4340 100644
--- a/stgit/commands/assimilate.py
+++ b/stgit/commands/assimilate.py
@@ -22,6 +22,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
help = 'StGIT-ify any GIT commits made on top of your StGIT stack'
diff --git a/stgit/commands/branch.py b/stgit/commands/branch.py
index 75a9046..2d491d5 100644
--- a/stgit/commands/branch.py
+++ b/stgit/commands/branch.py
@@ -23,6 +23,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git, basedir
diff --git a/stgit/commands/clean.py b/stgit/commands/clean.py
index 7b57526..2e3b202 100644
--- a/stgit/commands/clean.py
+++ b/stgit/commands/clean.py
@@ -20,6 +20,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/commands/commit.py b/stgit/commands/commit.py
index 2b8d7ce..0b76c56 100644
--- a/stgit/commands/commit.py
+++ b/stgit/commands/commit.py
@@ -20,6 +20,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
help = 'permanently store the applied patches into stack base'
diff --git a/stgit/commands/common.py b/stgit/commands/common.py
index dddee85..f3fa89d 100644
--- a/stgit/commands/common.py
+++ b/stgit/commands/common.py
@@ -22,6 +22,7 @@ import sys, os, os.path, re
from optparse import OptionParser, make_option
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git, basedir
from stgit.config import config, file_extensions
diff --git a/stgit/commands/delete.py b/stgit/commands/delete.py
index a9e2744..2121015 100644
--- a/stgit/commands/delete.py
+++ b/stgit/commands/delete.py
@@ -21,6 +21,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/commands/diff.py b/stgit/commands/diff.py
index f8b19f8..aeca4ab 100644
--- a/stgit/commands/diff.py
+++ b/stgit/commands/diff.py
@@ -22,6 +22,7 @@ from pydoc import pager
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/commands/export.py b/stgit/commands/export.py
index 8424f9d..5ca07d3 100644
--- a/stgit/commands/export.py
+++ b/stgit/commands/export.py
@@ -23,6 +23,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git, templates
diff --git a/stgit/commands/files.py b/stgit/commands/files.py
index 659a82b..1a10023 100644
--- a/stgit/commands/files.py
+++ b/stgit/commands/files.py
@@ -21,6 +21,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/commands/fold.py b/stgit/commands/fold.py
index 297dfbf..d97185e 100644
--- a/stgit/commands/fold.py
+++ b/stgit/commands/fold.py
@@ -20,6 +20,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/commands/hide.py b/stgit/commands/hide.py
index 39cbd67..de19c19 100644
--- a/stgit/commands/hide.py
+++ b/stgit/commands/hide.py
@@ -20,6 +20,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/commands/id.py b/stgit/commands/id.py
index 8c717e8..f72d2f3 100644
--- a/stgit/commands/id.py
+++ b/stgit/commands/id.py
@@ -20,6 +20,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/commands/imprt.py b/stgit/commands/imprt.py
index 98fe708..57bf2c8 100644
--- a/stgit/commands/imprt.py
+++ b/stgit/commands/imprt.py
@@ -23,6 +23,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/commands/log.py b/stgit/commands/log.py
index ebd8cab..4d6b022 100644
--- a/stgit/commands/log.py
+++ b/stgit/commands/log.py
@@ -20,6 +20,7 @@ from optparse import OptionParser, make_option
from pydoc import pager
from stgit.commands.common import *
from stgit import stack, git
+from stgit.out import *
help = 'display the patch changelog'
usage = """%prog [options] [patch]
diff --git a/stgit/commands/mail.py b/stgit/commands/mail.py
index 71a6e4e..6202fc5 100644
--- a/stgit/commands/mail.py
+++ b/stgit/commands/mail.py
@@ -21,6 +21,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git, version, templates
from stgit.config import config
diff --git a/stgit/commands/patches.py b/stgit/commands/patches.py
index 23b3aa7..b3defb6 100644
--- a/stgit/commands/patches.py
+++ b/stgit/commands/patches.py
@@ -21,6 +21,7 @@ from pydoc import pager
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/commands/pick.py b/stgit/commands/pick.py
index 27fdf9c..1c3ef11 100644
--- a/stgit/commands/pick.py
+++ b/stgit/commands/pick.py
@@ -20,6 +20,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
from stgit.stack import Series
diff --git a/stgit/commands/pull.py b/stgit/commands/pull.py
index fe3b67d..ad485a5 100644
--- a/stgit/commands/pull.py
+++ b/stgit/commands/pull.py
@@ -20,6 +20,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit.config import GitConfigException
from stgit import stack, git
diff --git a/stgit/commands/push.py b/stgit/commands/push.py
index 17b32f6..53cdb8f 100644
--- a/stgit/commands/push.py
+++ b/stgit/commands/push.py
@@ -21,6 +21,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/commands/refresh.py b/stgit/commands/refresh.py
index 8277388..218075b 100644
--- a/stgit/commands/refresh.py
+++ b/stgit/commands/refresh.py
@@ -21,6 +21,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
from stgit.config import config
diff --git a/stgit/commands/rename.py b/stgit/commands/rename.py
index d6c53be..2830e72 100644
--- a/stgit/commands/rename.py
+++ b/stgit/commands/rename.py
@@ -20,6 +20,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/commands/series.py b/stgit/commands/series.py
index 9e0b0ff..00a3372 100644
--- a/stgit/commands/series.py
+++ b/stgit/commands/series.py
@@ -22,6 +22,7 @@ from optparse import OptionParser, make_option
import stgit.commands.common
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/commands/sync.py b/stgit/commands/sync.py
index 5e33324..580b5bd 100644
--- a/stgit/commands/sync.py
+++ b/stgit/commands/sync.py
@@ -21,6 +21,7 @@ from optparse import OptionParser, make_option
import stgit.commands.common
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/commands/top.py b/stgit/commands/top.py
index 7cc92ca..1a9267a 100644
--- a/stgit/commands/top.py
+++ b/stgit/commands/top.py
@@ -21,6 +21,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/commands/unapplied.py b/stgit/commands/unapplied.py
index 0d330a1..c6408a3 100644
--- a/stgit/commands/unapplied.py
+++ b/stgit/commands/unapplied.py
@@ -21,6 +21,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/commands/uncommit.py b/stgit/commands/uncommit.py
index f611d29..e3bf0d8 100644
--- a/stgit/commands/uncommit.py
+++ b/stgit/commands/uncommit.py
@@ -22,6 +22,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
help = 'turn regular GIT commits into StGIT patches'
diff --git a/stgit/commands/unhide.py b/stgit/commands/unhide.py
index b6a2297..0a1dcaf 100644
--- a/stgit/commands/unhide.py
+++ b/stgit/commands/unhide.py
@@ -20,6 +20,7 @@ from optparse import OptionParser, make_option
from stgit.commands.common import *
from stgit.utils import *
+from stgit.out import *
from stgit import stack, git
diff --git a/stgit/git.py b/stgit/git.py
index 14b4c81..827bd61 100644
--- a/stgit/git.py
+++ b/stgit/git.py
@@ -23,6 +23,7 @@ from shutil import copyfile
from stgit import basedir
from stgit.utils import *
+from stgit.out import *
from stgit.config import config
from sets import Set
diff --git a/stgit/gitmergeonefile.py b/stgit/gitmergeonefile.py
index 5e51b8a..e76f9b1 100644
--- a/stgit/gitmergeonefile.py
+++ b/stgit/gitmergeonefile.py
@@ -21,7 +21,8 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
import sys, os
from stgit import basedir
from stgit.config import config, file_extensions, ConfigOption
-from stgit.utils import append_string, out
+from stgit.utils import append_string
+from stgit.out import *
class GitMergeException(Exception):
diff --git a/stgit/main.py b/stgit/main.py
index 2390110..5b9d7c4 100644
--- a/stgit/main.py
+++ b/stgit/main.py
@@ -22,7 +22,7 @@ import sys, os
from optparse import OptionParser
import stgit.commands
-from stgit.utils import out
+from stgit.out import *
#
# The commands map
diff --git a/stgit/out.py b/stgit/out.py
new file mode 100644
index 0000000..f80daf2
--- /dev/null
+++ b/stgit/out.py
@@ -0,0 +1,100 @@
+# -*- coding: utf-8 -*-
+
+__copyright__ = """
+Copyright (C) 2007, Karl Hasselström <kha@treskal.com>
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 2 as
+published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+"""
+
+import sys
+
+class MessagePrinter(object):
+ def __init__(self):
+ class Output(object):
+ def __init__(self, write, flush):
+ self.write = write
+ self.flush = flush
+ self.at_start_of_line = True
+ self.level = 0
+ def new_line(self):
+ """Ensure that we're at the beginning of a line."""
+ if not self.at_start_of_line:
+ self.write('\n')
+ self.at_start_of_line = True
+ def single_line(self, msg, print_newline = True,
+ need_newline = True):
+ """Write a single line. Newline before and after are
+ separately configurable."""
+ if need_newline:
+ self.new_line()
+ if self.at_start_of_line:
+ self.write(' '*self.level)
+ self.write(msg)
+ if print_newline:
+ self.write('\n')
+ self.at_start_of_line = True
+ else:
+ self.flush()
+ self.at_start_of_line = False
+ def tagged_lines(self, tag, lines):
+ tag += ': '
+ for line in lines:
+ self.single_line(tag + line)
+ tag = ' '*len(tag)
+ def write_line(self, line):
+ """Write one line of text on a lines of its own, not
+ indented."""
+ self.new_line()
+ self.write('%s\n' % line)
+ self.at_start_of_line = True
+ def write_raw(self, string):
+ """Write an arbitrary string, possibly containing
+ newlines."""
+ self.new_line()
+ self.write(string)
+ self.at_start_of_line = string.endswith('\n')
+ self.__stdout = Output(sys.stdout.write, sys.stdout.flush)
+ if sys.stdout.isatty():
+ self.__out = self.__stdout
+ else:
+ self.__out = Output(lambda msg: None, lambda: None)
+ def stdout(self, line):
+ """Write a line to stdout."""
+ self.__stdout.write_line(line)
+ def stdout_raw(self, string):
+ """Write a string possibly containing newlines to stdout."""
+ self.__stdout.write_raw(string)
+ def info(self, *msgs):
+ for msg in msgs:
+ self.__out.single_line(msg)
+ def note(self, *msgs):
+ self.__out.tagged_lines('Notice', msgs)
+ def warn(self, *msgs):
+ self.__out.tagged_lines('Warning', msgs)
+ def error(self, *msgs):
+ self.__out.tagged_lines('Error', msgs)
+ def start(self, msg):
+ """Start a long-running operation."""
+ self.__out.single_line('%s ... ' % msg, print_newline = False)
+ self.__out.level += 1
+ def done(self, extramsg = None):
+ """Finish long-running operation."""
+ self.__out.level -= 1
+ if extramsg:
+ msg = 'done (%s)' % extramsg
+ else:
+ msg = 'done'
+ self.__out.single_line(msg, need_newline = False)
+
+out = MessagePrinter()
diff --git a/stgit/stack.py b/stgit/stack.py
index 1ab10c5..12c5091 100644
--- a/stgit/stack.py
+++ b/stgit/stack.py
@@ -21,6 +21,7 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
import sys, os, re
from stgit.utils import *
+from stgit.out import *
from stgit import git, basedir, templates
from stgit.config import config
from shutil import copyfile
diff --git a/stgit/utils.py b/stgit/utils.py
index 039b433..38dd474 100644
--- a/stgit/utils.py
+++ b/stgit/utils.py
@@ -3,6 +3,7 @@
import errno, os, os.path, re, sys
from stgit.config import config
+from stgit.out import *
__copyright__ = """
Copyright (C) 2005, Catalin Marinas <catalin.marinas@gmail.com>
@@ -21,86 +22,6 @@ along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
-class MessagePrinter(object):
- def __init__(self):
- class Output(object):
- def __init__(self, write, flush):
- self.write = write
- self.flush = flush
- self.at_start_of_line = True
- self.level = 0
- def new_line(self):
- """Ensure that we're at the beginning of a line."""
- if not self.at_start_of_line:
- self.write('\n')
- self.at_start_of_line = True
- def single_line(self, msg, print_newline = True,
- need_newline = True):
- """Write a single line. Newline before and after are
- separately configurable."""
- if need_newline:
- self.new_line()
- if self.at_start_of_line:
- self.write(' '*self.level)
- self.write(msg)
- if print_newline:
- self.write('\n')
- self.at_start_of_line = True
- else:
- self.flush()
- self.at_start_of_line = False
- def tagged_lines(self, tag, lines):
- tag += ': '
- for line in lines:
- self.single_line(tag + line)
- tag = ' '*len(tag)
- def write_line(self, line):
- """Write one line of text on a lines of its own, not
- indented."""
- self.new_line()
- self.write('%s\n' % line)
- self.at_start_of_line = True
- def write_raw(self, string):
- """Write an arbitrary string, possibly containing
- newlines."""
- self.new_line()
- self.write(string)
- self.at_start_of_line = string.endswith('\n')
- self.__stdout = Output(sys.stdout.write, sys.stdout.flush)
- if sys.stdout.isatty():
- self.__out = self.__stdout
- else:
- self.__out = Output(lambda msg: None, lambda: None)
- def stdout(self, line):
- """Write a line to stdout."""
- self.__stdout.write_line(line)
- def stdout_raw(self, string):
- """Write a string possibly containing newlines to stdout."""
- self.__stdout.write_raw(string)
- def info(self, *msgs):
- for msg in msgs:
- self.__out.single_line(msg)
- def note(self, *msgs):
- self.__out.tagged_lines('Notice', msgs)
- def warn(self, *msgs):
- self.__out.tagged_lines('Warning', msgs)
- def error(self, *msgs):
- self.__out.tagged_lines('Error', msgs)
- def start(self, msg):
- """Start a long-running operation."""
- self.__out.single_line('%s ... ' % msg, print_newline = False)
- self.__out.level += 1
- def done(self, extramsg = None):
- """Finish long-running operation."""
- self.__out.level -= 1
- if extramsg:
- msg = 'done (%s)' % extramsg
- else:
- msg = 'done'
- self.__out.single_line(msg, need_newline = False)
-
-out = MessagePrinter()
-
def mkdir_file(filename, mode):
"""Opens filename with the given mode, creating the directory it's
in if it doesn't already exist."""
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [StGIT PATCH 2/4] Refactor subprocess creation
2007-08-26 20:33 [StGIT PATCH 0/4] Clean up subprocess calling Karl Hasselström
2007-08-26 20:33 ` [StGIT PATCH 1/4] Refactor output handling to break circular dependency Karl Hasselström
@ 2007-08-26 20:33 ` Karl Hasselström
2007-08-26 20:33 ` [StGIT PATCH 3/4] Assert that the argument to Run is a sequence of strings Karl Hasselström
2007-08-26 20:33 ` [StGIT PATCH 4/4] Add optional logging of subprocess execution Karl Hasselström
3 siblings, 0 replies; 14+ messages in thread
From: Karl Hasselström @ 2007-08-26 20:33 UTC (permalink / raw)
To: Catalin Marinas; +Cc: git
Instead of having a gazillion different little functions that wrap
os.spawnvp and popen2.Popen3, make a single wrapper that's flexible
enough to cater to everyone. Apart from making the code cleaner and
easier to read, this has the added benefit of making it possible to
add little things such as extra safety checks and logging.
Ideally, I'd have liked this wrapper to wrap subprocess.Popen, but
that's only available in Python 2.4 and up, so we'll have to make do
with os.spawnvp and popen2.Popen3 for now, even though they suck.
Signed-off-by: Karl Hasselström <kha@treskal.com>
---
stgit/basedir.py | 13 +-
stgit/commands/export.py | 9 -
stgit/config.py | 56 ++------
stgit/git.py | 330 ++++++++++++++++++----------------------------
stgit/gitmergeonefile.py | 18 +--
stgit/run.py | 111 +++++++++++++++
6 files changed, 272 insertions(+), 265 deletions(-)
diff --git a/stgit/basedir.py b/stgit/basedir.py
index 81f2b40..30f45f9 100644
--- a/stgit/basedir.py
+++ b/stgit/basedir.py
@@ -19,13 +19,7 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
import os
-
-def __output(cmd):
- f = os.popen(cmd, 'r')
- string = f.readline().rstrip()
- if f.close():
- return ''
- return string
+from stgit.run import *
# GIT_DIR value cached
__base_dir = None
@@ -39,7 +33,10 @@ def get():
if 'GIT_DIR' in os.environ:
__base_dir = os.environ['GIT_DIR']
else:
- __base_dir = __output('git-rev-parse --git-dir 2> /dev/null')
+ try:
+ __base_dir = Run('git-rev-parse', '--git-dir').output_one_line()
+ except RunException:
+ __base_dir = ''
return __base_dir
diff --git a/stgit/commands/export.py b/stgit/commands/export.py
index 5ca07d3..62be394 100644
--- a/stgit/commands/export.py
+++ b/stgit/commands/export.py
@@ -175,13 +175,10 @@ def func(parser, options, args):
print patch.get_name()
print '-'*79
- # write description
f.write(descr)
- # write the diff
- git.diff(rev1 = patch.get_bottom(),
- rev2 = patch.get_top(),
- out_fd = f,
- diff_flags = diff_flags )
+ f.write(git.diff(rev1 = patch.get_bottom(),
+ rev2 = patch.get_top(),
+ diff_flags = diff_flags))
if not options.stdout:
f.close()
patch_no += 1
diff --git a/stgit/config.py b/stgit/config.py
index 2fd1273..799e1d7 100644
--- a/stgit/config.py
+++ b/stgit/config.py
@@ -20,6 +20,7 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
import os, re
from stgit import basedir
+from stgit.run import *
class GitConfigException(Exception):
pass
@@ -43,50 +44,21 @@ class GitConfig:
__cache={}
- def __run(self, cmd, args=None):
- """__run: runs cmd using spawnvp.
-
- Runs cmd using spawnvp. The shell is avoided so it won't mess up
- our arguments. If args is very large, the command is run multiple
- times; args is split xargs style: cmd is passed on each
- invocation. Unlike xargs, returns immediately if any non-zero
- return code is received.
- """
-
- args_l=cmd.split()
- if args is None:
- args = []
- for i in range(0, len(args)+1, 100):
- r=os.spawnvp(os.P_WAIT, args_l[0], args_l + args[i:min(i+100, len(args))])
- if r:
- return r
- return 0
-
def get(self, name):
if self.__cache.has_key(name):
return self.__cache[name]
-
- stream = os.popen('git repo-config --get %s' % name, 'r')
- value = stream.readline().strip()
- stream.close()
- if len(value) > 0:
- pass
- elif (self.__defaults.has_key(name)):
- value = self.__defaults[name]
- else:
- value = None
-
+ try:
+ value = Run('git-repo-config', '--get', name).output_one_line()
+ except RunException:
+ value = self.__defaults.get(name, None)
self.__cache[name] = value
return value
def getall(self, name):
if self.__cache.has_key(name):
return self.__cache[name]
-
- stream = os.popen('git repo-config --get-all %s' % name, 'r')
- values = [line.strip() for line in stream]
- stream.close()
-
+ values = Run('git-repo-config', '--get-all', name
+ ).returns([0, 1]).output_lines()
self.__cache[name] = values
return values
@@ -98,15 +70,18 @@ class GitConfig:
raise GitConfigException, 'Value for "%s" is not an integer: "%s"' % (name, value)
def rename_section(self, from_name, to_name):
- self.__run('git-repo-config --rename-section', [from_name, to_name])
+ """Rename a section in the config file. Silently do nothing if
+ the section doesn't exist."""
+ Run('git-repo-config', '--rename-section', from_name, to_name
+ ).returns([0, 1]).run()
self.__cache.clear()
def set(self, name, value):
- self.__run('git-repo-config', [name, value])
+ Run('git-repo-config', name, value).run()
self.__cache[name] = value
def unset(self, name):
- self.__run('git-repo-config --unset', [name])
+ Run('git-repo-config', '--unset', name)
self.__cache[name] = None
def sections_matching(self, regexp):
@@ -115,12 +90,11 @@ class GitConfig:
group contents, for all variable names matching the regexp.
"""
result = []
- stream = os.popen('git repo-config --get-regexp "^%s$"' % regexp, 'r')
- for line in stream:
+ for line in Run('git-repo-config', '--get-regexp', '"^%s$"' % regexp
+ ).returns([0, 1]).output_lines():
m = re.match('^%s ' % regexp, line)
if m:
result.append(m.group(1))
- stream.close()
return result
config=GitConfig()
diff --git a/stgit/git.py b/stgit/git.py
index 827bd61..7962cdb 100644
--- a/stgit/git.py
+++ b/stgit/git.py
@@ -18,12 +18,13 @@ along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
-import sys, os, popen2, re, gitmergeonefile
+import sys, os, re, gitmergeonefile
from shutil import copyfile
from stgit import basedir
from stgit.utils import *
from stgit.out import *
+from stgit.run import *
from stgit.config import config
from sets import Set
@@ -31,6 +32,12 @@ from sets import Set
class GitException(Exception):
pass
+# When a subprocess has a problem, we want the exception to be a
+# subclass of GitException.
+class GitRunException(GitException):
+ pass
+class GRun(Run):
+ exc = GitRunException
#
@@ -75,19 +82,21 @@ class Commit:
def __init__(self, id_hash):
self.__id_hash = id_hash
- lines = _output_lines(['git-cat-file', 'commit', id_hash])
+ lines = GRun('git-cat-file', 'commit', id_hash).output_lines()
for i in range(len(lines)):
line = lines[i]
- if line == '\n':
- break
- field = line.strip().split(' ', 1)
- if field[0] == 'tree':
- self.__tree = field[1]
- if field[0] == 'author':
- self.__author = field[1]
- if field[0] == 'committer':
- self.__committer = field[1]
- self.__log = ''.join(lines[i+1:])
+ if not line:
+ break # we've seen all the header fields
+ key, val = line.split(' ', 1)
+ if key == 'tree':
+ self.__tree = val
+ elif key == 'author':
+ self.__author = val
+ elif key == 'committer':
+ self.__committer = val
+ else:
+ pass # ignore other headers
+ self.__log = '\n'.join(lines[i+1:])
def get_id_hash(self):
return self.__id_hash
@@ -103,8 +112,8 @@ class Commit:
return None
def get_parents(self):
- return _output_lines(['git-rev-list', '--parents', '--max-count=1',
- self.__id_hash])[0].split()[1:]
+ return GRun('git-rev-list', '--parents', '--max-count=1', self.__id_hash
+ ).output_one_line().split()[1:]
def get_author(self):
return self.__author
@@ -150,73 +159,6 @@ def get_conflicts():
else:
return None
-def _input(cmd, file_desc):
- p = popen2.Popen3(cmd, True)
- while True:
- line = file_desc.readline()
- if not line:
- break
- p.tochild.write(line)
- p.tochild.close()
- if p.wait():
- raise GitException, '%s failed (%s)' % (' '.join(cmd),
- p.childerr.read().strip())
-
-def _input_str(cmd, string):
- p = popen2.Popen3(cmd, True)
- p.tochild.write(string)
- p.tochild.close()
- if p.wait():
- raise GitException, '%s failed (%s)' % (' '.join(cmd),
- p.childerr.read().strip())
-
-def _output(cmd):
- p=popen2.Popen3(cmd, True)
- output = p.fromchild.read()
- if p.wait():
- raise GitException, '%s failed (%s)' % (' '.join(cmd),
- p.childerr.read().strip())
- return output
-
-def _output_one_line(cmd, file_desc = None):
- p=popen2.Popen3(cmd, True)
- if file_desc != None:
- for line in file_desc:
- p.tochild.write(line)
- p.tochild.close()
- output = p.fromchild.readline().strip()
- if p.wait():
- raise GitException, '%s failed (%s)' % (' '.join(cmd),
- p.childerr.read().strip())
- return output
-
-def _output_lines(cmd):
- p=popen2.Popen3(cmd, True)
- lines = p.fromchild.readlines()
- if p.wait():
- raise GitException, '%s failed (%s)' % (' '.join(cmd),
- p.childerr.read().strip())
- return lines
-
-def __run(cmd, args=None):
- """__run: runs cmd using spawnvp.
-
- Runs cmd using spawnvp. The shell is avoided so it won't mess up
- our arguments. If args is very large, the command is run multiple
- times; args is split xargs style: cmd is passed on each
- invocation. Unlike xargs, returns immediately if any non-zero
- return code is received.
- """
-
- args_l=cmd.split()
- if args is None:
- args = []
- for i in range(0, len(args)+1, 100):
- r=os.spawnvp(os.P_WAIT, args_l[0], args_l + args[i:min(i+100, len(args))])
- if r:
- return r
- return 0
-
def exclude_files():
files = [os.path.join(basedir.get(), 'info', 'exclude')]
user_exclude = config.get('core.excludesfile')
@@ -247,9 +189,9 @@ def tree_status(files = None, tree_id = 'HEAD', unknown = False,
+ ['--exclude-per-directory=.gitignore']
+ ['--exclude-from=%s' % fn for fn in exclude_files()
if os.path.exists(fn)])
- lines = _output_lines(['git-ls-files', '--others', '--directory']
- + exclude)
- cache_files += [('?', line.strip()) for line in lines]
+ lines = GRun('git-ls-files', '--others', '--directory', *exclude
+ ).output_lines()
+ cache_files += [('?', line) for line in lines]
# conflicted files
conflicts = get_conflicts()
@@ -258,8 +200,8 @@ def tree_status(files = None, tree_id = 'HEAD', unknown = False,
cache_files += [('C', filename) for filename in conflicts]
# the rest
- for line in _output_lines(['git-diff-index'] + diff_flags +
- [ tree_id, '--'] + files):
+ for line in GRun('git-diff-index', *(diff_flags + [tree_id, '--'] + files)
+ ).output_lines():
fs = tuple(line.rstrip().split(' ',4)[-1].split('\t',1))
if fs[1] not in conflicts:
cache_files.append(fs)
@@ -277,7 +219,7 @@ def local_changes(verbose = True):
def get_heads():
heads = []
hr = re.compile(r'^[0-9a-f]{40} refs/heads/(.+)$')
- for line in _output_lines(['git-show-ref', '--heads']):
+ for line in GRun('git-show-ref', '--heads').output_lines():
m = hr.match(line)
heads.append(m.group(1))
return heads
@@ -298,7 +240,7 @@ def get_head_file():
"""Returns the name of the file pointed to by the HEAD link
"""
return strip_prefix('refs/heads/',
- _output_one_line(['git-symbolic-ref', 'HEAD']))
+ GRun('git-symbolic-ref', 'HEAD').output_one_line())
def set_head_file(ref):
"""Resets HEAD to point to a new ref
@@ -306,12 +248,16 @@ def set_head_file(ref):
# head cache flushing is needed since we might have a different value
# in the new head
__clear_head_cache()
- if __run('git-symbolic-ref HEAD', ['refs/heads/%s' % ref]) != 0:
+ try:
+ GRun('git-symbolic-ref', 'HEAD', 'refs/heads/%s' % ref).run()
+ except GitRunException:
raise GitException, 'Could not set head to "%s"' % ref
def set_ref(ref, val):
"""Point ref at a new commit object."""
- if __run('git-update-ref', [ref, val]) != 0:
+ try:
+ GRun('git-update-ref', ref, val).run()
+ except GitRunException:
raise GitException, 'Could not update %s to "%s".' % (ref, val)
def set_branch(branch, val):
@@ -339,14 +285,14 @@ def __clear_head_cache():
def refresh_index():
"""Refresh index with stat() information from the working directory.
"""
- __run('git-update-index -q --unmerged --refresh')
+ GRun('git-update-index', '-q', '--unmerged', '--refresh').run()
def rev_parse(git_id):
"""Parse the string and return a verified SHA1 id
"""
try:
- return _output_one_line(['git-rev-parse', '--verify', git_id])
- except GitException:
+ return GRun('git-rev-parse', '--verify', git_id).output_one_line()
+ except GitRunException:
raise GitException, 'Unknown revision: %s' % git_id
def ref_exists(ref):
@@ -387,7 +333,9 @@ def switch_branch(new_branch):
tree_id = rev_parse('refs/heads/%s^{commit}' % new_branch)
if tree_id != get_head():
refresh_index()
- if __run('git-read-tree -u -m', [get_head(), tree_id]) != 0:
+ try:
+ GRun('git-read-tree', '-u', '-m', get_head(), tree_id).run()
+ except GitRunException:
raise GitException, 'git-read-tree failed (local changes maybe?)'
__head = tree_id
set_head_file(new_branch)
@@ -398,8 +346,10 @@ def switch_branch(new_branch):
def delete_ref(ref):
if not ref_exists(ref):
raise GitException, '%s does not exist' % ref
- sha1 = _output_one_line(['git-show-ref', '-s', ref])
- if __run('git-update-ref -d %s %s' % (ref, sha1)):
+ sha1 = GRun('git-show-ref', '-s', ref).output_one_line()
+ try:
+ GRun('git-update-ref', '-d', ref, sha1).run()
+ except GitRunException:
raise GitException, 'Failed to delete ref %s' % ref
def delete_branch(name):
@@ -411,10 +361,14 @@ def rename_ref(from_ref, to_ref):
if ref_exists(to_ref):
raise GitException, '"%s" already exists' % to_ref
- sha1 = _output_one_line(['git-show-ref', '-s', from_ref])
- if __run('git-update-ref %s %s %s' % (to_ref, sha1, '0'*40)):
+ sha1 = GRun('git-show-ref', '-s', from_ref).output_one_line()
+ try:
+ GRun('git-update-ref', to_ref, sha1, '0'*40).run()
+ except GitRunException:
raise GitException, 'Failed to create new ref %s' % to_ref
- if __run('git-update-ref -d %s %s' % (from_ref, sha1)):
+ try:
+ GRun('git-update-ref', '-d', from_ref, sha1).run()
+ except GitRunException:
raise GitException, 'Failed to delete ref %s' % from_ref
def rename_branch(from_name, to_name):
@@ -448,14 +402,16 @@ def add(names):
raise GitException, '%s is not a file or directory' % i
if files:
- if __run('git-update-index --add --', files):
+ try:
+ GRun('git-update-index', '--add', '--').xargs(files)
+ except GitRunException:
raise GitException, 'Unable to add file'
def __copy_single(source, target, target2=''):
"""Copy file or dir named 'source' to name target+target2"""
# "source" (file or dir) must match one or more git-controlled file
- realfiles = _output_lines(['git-ls-files', source])
+ realfiles = GRun('git-ls-files', source).output_lines()
if len(realfiles) == 0:
raise GitException, '"%s" matches no git-controled files' % source
@@ -523,10 +479,10 @@ def rm(files, force = False):
if os.path.exists(f):
raise GitException, '%s exists. Remove it first' %f
if files:
- __run('git-update-index --remove --', files)
+ GRun('git-update-index', '--remove', '--').xargs(files)
else:
if files:
- __run('git-update-index --force-remove --', files)
+ GRun('git-update-index', '--force-remove', '--').xargs(files)
# Persons caching
__user = None
@@ -601,12 +557,9 @@ def update_cache(files = None, force = False):
rm_files = [x[1] for x in cache_files if x[0] in ['D']]
m_files = [x[1] for x in cache_files if x[0] in ['M']]
- if add_files and __run('git-update-index --add --', add_files) != 0:
- raise GitException, 'Failed git-update-index --add'
- if rm_files and __run('git-update-index --force-remove --', rm_files) != 0:
- raise GitException, 'Failed git-update-index --rm'
- if m_files and __run('git-update-index --', m_files) != 0:
- raise GitException, 'Failed git-update-index'
+ GRun('git-update-index', '--add', '--').xargs(add_files)
+ GRun('git-update-index', '--force-remove', '--').xargs(rm_files)
+ GRun('git-update-index', '--').xargs(m_files)
return True
@@ -635,28 +588,24 @@ def commit(message, files = None, parents = None, allowempty = False,
# write the index to repository
if tree_id == None:
- tree_id = _output_one_line(['git-write-tree'])
+ tree_id = GRun('git-write-tree').output_one_line()
set_head = True
# the commit
- cmd = ['env']
+ env = {}
if author_name:
- cmd += ['GIT_AUTHOR_NAME=%s' % author_name]
+ env['GIT_AUTHOR_NAME'] = author_name
if author_email:
- cmd += ['GIT_AUTHOR_EMAIL=%s' % author_email]
+ env['GIT_AUTHOR_EMAIL'] = author_email
if author_date:
- cmd += ['GIT_AUTHOR_DATE=%s' % author_date]
+ env['GIT_AUTHOR_DATE'] = author_date
if committer_name:
- cmd += ['GIT_COMMITTER_NAME=%s' % committer_name]
+ env['GIT_COMMITTER_NAME'] = committer_name
if committer_email:
- cmd += ['GIT_COMMITTER_EMAIL=%s' % committer_email]
- cmd += ['git-commit-tree', tree_id]
-
- # get the parents
- for p in parents:
- cmd += ['-p', p]
-
- commit_id = _output_one_line(cmd, message)
+ env['GIT_COMMITTER_EMAIL'] = committer_email
+ commit_id = GRun('git-commit-tree', tree_id,
+ *sum([['-p', p] for p in parents], [])
+ ).env(env).raw_input(message).output_one_line()
if set_head:
__set_head(commit_id)
@@ -679,8 +628,8 @@ def apply_diff(rev1, rev2, check_index = True, files = None):
diff_str = diff(files, rev1, rev2)
if diff_str:
try:
- _input_str(['git-apply'] + index_opt, diff_str)
- except GitException:
+ GRun('git-apply', *index_opt).raw_input(diff_str).no_output()
+ except GitRunException:
return False
return True
@@ -696,24 +645,28 @@ def merge(base, head1, head2, recursive = False):
# this operation tracks renames but it is slower (used in
# general when pushing or picking patches)
try:
- # use _output() to mask the verbose prints of the tool
- _output(['git-merge-recursive', base, '--', head1, head2])
- except GitException, ex:
+ # discard output to mask the verbose prints of the tool
+ GRun('git-merge-recursive', base, '--', head1, head2
+ ).discard_output()
+ except GitRunException, ex:
err_output = str(ex)
pass
else:
# the fast case where we don't track renames (used when the
# distance between base and heads is small, i.e. folding or
# synchronising patches)
- if __run('git-read-tree -u -m --aggressive',
- [base, head1, head2]) != 0:
+ try:
+ GRun('git-read-tree', '-u', '-m', '--aggressive',
+ base, head1, head2).run()
+ except GitRunException:
raise GitException, 'git-read-tree failed (local changes maybe?)'
# check the index for unmerged entries
files = {}
stages_re = re.compile('^([0-7]+) ([0-9a-f]{40}) ([1-3])\t(.*)$', re.S)
- for line in _output(['git-ls-files', '--unmerged', '--stage', '-z']).split('\0'):
+ for line in GRun('git-ls-files', '--unmerged', '--stage', '-z'
+ ).raw_output().split('\0'):
if not line:
continue
@@ -788,98 +741,85 @@ def status(files = None, modified = False, new = False, deleted = False,
else:
out.stdout('%s' % fs[1])
-def diff(files = None, rev1 = 'HEAD', rev2 = None, out_fd = None,
- diff_flags = []):
+def diff(files = None, rev1 = 'HEAD', rev2 = None, diff_flags = []):
"""Show the diff between rev1 and rev2
"""
if not files:
files = []
if rev1 and rev2:
- diff_str = _output(['git-diff-tree', '-p'] + diff_flags
- + [rev1, rev2, '--'] + files)
+ return GRun('git-diff-tree', '-p',
+ *(diff_flags + [rev1, rev2, '--'] + files)).raw_output()
elif rev1 or rev2:
refresh_index()
if rev2:
- diff_str = _output(['git-diff-index', '-p', '-R']
- + diff_flags + [rev2, '--'] + files)
+ return GRun('git-diff-index', '-p', '-R',
+ *(diff_flags + [rev2, '--'] + files)).raw_output()
else:
- diff_str = _output(['git-diff-index', '-p']
- + diff_flags + [rev1, '--'] + files)
+ return GRun('git-diff-index', '-p',
+ *(diff_flags + [rev1, '--'] + files)).raw_output()
else:
- diff_str = ''
-
- if out_fd:
- out_fd.write(diff_str)
- else:
- return diff_str
+ return ''
def diffstat(files = None, rev1 = 'HEAD', rev2 = None):
- """Return the diffstat between rev1 and rev2
- """
- if not files:
- files = []
-
- p=popen2.Popen3('git-apply --stat')
- diff(files, rev1, rev2, p.tochild)
- p.tochild.close()
- diff_str = p.fromchild.read().rstrip()
- if p.wait():
- raise GitException, 'git.diffstat failed'
- return diff_str
+ """Return the diffstat between rev1 and rev2."""
+ return GRun('git-apply', '--stat'
+ ).raw_input(diff(files, rev1, rev2)).raw_output()
def files(rev1, rev2, diff_flags = []):
"""Return the files modified between rev1 and rev2
"""
- result = ''
- for line in _output_lines(['git-diff-tree'] + diff_flags + ['-r', rev1, rev2]):
- result += '%s %s\n' % tuple(line.rstrip().split(' ',4)[-1].split('\t',1))
+ result = []
+ for line in GRun('git-diff-tree', *(diff_flags + ['-r', rev1, rev2])
+ ).output_lines():
+ result.append('%s %s' % tuple(line.split(' ', 4)[-1].split('\t', 1)))
- return result.rstrip()
+ return '\n'.join(result)
def barefiles(rev1, rev2):
"""Return the files modified between rev1 and rev2, without status info
"""
- result = ''
- for line in _output_lines(['git-diff-tree', '-r', rev1, rev2]):
- result += '%s\n' % line.rstrip().split(' ',4)[-1].split('\t',1)[-1]
+ result = []
+ for line in GRun('git-diff-tree', '-r', rev1, rev2).output_lines():
+ result.append(line.split(' ', 4)[-1].split('\t', 1)[-1])
- return result.rstrip()
+ return '\n'.join(result)
def pretty_commit(commit_id = 'HEAD', diff_flags = []):
"""Return a given commit (log + diff)
"""
- return _output(['git-diff-tree'] + diff_flags +
- ['--cc', '--always', '--pretty', '-r', commit_id])
+ return GRun('git-diff-tree',
+ *(diff_flags
+ + ['--cc', '--always', '--pretty', '-r', commit_id])
+ ).raw_output()
def checkout(files = None, tree_id = None, force = False):
"""Check out the given or all files
"""
- if not files:
- files = []
-
- if tree_id and __run('git-read-tree --reset', [tree_id]) != 0:
- raise GitException, 'Failed git-read-tree --reset %s' % tree_id
+ if tree_id:
+ try:
+ GRun('git-read-tree', '--reset', tree_id).run()
+ except GitRunException:
+ raise GitException, 'Failed git-read-tree --reset %s' % tree_id
- checkout_cmd = 'git-checkout-index -q -u'
+ cmd = ['git-checkout-index', '-q', '-u']
if force:
- checkout_cmd += ' -f'
- if len(files) == 0:
- checkout_cmd += ' -a'
+ cmd.append('-f')
+ if files:
+ GRun(*(cmd + ['--'])).xargs(files)
else:
- checkout_cmd += ' --'
-
- if __run(checkout_cmd, files) != 0:
- raise GitException, 'Failed git-checkout-index'
+ GRun(*(cmd + ['-a'])).run()
def switch(tree_id, keep = False):
"""Switch the tree to the given id
"""
if not keep:
refresh_index()
- if __run('git-read-tree -u -m', [get_head(), tree_id]) != 0:
+ try:
+ GRun('git-read-tree', '-u', '-m', get_head(), tree_id).run()
+ except GitRunException:
raise GitException, 'git-read-tree failed (local changes maybe?)'
__set_head(tree_id)
@@ -917,8 +857,7 @@ def fetch(repository = 'origin', refspec = None):
command = config.get('branch.%s.stgit.fetchcmd' % get_head_file()) or \
config.get('stgit.fetchcmd')
- if __run(command, args) != 0:
- raise GitException, 'Failed "%s %s"' % (command, repository)
+ GRun(*(command.split() + args)).run()
def pull(repository = 'origin', refspec = None):
"""Fetches changes from the remote repository, using 'git-pull'
@@ -933,13 +872,12 @@ def pull(repository = 'origin', refspec = None):
command = config.get('branch.%s.stgit.pullcmd' % get_head_file()) or \
config.get('stgit.pullcmd')
- if __run(command, args) != 0:
- raise GitException, 'Failed "%s %s"' % (command, repository)
+ GRun(*(command.split() + args)).run()
def repack():
"""Repack all objects into a single pack
"""
- __run('git-repack -a -d -f')
+ GRun('git-repack', '-a', '-d', '-f').run()
def apply_patch(filename = None, diff = None, base = None,
fail_dump = True):
@@ -962,8 +900,8 @@ def apply_patch(filename = None, diff = None, base = None,
refresh_index()
try:
- _input_str(['git-apply', '--index'], diff)
- except GitException:
+ GRun('git-apply', '--index').raw_input(diff).no_output()
+ except GitRunException:
if base:
switch(orig_head)
if fail_dump:
@@ -985,18 +923,12 @@ def clone(repository, local_dir):
"""Clone a remote repository. At the moment, just use the
'git-clone' script
"""
- if __run('git-clone', [repository, local_dir]) != 0:
- raise GitException, 'Failed "git-clone %s %s"' \
- % (repository, local_dir)
+ GRun('git-clone', repository, local_dir).run()
def modifying_revs(files, base_rev, head_rev):
- """Return the revisions from the list modifying the given files
- """
- cmd = ['git-rev-list', '%s..%s' % (base_rev, head_rev), '--']
- revs = [line.strip() for line in _output_lines(cmd + files)]
-
- return revs
-
+ """Return the revisions from the list modifying the given files."""
+ return GRun('git-rev-list', '%s..%s' % (base_rev, head_rev), '--', *files
+ ).output_lines()
def refspec_localpart(refspec):
m = re.match('^[^:]*:([^:]*)$', refspec)
@@ -1091,4 +1023,4 @@ def all_refs():
"""Return a list of all refs in the current repository.
"""
- return [line.split()[1] for line in _output_lines(['git-show-ref'])]
+ return [line.split()[1] for line in GRun('git-show-ref').output_lines()]
diff --git a/stgit/gitmergeonefile.py b/stgit/gitmergeonefile.py
index e76f9b1..2aa5ef8 100644
--- a/stgit/gitmergeonefile.py
+++ b/stgit/gitmergeonefile.py
@@ -23,7 +23,7 @@ from stgit import basedir
from stgit.config import config, file_extensions, ConfigOption
from stgit.utils import append_string
from stgit.out import *
-
+from stgit.run import *
class GitMergeException(Exception):
pass
@@ -44,12 +44,8 @@ def __str2none(x):
else:
return x
-def __output(cmd):
- f = os.popen(cmd, 'r')
- string = f.readline().rstrip()
- if f.close():
- raise GitMergeException, 'Error: failed to execute "%s"' % cmd
- return string
+class MRun(Run):
+ exc = GitMergeException # use a custom exception class on errors
def __checkout_files(orig_hash, file1_hash, file2_hash,
path,
@@ -62,24 +58,24 @@ def __checkout_files(orig_hash, file1_hash, file2_hash,
if orig_hash:
orig = path + extensions['ancestor']
- tmp = __output('git-unpack-file %s' % orig_hash)
+ tmp = MRun('git-unpack-file', orig_hash).output_one_line()
os.chmod(tmp, int(orig_mode, 8))
os.renames(tmp, orig)
if file1_hash:
src1 = path + extensions['current']
- tmp = __output('git-unpack-file %s' % file1_hash)
+ tmp = MRun('git-unpack-file', file1_hash).output_one_line()
os.chmod(tmp, int(file1_mode, 8))
os.renames(tmp, src1)
if file2_hash:
src2 = path + extensions['patched']
- tmp = __output('git-unpack-file %s' % file2_hash)
+ tmp = MRun('git-unpack-file', file2_hash).output_one_line()
os.chmod(tmp, int(file2_mode, 8))
os.renames(tmp, src2)
if file1_hash and not os.path.exists(path):
# the current file might be removed by GIT when it is a new
# file added in both branches. Just re-generate it
- tmp = __output('git-unpack-file %s' % file1_hash)
+ tmp = MRun('git-unpack-file', file1_hash).output_one_line()
os.chmod(tmp, int(file1_mode, 8))
os.renames(tmp, path)
diff --git a/stgit/run.py b/stgit/run.py
new file mode 100644
index 0000000..d925cce
--- /dev/null
+++ b/stgit/run.py
@@ -0,0 +1,111 @@
+# -*- coding: utf-8 -*-
+
+__copyright__ = """
+Copyright (C) 2007, Karl Hasselström <kha@treskal.com>
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 2 as
+published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+"""
+
+# popen2 and os.spawn* suck. We should really use subprocess instead,
+# but that's only available in Python 2.4 and up, and we try our best
+# to stay Python 2.3 compatible.
+import popen2, os
+
+class RunException(Exception):
+ """Thrown when something bad happened when we tried to run the
+ subprocess."""
+ pass
+
+class Run:
+ exc = RunException
+ def __init__(self, *cmd):
+ self.__cmd = list(cmd)
+ self.__good_retvals = [0]
+ self.__env = None
+ self.__indata = None
+ def __run_io(self, cmd):
+ """Run with captured IO. Note: arguments are parsed by the
+ shell. We single-quote them, so don't use anything with single
+ quotes in it."""
+ if self.__env == None:
+ ecmd = cmd
+ else:
+ ecmd = (['env'] + ['%s=%s' % (key, val)
+ for key, val in self.__env.iteritems()]
+ + cmd)
+ p = popen2.Popen3(' '.join(["'%s'" % c for c in ecmd]), True)
+ if self.__indata != None:
+ p.tochild.write(self.__indata)
+ p.tochild.close()
+ outdata = p.fromchild.read()
+ errdata = p.childerr.read()
+ self.exitcode = p.wait() >> 8
+ if errdata or self.exitcode not in self.__good_retvals:
+ raise self.exc('%s failed with code %d:\n%s'
+ % (cmd[0], self.exitcode, errdata))
+ return outdata
+ def __run_noshell(self, cmd):
+ """Run without captured IO. Note: arguments are not parsed by
+ the shell."""
+ assert self.__env == None
+ assert self.__indata == None
+ self.exitcode = os.spawnvp(os.P_WAIT, cmd[0], cmd)
+ if not self.exitcode in self.__good_retvals:
+ raise self.exc('%s failed with code %d'
+ % (cmd[0], self.exitcode))
+ def returns(self, retvals):
+ self.__good_retvals = retvals
+ return self
+ def env(self, env):
+ self.__env = env
+ return self
+ def raw_input(self, indata):
+ self.__indata = indata
+ return self
+ def input_lines(self, lines):
+ self.__indata = ''.join(['%s\n' % line for line in lines])
+ return self
+ def no_output(self):
+ outdata = self.__run_io(self.__cmd)
+ if outdata:
+ raise self.exc, '%s produced output' % self.__cmd[0]
+ def discard_output(self):
+ self.__run_io(self.__cmd)
+ def raw_output(self):
+ return self.__run_io(self.__cmd)
+ def output_lines(self):
+ outdata = self.__run_io(self.__cmd)
+ if outdata.endswith('\n'):
+ outdata = outdata[:-1]
+ if outdata:
+ return outdata.split('\n')
+ else:
+ return []
+ def output_one_line(self):
+ outlines = self.output_lines()
+ if len(outlines) == 1:
+ return outlines[0]
+ else:
+ raise self.exc('%s produced %d lines, expected 1'
+ % (self.__cmd[0], len(outlines)))
+ def run(self):
+ """Just run, with no IO redirection."""
+ self.__run_noshell(self.__cmd)
+ def xargs(self, xargs):
+ """Just run, with no IO redirection. The extra arguments are
+ appended to the command line a few at a time; the command is
+ run as many times as needed to consume them all."""
+ step = 100
+ for i in xrange(0, len(xargs), step):
+ self.__run_noshell(self.__cmd + xargs[i:i+step])
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [StGIT PATCH 3/4] Assert that the argument to Run is a sequence of strings
2007-08-26 20:33 [StGIT PATCH 0/4] Clean up subprocess calling Karl Hasselström
2007-08-26 20:33 ` [StGIT PATCH 1/4] Refactor output handling to break circular dependency Karl Hasselström
2007-08-26 20:33 ` [StGIT PATCH 2/4] Refactor subprocess creation Karl Hasselström
@ 2007-08-26 20:33 ` Karl Hasselström
2007-08-26 20:33 ` [StGIT PATCH 4/4] Add optional logging of subprocess execution Karl Hasselström
3 siblings, 0 replies; 14+ messages in thread
From: Karl Hasselström @ 2007-08-26 20:33 UTC (permalink / raw)
To: Catalin Marinas; +Cc: git
This runtime assertion makes bugs easier to find.
In most other languages, we'd have been able to check this at compile
time. But this is Python. Yay!
Signed-off-by: Karl Hasselström <kha@treskal.com>
---
stgit/run.py | 3 +++
1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/stgit/run.py b/stgit/run.py
index d925cce..1bc4759 100644
--- a/stgit/run.py
+++ b/stgit/run.py
@@ -31,6 +31,9 @@ class Run:
exc = RunException
def __init__(self, *cmd):
self.__cmd = list(cmd)
+ for c in cmd:
+ if type(c) != str:
+ raise Exception, 'Bad command: %r' % cmd
self.__good_retvals = [0]
self.__env = None
self.__indata = None
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [StGIT PATCH 4/4] Add optional logging of subprocess execution
2007-08-26 20:33 [StGIT PATCH 0/4] Clean up subprocess calling Karl Hasselström
` (2 preceding siblings ...)
2007-08-26 20:33 ` [StGIT PATCH 3/4] Assert that the argument to Run is a sequence of strings Karl Hasselström
@ 2007-08-26 20:33 ` Karl Hasselström
2007-08-29 10:50 ` Catalin Marinas
3 siblings, 1 reply; 14+ messages in thread
From: Karl Hasselström @ 2007-08-26 20:33 UTC (permalink / raw)
To: Catalin Marinas; +Cc: git
Now that the subprocess calling has been refactored and is in a nice
shape, it's quite simple to add some logging facilities. This patch
adds two separate log modes, switched by the STG_SUBPROCESS_LOG
environment variable:
* Setting it to "debug" prints the executable name and all
arguments, and the subprocess return value.
* Setting it to "profile" prints just the executable name, and the
(wallclock) time elapsed during the call.
* Not setting it will disable logging, of course.
Signed-off-by: Karl Hasselström <kha@treskal.com>
---
stgit/run.py | 27 +++++++++++++++++++++++++++
1 files changed, 27 insertions(+), 0 deletions(-)
diff --git a/stgit/run.py b/stgit/run.py
index 1bc4759..719ecb7 100644
--- a/stgit/run.py
+++ b/stgit/run.py
@@ -22,11 +22,22 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
# to stay Python 2.3 compatible.
import popen2, os
+import datetime
+
+from stgit.out import *
+
class RunException(Exception):
"""Thrown when something bad happened when we tried to run the
subprocess."""
pass
+_all_log_modes = ['debug', 'profile']
+_log_mode = os.environ.get('STG_SUBPROCESS_LOG', '')
+if _log_mode and not _log_mode in _all_log_modes:
+ out.warn(('Unknown log mode "%s" specified in $STG_SUBPROCESS_LOG.'
+ % _log_mode),
+ 'Valid values are: %s' % ', '.join(_all_log_modes))
+
class Run:
exc = RunException
def __init__(self, *cmd):
@@ -37,6 +48,18 @@ class Run:
self.__good_retvals = [0]
self.__env = None
self.__indata = None
+ def __log_start(self, cmd):
+ if _log_mode == 'debug':
+ out.start('Running subprocess %s' % cmd)
+ elif _log_mode == 'profile':
+ out.start('Running subprocess %s' % cmd[0])
+ self.__starttime = datetime.datetime.now()
+ def __log_end(self, retcode):
+ if _log_mode == 'debug':
+ out.done('return code: %d' % retcode)
+ elif _log_mode == 'profile':
+ duration = datetime.datetime.now() - self.__starttime
+ out.done('%1.3f s' % (duration.microseconds/1e6 + duration.seconds))
def __run_io(self, cmd):
"""Run with captured IO. Note: arguments are parsed by the
shell. We single-quote them, so don't use anything with single
@@ -47,6 +70,7 @@ class Run:
ecmd = (['env'] + ['%s=%s' % (key, val)
for key, val in self.__env.iteritems()]
+ cmd)
+ self.__log_start(ecmd)
p = popen2.Popen3(' '.join(["'%s'" % c for c in ecmd]), True)
if self.__indata != None:
p.tochild.write(self.__indata)
@@ -54,6 +78,7 @@ class Run:
outdata = p.fromchild.read()
errdata = p.childerr.read()
self.exitcode = p.wait() >> 8
+ self.__log_end(self.exitcode)
if errdata or self.exitcode not in self.__good_retvals:
raise self.exc('%s failed with code %d:\n%s'
% (cmd[0], self.exitcode, errdata))
@@ -63,7 +88,9 @@ class Run:
the shell."""
assert self.__env == None
assert self.__indata == None
+ self.__log_start(cmd)
self.exitcode = os.spawnvp(os.P_WAIT, cmd[0], cmd)
+ self.__log_end(self.exitcode)
if not self.exitcode in self.__good_retvals:
raise self.exc('%s failed with code %d'
% (cmd[0], self.exitcode))
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [StGIT PATCH 4/4] Add optional logging of subprocess execution
2007-08-26 20:33 ` [StGIT PATCH 4/4] Add optional logging of subprocess execution Karl Hasselström
@ 2007-08-29 10:50 ` Catalin Marinas
2007-08-29 11:11 ` Karl Hasselström
0 siblings, 1 reply; 14+ messages in thread
From: Catalin Marinas @ 2007-08-29 10:50 UTC (permalink / raw)
To: Karl Hasselström; +Cc: git
On 26/08/2007, Karl Hasselström <kha@treskal.com> wrote:
> Now that the subprocess calling has been refactored and is in a nice
> shape, it's quite simple to add some logging facilities. This patch
> adds two separate log modes, switched by the STG_SUBPROCESS_LOG
> environment variable:
Any objection to calling this variable STGIT_SUBPROCESS_LOG? We
already have STGIT_DEBUG_LEVEL (used in stgit.main). I can do it in my
tree before pushing as I already merged your branches.
BTW, thanks for refactoring the subprocess calling.
--
Catalin
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [StGIT PATCH 4/4] Add optional logging of subprocess execution
2007-08-29 10:50 ` Catalin Marinas
@ 2007-08-29 11:11 ` Karl Hasselström
2007-08-29 17:16 ` Karl Hasselström
2007-09-03 8:36 ` Catalin Marinas
0 siblings, 2 replies; 14+ messages in thread
From: Karl Hasselström @ 2007-08-29 11:11 UTC (permalink / raw)
To: Catalin Marinas; +Cc: git
On 2007-08-29 11:50:11 +0100, Catalin Marinas wrote:
> On 26/08/2007, Karl Hasselström <kha@treskal.com> wrote:
>
> > Now that the subprocess calling has been refactored and is in a
> > nice shape, it's quite simple to add some logging facilities. This
> > patch adds two separate log modes, switched by the
> > STG_SUBPROCESS_LOG environment variable:
>
> Any objection to calling this variable STGIT_SUBPROCESS_LOG? We
> already have STGIT_DEBUG_LEVEL (used in stgit.main). I can do it in
> my tree before pushing as I already merged your branches.
No, no objection at all. I was simply too lazy to actually check what
the existing naming convention was, and misremembered.
> BTW, thanks for refactoring the subprocess calling.
It needed doing, and was actually fun.
Any chance we can drop Python 2.3 support any time soon, by the way?
I've confined all the ickyness to one place, but it would still be
good to get rid of it (not to mention being able to use sets and
generator expressions).
--
Karl Hasselström, kha@treskal.com
www.treskal.com/kalle
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [StGIT PATCH 4/4] Add optional logging of subprocess execution
2007-08-29 11:11 ` Karl Hasselström
@ 2007-08-29 17:16 ` Karl Hasselström
2007-09-03 8:34 ` Catalin Marinas
2007-09-03 8:36 ` Catalin Marinas
1 sibling, 1 reply; 14+ messages in thread
From: Karl Hasselström @ 2007-08-29 17:16 UTC (permalink / raw)
To: Catalin Marinas; +Cc: git
On 2007-08-29 13:11:02 +0200, Karl Hasselström wrote:
> On 2007-08-29 11:50:11 +0100, Catalin Marinas wrote:
>
> > Any objection to calling this variable STGIT_SUBPROCESS_LOG? We
> > already have STGIT_DEBUG_LEVEL (used in stgit.main). I can do it
> > in my tree before pushing as I already merged your branches.
>
> No, no objection at all. I was simply too lazy to actually check
> what the existing naming convention was, and misremembered.
You changed it in the code, but forgot to change it in the commit
message. :-P
--
Karl Hasselström, kha@treskal.com
www.treskal.com/kalle
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [StGIT PATCH 4/4] Add optional logging of subprocess execution
2007-08-29 17:16 ` Karl Hasselström
@ 2007-09-03 8:34 ` Catalin Marinas
0 siblings, 0 replies; 14+ messages in thread
From: Catalin Marinas @ 2007-09-03 8:34 UTC (permalink / raw)
To: Karl Hasselström; +Cc: git
On 29/08/07, Karl Hasselström <kha@treskal.com> wrote:
> On 2007-08-29 13:11:02 +0200, Karl Hasselström wrote:
>
> > On 2007-08-29 11:50:11 +0100, Catalin Marinas wrote:
> >
> > > Any objection to calling this variable STGIT_SUBPROCESS_LOG? We
> > > already have STGIT_DEBUG_LEVEL (used in stgit.main). I can do it
> > > in my tree before pushing as I already merged your branches.
> >
> > No, no objection at all. I was simply too lazy to actually check
> > what the existing naming convention was, and misremembered.
>
> You changed it in the code, but forgot to change it in the commit
> message. :-P
I changed it on my office PC and the locale setup breaks your name
encoding :-). I tried to avoid it.
--
Catalin
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [StGIT PATCH 4/4] Add optional logging of subprocess execution
2007-08-29 11:11 ` Karl Hasselström
2007-08-29 17:16 ` Karl Hasselström
@ 2007-09-03 8:36 ` Catalin Marinas
2007-09-03 9:04 ` Karl Hasselström
2007-09-03 21:48 ` [StGit PATCH 0/2] Break Python 2.3 compatibility Karl Hasselström
1 sibling, 2 replies; 14+ messages in thread
From: Catalin Marinas @ 2007-09-03 8:36 UTC (permalink / raw)
To: Karl Hasselström; +Cc: git
On 29/08/07, Karl Hasselström <kha@treskal.com> wrote:
> Any chance we can drop Python 2.3 support any time soon, by the way?
> I've confined all the ickyness to one place, but it would still be
> good to get rid of it (not to mention being able to use sets and
> generator expressions).
Unless there is no objection, I'm actually OK with this. Feel free to
break the 2.3 support from now on. I'll add a note stating the minimum
versions for Python and GIT in the 0.14 release.
--
Catalin
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [StGIT PATCH 4/4] Add optional logging of subprocess execution
2007-09-03 8:36 ` Catalin Marinas
@ 2007-09-03 9:04 ` Karl Hasselström
2007-09-03 21:48 ` [StGit PATCH 0/2] Break Python 2.3 compatibility Karl Hasselström
1 sibling, 0 replies; 14+ messages in thread
From: Karl Hasselström @ 2007-09-03 9:04 UTC (permalink / raw)
To: Catalin Marinas; +Cc: git
On 2007-09-03 09:36:26 +0100, Catalin Marinas wrote:
> On 29/08/07, Karl Hasselström <kha@treskal.com> wrote:
>
> > Any chance we can drop Python 2.3 support any time soon, by the
> > way? I've confined all the ickyness to one place, but it would
> > still be good to get rid of it (not to mention being able to use
> > sets and generator expressions).
>
> Unless there is no objection, I'm actually OK with this. Feel free
> to break the 2.3 support from now on. I'll add a note stating the
> minimum versions for Python and GIT in the 0.14 release.
Great!
--
Karl Hasselström, kha@treskal.com
www.treskal.com/kalle
^ permalink raw reply [flat|nested] 14+ messages in thread
* [StGit PATCH 0/2] Break Python 2.3 compatibility
2007-09-03 8:36 ` Catalin Marinas
2007-09-03 9:04 ` Karl Hasselström
@ 2007-09-03 21:48 ` Karl Hasselström
2007-09-03 21:48 ` [StGit PATCH 1/2] Use subprocess.Popen to call git executables Karl Hasselström
2007-09-03 21:48 ` [StGit PATCH 2/2] Use the builtin set() instead of sets.Set() Karl Hasselström
1 sibling, 2 replies; 14+ messages in thread
From: Karl Hasselström @ 2007-09-03 21:48 UTC (permalink / raw)
To: Catalin Marinas; +Cc: git
On 2007-09-03 09:36:26 +0100, Catalin Marinas wrote:
> On 29/08/07, Karl Hasselström <kha@treskal.com> wrote:
>
> > Any chance we can drop Python 2.3 support any time soon, by the
> > way? I've confined all the ickyness to one place, but it would
> > still be good to get rid of it (not to mention being able to use
> > sets and generator expressions).
>
> Unless there is no objection, I'm actually OK with this. Feel free
> to break the 2.3 support from now on. I'll add a note stating the
> minimum versions for Python and GIT in the 0.14 release.
And here it is. We actually used sets in much fewer places than I
thought -- but then again, there are a number of places where we ought
to be using them but aren't.
--
Karl Hasselström, kha@treskal.com
www.treskal.com/kalle
^ permalink raw reply [flat|nested] 14+ messages in thread
* [StGit PATCH 1/2] Use subprocess.Popen to call git executables
2007-09-03 21:48 ` [StGit PATCH 0/2] Break Python 2.3 compatibility Karl Hasselström
@ 2007-09-03 21:48 ` Karl Hasselström
2007-09-03 21:48 ` [StGit PATCH 2/2] Use the builtin set() instead of sets.Set() Karl Hasselström
1 sibling, 0 replies; 14+ messages in thread
From: Karl Hasselström @ 2007-09-03 21:48 UTC (permalink / raw)
To: Catalin Marinas; +Cc: git
Replace popen2.Popen3 and os.spawn* with the superior
subprocess.Popen. Now we can pass a modified environment to any
subprocess, redirect input and output as we please, and the shell is
no longer involved, meaning we don't have to worry about argument
quoting. (We could already do all of that, just not at the same time.)
This is a Python 2.4 library, so StGit now officially requires Python
2.4 or later.
Signed-off-by: Karl Hasselström <kha@treskal.com>
---
stgit/run.py | 89 ++++++++++++++++++++++++++--------------------------------
1 files changed, 40 insertions(+), 49 deletions(-)
diff --git a/stgit/run.py b/stgit/run.py
index 94dd98e..29f8f71 100644
--- a/stgit/run.py
+++ b/stgit/run.py
@@ -17,12 +17,7 @@ along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
"""
-# popen2 and os.spawn* suck. We should really use subprocess instead,
-# but that's only available in Python 2.4 and up, and we try our best
-# to stay Python 2.3 compatible.
-import popen2, os
-
-import datetime
+import datetime, os, subprocess
from stgit.out import *
@@ -48,11 +43,11 @@ class Run:
self.__good_retvals = [0]
self.__env = None
self.__indata = None
- def __log_start(self, cmd):
+ def __log_start(self):
if _log_mode == 'debug':
- out.start('Running subprocess %s' % cmd)
+ out.start('Running subprocess %s' % self.__cmd)
elif _log_mode == 'profile':
- out.start('Running subprocess %s' % cmd[0])
+ out.start('Running subprocess %s' % self.__cmd[0])
self.__starttime = datetime.datetime.now()
def __log_end(self, retcode):
if _log_mode == 'debug':
@@ -60,48 +55,41 @@ class Run:
elif _log_mode == 'profile':
duration = datetime.datetime.now() - self.__starttime
out.done('%1.3f s' % (duration.microseconds/1e6 + duration.seconds))
- def __run_io(self, cmd):
- """Run with captured IO. Note: arguments are parsed by the
- shell. We single-quote them, so don't use anything with single
- quotes in it."""
- if self.__env == None:
- ecmd = cmd
- else:
- ecmd = (['env'] + ['%s=%s' % (key, val)
- for key, val in self.__env.iteritems()]
- + cmd)
- self.__log_start(ecmd)
- p = popen2.Popen3(' '.join(["'%s'" % c for c in ecmd]), True)
- if self.__indata != None:
- p.tochild.write(self.__indata)
- p.tochild.close()
- outdata = p.fromchild.read()
- errdata = p.childerr.read()
- self.exitcode = p.wait() >> 8
- self.__log_end(self.exitcode)
+ def __check_exitcode(self):
if self.exitcode not in self.__good_retvals:
- raise self.exc('%s failed with code %d:\n%s'
- % (cmd[0], self.exitcode, errdata))
- if errdata:
- out.warn('call to %s succeeded, but generated a warning:' % cmd[0])
- out.err_raw(errdata)
+ raise self.exc('%s failed with code %d'
+ % (self.__cmd[0], self.exitcode))
+ def __run_io(self):
+ """Run with captured IO."""
+ self.__log_start()
+ try:
+ p = subprocess.Popen(self.__cmd, env = self.__env,
+ stdin = subprocess.PIPE,
+ stdout = subprocess.PIPE)
+ outdata, errdata = p.communicate(self.__indata)
+ self.exitcode = p.returncode
+ except OSError, e:
+ raise self.exc('%s failed: %s' % (self.__cmd[0], e))
+ self.__log_end(self.exitcode)
+ self.__check_exitcode()
return outdata
- def __run_noshell(self, cmd):
- """Run without captured IO. Note: arguments are not parsed by
- the shell."""
- assert self.__env == None
+ def __run_noio(self):
+ """Run without captured IO."""
assert self.__indata == None
- self.__log_start(cmd)
- self.exitcode = os.spawnvp(os.P_WAIT, cmd[0], cmd)
+ self.__log_start()
+ try:
+ p = subprocess.Popen(self.__cmd, env = self.__env)
+ self.exitcode = p.wait()
+ except OSError, e:
+ raise self.exc('%s failed: %s' % (self.__cmd[0], e))
self.__log_end(self.exitcode)
- if not self.exitcode in self.__good_retvals:
- raise self.exc('%s failed with code %d'
- % (cmd[0], self.exitcode))
+ self.__check_exitcode()
def returns(self, retvals):
self.__good_retvals = retvals
return self
def env(self, env):
- self.__env = env
+ self.__env = dict(os.environ)
+ self.__env.update(env)
return self
def raw_input(self, indata):
self.__indata = indata
@@ -110,15 +98,15 @@ class Run:
self.__indata = ''.join(['%s\n' % line for line in lines])
return self
def no_output(self):
- outdata = self.__run_io(self.__cmd)
+ outdata = self.__run_io()
if outdata:
raise self.exc, '%s produced output' % self.__cmd[0]
def discard_output(self):
- self.__run_io(self.__cmd)
+ self.__run_io()
def raw_output(self):
- return self.__run_io(self.__cmd)
+ return self.__run_io()
def output_lines(self):
- outdata = self.__run_io(self.__cmd)
+ outdata = self.__run_io()
if outdata.endswith('\n'):
outdata = outdata[:-1]
if outdata:
@@ -134,11 +122,14 @@ class Run:
% (self.__cmd[0], len(outlines)))
def run(self):
"""Just run, with no IO redirection."""
- self.__run_noshell(self.__cmd)
+ self.__run_noio()
def xargs(self, xargs):
"""Just run, with no IO redirection. The extra arguments are
appended to the command line a few at a time; the command is
run as many times as needed to consume them all."""
step = 100
+ basecmd = self.__cmd
for i in xrange(0, len(xargs), step):
- self.__run_noshell(self.__cmd + xargs[i:i+step])
+ self.__cmd = basecmd + xargs[i:i+step]
+ self.__run_noio()
+ self.__cmd = basecmd
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [StGit PATCH 2/2] Use the builtin set() instead of sets.Set()
2007-09-03 21:48 ` [StGit PATCH 0/2] Break Python 2.3 compatibility Karl Hasselström
2007-09-03 21:48 ` [StGit PATCH 1/2] Use subprocess.Popen to call git executables Karl Hasselström
@ 2007-09-03 21:48 ` Karl Hasselström
1 sibling, 0 replies; 14+ messages in thread
From: Karl Hasselström @ 2007-09-03 21:48 UTC (permalink / raw)
To: Catalin Marinas; +Cc: git
We can do that now that we're guaranteed to have Python 2.4 or later.
Signed-off-by: Karl Hasselström <kha@treskal.com>
---
stgit/git.py | 8 +++-----
1 files changed, 3 insertions(+), 5 deletions(-)
diff --git a/stgit/git.py b/stgit/git.py
index 8857209..4b4c626 100644
--- a/stgit/git.py
+++ b/stgit/git.py
@@ -26,7 +26,6 @@ from stgit.utils import *
from stgit.out import *
from stgit.run import *
from stgit.config import config
-from sets import Set
# git exception class
class GitException(Exception):
@@ -959,10 +958,9 @@ def __remotes_from_dir(dir):
def remotes_list():
"""Return the list of remotes in the repository
"""
-
- return Set(__remotes_from_config()) | \
- Set(__remotes_from_dir('remotes')) | \
- Set(__remotes_from_dir('branches'))
+ return (set(__remotes_from_config())
+ | set(__remotes_from_dir('remotes'))
+ | set(__remotes_from_dir('branches')))
def remotes_local_branches(remote):
"""Returns the list of local branches fetched from given remote
^ permalink raw reply related [flat|nested] 14+ messages in thread
end of thread, other threads:[~2007-09-03 21:49 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-08-26 20:33 [StGIT PATCH 0/4] Clean up subprocess calling Karl Hasselström
2007-08-26 20:33 ` [StGIT PATCH 1/4] Refactor output handling to break circular dependency Karl Hasselström
2007-08-26 20:33 ` [StGIT PATCH 2/4] Refactor subprocess creation Karl Hasselström
2007-08-26 20:33 ` [StGIT PATCH 3/4] Assert that the argument to Run is a sequence of strings Karl Hasselström
2007-08-26 20:33 ` [StGIT PATCH 4/4] Add optional logging of subprocess execution Karl Hasselström
2007-08-29 10:50 ` Catalin Marinas
2007-08-29 11:11 ` Karl Hasselström
2007-08-29 17:16 ` Karl Hasselström
2007-09-03 8:34 ` Catalin Marinas
2007-09-03 8:36 ` Catalin Marinas
2007-09-03 9:04 ` Karl Hasselström
2007-09-03 21:48 ` [StGit PATCH 0/2] Break Python 2.3 compatibility Karl Hasselström
2007-09-03 21:48 ` [StGit PATCH 1/2] Use subprocess.Popen to call git executables Karl Hasselström
2007-09-03 21:48 ` [StGit PATCH 2/2] Use the builtin set() instead of sets.Set() Karl Hasselström
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).