[PATCH V3 0/4] git-remote-mediawiki: new tool to preview local changes without pushing

[PATCH V3 0/4] git-remote-mediawiki: new tool to preview local changes without pushing

[benoit.person]

From: Benoit Person <benoit.person@ensimag.fr>

The #7 issue on git-mediawiki's issue tracker [1] states that the ability to
preview content without pushing would be a nice thing to have.

changes from V2:
  - Add a way to test, without installation, code that uses GitMediawiki.pm.
  - Move more constants to GitMediawiki.pm
  - Remove the encapsulation of Git::config calls into a git_cmd_try one.
  - Remove the --blob option, distinction between files and blobs is now 
    automatic.
  - Add a --verbose option to output more information on what's going on.
  - Rewrote the doc and the commit message.
  - Rewrote of the template retrieving code (see 'get_template' sub).
  - Use a configuration variable to define the content ID search in the
    template. Default value set as 'bodyContent' since it seems more standard
    than 'mw-content-text'.
  - Final content is now saved as utf-8 to solve encoding issues.
  - Perlcritic changes: 
    - Update for loops style to a more perlish one.
    - All 'print's specify their output streams.
    --> Same useless warnings left in git-remote-mediawiki.perl after célestin's 
        work and git-mw.perl after this patch :) .

changes from V1:
  - add new package GitMediawiki
    - move some of git-remote-mediawiki functions into the package
    - update git-remote-mediawiki to use those "moved" functions
    - add a hacky-way to install it in the Makefile
    - use it in the new git mw tool
  - add a way to give to the preview tool blobs as argument
  - add a fallback when the upstream's branch remote is not a mediawiki remote
  - update the `autoload` option to use `git web--browse` and not `xdg-open`
  - update the way we find the upstream's branch remote name

This serie is based on the 'master' branch merged with célestin's patch series.

[1] https://github.com/moy/Git-Mediawiki/issues/7

Benoit Person (4):
  git-mw: Introduction of GitMediawiki.pm
  git-mw: Move some functions from git-remote-mediawiki.perl to
    GitMediawiki.pm
  git-mw: Adding git-mw command
  git-mw: Add preview subcommand into git mw.

 contrib/mw-to-git/GitMediawiki.pm           | 100 ++++++++
 contrib/mw-to-git/Makefile                  |  29 ++-
 contrib/mw-to-git/git-mw.perl               | 347 ++++++++++++++++++++++++++++
 contrib/mw-to-git/git-remote-mediawiki.perl |  85 ++-----
 4 files changed, 485 insertions(+), 76 deletions(-)
 create mode 100644 contrib/mw-to-git/GitMediawiki.pm
 create mode 100644 contrib/mw-to-git/git-mw.perl

-- 
1.8.3.GIT

[PATCH V3 1/4] git-mw: Introduction of GitMediawiki.pm

[benoit.person]

From: Benoit Person <benoit.person@ensimag.fr>

GitMediawiki.pm enable code factoring between several scripts in mw-to-git.

To make it usable in scripts (ie: accessible in @INC) it adds two targets
(copy_pm & install_pm) into the Makefile, one for tests and one for
installation.

Signed-off-by: Benoit Person <benoit.person@ensimag.fr>
Signed-off-by: Matthieu Moy <matthieu.moy@grenoble-inp.fr>

---

changes from the V2:
  - Add a way to test, without installation, code that uses GitMediawiki.pm.

 contrib/mw-to-git/GitMediawiki.pm | 24 ++++++++++++++++++++++++
 contrib/mw-to-git/Makefile        | 26 +++++++++++++++++++++++---
 2 files changed, 47 insertions(+), 3 deletions(-)
 create mode 100644 contrib/mw-to-git/GitMediawiki.pm

diff --git a/contrib/mw-to-git/GitMediawiki.pm b/contrib/mw-to-git/GitMediawiki.pm
new file mode 100644
index 0000000..8a0ffc7
--- /dev/null
+++ b/contrib/mw-to-git/GitMediawiki.pm
@@ -0,0 +1,24 @@
+package GitMediawiki;
+
+use 5.008;
+use strict;
+use Git;
+
+BEGIN {
+
+our ($VERSION, @ISA, @EXPORT, @EXPORT_OK);
+
+# Totally unstable API.
+$VERSION = '0.01';
+
+require Exporter;
+
+@ISA = qw(Exporter);
+
+@EXPORT = ();
+
+# Methods which can be called as standalone functions as well:
+@EXPORT_OK = ();
+}
+
+1; # Famous last words
\ No newline at end of file
diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile
index 1fb2424..b0c7cf2 100644
--- a/contrib/mw-to-git/Makefile
+++ b/contrib/mw-to-git/Makefile
@@ -4,16 +4,36 @@
 #
 ## Build git-remote-mediawiki
 
+GIT_MEDIAWIKI_PM=GitMediawiki.pm
 SCRIPT_PERL=git-remote-mediawiki.perl
 GIT_ROOT_DIR=../..
 HERE=contrib/mw-to-git/
 
 SCRIPT_PERL_FULL=$(patsubst %,$(HERE)/%,$(SCRIPT_PERL))
+INSTLIBDIR=$(shell $(MAKE) -C $(GIT_ROOT_DIR)/perl \
+                -s --no-print-directory instlibdir)
 
 all: build
 
-build install clean:
+copy_pm:
+	cp $(GIT_MEDIAWIKI_PM) $(GIT_ROOT_DIR)/perl/blib/lib/
+
+install_pm:
+	cp $(GIT_MEDIAWIKI_PM) $(INSTLIBDIR)
+
+build: copy_pm
+	$(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \
+                build-perl-script
+
+install: install_pm
 	$(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \
-                $@-perl-script
+                install-perl-script
+
+clean:
+	$(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \
+                clean-perl-script
+	rm $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) \
+       $(GIT_ROOT_DIR)/perl/blib/lib/$(GIT_MEDIAWIKI_PM)
+
 perlcritic:
-	perlcritic -2 *.perl
+	perlcritic -2 *.perl
\ No newline at end of file
-- 
1.8.3.GIT

[PATCH V3 2/4] git-mw: Move some functions from git-remote-mediawiki.perl to GitMediawiki.pm

[benoit.person]

From: Benoit Person <benoit.person@ensimag.fr>

Move and rename subs 'mediawiki_clean_filename' into 'clean_filename',
'mediawiki_smudge_filename' into 'smudge_filename' and
'mw_connect_maybe' into 'connect_maybe' since we have a cleaner namespace in
a perl module.
Move constants 'EMPTY', 'SLASH_REPLACEMENT' and 'HTTP_CODE_OK'.
Remove side effects in those functions to provide a cleaner API.
Update git-remote-mediawiki for those changes.

Signed-off-by: Benoit Person <benoit.person@ensimag.fr>
Signed-off-by: Matthieu Moy <matthieu.moy@grenoble-inp.fr>

---

changes from the V2:
  - Move more constants to GitMediawiki.pm.
  - Remove the encapsulation of Git::config calls into a git_cmd_try one.

 contrib/mw-to-git/GitMediawiki.pm           | 77 +++++++++++++++++++++++++-
 contrib/mw-to-git/git-remote-mediawiki.perl | 85 +++++------------------------
 2 files changed, 89 insertions(+), 73 deletions(-)

diff --git a/contrib/mw-to-git/GitMediawiki.pm b/contrib/mw-to-git/GitMediawiki.pm
index 8a0ffc7..beae6d0 100644
--- a/contrib/mw-to-git/GitMediawiki.pm
+++ b/contrib/mw-to-git/GitMediawiki.pm
@@ -18,7 +18,82 @@ require Exporter;
 @EXPORT = ();
 
 # Methods which can be called as standalone functions as well:
-@EXPORT_OK = ();
+@EXPORT_OK = qw(clean_filename smudge_filename connect_maybe
+				EMPTY HTTP_CODE_OK);
+}
+
+# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced
+use constant SLASH_REPLACEMENT => '%2F';
+
+# Used to test for empty strings
+use constant EMPTY => q{};
+
+# HTTP codes
+use constant HTTP_CODE_OK => 200;
+
+sub clean_filename {
+	my $filename = shift;
+	$filename =~ s{@{[SLASH_REPLACEMENT]}}{/}g;
+	# [, ], |, {, and } are forbidden by MediaWiki, even URL-encoded.
+	# Do a variant of URL-encoding, i.e. looks like URL-encoding,
+	# but with _ added to prevent MediaWiki from thinking this is
+	# an actual special character.
+	$filename =~ s/[\[\]\{\}\|]/sprintf("_%%_%x", ord($&))/ge;
+	# If we use the uri escape before
+	# we should unescape here, before anything
+
+	return $filename;
+}
+
+sub smudge_filename {
+	my $filename = shift;
+	$filename =~ s{/}{@{[SLASH_REPLACEMENT]}}g;
+	$filename =~ s/ /_/g;
+	# Decode forbidden characters encoded in clean_filename
+	$filename =~ s/_%_([0-9a-fA-F][0-9a-fA-F])/sprintf('%c', hex($1))/ge;
+	return $filename;
+}
+
+sub connect_maybe {
+	my $wiki = shift;
+	if ($wiki) {
+		return $wiki;
+	}
+
+	my $remote_name = shift;
+	my $remote_url = shift;
+	my ($wiki_login, $wiki_password, $wiki_domain);
+
+	$wiki_login = Git::config("remote.${remote_name}.mwLogin");
+	$wiki_password = Git::config("remote.${remote_name}.mwPassword");
+	$wiki_domain = Git::config("remote.${remote_name}.mwDomain");
+
+	$wiki = MediaWiki::API->new;
+	$wiki->{config}->{api_url} = "${remote_url}/api.php";
+	if ($wiki_login) {
+		my %credential = (
+			'url' => $remote_url,
+			'username' => $wiki_login,
+			'password' => $wiki_password
+		);
+		Git::credential(\%credential);
+		my $request = {lgname => $credential{username},
+			       lgpassword => $credential{password},
+			       lgdomain => $wiki_domain};
+		if ($wiki->login($request)) {
+			Git::credential(\%credential, 'approve');
+			print {*STDERR} qq(Logged in mediawiki user "$credential{username}".\n);
+		} else {
+			print {*STDERR} qq(Failed to log in mediawiki user "$credential{username}" on ${remote_url}\n);
+			print {*STDERR} '  (error ' .
+				$wiki->{error}->{code} . ': ' .
+				$wiki->{error}->{details} . ")\n";
+			Git::credential(\%credential, 'reject');
+			exit 1;
+		}
+	}
+
+	return $wiki;
 }
 
 1; # Famous last words
\ No newline at end of file
diff --git a/contrib/mw-to-git/git-remote-mediawiki.perl b/contrib/mw-to-git/git-remote-mediawiki.perl
index 9ff45fd..3b6422b 100755
--- a/contrib/mw-to-git/git-remote-mediawiki.perl
+++ b/contrib/mw-to-git/git-remote-mediawiki.perl
@@ -14,6 +14,8 @@
 use strict;
 use MediaWiki::API;
 use Git;
+use GitMediawiki qw(clean_filename smudge_filename connect_maybe
+					EMPTY HTTP_CODE_OK);
 use DateTime::Format::ISO8601;
 use warnings;
 
@@ -23,9 +25,6 @@ binmode STDOUT, ':encoding(UTF-8)';
 
 use URI::Escape;
 
-# Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced
-use constant SLASH_REPLACEMENT => '%2F';
-
 # It's not always possible to delete pages (may require some
 # privileges). Deleted pages are replaced with this content.
 use constant DELETED_CONTENT => "[[Category:Deleted]]\n";
@@ -40,8 +39,6 @@ use constant NULL_SHA1 => '0000000000000000000000000000000000000000';
 # Used on Git's side to reflect empty edit messages on the wiki
 use constant EMPTY_MESSAGE => '*Empty MediaWiki Message*';
 
-use constant EMPTY => q{};
-
 # Number of pages taken into account at once in submodule get_mw_page_list
 use constant SLICE_SIZE => 50;
 
@@ -50,8 +47,6 @@ use constant SLICE_SIZE => 50;
 # the number of links to be returned (500 links max).
 use constant BATCH_SIZE => 10;
 
-use constant HTTP_CODE_OK => 200;
-
 my $remotename = $ARGV[0];
 my $url = $ARGV[1];
 
@@ -184,37 +179,6 @@ sub parse_command {
 # MediaWiki API instance, created lazily.
 my $mediawiki;
 
-sub mw_connect_maybe {
-	if ($mediawiki) {
-		return;
-	}
-	$mediawiki = MediaWiki::API->new;
-	$mediawiki->{config}->{api_url} = "${url}/api.php";
-	if ($wiki_login) {
-		my %credential = (
-			'url' => $url,
-			'username' => $wiki_login,
-			'password' => $wiki_passwd
-		);
-		Git::credential(\%credential);
-		my $request = {lgname => $credential{username},
-			       lgpassword => $credential{password},
-			       lgdomain => $wiki_domain};
-		if ($mediawiki->login($request)) {
-			Git::credential(\%credential, 'approve');
-			print {*STDERR} qq(Logged in mediawiki user "$credential{username}".\n);
-		} else {
-			print {*STDERR} qq(Failed to log in mediawiki user "$credential{username}" on ${url}\n);
-			print {*STDERR} '  (error ' .
-				$mediawiki->{error}->{code} . ': ' .
-				$mediawiki->{error}->{details} . ")\n";
-			Git::credential(\%credential, 'reject');
-			exit 1;
-		}
-	}
-	return;
-}
-
 sub fatal_mw_error {
 	my $action = shift;
 	print STDERR "fatal: could not $action.\n";
@@ -324,7 +288,7 @@ sub get_mw_first_pages {
 
 # Get the list of pages to be fetched according to configuration.
 sub get_mw_pages {
-	mw_connect_maybe();
+	$mediawiki = connect_maybe($mediawiki, $remotename, $url);
 
 	print {*STDERR} "Listing pages on remote wiki...\n";
 
@@ -514,7 +478,7 @@ sub get_last_local_revision {
 # avoid a loop onto all tracked pages. This is useful for the fetch-by-rev
 # option.
 sub get_last_global_remote_rev {
-	mw_connect_maybe();
+	$mediawiki = connect_maybe($mediawiki, $remotename, $url);
 
 	my $query = {
 		action => 'query',
@@ -530,7 +494,7 @@ sub get_last_global_remote_rev {
 # Get the last remote revision concerning the tracked pages and the tracked
 # categories.
 sub get_last_remote_revision {
-	mw_connect_maybe();
+	$mediawiki = connect_maybe($mediawiki, $remotename, $url);
 
 	my %pages_hash = get_mw_pages();
 	my @pages = values(%pages_hash);
@@ -586,29 +550,6 @@ sub mediawiki_smudge {
 	return "${string}\n";
 }
 
-sub mediawiki_clean_filename {
-	my $filename = shift;
-	$filename =~ s{@{[SLASH_REPLACEMENT]}}{/}g;
-	# [, ], |, {, and } are forbidden by MediaWiki, even URL-encoded.
-	# Do a variant of URL-encoding, i.e. looks like URL-encoding,
-	# but with _ added to prevent MediaWiki from thinking this is
-	# an actual special character.
-	$filename =~ s/[\[\]\{\}\|]/sprintf("_%%_%x", ord($&))/ge;
-	# If we use the uri escape before
-	# we should unescape here, before anything
-
-	return $filename;
-}
-
-sub mediawiki_smudge_filename {
-	my $filename = shift;
-	$filename =~ s{/}{@{[SLASH_REPLACEMENT]}}g;
-	$filename =~ s/ /_/g;
-	# Decode forbidden characters encoded in mediawiki_clean_filename
-	$filename =~ s/_%_([0-9a-fA-F][0-9a-fA-F])/sprintf('%c', hex($1))/ge;
-	return $filename;
-}
-
 sub literal_data {
 	my ($content) = @_;
 	print {*STDOUT} 'data ', bytes::length($content), "\n", $content;
@@ -816,7 +757,7 @@ sub mw_import_ref {
 		return;
 	}
 
-	mw_connect_maybe();
+	$mediawiki = connect_maybe($mediawiki, $remotename, $url);
 
 	print {*STDERR} "Searching revisions...\n";
 	my $last_local = get_last_local_revision();
@@ -930,7 +871,7 @@ sub mw_import_revids {
 		my %commit;
 		$commit{author} = $rev->{user} || 'Anonymous';
 		$commit{comment} = $rev->{comment} || EMPTY_MESSAGE;
-		$commit{title} = mediawiki_smudge_filename($page_title);
+		$commit{title} = smudge_filename($page_title);
 		$commit{mw_revision} = $rev->{revid};
 		$commit{content} = mediawiki_smudge($rev->{'*'});
 
@@ -991,7 +932,7 @@ sub mw_upload_file {
 	}
 	# Deleting and uploading a file requires a priviledged user
 	if ($file_deleted) {
-		mw_connect_maybe();
+		$mediawiki = connect_maybe($mediawiki, $remotename, $url);
 		my $query = {
 			action => 'delete',
 			title => $path,
@@ -1007,7 +948,7 @@ sub mw_upload_file {
 		# Don't let perl try to interpret file content as UTF-8 => use "raw"
 		my $content = run_git("cat-file blob ${new_sha1}", 'raw');
 		if ($content ne EMPTY) {
-			mw_connect_maybe();
+			$mediawiki = connect_maybe($mediawiki, $remotename, $url);
 			$mediawiki->{config}->{upload_url} =
 				"${url}/index.php/Special:Upload";
 			$mediawiki->edit({
@@ -1055,7 +996,7 @@ sub mw_push_file {
 	my $old_sha1 = $diff_info_split[2];
 	my $page_created = ($old_sha1 eq NULL_SHA1);
 	my $page_deleted = ($new_sha1 eq NULL_SHA1);
-	$complete_file_name = mediawiki_clean_filename($complete_file_name);
+	$complete_file_name = clean_filename($complete_file_name);
 
 	my ($title, $extension) = $complete_file_name =~ /^(.*)\.([^\.]*)$/;
 	if (!defined($extension)) {
@@ -1078,7 +1019,7 @@ sub mw_push_file {
 			$file_content = run_git("cat-file blob ${new_sha1}");
 		}
 
-		mw_connect_maybe();
+		$mediawiki = connect_maybe($mediawiki, $remotename, $url);
 
 		my $result = $mediawiki->edit( {
 			action => 'edit',
@@ -1264,7 +1205,7 @@ sub mw_push_revision {
 }
 
 sub get_allowed_file_extensions {
-	mw_connect_maybe();
+	$mediawiki = connect_maybe($mediawiki, $remotename, $url);
 
 	my $query = {
 		action => 'query',
@@ -1288,7 +1229,7 @@ my %cached_mw_namespace_id;
 # Return MediaWiki id for a canonical namespace name.
 # Ex.: "File", "Project".
 sub get_mw_namespace_id {
-	mw_connect_maybe();
+	$mediawiki = connect_maybe($mediawiki, $remotename, $url);
 	my $name = shift;
 
 	if (!exists $namespace_id{$name}) {
-- 
1.8.3.GIT

[PATCH V3 3/4] git-mw: Adding git-mw command

[benoit.person]

From: Benoit Person <benoit.person@ensimag.fr>

Add the new git-mw command, with its 'help' subcommand as an example. Argument
parsing and subcommand choice is based on git-svn.perl.
Update Makefile to build, install and clean both scripts now (git-mw.perl and
git-remote-mediawiki.perl).

Signed-off-by: Benoit Person <benoit.person@ensimag.fr>
Signed-off-by: Matthieu Moy <matthieu.moy@grenoble-inp.fr>

---

changes from V2:
  - Perlcritic changes: 
    - Update for loops style to a more perlish one.
    - All 'print's specify their output streams.

 contrib/mw-to-git/Makefile    |  7 ++++---
 contrib/mw-to-git/git-mw.perl | 46 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+), 3 deletions(-)
 create mode 100644 contrib/mw-to-git/git-mw.perl

diff --git a/contrib/mw-to-git/Makefile b/contrib/mw-to-git/Makefile
index b0c7cf2..fe02c7e 100644
--- a/contrib/mw-to-git/Makefile
+++ b/contrib/mw-to-git/Makefile
@@ -6,6 +6,7 @@
 
 GIT_MEDIAWIKI_PM=GitMediawiki.pm
 SCRIPT_PERL=git-remote-mediawiki.perl
+SCRIPT_PERL+=git-mw.perl
 GIT_ROOT_DIR=../..
 HERE=contrib/mw-to-git/
 
@@ -22,15 +23,15 @@ install_pm:
 	cp $(GIT_MEDIAWIKI_PM) $(INSTLIBDIR)
 
 build: copy_pm
-	$(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \
+	$(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \
                 build-perl-script
 
 install: install_pm
-	$(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \
+	$(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \
                 install-perl-script
 
 clean:
-	$(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL=$(SCRIPT_PERL_FULL) \
+	$(MAKE) -C $(GIT_ROOT_DIR) SCRIPT_PERL="$(SCRIPT_PERL_FULL)" \
                 clean-perl-script
 	rm $(INSTLIBDIR)/$(GIT_MEDIAWIKI_PM) \
        $(GIT_ROOT_DIR)/perl/blib/lib/$(GIT_MEDIAWIKI_PM)
diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl
new file mode 100644
index 0000000..320c00e
--- /dev/null
+++ b/contrib/mw-to-git/git-mw.perl
@@ -0,0 +1,46 @@
+#!/usr/bin/perl
+
+# Copyright (C) 2013
+#     Benoit Person <benoit.person@ensimag.imag.fr>
+#     Celestin Matte <celestin.matte@ensimag.imag.fr>
+# License: GPL v2 or later
+
+# Set of tools for git repo with a mediawiki remote.
+# Documentation & bugtracker: https://github.com/moy/Git-Mediawiki/
+
+use strict;
+use warnings;
+
+use Getopt::Long;
+
+my %commands = (
+	'help' =>
+		[\&help, {}, \&help]
+);
+
+# Search for sub-command
+my $cmd = $commands{'help'};
+for (0..@ARGV) {
+	if (defined $commands{$ARGV[$_]}) {
+		$cmd = $commands{$ARGV[$_]};
+		splice @ARGV, $_, 1;
+		last;
+	}
+};
+GetOptions( %{$cmd->[1]},
+	'help|h' => \&{$cmd->[2]});
+
+# Launch command
+&{$cmd->[0]};
+
+############################## Help Functions ##################################
+
+sub help {
+	print {*STDOUT} <<'END';
+usage: git mw <command> <args>
+
+git mw commands are:
+    help        Display help information about git mw
+END
+	exit;
+}
\ No newline at end of file
-- 
1.8.3.GIT

[PATCH V3 4/4] git-mw: Add preview subcommand into git mw.

[benoit.person]

From: Benoit Person <benoit.person@ensimag.fr>

Add the subcommand to 'git-mw.perl'.
Add a new constant in GitMediawiki.pm 'HTTP_CODE_PAGE_NOT_FOUND'.

Signed-off-by: Benoit Person <benoit.person@ensimag.fr>
Signed-off-by: Matthieu Moy <matthieu.moy@grenoble-inp.fr>

---

changes from V2:
  - Remove the --blob option, distinction between files and blobs is now 
    automatic.
  - Add a --verbose option to output more information on what's going on.
  - Rewrote the doc and the commit message.
  - Rewrote of the template retrieving code (see 'get_template' sub).
  - Use a configuration variable to define the content ID search in the
    template. Default value set as 'bodyContent' since it seems more standard
    than 'mw-content-text'.
  - Final content is now saved as utf-8 to solve encoding issues.
  - Perlcritic changes:
    - All 'print's specify their output streams.
    --> Same useless warnings left in git-remote-mediawiki.perl after célestin's 
        work and git-mw.perl after this patch :) .

 contrib/mw-to-git/GitMediawiki.pm |   3 +-
 contrib/mw-to-git/git-mw.perl     | 303 +++++++++++++++++++++++++++++++++++++-
 2 files changed, 304 insertions(+), 2 deletions(-)

diff --git a/contrib/mw-to-git/GitMediawiki.pm b/contrib/mw-to-git/GitMediawiki.pm
index beae6d0..d1f2c41 100644
--- a/contrib/mw-to-git/GitMediawiki.pm
+++ b/contrib/mw-to-git/GitMediawiki.pm
@@ -19,7 +19,7 @@ require Exporter;
 
 # Methods which can be called as standalone functions as well:
 @EXPORT_OK = qw(clean_filename smudge_filename connect_maybe
-				EMPTY HTTP_CODE_OK);
+				EMPTY HTTP_CODE_OK HTTP_CODE_PAGE_NOT_FOUND);
 }
 
 # Mediawiki filenames can contain forward slashes. This variable decides by which pattern they should be replaced
@@ -30,6 +30,7 @@ use constant EMPTY => q{};
 
 # HTTP codes
 use constant HTTP_CODE_OK => 200;
+use constant HTTP_CODE_PAGE_NOT_FOUND => 404;
 
 sub clean_filename {
 	my $filename = shift;
diff --git a/contrib/mw-to-git/git-mw.perl b/contrib/mw-to-git/git-mw.perl
index 320c00e..0b83108 100644
--- a/contrib/mw-to-git/git-mw.perl
+++ b/contrib/mw-to-git/git-mw.perl
@@ -12,10 +12,41 @@ use strict;
 use warnings;
 
 use Getopt::Long;
+use URI::URL qw(url);
+use LWP::UserAgent;
+use HTML::TreeBuilder;
+
+use Git;
+use MediaWiki::API;
+use GitMediawiki qw(smudge_filename connect_maybe
+					EMPTY HTTP_CODE_PAGE_NOT_FOUND);
+
+# By default, use UTF-8 to communicate with Git and the user
+binmode STDERR, ':encoding(UTF-8)';
+binmode STDOUT, ':encoding(UTF-8)';
+
+#preview parameters
+my $file_name = EMPTY;
+my $remote_name = EMPTY;
+my $preview_file_name = EMPTY;
+my $autoload = 0;
+my $verbose = 0;
+sub file {
+	$file_name = shift;
+	return $file_name;
+}
 
 my %commands = (
 	'help' =>
-		[\&help, {}, \&help]
+		[\&help, {}, \&help],
+	'preview' =>
+		[\&preview, {
+			'<>' => \&file,
+			'output|o=s' => \$preview_file_name,
+			'remote|r=s' => \$remote_name,
+			'autoload|a' => \$autoload,
+			'verbose|v'  => \$verbose
+		}, \&preview_help]
 );
 
 # Search for sub-command
@@ -33,6 +64,275 @@ GetOptions( %{$cmd->[1]},
 # Launch command
 &{$cmd->[0]};
 
+############################# Preview Functions ################################
+
+# @TODO : add documentation for verbose option
+sub preview_help {
+	print {*STDOUT} <<'END';
+USAGE: git mw preview [--remote|-r <remote name>] [--autoload|-a]
+                      [--output|-o <output filename>] [--verbose|-v]
+                      <blob> | <filename>
+
+DESCRIPTION:
+Preview is an utiliy to preview local content of a mediawiki repo as if it was
+pushed on the remote.
+
+For that, preview searches for the remote name of the current branch's upstream
+if --remote is not set. If that remote is not found or if it is not a mediawiki,
+it lists all mediawiki remotes configured and asks you to replay your command
+with the --remote option set properly.
+
+Then, it searches for a file named 'filename'. If it's not found in the current
+dir, it will assume it's a blob.
+
+The content retrieved in the file (or in the blob) will then be parsed by the
+distant mediawiki and combined with a template retrieved from the mediawiki.
+
+Finally, preview will save the HTML result in a file. and autoload it in your
+default web browser if the option --autoload is present.
+
+OPTIONS:
+	-r <remote name>, --remote <remote name>
+		If the remote is a mediawiki, the template and the parse engine used for
+		the preview will be those of that remote.
+		If not, a list of valid remotes will be shown.
+
+	-a, --autoload
+		Try to load the HTML output in a new tab (or new window) of your default
+		web browser.
+
+	-o <output filename>, --output <output filename>
+		Change the HTML output filename. Default filename is based on the input
+		filename with its extension replaced by '.html'.
+
+	-v, --verbose
+		Show more information on what's going on under the hood.
+END
+	exit;
+}
+
+sub preview {
+	my $wiki;
+	my ($remote_url, $wiki_page_name);
+	my ($content, $content_tree, $template, $html_tree, $mw_content_text);
+	my $file_content;
+	my $template_content_id = 'bodyContent';
+
+	# file_name argumeent is mandatory
+	if (!defined $file_name) {
+		die "File not set, see `git mw help` \n";
+	}
+
+	v_print("### SELECTING REMOTE ###\n");
+
+	if ($remote_name eq EMPTY) {
+		# Search current branch upstream branch remote
+		my $current_branch = git_cmd_try {
+			Git::command_oneline('symbolic-ref', '--short', 'HEAD') }
+			"%s failed w/ code %d";
+		$remote_name = Git::config("branch.${current_branch}.remote");
+
+		if ($remote_name) {
+			$remote_url = mediawiki_remote_url_maybe($remote_name);
+		}
+
+		# Search all possibles mediawiki remotes
+		if (! $remote_url) {
+			my @remotes = git_cmd_try {
+				Git::command('remote'); }
+				"%s failed w/ code %d";
+
+			my @valid_remotes = ();
+			foreach my $remote (@remotes) {
+				$remote_url = mediawiki_remote_url_maybe($remote);
+				if ($remote_url) {
+					push(@valid_remotes, $remote);
+				}
+			}
+
+			if ($#valid_remotes == 0) {
+				print {*STDERR} "Can not find any mediawiki remote in this repo. \n";
+				exit 1;
+			} else {
+				print {*STDERR} "There are multiple mediawiki remotes, which of:\n";
+				foreach my $remote (@remotes) {
+					print {*STDERR} "\t${remote}\n";
+				}
+				print {*STDERR} "do you want ? Use the -r option to specify the remote\n";
+			}
+
+			exit 0;
+		}
+	} else {
+		# Check remote name
+		my @remotes = git_cmd_try {
+			Git::command('remote') }
+			"%s failed w/ code %d";
+		my $found_remote = 0;
+		foreach my $remote (@remotes) {
+			if ($remote eq $remote_name) {
+				$found_remote = 1;
+				last;
+			}
+		}
+		if (!$found_remote) {
+			die "${remote_name} is not a remote\n";
+		}
+
+		# Find remote url
+		$remote_url = mediawiki_remote_url_maybe($remote_name);
+		if (! $remote_url) {
+			die "the remote you selected is not a mediawiki remote\n";
+		}
+	}
+	v_print("selected remote:\n\tname: ${remote_name}\n\turl: ${remote_url}\n");
+
+	# Create and connect to the wiki if necessary
+	$wiki = connect_maybe($wiki, $remote_name, $remote_url);
+
+	# Read file content
+	if (! -e $file_name) {
+		$file_content = git_cmd_try {
+			Git::command('cat-file', 'blob', $file_name); }
+			"%s failed w/ code %d";
+
+		if ($file_name =~ /(.+):(.+)/) {
+			$file_name = $2;
+		}
+	} else {
+		open my $read_fh, "<", $file_name
+			or die "could not open ${file_name}: $!\n";
+		$file_content = do { local $/ = undef; <$read_fh> };
+		close $read_fh
+			or die "unable to close: $!\n";
+	}
+
+	# Transform file_name into a mediawiki page name
+	$wiki_page_name = smudge_filename($file_name);
+	$wiki_page_name =~ s/\.[^.]+$//;
+
+	# Default preview_file_name is file_name with .html ext
+	if ($preview_file_name eq EMPTY) {
+		$preview_file_name = $file_name;
+		$preview_file_name =~ s/\.[^.]+$/.html/;
+	}
+
+	v_print("### Retrieving template\n");
+	$template = get_template($remote_url, $wiki_page_name);
+
+	v_print("### Parsing & merging contents\n");
+	# Parsing template page
+	$html_tree = HTML::TreeBuilder->new;
+	$html_tree->parse($template);
+
+	# Load new content
+	$content = $wiki->api({
+		action => 'parse',
+		text => $file_content,
+		title => $wiki_page_name
+	}, {
+		skip_encoding => 1
+	}) or die "No response from distant mediawiki\n";
+	$content = $content->{'parse'}->{'text'}->{'*'};
+	$content_tree = HTML::TreeBuilder->new;
+	$content_tree->parse($content);
+
+	# Replace old content with new one
+	$template_content_id = Git::config('mediawiki.IDContent')
+		|| $template_content_id;
+	v_print("Using '${template_content_id}' as the content ID\n");
+	$mw_content_text = $html_tree->look_down('id', $template_content_id);
+	if (!defined $mw_content_text) {
+		print {*STDERR} <<"CONFIG";
+Could not combine the new parsed content with the template. You might want to
+configure `mediawiki.IDContent` in your config:
+	git config --add mediawiki.IDContent <your_template_content_element_id>
+CONFIG
+	}
+	$mw_content_text->delete_content();
+	$mw_content_text->push_content($content_tree);
+
+	# Transform relative links into absolute ones
+	for (@{ $html_tree->extract_links() }) {
+		my ($link, $element, $attr) = @{ $_ };
+		my $url = url($link)->canonical;
+		$element->attr($attr, URI->new_abs($url, $remote_url));
+	}
+
+	# Save the preview file
+	open(my $save_fh, '>:encoding(UTF-8)', $preview_file_name)
+		or die "Could not open: $!\n";
+	print {$save_fh} $html_tree->as_HTML;
+	close($save_fh)
+		or die "Could not close: $!\n";
+
+	# Auto-loading in browser
+	v_print("### Results\n");
+	if ($autoload) {
+		v_print("Launching browser w/ file: ${preview_file_name}");
+		system('git', 'web--browse', $preview_file_name);
+	} else {
+		print {*STDERR} "Preview file saved as: ${preview_file_name}\n";
+	}
+
+	exit;
+}
+
+sub mediawiki_remote_url_maybe {
+	my $remote = shift;
+
+	# Find remote url
+	my $remote_url = Git::config("remote.${remote}.url");
+	if ($remote_url =~ s/mediawiki::(.*)/$1/) {
+		return url($remote_url)->canonical;
+	}
+
+	return;
+}
+
+sub get_template {
+	my $url = shift;
+	my $page_name = shift;
+	my ($req, $res, $code, $url_after);
+
+	$req = LWP::UserAgent->new;
+	if ($verbose) {
+		$req->show_progress(1);
+	}
+
+	$res = $req->get("${url}/index.php?title=${page_name}");
+	if (!$res->is_success) {
+		$code = $res->code;
+		$url_after = $res->request()->uri(); # resolve all redirections
+		if ($code == HTTP_CODE_PAGE_NOT_FOUND) {
+			if ($verbose) {
+				print {*STDERR} <<"WARNING";
+Warning: Failed to retrieve '$page_name'. Create it on the mediawiki if you want
+all the links to work properly.
+Trying to use the mediawiki homepage as a fallback template ...
+WARNING
+			}
+
+			# LWP automatically redirects GET request
+			$res = $req->get("${url}/index.php");
+			if (!$res->is_success) {
+				$url_after = $res->request()->uri(); # resolve all redirections
+				die "Failed to get homepage @ ${url_after} w/ code ${code}\n";
+			}
+		} else {
+			die "Failed to get '${page_name}' @ ${url_after} w/ code ${code}\n";
+		}
+	}
+
+	return $res->decoded_content;
+}
+
+sub v_print {
+	if ($verbose) {
+		print {*STDERR} @_;
+	}
+}
+
 ############################## Help Functions ##################################
 
 sub help {
@@ -41,6 +341,7 @@ usage: git mw <command> <args>
 
 git mw commands are:
     help        Display help information about git mw
+    preview 	Parse and render local file into HTML
 END
 	exit;
 }
\ No newline at end of file
-- 
1.8.3.GIT

Re: [PATCH V3 0/4] git-remote-mediawiki: new tool to preview local changes without pushing

[Ramkumar Ramachandra]

<benoit.person@ensimag.fr> wrote:
> Subject: [PATCH V3 0/4] git-remote-mediawiki: new tool to preview local changes without pushing

Quick tip: use git format-patch -v3.  The uppercase 'V' indicates that
you probably typed it out yourself, or used --subject-prefix="PATCH
V3".

Re: [PATCH V3 1/4] git-mw: Introduction of GitMediawiki.pm

[Matthieu Moy]

benoit.person@ensimag.fr writes:

> changes from the V2:
>   - Add a way to test, without installation, code that uses GitMediawiki.pm.

This still needs to be documented, even very quickly, somewhere in the
code (e.g a comment in the Makefile).

> -build install clean:
> +copy_pm:
> +	cp $(GIT_MEDIAWIKI_PM) $(GIT_ROOT_DIR)/perl/blib/lib/

I already commented on this:

http://thread.gmane.org/gmane.comp.version-control.git/227711/focus=227721

Also, it seems to be only part of the solution. With your change, from
contrib/mw-to-git/ and after running only "make",

./git-mw takes the installed version of GitMediawiki.pm in priority

../../bin-wrappers/git takes the installed version of git-mw only (i.e.
does not know "git mw" if "make install" hasn't been ran).

>  perlcritic:
> -	perlcritic -2 *.perl
> +	perlcritic -2 *.perl
> \ No newline at end of file

Please, avoid these whitespace-only changes. They create noise during
review, and more potential conflicts.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [PATCH V3 4/4] git-mw: Add preview subcommand into git mw.

[Matthieu Moy]

[ Just a quick look, no time for a detailed review ]

benoit.person@ensimag.fr writes:

> From: Benoit Person <benoit.person@ensimag.fr>
>
> Add the subcommand to 'git-mw.perl'.

That's already said in the Subject field.

> Add a new constant in GitMediawiki.pm 'HTTP_CODE_PAGE_NOT_FOUND'.

And this brings zero information compared to

> --- a/contrib/mw-to-git/GitMediawiki.pm
> +++ b/contrib/mw-to-git/GitMediawiki.pm
> -				EMPTY HTTP_CODE_OK);
> +				EMPTY HTTP_CODE_OK HTTP_CODE_PAGE_NOT_FOUND);

I'd say your commit messages looks like a GNU-style changelog entry,
which I do not consider a compliment ;-).

Still, after removing rendundant information, you may notice that the
reader has no idea what "preview" is or does, and *why* it is a good
thing to have it. How about:

"
In the current state, a user of git-remote-mediawiki can edit the markup
text locally, but has to push to the remote wiki to see how the page is
rendered. Add a new "git mw preview" command that allows rendering the
markup text on the remote wiki without actually doing any change on the
wiki.

This uses MediaWiki's API to render the markup, and inserts the result
in an actual HTML page from the wiki so that CSS be rendered properly.
"

?

(The first paragraph answers "*why* did you do this?" and the second
"*why* did you do it this way?")

(did I put enough emphasis on the "why"? ;-) )

> +	# file_name argumeent is mandatory

argumeent -> argument

> +	if (!defined $file_name) {
> +		die "File not set, see `git mw help` \n";

Perhaps "missing file argument"?

> +		# Search all possibles mediawiki remotes
> +		if (! $remote_url) {

The "why" thing about commit message also applies to comments: when you
start saying what you're doing in a comment, it usually means your code
should be refactored.

Wouldn't it better to add a function here? The name of the function
would probably carry the same information as the comment.

> +				print {*STDERR} "do you want ? Use the -r option to specify the remote\n";

Missing period (.).

> +	}) or die "No response from distant mediawiki\n";

distant -> remote.

> +	$template_content_id = Git::config('mediawiki.IDContent')
> +		|| $template_content_id;

It would be cool to have also remote.<name>.IDContent or something like
this in case you have several remotes with different div ids. But this
can be added later.

> @@ -41,6 +341,7 @@ usage: git mw <command> <args>
>  
>  git mw commands are:
>      help        Display help information about git mw
> +    preview 	Parse and render local file into HTML

Space/tab mix after preview.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [PATCH V3 1/4] git-mw: Introduction of GitMediawiki.pm

[Benoît Person]

On 16 June 2013 22:18, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote:
> benoit.person@ensimag.fr writes:
>
>> changes from the V2:
>>   - Add a way to test, without installation, code that uses GitMediawiki.pm.
>
> This still needs to be documented, even very quickly, somewhere in the
> code (e.g a comment in the Makefile).
Well, I think I will have to re-read some docs (and some earlier
reviews) about what to write in commit messages, in the emails, in the
code as comments and in the documentation ... I am just totally lost
right now :/ .

>> -build install clean:
>> +copy_pm:
>> +     cp $(GIT_MEDIAWIKI_PM) $(GIT_ROOT_DIR)/perl/blib/lib/
>
> I already commented on this:
>
> http://thread.gmane.org/gmane.comp.version-control.git/227711/focus=227721
Oops, I forgot that one, so sorry :/ .

> Also, it seems to be only part of the solution. With your change, from
> contrib/mw-to-git/ and after running only "make",
>
> ./git-mw takes the installed version of GitMediawiki.pm in priority
>
> ../../bin-wrappers/git takes the installed version of git-mw only (i.e.
> does not know "git mw" if "make install" hasn't been ran).
Same thing as the documentation point, I think I am a bit lost in that
whole thing. I will re-look into it for the next version :/ .

>>  perlcritic:
>> -     perlcritic -2 *.perl
>> +     perlcritic -2 *.perl
>> \ No newline at end of file
>
> Please, avoid these whitespace-only changes. They create noise during
> review, and more potential conflicts.
For that one, I don't know why git assumes there is a change in it :
from what I see there is absolutely no whitespaces changes but maybe
my editor added some weird character somewhere ? I will look into that
for the next version ...

Thank you,

Benoit Person

Re: [PATCH V3 1/4] git-mw: Introduction of GitMediawiki.pm

[Matthieu Moy]

Benoît Person <benoit.person@ensimag.fr> writes:

> On 16 June 2013 22:18, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote:
>> benoit.person@ensimag.fr writes:
>>
>>> changes from the V2:
>>>   - Add a way to test, without installation, code that uses GitMediawiki.pm.
>>
>> This still needs to be documented, even very quickly, somewhere in the
>> code (e.g a comment in the Makefile).
> Well, I think I will have to re-read some docs (and some earlier
> reviews) about what to write in commit messages, in the emails, in the
> code as comments and in the documentation ... I am just totally lost
> right now :/ .

Don't worry, reviews are meant to improve your code (present and
future), not to blame you ;-).

Just think about what you expect as a user or developer. Would you run
"git log Makefile" or "git blame Makefile" to know how to use the
Makefile? Commit messages are primarily meant for reviewers ("here's
some code, and here's why it's good and why you should merge it"), and
can be very useful when bisecting a regression or blaming a source file.

Right now, git-remote-mediawiki has only little doc, and the user manual
is hosted on a GitHub wiki, not in the source. So there's no ideal place 
to say how to use the tool as a developer, but a comment in the Makefile
should be OK.

>> Also, it seems to be only part of the solution. With your change, from
>> contrib/mw-to-git/ and after running only "make",
>>
>> ./git-mw takes the installed version of GitMediawiki.pm in priority
>>
>> ../../bin-wrappers/git takes the installed version of git-mw only (i.e.
>> does not know "git mw" if "make install" hasn't been ran).
> Same thing as the documentation point, I think I am a bit lost in that
> whole thing. I will re-look into it for the next version :/ .

In short, the include path should contain both the *.pm file and the
git-<foo> ones.

>>>  perlcritic:
>>> -     perlcritic -2 *.perl
>>> +     perlcritic -2 *.perl
>>> \ No newline at end of file
>>
>> Please, avoid these whitespace-only changes. They create noise during
>> review, and more potential conflicts.
> For that one, I don't know why git assumes there is a change in it :

I think you removed a newline from the end of the file. It's usually
considered good practice to have this trailing newline (e.g. so that
"cat file" in a terminal doesn't put your prompt after the last line).
IIRC, it's actually required to call the file a "text file" according to
POSIX.

> I will look into that for the next version ...

In any case, using "git add -p" and if needed its "s" command avoids
introducing unwanted things in the commit.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [PATCH V3 1/4] git-mw: Introduction of GitMediawiki.pm

[Benoît Person]

On 17 June 2013 09:12, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote:
>>> Also, it seems to be only part of the solution. With your change, from
>>> contrib/mw-to-git/ and after running only "make",
>>>
>>> ./git-mw takes the installed version of GitMediawiki.pm in priority
>>>
>>> ../../bin-wrappers/git takes the installed version of git-mw only (i.e.
>>> does not know "git mw" if "make install" hasn't been ran).
>> Same thing as the documentation point, I think I am a bit lost in that
>> whole thing. I will re-look into it for the next version :/ .
>
> In short, the include path should contain both the *.pm file and the
> git-<foo> ones.
The fact is, for now, is there a way to test changes in
git-remote-mediawiki.perl without 'make install'-ing it ? I could not find one

So maybe in the "build-perl-script" of the toplevel Makefile we could add
something copying the script at the toplevel ?

And in GitMediawiki's Makefile, we let everything stay as is : copying *.pm
into /perl/blib/lib when building and copying it in installdir when installing ?

> I think you removed a newline from the end of the file. It's usually
> considered good practice to have this trailing newline (e.g. so that
> "cat file" in a terminal doesn't put your prompt after the last line).
> IIRC, it's actually required to call the file a "text file" according to
> POSIX.
That catch oO, thanks for the explanations.
From my point of view, this could definitely be improved from:
> -     perlcritic -2 *.perl
> +     perlcritic -2 *.perl
> \ No newline at end of file
to something like that:
> -     perlcritic -2 *.perl
> +     perlcritic -2 *.perl
> \ removed newline at end of file
which gives more insights into why the line is considered "edited".

Benoit Person

Re: [PATCH V3 1/4] git-mw: Introduction of GitMediawiki.pm

[Matthieu Moy]

Benoît Person <benoit.person@ensimag.fr> writes:

>>From my point of view, this could definitely be improved from:
>> -     perlcritic -2 *.perl
>> +     perlcritic -2 *.perl
>> \ No newline at end of file
> to something like that:
>> -     perlcritic -2 *.perl
>> +     perlcritic -2 *.perl
>> \ removed newline at end of file
> which gives more insights into why the line is considered "edited".

Too late ;-). This is a diff/patch thing, and Git is compatible with
them.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

Re: [PATCH V3 1/4] git-mw: Introduction of GitMediawiki.pm

[Matthieu Moy]

Benoît Person <benoit.person@ensimag.fr> writes:

> On 17 June 2013 09:12, Matthieu Moy <Matthieu.Moy@grenoble-inp.fr> wrote:
>>>> Also, it seems to be only part of the solution. With your change, from
>>>> contrib/mw-to-git/ and after running only "make",
>>>>
>>>> ./git-mw takes the installed version of GitMediawiki.pm in priority
>>>>
>>>> ../../bin-wrappers/git takes the installed version of git-mw only (i.e.
>>>> does not know "git mw" if "make install" hasn't been ran).
>>> Same thing as the documentation point, I think I am a bit lost in that
>>> whole thing. I will re-look into it for the next version :/ .
>>
>> In short, the include path should contain both the *.pm file and the
>> git-<foo> ones.
> The fact is, for now, is there a way to test changes in
> git-remote-mediawiki.perl without 'make install'-ing it ? I could not find one

You can put contrib/mw-to-git/ in your $PATH, and then run git normally.

> So maybe in the "build-perl-script" of the toplevel Makefile we could add
> something copying the script at the toplevel ?

I find this a bit dirty, as it polutes the toplevel with untracked files
(that are not in the .gitignore file). But that's what the testsuite
already does (IIRC, it does a symlink), so I'd say it's OK for now.

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/