[PATCHv2 0/2] gitweb: Beginnings of splitting gitweb into modules

[PATCHv2 0/2] gitweb: Beginnings of splitting gitweb into modules

[Jakub Narebski]

Gitweb is currently next to largest file (after gitk) in git sources,
more than 225KB with more than 7,000 lines.  Therefore adding any
large feature that would require large amount of code added, like
gitweb caching by J.H. and my rewrite of it, or "gitweb admin/write"
[failed] GSoC 2010 project by Pavan Kumar Sunkara, would require for
new code to be added as a separate module.  Otherwise gitweb would
fast become unmaintainable.

Note that there is already patch series sent which as one of side
effects splits the JavaScript side of gitweb into many smaller
self-contained files:

  [PATCH 03/13] gitweb: Split JavaScript for maintability, combining on build
  http://thread.gmane.org/gmane.comp.version-control.git/172384/focus=172385

Not in all cases splitting gitweb upfront would be required.  At least
in the case of gitweb caching it doesn't.  What must be done however
is preparing the infrastructure for modular gitweb sources; to
properly test such infrastructure we need at least one split gitweb
module.  With patch series preparing for splitting or true splitting of
gitweb sent upfront the future patch series that implements
code-extensive feature (like e.g. output caching) would be smaller and
easier to review.


This series is intended to bring such infrastructure to gitweb, to
prepare way for adding output caching to gitweb. Alternatively it can
be thought as beginning of splitting gitweb into smaller submodules,
for better maintainability.

This patch series was sent to git mailing list as

  [PATCH 0/2] gitweb: Begin splitting gitweb
  http://thread.gmane.org/gmane.comp.version-control.git/165824

In the above mentioned first version of this series, the first patch
that prepared the way for splitting gitweb was in three versions: A, B
and C.  In this second version of this series the first patch in
series most closely resembles version C in v1 series.

In this version gitweb uses _both_ 'use lib __DIR__."/lib";' and
'use lib "++GITWEBLIBDIR++";', in correct order (as compared to
version C of v1 series), so that __DIR__."/lib" is checked first,
i.e. modules installed alongside gitweb.cgi win.

Pull request:
~~~~~~~~~~~~~
This series is available in the git repository at:
  git://repo.or.cz/git/jnareb-git.git gitweb/split
  git://github.com/jnareb/git.git gitweb/split

Well, those patches has a few minor cosmetic fixups...

Table of contents:
~~~~~~~~~~~~~~~~~~
* [PATCHv2 1/2] gitweb: Prepare for splitting gitweb
  [PATCHv2 2/2] gitweb: Create Gitweb::Util module

  First patch implements infrastructure, second proof of concept patch uses
  this infrastructure, testing it ("make -C gitweb test-installed").

Shortlog:
~~~~~~~~~
Jakub Narebski (1):
  gitweb: Prepare for splitting gitweb

Pavan Kumar Sunkara (1):
  gitweb: Create Gitweb::Util module

Diffstat:
~~~~~~~~~
 gitweb/INSTALL            |    7 ++
 gitweb/Makefile           |   23 ++++++-
 gitweb/gitweb.perl        |  151 ++++-----------------------------------
 gitweb/lib/Gitweb/Util.pm |  177 +++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 219 insertions(+), 139 deletions(-)
 create mode 100755 gitweb/lib/Gitweb/Util.pm

-- 
1.7.3

[PATCHv2 1/2 (RFC?)] gitweb: Prepare for splitting gitweb

[Jakub Narebski]

Prepare gitweb for being split into modules that would be installed in
$(gitweblibdir), by default alongside gitweb in 'lib/' subdirectory.

Gitweb would search first in 'lib/' subdirectory from where gitweb.cgi
is installed, via

  use lib __DIR__.'/lib';

(This allow for tests to work with source version of gitweb without
changes.)  Then it searches in $(gitweblibdir) directory (set during
build time), by default "$(gitwebdir)/lib", via

  use lib "++GITWEBLIBDIR++";

which is set to requested dir during building of gitweb.cgi.  Note
that 'use lib' assures no trailing duplicate entries in @INC.


This preparatory work allows to add new module to gitweb by simply
adding

  GITWEB_MODULES += <module>

to gitweb/Makefile (assuming that the module is in 'gitweb/lib/'
directory).

While at it pass GITWEBLIBDIR in addition to GITWEB_TEST_INSTALLED to
allow testing installed version of gitweb and installed version of
modules (for future tests which would check individual (sub)modules).


At Pavan Kumar Sankara suggestion gitweb/Makefile uses

  install [OPTION]... SOURCE... DIRECTORY

format (2nd format) with single SOURCE rather than

  install [OPTION]... SOURCE DEST

format (1st format) because of security reasons (race conditions).
Modern GNU install has `-T' / `--no-target-directory' option, but we
cannot rely that the $(INSTALL) we are using supports this option.

The install-modules target in gitweb/Makefile uses shell 'for' loop,
instead of make's $(foreach) function, to avoid possible problem with
generating a command line that exceeded the maximum argument list
length.

Helped-by: Pavan Kumar Sunkara <pavan.sss1991@gmail.com>
Helped-by: Alejandro R. Sedeño <asedeno@mit.edu>
Signed-off-by: Jakub Narębski <jnareb@gmail.com>
---
This patch is closest to

  [PATCH (version C) 1/2] gitweb: Prepare for splitting gitweb
  http://thread.gmane.org/gmane.comp.version-control.git/165824/focus=165826

from previous (v1) version of this series.

  Advantages:
  - can run source version of gitweb (gitweb/gitweb.perl) as a script simply
  - supports relocating gitweb modules (to gitweblibdir)
  - allow users to simply install extra modules by hand alongside gitweb

  Disadvantages:
  - most complicated code of all cases

In this version modules from __DIR__."/lib", i.e. installed alongside
gitweb.cgi take preference over common modules installed in
"++GITWEBLIBDIR++", as suggested / requested by Alejandro:

  http://thread.gmane.org/gmane.comp.version-control.git/165824/focus=165926  


This patch is marked as possible RFC because I am not sure if
"$(gitwebdir)/lib" directory to install modules alongside gitweb.cgi
should be created unconditionally; it might be empty if
$(gitweblibdir) is changed from its default version.

This version also include update to gitweb/INSTALL.


Side-note: I have thought about adding sanity check for empty
"++GITWEBLIBDIR++" in the form of

  use if "++GITWEBLIBDIR++", lib => "++GITWEBLIBDIR++";

or

  use lib "++GITWEBLIBDIR++" || '.';

But because default value of "++GITWEBLIBDIR++" is never empty, I
don't think it is worth complicating code protecting against unlikely
user error; Perl would give the following warning:

  Empty compile time value given to use lib

 gitweb/INSTALL     |    7 +++++++
 gitweb/Makefile    |   20 ++++++++++++++++++--
 gitweb/gitweb.perl |   11 +++++++++++
 3 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/gitweb/INSTALL b/gitweb/INSTALL
index 4964a67..7af343a 100644
--- a/gitweb/INSTALL
+++ b/gitweb/INSTALL
@@ -243,6 +243,13 @@ The following optional Perl modules are required for extra features
  - HTML::TagCloud - for fancy tag cloud in project list view
  - HTTP::Date or Time::ParseDate - to support If-Modified-Since for feeds
 
+Those modules can be installed, in order of search, alongside
+gitweb.cgi in 'lib/' subdirectory, in '$(gitweblibdir)' directory
+(given during build), or in one of directories in which Perl looks for
+library files (PERL5LIB, PERLLIB, standard places, current directory,
+etc.).  Note that the first two places are by default the same
+directory; "$(gitweblibdir)" is "$(gitwebdir)/lib".
+
 
 Example web server configuration
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/gitweb/Makefile b/gitweb/Makefile
index 0a6ac00..b353d15 100644
--- a/gitweb/Makefile
+++ b/gitweb/Makefile
@@ -13,6 +13,7 @@ all::
 prefix ?= $(HOME)
 bindir ?= $(prefix)/bin
 gitwebdir ?= /var/www/cgi-bin
+gitweblibdir ?= $(gitwebdir)/lib
 
 RM ?= rm -f
 INSTALL ?= install
@@ -57,6 +58,7 @@ PERL_PATH  ?= /usr/bin/perl
 bindir_SQ = $(subst ','\'',$(bindir))#'
 gitwebdir_SQ = $(subst ','\'',$(gitwebdir))#'
 gitwebstaticdir_SQ = $(subst ','\'',$(gitwebdir)/static)#'
+gitweblibdir_SQ = $(subst ','\'',$(gitweblibdir))#'
 SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))#'
 PERL_PATH_SQ  = $(subst ','\'',$(PERL_PATH))#'
 DESTDIR_SQ    = $(subst ','\'',$(DESTDIR))#'
@@ -115,6 +117,7 @@ GITWEB_FILES += static/git-logo.png static/git-favicon.png
 GITWEB_REPLACE = \
 	-e 's|++GIT_VERSION++|$(GIT_VERSION)|g' \
 	-e 's|++GIT_BINDIR++|$(bindir)|g' \
+	-e 's|++GITWEBLIBDIR++|$(gitweblibdir)|g' \
 	-e 's|++GITWEB_CONFIG++|$(GITWEB_CONFIG)|g' \
 	-e 's|++GITWEB_CONFIG_SYSTEM++|$(GITWEB_CONFIG_SYSTEM)|g' \
 	-e 's|++GITWEB_HOME_LINK_STR++|$(GITWEB_HOME_LINK_STR)|g' \
@@ -153,20 +156,33 @@ test:
 
 test-installed:
 	GITWEB_TEST_INSTALLED='$(DESTDIR_SQ)$(gitwebdir_SQ)' \
+	GITWEBLIBDIR='$(DESTDIR_SQ)$(gitweblibdir_SQ)' \
 		$(MAKE) -C ../t gitweb-test
 
 ### Installation rules
 
-install: all
+install: all install-modules
 	$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitwebdir_SQ)'
 	$(INSTALL) -m 755 $(GITWEB_PROGRAMS) '$(DESTDIR_SQ)$(gitwebdir_SQ)'
 	$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitwebstaticdir_SQ)'
 	$(INSTALL) -m 644 $(GITWEB_FILES) '$(DESTDIR_SQ)$(gitwebstaticdir_SQ)'
 
+install-modules:
+	$(INSTALL) -m 755 $(GITWEB_PROGRAMS) '$(DESTDIR_SQ)$(gitwebdir_SQ)/lib'
+	install_dirs="$(sort $(dir $(GITWEB_MODULES)))" && \
+	for dir in $$install_dirs; do \
+		test -d '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/"$$dir" || \
+		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/"$$dir"; \
+	done
+	gitweb_modules="$(GITWEB_MODULES)" && \
+	for mod in $$gitweb_modules; do \
+		$(INSTALL) -m 644 "lib/$$mod" '$(DESTDIR_SQ)$(gitweblibdir_SQ)'/"$$(dirname $$mod)"; \
+	done
+
 ### Cleaning rules
 
 clean:
 	$(RM) gitweb.cgi static/gitweb.min.js static/gitweb.min.css GITWEB-BUILD-OPTIONS
 
-.PHONY: all clean install test test-installed .FORCE-GIT-VERSION-FILE FORCE
+.PHONY: all clean install install-modules test test-installed .FORCE-GIT-VERSION-FILE FORCE
 
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index ee69ea6..f094471 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -10,6 +10,17 @@
 use 5.008;
 use strict;
 use warnings;
+
+use File::Spec;
+
+# __DIR__ is excerpt from Dir::Self
+sub __DIR__ () {
+	File::Spec->rel2abs(join '', (File::Spec->splitpath(__FILE__))[0, 1]);
+}
+use lib "++GITWEBLIBDIR++";
+use lib __DIR__ . '/lib';
+
+
 use CGI qw(:standard :escapeHTML -nosticky);
 use CGI::Util qw(unescape);
 use CGI::Carp qw(fatalsToBrowser set_message);
-- 
1.7.3

[PATCHv2 2/2 (PoC)] gitweb: Create Gitweb::Util module

[Jakub Narebski]

From: Pavan Kumar Sunkara <pavan.sss1991@gmail.com>

Create a Gitweb::Util module, which is meant to contain internal
utilities used by gitweb.  Currently it includes all the
quoting/unquoting and escaping subroutines that are used by the
gitweb.

Update gitweb/Makefile to install Gitweb::Util module alongside gitweb

This was sort of part of [unfinished] Google Summer of Code 2010
project "Splitting gitweb and developing write functionalities
(Integrated web client for git)" by Pavan Kumar Sunkara, mentored by
Christian Couder and co-mentored by Petr Baudis.

Signed-off-by: Pavan Kumar Sunkara <pavan.sss1991@gmail.com>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
This patch is unchanged from previous version:

  [PATCH (proof of concept) 2/2] gitweb: Create Gitweb::Util module
  http://thread.gmane.org/gmane.comp.version-control.git/165824/focus=165828


This patch serves two purposes.  First, it serves as test that earlier
"gitweb: Prepare for splitting gitweb" patch actually work correctly.
This can be checked by running "make -C gitweb test-installed" after
installing gitweb ("make install-gitweb" or "make -C gitweb install").

Second, it might be good starting point to splitting gitweb.
Refactoring well defined parts into separate modules (Perl packages)
could be a better, easier way than trying to come with good separation
(split) into modules upfront.  Such leisure approach to splitting
gitweb has more chance to be accepted.  Perhaps if such approach were
proposed on GSoC 2010, maybe "gitweb write" project wouldn't fail
midterm evaluations...


This module was taken out of unfinished GSoC 2010 project with
Pavan Kumar Sunkara as a student

  git://repo.or.cz/git/gsoc2010-gitweb.git

The module was renamed from Gitweb::Escape to Gitweb::Util.  Currently
the contents is the same, but it might change.

Code was updated to more modern codebase; since then esc_path_info and
esc_attr were added to gitweb - both of those are now in Gitweb::Util.

There were also required some changes and conflicts resolved due to
the fact that creating Gitweb::Util (formerly Gitweb::Escape) is no
longer in the middle of larger patch series.  In particular lack of
Gitweb::Config means that $fallback_encoding needed to be added to
Gitweb::Util module.

While at it do not export quot_cec and quot_upr helper subroutines by
default, but mark them exportable nevertheless.

 gitweb/Makefile           |    3 +
 gitweb/gitweb.perl        |  140 +-----------------------------------
 gitweb/lib/Gitweb/Util.pm |  177 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 183 insertions(+), 137 deletions(-)
 create mode 100644 gitweb/lib/Gitweb/Util.pm

diff --git a/gitweb/Makefile b/gitweb/Makefile
index b353d15..1b35808 100644
--- a/gitweb/Makefile
+++ b/gitweb/Makefile
@@ -114,6 +114,9 @@ endif
 
 GITWEB_FILES += static/git-logo.png static/git-favicon.png
 
+# Modules: Gitweb::*
+GITWEB_MODULES += Gitweb/Util.pm
+
 GITWEB_REPLACE = \
 	-e 's|++GIT_VERSION++|$(GIT_VERSION)|g' \
 	-e 's|++GIT_BINDIR++|$(bindir)|g' \
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index f094471..967ad64 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -24,11 +24,13 @@ use lib __DIR__ . '/lib';
 use CGI qw(:standard :escapeHTML -nosticky);
 use CGI::Util qw(unescape);
 use CGI::Carp qw(fatalsToBrowser set_message);
-use Encode;
 use Fcntl ':mode';
 use File::Find qw();
 use File::Basename qw(basename);
 use Time::HiRes qw(gettimeofday tv_interval);
+
+use Gitweb::Util;
+
 binmode STDOUT, ':utf8';
 
 our $t0 = [ gettimeofday() ];
@@ -1391,128 +1393,6 @@ sub validate_refname {
 	return $input;
 }
 
-# decode sequences of octets in utf8 into Perl's internal form,
-# which is utf-8 with utf8 flag set if needed.  gitweb writes out
-# in utf-8 thanks to "binmode STDOUT, ':utf8'" at beginning
-sub to_utf8 {
-	my $str = shift;
-	return undef unless defined $str;
-	if (utf8::valid($str)) {
-		utf8::decode($str);
-		return $str;
-	} else {
-		return decode($fallback_encoding, $str, Encode::FB_DEFAULT);
-	}
-}
-
-# quote unsafe chars, but keep the slash, even when it's not
-# correct, but quoted slashes look too horrible in bookmarks
-sub esc_param {
-	my $str = shift;
-	return undef unless defined $str;
-	$str =~ s/([^A-Za-z0-9\-_.~()\/:@ ]+)/CGI::escape($1)/eg;
-	$str =~ s/ /\+/g;
-	return $str;
-}
-
-# the quoting rules for path_info fragment are slightly different
-sub esc_path_info {
-	my $str = shift;
-	return undef unless defined $str;
-
-	# path_info doesn't treat '+' as space (specially), but '?' must be escaped
-	$str =~ s/([^A-Za-z0-9\-_.~();\/;:@&= +]+)/CGI::escape($1)/eg;
-
-	return $str;
-}
-
-# quote unsafe chars in whole URL, so some characters cannot be quoted
-sub esc_url {
-	my $str = shift;
-	return undef unless defined $str;
-	$str =~ s/([^A-Za-z0-9\-_.~();\/;?:@&= ]+)/CGI::escape($1)/eg;
-	$str =~ s/ /\+/g;
-	return $str;
-}
-
-# quote unsafe characters in HTML attributes
-sub esc_attr {
-
-	# for XHTML conformance escaping '"' to '&quot;' is not enough
-	return esc_html(@_);
-}
-
-# replace invalid utf8 character with SUBSTITUTION sequence
-sub esc_html {
-	my $str = shift;
-	my %opts = @_;
-
-	return undef unless defined $str;
-
-	$str = to_utf8($str);
-	$str = $cgi->escapeHTML($str);
-	if ($opts{'-nbsp'}) {
-		$str =~ s/ /&nbsp;/g;
-	}
-	$str =~ s|([[:cntrl:]])|(($1 ne "\t") ? quot_cec($1) : $1)|eg;
-	return $str;
-}
-
-# quote control characters and escape filename to HTML
-sub esc_path {
-	my $str = shift;
-	my %opts = @_;
-
-	return undef unless defined $str;
-
-	$str = to_utf8($str);
-	$str = $cgi->escapeHTML($str);
-	if ($opts{'-nbsp'}) {
-		$str =~ s/ /&nbsp;/g;
-	}
-	$str =~ s|([[:cntrl:]])|quot_cec($1)|eg;
-	return $str;
-}
-
-# Make control characters "printable", using character escape codes (CEC)
-sub quot_cec {
-	my $cntrl = shift;
-	my %opts = @_;
-	my %es = ( # character escape codes, aka escape sequences
-		"\t" => '\t',   # tab            (HT)
-		"\n" => '\n',   # line feed      (LF)
-		"\r" => '\r',   # carrige return (CR)
-		"\f" => '\f',   # form feed      (FF)
-		"\b" => '\b',   # backspace      (BS)
-		"\a" => '\a',   # alarm (bell)   (BEL)
-		"\e" => '\e',   # escape         (ESC)
-		"\013" => '\v', # vertical tab   (VT)
-		"\000" => '\0', # nul character  (NUL)
-	);
-	my $chr = ( (exists $es{$cntrl})
-		    ? $es{$cntrl}
-		    : sprintf('\%2x', ord($cntrl)) );
-	if ($opts{-nohtml}) {
-		return $chr;
-	} else {
-		return "<span class=\"cntrl\">$chr</span>";
-	}
-}
-
-# Alternatively use unicode control pictures codepoints,
-# Unicode "printable representation" (PR)
-sub quot_upr {
-	my $cntrl = shift;
-	my %opts = @_;
-
-	my $chr = sprintf('&#%04d;', 0x2400+ord($cntrl));
-	if ($opts{-nohtml}) {
-		return $chr;
-	} else {
-		return "<span class=\"cntrl\">$chr</span>";
-	}
-}
-
 # git may return quoted and escaped filenames
 sub unquote {
 	my $str = shift;
@@ -1549,20 +1429,6 @@ sub unquote {
 	return $str;
 }
 
-# escape tabs (convert tabs to spaces)
-sub untabify {
-	my $line = shift;
-
-	while ((my $pos = index($line, "\t")) != -1) {
-		if (my $count = (8 - ($pos % 8))) {
-			my $spaces = ' ' x $count;
-			$line =~ s/\t/$spaces/;
-		}
-	}
-
-	return $line;
-}
-
 sub project_in_list {
 	my $project = shift;
 	my @list = git_get_projects_list();
diff --git a/gitweb/lib/Gitweb/Util.pm b/gitweb/lib/Gitweb/Util.pm
new file mode 100644
index 0000000..a213d3f
--- /dev/null
+++ b/gitweb/lib/Gitweb/Util.pm
@@ -0,0 +1,177 @@
+# Gitweb::Util -- Internal utilities used by gitweb (git web interface)
+#
+# This module is licensed under the GPLv2
+
+package Gitweb::Util;
+
+use strict;
+use warnings;
+use Exporter qw(import);
+
+our @EXPORT = qw(to_utf8
+                 esc_param esc_path_info esc_url
+                 esc_html esc_path esc_attr
+                 untabify
+                 $fallback_encoding);
+our @EXPORT_OK = qw(quot_cec quot_upr);
+
+use Encode;
+use CGI;
+
+# ......................................................................
+# Perl encoding (utf-8)
+
+# decode sequences of octets in utf8 into Perl's internal form,
+# which is utf-8 with utf8 flag set if needed.  gitweb writes out
+# in utf-8 thanks to "binmode STDOUT, ':utf8'" at beginning of gitweb.perl
+our $fallback_encoding = 'latin1';
+sub to_utf8 {
+	my $str = shift;
+	return undef unless defined $str;
+	if (utf8::valid($str)) {
+		utf8::decode($str);
+		return $str;
+	} else {
+		return decode($fallback_encoding, $str, Encode::FB_DEFAULT);
+	}
+}
+
+# ......................................................................
+# CGI encoding
+
+# quote unsafe chars, but keep the slash, even when it's not
+# correct, but quoted slashes look too horrible in bookmarks
+sub esc_param {
+	my $str = shift;
+	return undef unless defined $str;
+
+	$str =~ s/([^A-Za-z0-9\-_.~()\/:@ ]+)/CGI::escape($1)/eg;
+	$str =~ s/ /\+/g;
+
+	return $str;
+}
+
+# the quoting rules for path_info fragment are slightly different
+sub esc_path_info {
+	my $str = shift;
+	return undef unless defined $str;
+
+	# path_info doesn't treat '+' as space (specially), but '?' must be escaped
+	$str =~ s/([^A-Za-z0-9\-_.~();\/;:@&= +]+)/CGI::escape($1)/eg;
+
+	return $str;
+}
+
+# quote unsafe chars in whole URL, so some characters cannot be quoted
+sub esc_url {
+	my $str = shift;
+	return undef unless defined $str;
+
+	$str =~ s/([^A-Za-z0-9\-_.~();\/;?:@&= ]+)/CGI::escape($1)/eg;
+	$str =~ s/ /\+/g;
+
+	return $str;
+}
+
+# ......................................................................
+# (X)HTML escaping
+
+# replace invalid utf8 character with SUBSTITUTION sequence
+sub esc_html {
+	my $str = shift;
+	my %opts = @_;
+
+	return undef unless defined $str;
+
+	$str = to_utf8($str);
+	$str = CGI::escapeHTML($str);
+	if ($opts{'-nbsp'}) {
+		$str =~ s/ /&nbsp;/g;
+	}
+	$str =~ s|([[:cntrl:]])|(($1 ne "\t") ? quot_cec($1) : $1)|eg;
+	return $str;
+}
+
+# quote unsafe characters in HTML attributes
+sub esc_attr {
+
+	# for XHTML conformance escaping '"' to '&quot;' is not enough
+	return esc_html(@_);
+}
+
+# quote control characters and escape filename to HTML
+sub esc_path {
+	my $str = shift;
+	my %opts = @_;
+
+	return undef unless defined $str;
+
+	$str = to_utf8($str);
+	$str = CGI::escapeHTML($str);
+	if ($opts{'-nbsp'}) {
+		$str =~ s/ /&nbsp;/g;
+	}
+	$str =~ s|([[:cntrl:]])|quot_cec($1)|eg;
+	return $str;
+}
+
+# ......................................................................
+# Other
+
+# escape tabs (convert tabs to spaces)
+sub untabify {
+	my $line = shift;
+
+	while ((my $pos = index($line, "\t")) != -1) {
+		if (my $count = (8 - ($pos % 8))) {
+			my $spaces = ' ' x $count;
+			$line =~ s/\t/$spaces/;
+		}
+	}
+
+	return $line;
+}
+
+# ----------------------------------------------------------------------
+# Showing "unprintable" characters (utility functions)
+
+# Make control characters "printable", using character escape codes (CEC)
+sub quot_cec {
+	my $cntrl = shift;
+	my %opts = @_;
+	my %es = ( # character escape codes, aka escape sequences
+		"\t" => '\t',   # tab            (HT)
+		"\n" => '\n',   # line feed      (LF)
+		"\r" => '\r',   # carrige return (CR)
+		"\f" => '\f',   # form feed      (FF)
+		"\b" => '\b',   # backspace      (BS)
+		"\a" => '\a',   # alarm (bell)   (BEL)
+		"\e" => '\e',   # escape         (ESC)
+		"\013" => '\v', # vertical tab   (VT)
+		"\000" => '\0', # nul character  (NUL)
+	);
+	my $chr = ( (exists $es{$cntrl})
+		    ? $es{$cntrl}
+		    : sprintf('\%2x', ord($cntrl)) );
+	if ($opts{-nohtml}) {
+		return $chr;
+	} else {
+		return "<span class=\"cntrl\">$chr</span>";
+	}
+}
+
+# Alternatively use unicode control pictures codepoints,
+# Unicode "printable representation" (PR)
+sub quot_upr {
+	my $cntrl = shift;
+	my %opts = @_;
+
+	my $chr = sprintf('&#%04d;', 0x2400+ord($cntrl));
+	if ($opts{-nohtml}) {
+		return $chr;
+	} else {
+		return "<span class=\"cntrl\">$chr</span>";
+	}
+}
+
+1;
-- 
1.7.3

Re: [PATCHv2 0/2] gitweb: Beginnings of splitting gitweb into modules

[Sylvain Rabot]

[-- Attachment #1: Type: text/plain, Size: 3610 bytes --]

Hi,

Just wanted to know what is the current status of splitting gitweb
sources into modules.

Regards.

On Tue, 2011-05-03 at 16:04 +0200, Jakub Narebski wrote:
> Gitweb is currently next to largest file (after gitk) in git sources,
> more than 225KB with more than 7,000 lines.  Therefore adding any
> large feature that would require large amount of code added, like
> gitweb caching by J.H. and my rewrite of it, or "gitweb admin/write"
> [failed] GSoC 2010 project by Pavan Kumar Sunkara, would require for
> new code to be added as a separate module.  Otherwise gitweb would
> fast become unmaintainable.
> 
> Note that there is already patch series sent which as one of side
> effects splits the JavaScript side of gitweb into many smaller
> self-contained files:
> 
>   [PATCH 03/13] gitweb: Split JavaScript for maintability, combining on build
>   http://thread.gmane.org/gmane.comp.version-control.git/172384/focus=172385
> 
> Not in all cases splitting gitweb upfront would be required.  At least
> in the case of gitweb caching it doesn't.  What must be done however
> is preparing the infrastructure for modular gitweb sources; to
> properly test such infrastructure we need at least one split gitweb
> module.  With patch series preparing for splitting or true splitting of
> gitweb sent upfront the future patch series that implements
> code-extensive feature (like e.g. output caching) would be smaller and
> easier to review.
> 
> 
> This series is intended to bring such infrastructure to gitweb, to
> prepare way for adding output caching to gitweb. Alternatively it can
> be thought as beginning of splitting gitweb into smaller submodules,
> for better maintainability.
> 
> This patch series was sent to git mailing list as
> 
>   [PATCH 0/2] gitweb: Begin splitting gitweb
>   http://thread.gmane.org/gmane.comp.version-control.git/165824
> 
> In the above mentioned first version of this series, the first patch
> that prepared the way for splitting gitweb was in three versions: A, B
> and C.  In this second version of this series the first patch in
> series most closely resembles version C in v1 series.
> 
> In this version gitweb uses _both_ 'use lib __DIR__."/lib";' and
> 'use lib "++GITWEBLIBDIR++";', in correct order (as compared to
> version C of v1 series), so that __DIR__."/lib" is checked first,
> i.e. modules installed alongside gitweb.cgi win.
> 
> Pull request:
> ~~~~~~~~~~~~~
> This series is available in the git repository at:
>   git://repo.or.cz/git/jnareb-git.git gitweb/split
>   git://github.com/jnareb/git.git gitweb/split
> 
> Well, those patches has a few minor cosmetic fixups...
> 
> Table of contents:
> ~~~~~~~~~~~~~~~~~~
> * [PATCHv2 1/2] gitweb: Prepare for splitting gitweb
>   [PATCHv2 2/2] gitweb: Create Gitweb::Util module
> 
>   First patch implements infrastructure, second proof of concept patch uses
>   this infrastructure, testing it ("make -C gitweb test-installed").
> 
> Shortlog:
> ~~~~~~~~~
> Jakub Narebski (1):
>   gitweb: Prepare for splitting gitweb
> 
> Pavan Kumar Sunkara (1):
>   gitweb: Create Gitweb::Util module
> 
> Diffstat:
> ~~~~~~~~~
>  gitweb/INSTALL            |    7 ++
>  gitweb/Makefile           |   23 ++++++-
>  gitweb/gitweb.perl        |  151 ++++-----------------------------------
>  gitweb/lib/Gitweb/Util.pm |  177 +++++++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 219 insertions(+), 139 deletions(-)
>  create mode 100755 gitweb/lib/Gitweb/Util.pm
> 

-- 
Sylvain Rabot <sylvain@abstraction.fr>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [PATCHv2 0/2] gitweb: Beginnings of splitting gitweb into modules

[J.H.]

Pretty much stalled like most things on gitweb.  I've been knee deep in
a lot of GSoC stuff, conferences and dealing with kernel bugs and Jakub
has been busy on other things so we haven't had a solid head bashing
session to get this all worked out yet.

Maybe while I'm over in Prague / Europe for KS & Linuxcon we can hash
things out.

- John 'Warthog9' Hawley

On 08/23/2011 01:35 PM, Sylvain Rabot wrote:
> Hi,
> 
> Just wanted to know what is the current status of splitting gitweb
> sources into modules.
> 
> Regards.
> 
> On Tue, 2011-05-03 at 16:04 +0200, Jakub Narebski wrote:
>> Gitweb is currently next to largest file (after gitk) in git sources,
>> more than 225KB with more than 7,000 lines.  Therefore adding any
>> large feature that would require large amount of code added, like
>> gitweb caching by J.H. and my rewrite of it, or "gitweb admin/write"
>> [failed] GSoC 2010 project by Pavan Kumar Sunkara, would require for
>> new code to be added as a separate module.  Otherwise gitweb would
>> fast become unmaintainable.
>>
>> Note that there is already patch series sent which as one of side
>> effects splits the JavaScript side of gitweb into many smaller
>> self-contained files:
>>
>>   [PATCH 03/13] gitweb: Split JavaScript for maintability, combining on build
>>   http://thread.gmane.org/gmane.comp.version-control.git/172384/focus=172385
>>
>> Not in all cases splitting gitweb upfront would be required.  At least
>> in the case of gitweb caching it doesn't.  What must be done however
>> is preparing the infrastructure for modular gitweb sources; to
>> properly test such infrastructure we need at least one split gitweb
>> module.  With patch series preparing for splitting or true splitting of
>> gitweb sent upfront the future patch series that implements
>> code-extensive feature (like e.g. output caching) would be smaller and
>> easier to review.
>>
>>
>> This series is intended to bring such infrastructure to gitweb, to
>> prepare way for adding output caching to gitweb. Alternatively it can
>> be thought as beginning of splitting gitweb into smaller submodules,
>> for better maintainability.
>>
>> This patch series was sent to git mailing list as
>>
>>   [PATCH 0/2] gitweb: Begin splitting gitweb
>>   http://thread.gmane.org/gmane.comp.version-control.git/165824
>>
>> In the above mentioned first version of this series, the first patch
>> that prepared the way for splitting gitweb was in three versions: A, B
>> and C.  In this second version of this series the first patch in
>> series most closely resembles version C in v1 series.
>>
>> In this version gitweb uses _both_ 'use lib __DIR__."/lib";' and
>> 'use lib "++GITWEBLIBDIR++";', in correct order (as compared to
>> version C of v1 series), so that __DIR__."/lib" is checked first,
>> i.e. modules installed alongside gitweb.cgi win.
>>
>> Pull request:
>> ~~~~~~~~~~~~~
>> This series is available in the git repository at:
>>   git://repo.or.cz/git/jnareb-git.git gitweb/split
>>   git://github.com/jnareb/git.git gitweb/split
>>
>> Well, those patches has a few minor cosmetic fixups...
>>
>> Table of contents:
>> ~~~~~~~~~~~~~~~~~~
>> * [PATCHv2 1/2] gitweb: Prepare for splitting gitweb
>>   [PATCHv2 2/2] gitweb: Create Gitweb::Util module
>>
>>   First patch implements infrastructure, second proof of concept patch uses
>>   this infrastructure, testing it ("make -C gitweb test-installed").
>>
>> Shortlog:
>> ~~~~~~~~~
>> Jakub Narebski (1):
>>   gitweb: Prepare for splitting gitweb
>>
>> Pavan Kumar Sunkara (1):
>>   gitweb: Create Gitweb::Util module
>>
>> Diffstat:
>> ~~~~~~~~~
>>  gitweb/INSTALL            |    7 ++
>>  gitweb/Makefile           |   23 ++++++-
>>  gitweb/gitweb.perl        |  151 ++++-----------------------------------
>>  gitweb/lib/Gitweb/Util.pm |  177 +++++++++++++++++++++++++++++++++++++++++++++
>>  4 files changed, 219 insertions(+), 139 deletions(-)
>>  create mode 100755 gitweb/lib/Gitweb/Util.pm
>>
> 

Re: [PATCHv2 0/2] gitweb: Beginnings of splitting gitweb into modules

[Pavan Kumar Sunkara]

I can try to restart it. I had a lot of health issues last year, But I
am free now to finish this series.

On Wed, Aug 24, 2011 at 2:10 AM, J.H. <warthog9@eaglescrag.net> wrote:
> Pretty much stalled like most things on gitweb.  I've been knee deep in
> a lot of GSoC stuff, conferences and dealing with kernel bugs and Jakub
> has been busy on other things so we haven't had a solid head bashing
> session to get this all worked out yet.
>
> Maybe while I'm over in Prague / Europe for KS & Linuxcon we can hash
> things out.
>
> - John 'Warthog9' Hawley
>
> On 08/23/2011 01:35 PM, Sylvain Rabot wrote:
>> Hi,
>>
>> Just wanted to know what is the current status of splitting gitweb
>> sources into modules.
>>
>> Regards.
>>
>> On Tue, 2011-05-03 at 16:04 +0200, Jakub Narebski wrote:
>>> Gitweb is currently next to largest file (after gitk) in git sources,
>>> more than 225KB with more than 7,000 lines.  Therefore adding any
>>> large feature that would require large amount of code added, like
>>> gitweb caching by J.H. and my rewrite of it, or "gitweb admin/write"
>>> [failed] GSoC 2010 project by Pavan Kumar Sunkara, would require for
>>> new code to be added as a separate module.  Otherwise gitweb would
>>> fast become unmaintainable.
>>>
>>> Note that there is already patch series sent which as one of side
>>> effects splits the JavaScript side of gitweb into many smaller
>>> self-contained files:
>>>
>>>   [PATCH 03/13] gitweb: Split JavaScript for maintability, combining on build
>>>   http://thread.gmane.org/gmane.comp.version-control.git/172384/focus=172385
>>>
>>> Not in all cases splitting gitweb upfront would be required.  At least
>>> in the case of gitweb caching it doesn't.  What must be done however
>>> is preparing the infrastructure for modular gitweb sources; to
>>> properly test such infrastructure we need at least one split gitweb
>>> module.  With patch series preparing for splitting or true splitting of
>>> gitweb sent upfront the future patch series that implements
>>> code-extensive feature (like e.g. output caching) would be smaller and
>>> easier to review.
>>>
>>>
>>> This series is intended to bring such infrastructure to gitweb, to
>>> prepare way for adding output caching to gitweb. Alternatively it can
>>> be thought as beginning of splitting gitweb into smaller submodules,
>>> for better maintainability.
>>>
>>> This patch series was sent to git mailing list as
>>>
>>>   [PATCH 0/2] gitweb: Begin splitting gitweb
>>>   http://thread.gmane.org/gmane.comp.version-control.git/165824
>>>
>>> In the above mentioned first version of this series, the first patch
>>> that prepared the way for splitting gitweb was in three versions: A, B
>>> and C.  In this second version of this series the first patch in
>>> series most closely resembles version C in v1 series.
>>>
>>> In this version gitweb uses _both_ 'use lib __DIR__."/lib";' and
>>> 'use lib "++GITWEBLIBDIR++";', in correct order (as compared to
>>> version C of v1 series), so that __DIR__."/lib" is checked first,
>>> i.e. modules installed alongside gitweb.cgi win.
>>>
>>> Pull request:
>>> ~~~~~~~~~~~~~
>>> This series is available in the git repository at:
>>>   git://repo.or.cz/git/jnareb-git.git gitweb/split
>>>   git://github.com/jnareb/git.git gitweb/split
>>>
>>> Well, those patches has a few minor cosmetic fixups...
>>>
>>> Table of contents:
>>> ~~~~~~~~~~~~~~~~~~
>>> * [PATCHv2 1/2] gitweb: Prepare for splitting gitweb
>>>   [PATCHv2 2/2] gitweb: Create Gitweb::Util module
>>>
>>>   First patch implements infrastructure, second proof of concept patch uses
>>>   this infrastructure, testing it ("make -C gitweb test-installed").
>>>
>>> Shortlog:
>>> ~~~~~~~~~
>>> Jakub Narebski (1):
>>>   gitweb: Prepare for splitting gitweb
>>>
>>> Pavan Kumar Sunkara (1):
>>>   gitweb: Create Gitweb::Util module
>>>
>>> Diffstat:
>>> ~~~~~~~~~
>>>  gitweb/INSTALL            |    7 ++
>>>  gitweb/Makefile           |   23 ++++++-
>>>  gitweb/gitweb.perl        |  151 ++++-----------------------------------
>>>  gitweb/lib/Gitweb/Util.pm |  177 +++++++++++++++++++++++++++++++++++++++++++++
>>>  4 files changed, 219 insertions(+), 139 deletions(-)
>>>  create mode 100755 gitweb/lib/Gitweb/Util.pm
>>>
>>
>
>



-- 
- Pavan Kumar Sunkara