user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
Search results ordered by [date|relevance]  view[summary|nested|Atom feed]
thread overview below | download mbox.gz: |
* Re: [PATCH v2] examples: cgit filter for use with WwwHighlight
  2019-04-25  3:19  4%   ` [PATCH v2] " Eric Wong
@ 2019-04-25  3:25  6%     ` Eric Wong
  0 siblings, 0 replies; 4+ results
From: Eric Wong @ 2019-04-25  3:25 UTC (permalink / raw)
  To: meta

>  interdiff:
>  $ U=https://public-inbox.org/meta
>  $ interdiff <(curl -s $U/dafb7a8/s/?b=x.lua) <(curl -s $U/a267d1c/s/?b=x.lua)

Oops, I got that wrong :x  It should be:

	diff -u <(curl -s $U/dafb7a8/s/x.lua) <(curl -s $U/a267d1c/s/x.lua)

^ permalink raw reply	[relevance 6%]

* [PATCH v2] examples: cgit filter for use with WwwHighlight
  2019-04-24 23:02  5% ` [PATCH 4/5] examples: cgit filter for use with WwwHighlight Eric Wong
@ 2019-04-25  3:19  4%   ` Eric Wong
  2019-04-25  3:25  6%     ` Eric Wong
  0 siblings, 1 reply; 4+ results
From: Eric Wong @ 2019-04-25  3:19 UTC (permalink / raw)
  To: meta

I'm using this as the cgit about-filter and source-filter
in https://80x24.org/public-inbox.git
---
 v2:
 * proper error handling and behavior w.r.t. large files
 * do not send Expect: header (and waiting for 100-continue)
 * correctly distinguish between about-filter and source-filter
   (v1 was creating nested <pre> when used as source-filter)

 interdiff:
 $ U=https://public-inbox.org/meta
 $ interdiff <(curl -s $U/dafb7a8/s/?b=x.lua) <(curl -s $U/a267d1c/s/?b=x.lua)

 MANIFEST                              |   1 +
 examples/cgit-wwwhighlight-filter.lua | 105 ++++++++++++++++++++++++++
 2 files changed, 106 insertions(+)
 create mode 100644 examples/cgit-wwwhighlight-filter.lua

diff --git a/MANIFEST b/MANIFEST
index 9858b9e..ed8ff49 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -39,6 +39,7 @@ examples/apache2_perl.conf
 examples/apache2_perl_old.conf
 examples/cgi-webrick.rb
 examples/cgit-commit-filter.lua
+examples/cgit-wwwhighlight-filter.lua
 examples/cgit.psgi
 examples/highlight.psgi
 examples/logrotate.conf
diff --git a/examples/cgit-wwwhighlight-filter.lua b/examples/cgit-wwwhighlight-filter.lua
new file mode 100644
index 0000000..a267d1c
--- /dev/null
+++ b/examples/cgit-wwwhighlight-filter.lua
@@ -0,0 +1,105 @@
+-- Copyright (C) 2019 all contributors <meta@public-inbox.org>
+-- License: GPL-2.0+ <https://www.gnu.org/licenses/gpl-2.0.txt>
+--
+-- This filter accesses the PublicInbox::WwwHighlight PSGI endpoint
+-- (see examples/highlight.psgi)
+--
+-- Dependencies: lua-http
+--
+-- disclaimer: written by someone who does not know Lua.
+--
+-- This requires cgit linked with Lua
+-- Usage (in your cgitrc(5) config file):
+--
+--   source-filter=lua:/path/to/this/script.lua
+--   about-filter=lua:/path/to/this/script.lua
+--
+local wwwhighlight_url = 'http://127.0.0.1:9090/'
+local req_timeout = 10
+local too_big = false
+
+-- match $PublicInbox::HTTP::MAX_REQUEST_BUFFER
+local max_len = 10 * 1024 * 1024
+
+-- about-filter needs surrounding <pre> tags if all we do is
+-- highlight and linkify
+local pre = true
+
+function filter_open(...)
+	req_body = ""
+
+	-- detect when we're used in an about-filter
+	local repo_url = os.getenv('CGIT_REPO_URL')
+	if repo_url then
+		local path_info = os.getenv('PATH_INFO')
+		rurl = path_info:match("^/(.+)/about/?$")
+		pre = rurl == repo_url
+	end
+
+	-- hand filename off for language detection
+	local fn = select(1, ...)
+	if fn then
+		local http_util = require 'http.util'
+		wwwhighlight_url = wwwhighlight_url .. http_util.encodeURI(fn)
+	end
+end
+
+-- try to buffer the entire source in memory
+function filter_write(str)
+	if too_big then
+		html(str)
+	elseif (req_body:len() + str:len()) > max_len then
+		too_big = true
+		req_body = ""
+		html(req_body)
+		html(str)
+	else
+		req_body = req_body .. str
+	end
+end
+
+function fail(err)
+	io.stderr:write(tostring(err), "\n")
+	if pre then
+		html("<pre>")
+	end
+	html_txt(req_body)
+	if pre then
+		html("</pre>")
+	end
+	return 1
+end
+
+function filter_close()
+	if too_big then
+		return 0
+	end
+	local request = require 'http.request'
+	local req = request.new_from_uri(wwwhighlight_url)
+	req.headers:upsert(':method', 'PUT')
+	req:set_body(req_body)
+
+	-- don't wait for 100-Continue message from the PSGI app
+	req.headers:delete('expect')
+
+	local headers, stream = req:go(req_timeout)
+	if headers == nil then
+		return fail(stream)
+	end
+	local status = headers:get(':status')
+	if status ~= '200' then
+		return fail('status ' .. status)
+	end
+	local body, err = stream:get_body_as_string()
+	if not body and err then
+		return fail(err)
+	end
+	if pre then
+		html("<pre>")
+	end
+	html(body)
+	if pre then
+		html("</pre>")
+	end
+	return 0
+end
-- 
EW

^ permalink raw reply related	[relevance 4%]

* [PATCH 0/5] cgit + WwwHighlight.pm integration
@ 2019-04-24 23:02  7% Eric Wong
  2019-04-24 23:02  5% ` [PATCH 4/5] examples: cgit filter for use with WwwHighlight Eric Wong
  0 siblings, 1 reply; 4+ results
From: Eric Wong @ 2019-04-24 23:02 UTC (permalink / raw)
  To: meta

cgit-wwwhighlight-filter.lua can benefit cgit installations by
allowing them to avoid spawning a new process for about-filter
and source-filter in cgitrc.

Instead of spawning highlight(1) or similar every time a cgit
filter is invoked, we can have a long-lived PSGI server running
WwwHighlight.pm(*) and rely on lua-http to access the server
over loopback.

WwwHighlight.pm is also updated to support the same
linkification logic we use for emails.


(*) see examples/highlight.psgi

Eric Wong (5):
  wwwhighlight: do linkification, too
  examples/cgit-commit-filter: remove unused variable
  wwwhighlight: deal with no-op highlight
  examples: cgit filter for use with WwwHighlight
  examples/cgit-commit-filter.lua: some doc updates

 MANIFEST                              |  1 +
 examples/cgit-commit-filter.lua       | 10 ++++-
 examples/cgit-wwwhighlight-filter.lua | 57 +++++++++++++++++++++++++++
 lib/PublicInbox/WwwHighlight.pm       | 11 +++++-
 4 files changed, 77 insertions(+), 2 deletions(-)
 create mode 100644 examples/cgit-wwwhighlight-filter.lua

^ permalink raw reply	[relevance 7%]

* [PATCH 4/5] examples: cgit filter for use with WwwHighlight
  2019-04-24 23:02  7% [PATCH 0/5] cgit + WwwHighlight.pm integration Eric Wong
@ 2019-04-24 23:02  5% ` Eric Wong
  2019-04-25  3:19  4%   ` [PATCH v2] " Eric Wong
  0 siblings, 1 reply; 4+ results
From: Eric Wong @ 2019-04-24 23:02 UTC (permalink / raw)
  To: meta

I'm using this as the cgit about-filter and source-filter
in https://80x24.org/public-inbox.git
---
 MANIFEST                              |  1 +
 examples/cgit-wwwhighlight-filter.lua | 57 +++++++++++++++++++++++++++
 2 files changed, 58 insertions(+)
 create mode 100644 examples/cgit-wwwhighlight-filter.lua

diff --git a/MANIFEST b/MANIFEST
index 9858b9e..ed8ff49 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -39,6 +39,7 @@ examples/apache2_perl.conf
 examples/apache2_perl_old.conf
 examples/cgi-webrick.rb
 examples/cgit-commit-filter.lua
+examples/cgit-wwwhighlight-filter.lua
 examples/cgit.psgi
 examples/highlight.psgi
 examples/logrotate.conf
diff --git a/examples/cgit-wwwhighlight-filter.lua b/examples/cgit-wwwhighlight-filter.lua
new file mode 100644
index 0000000..dafb7a8
--- /dev/null
+++ b/examples/cgit-wwwhighlight-filter.lua
@@ -0,0 +1,57 @@
+-- Copyright (C) 2019 all contributors <meta@public-inbox.org>
+-- License: GPL-2.0+ <https://www.gnu.org/licenses/gpl-2.0.txt>
+--
+-- This filter accesses the PublicInbox::WwwHighlight PSGI endpoint
+-- (see examples/highlight.psgi)
+--
+-- Dependencies: lua-http
+--
+-- disclaimer: written by someone who does not know Lua.
+--
+-- This requires cgit linked with Lua
+-- Usage (in your cgitrc(5) config file):
+--
+--   source-filter=lua:/path/to/this/script.lua
+--   about-filter=lua:/path/to/this/script.lua
+--
+local wwwhighlight_url = 'http://127.0.0.1:9090/'
+local req_timeout = 10
+
+-- this is nil when used as about-filter
+local fn
+
+function filter_open(...)
+	req_body = ""
+	fn = select(1, ...)
+	if fn then
+		local http_util = require 'http.util'
+		wwwhighlight_url = wwwhighlight_url .. http_util.encodeURI(fn)
+	end
+end
+
+-- buffer the entire source in memory
+function filter_write(str)
+	req_body = req_body .. str
+end
+
+function filter_close()
+	local request = require 'http.request'
+	local req = request.new_from_uri(wwwhighlight_url)
+	req.headers:upsert(':method', 'PUT')
+	req:set_body(req_body)
+	local headers, stream = req:go(req_timeout)
+	local body, err = stream:get_body_as_string()
+	if fn then
+		html("<pre>")
+	end
+	if not body and err then
+		io.stderr:write(tostring(err), "\n")
+		html_txt(req_body)
+	else
+		html(body)
+	end
+	if fn then
+		html("</pre>")
+	end
+	return 0
+end
-- 
EW


^ permalink raw reply related	[relevance 5%]

Results 1-4 of 4 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2019-04-24 23:02  7% [PATCH 0/5] cgit + WwwHighlight.pm integration Eric Wong
2019-04-24 23:02  5% ` [PATCH 4/5] examples: cgit filter for use with WwwHighlight Eric Wong
2019-04-25  3:19  4%   ` [PATCH v2] " Eric Wong
2019-04-25  3:25  6%     ` Eric Wong

Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).