about summary refs log tree commit homepage
path: root/lib/PublicInbox/Git.pm
DateCommit message (Collapse)
2019-01-20git: support 'ambiguous' result from --batch-check
David Turner's patch to return "ambiguous" seems like a reasonable patch for future versions of git: https://public-inbox.org/git/672a6fb9e480becbfcb5df23ae37193784811b6b.camel@novalis.org/
2019-01-19git: disable abbreviations with cat-file hints
Ambiguity is not worth it for internal usage with the solver.
2019-01-19git: check saves error on disambiguation
This will be useful for disambiguating short OIDs in older emails when abbreviations were shorter. Tested against the following script with /path/to/git.git ==> t.perl <== use strict; use PublicInbox::Git; use Data::Dumper; my $dir = shift or die "Usage: $0 GIT_DIR # (of git.git)"; my $git = PublicInbox::Git->new($dir); my @res = $git->check('dead'); print Dumper({res => \@res, err=> $git->last_check_err}); @res = $git->check('5335669531d83d7d6c905bcfca9b5f8e182dc4d4'); print Dumper({res => \@res, err=> $git->last_check_err});
2019-01-19git: add git_quote
It'll be helpful for displaying progress in SolverGit output.
2019-01-19git: support multiple URL endpoints
For redundancy and centralization resistance.
2019-01-19solver: initial Perl implementation
This will lookup git blobs from associated git source code repositories. If the blobs can't be found, an attempt to "solve" them via patch application will be performed. Eventually, this may become the basis of a type-agnostic frontend similar to "git show"
2019-01-18git: git_unquote handles double-quote and backslash
We need to work with 0x22 (double-quote) and 0x5c (backslash); even if they're oddball characters in filenames which wouldn't be used by projects I'd want to work on.
2019-01-15git_unquote: perform modifications in-place
This function doesn't have a lot of callers at the moment so none of them are affected by this change. But the plan is to use this in our WWW code for things, so do it now before we call it in more places. Results from a Thinkpad X200 with a Core2Duo P8600 @ 2.4GHz: Benchmark: timing 10 iterations of cp, ip... cp: 12.868 wallclock secs (12.86 usr + 0.00 sys = 12.86 CPU) @ 0.78/s (n=10) ip: 10.9137 wallclock secs (10.91 usr + 0.00 sys = 10.91 CPU) @ 0.92/s (n=10) Note: I mainly care about unquoted performance because that's the common case for the target audience of public-inbox. Script used to get benchmark results against the Linux source tree: ==> bench_unquote.perl <== use strict; use warnings; use Benchmark ':hireswallclock'; my $nr = 50; my %GIT_ESC = ( a => "\a", b => "\b", f => "\f", n => "\n", r => "\r", t => "\t", v => "\013", ); sub git_unquote_ip ($) { return $_[0] unless ($_[0] =~ /\A"(.*)"\z/); $_[0] = $1; $_[0] =~ s/\\([abfnrtv])/$GIT_ESC{$1}/g; $_[0] =~ s/\\([0-7]{1,3})/chr(oct($1))/ge; $_[0]; } sub git_unquote_cp ($) { my ($s) = @_; return $s unless ($s =~ /\A"(.*)"\z/); $s = $1; $s =~ s/\\([abfnrtv])/$GIT_ESC{$1}/g; $s =~ s/\\([0-7]{1,3})/chr(oct($1))/ge; $s; } chomp(my @files = `git -C ~/linux ls-tree --name-only -r v4.19.13`); timethese(10, { cp => sub { for (0..$nr) { git_unquote_cp($_) for @files } }, ip => sub { for (0..$nr) { git_unquote_ip($_) for @files } }, });
2019-01-15searchidx: move git_unquote to PublicInbox::Git
We'll be using it outside of searchidx...
2019-01-05shrink low-bandwidth pipes under Linux
I've hit /proc/sys/fs/pipe-user-pages-* limits on some systems. So stop hogging resources on pipes which don't benefit from giant sizes. Some of these can use eventfd in the future to further reduce resource use.
2018-02-19git: reload alternates file on missing blob
Since we'll be adding new repositories to the `alternates' file in git, we must restart the `git cat-file --batch' process as git currently does not detect changes to the alternates file in long-running cat-file processes. Don't bother with the `--batch-check' process since we won't be using it with v2.
2018-02-19v2writable: initial cut for repo-rotation
Wrap the old Import package to enable creating new repos based on size thresholds. This is better than relying on time-based rotation as LKML traffic seems to be increasing.
2018-02-07update copyrights for 2018
Using update-copyrights from gnulib While we're at it, use the SPDX identifier for AGPL-3.0+ to ease mechanical processing.
2016-08-09searchidx: avoid holding Xapian lock in cat-file
We must ensure cat-file process is launched before Xapian grabs lock, too. Our use of "git cat-file --batch" has the same problem as "git log" did, (which was fixed in commit 3713c727cda431a0dc2865a7878c13ecf9f21851) "searchidx: release Xapian FDs before spawning git log"
2016-06-21spawn: improve error checking for fork failures
fork failures are unfortunately common when Xapian has gigabytes and gigabytes mmapped.
2016-05-21localize $/ in more places to avoid potential problems
This hopefully makes the intent of the code clearer, too. The the HTTP use of the numeric reference for getline caused problems in Git.pm, already.
2016-05-21mbox: switch generation over to pull model
This allows us to easily provide gigantic inboxes with proper backpressure handling for slow clients. It also eliminates public-inbox-httpd and Danga::Socket-specific knowledge from this class, making it easier to follow for those used to generic PSGI applications.
2016-04-27import: document API for public consumption
This is probably trivial enough to be final?
2016-04-11git: add support for qx wrapper
This lets us one-line git commands easily like ``, but without having to remember --git-dir or escape arguments.
2016-02-28reduce calls to close unless error checks are needed
We can rely on timely auto-destruction based on reference counting; reducing the chance of redundant close(2) calls which may hit the wront FD. We do care about certain close calls (e.g. writing to a buffered IO handle) if we require error-checking for write-integrity. In other cases, let things go out-of-scope so it can be freed automatically after use.
2016-02-27git: use built-in spawn implementation for vfork
This should reduce overhead of spawning git processes from our long-running httpd and nntpd servers.
2015-12-22rename 'GitCatFile' package to 'Git'
We'll be using it for more than just cat-file. Adding a `popen' API for internal use allows us to save a bunch of code in other places.