From: Thomas Adam <thomas@xteddy.org>
To: Jakub Narebski <jnareb@gmail.com>
Cc: git@vger.kernel.org,
"John 'Warthog9' Hawley" <warthog9@kernel.org>,
Petr Baudis <pasky@ucw.cz>,
admin@repo.or.cz
Subject: Re: [RFC/PATCHv5 18/17] gitweb/lib - Add clear() and size() methods to caching interface
Date: Wed, 6 Oct 2010 23:56:42 +0100 [thread overview]
Message-ID: <AANLkTimpXJT-R5vgdS4SvpuvhVOL-HCJxELMbfRcuqx5@mail.gmail.com> (raw)
In-Reply-To: <1286402526-13143-19-git-send-email-jnareb@gmail.com>
On 6 October 2010 23:02, Jakub Narebski <jnareb@gmail.com> wrote:
> Add ->size() method, which following Cache::Cache interface returns
> estimated total size of all entries in whole cache (in the namsepace
> assiciated with give cache instance). Note that ->get_size($key)
> returns size of a single entry!
>
> Add ->clear() method, which removes all entries from the namespace
> associated with given cache instance. For safety it requires
> namespace to be set to true value, which means that it cannot be
> empty; therefore default namespace is changed to 'gitweb'.
>
> The ->clear() method should be fairly safe, because it first renames
> directory (which should be atomic), and only then removes it
> (following code from CGI::Driver::File).
>
> Signed-off-by: Jakub Narebski <jnareb@gmail.com>
> ---
> The only difference from previous version is that a comment about some
> method was moved to earlier commit that introduced it, where it
> belongs.
>
> This patch (implementing "backend") together with next patch
> (implementing "interface") are meant to provide nice web interface to
> safely cleaning (pruning) cache, as described in comments and
> documentation (how to do this manually).
>
> gitweb/lib/GitwebCache/SimpleFileCache.pm | 48 ++++++++++++++++++++++++++--
> t/t9503/test_cache_interface.pl | 1 -
> 2 files changed, 44 insertions(+), 5 deletions(-)
>
> diff --git a/gitweb/lib/GitwebCache/SimpleFileCache.pm b/gitweb/lib/GitwebCache/SimpleFileCache.pm
> index 2a3d9cf..8f172cb 100644
> --- a/gitweb/lib/GitwebCache/SimpleFileCache.pm
> +++ b/gitweb/lib/GitwebCache/SimpleFileCache.pm
> @@ -20,8 +20,9 @@ package GitwebCache::SimpleFileCache;
> use strict;
> use warnings;
>
> -use File::Path qw(mkpath);
> -use File::Temp qw(tempfile);
> +use File::Path qw(mkpath rmtree);
> +use File::Temp qw(tempfile mktemp);
> +use File::Find qw(find);
> use Digest::MD5 qw(md5_hex);
>
> # by default, the cache nests all entries on the filesystem single
> @@ -37,7 +38,7 @@ our $DEFAULT_CACHE_ROOT = "cache";
> # by default we don't use cache namespace (empty namespace);
> # empty namespace does not allow for simple implementation of clear() method.
> #
> -our $DEFAULT_NAMESPACE = '';
> +our $DEFAULT_NAMESPACE = "gitweb";
>
> # ......................................................................
> # constructor
> @@ -334,7 +335,7 @@ sub get_size {
> }
>
> # ......................................................................
> -# interface methods
> +# interface methods dealing with single item
>
> # Removing and expiring
>
> @@ -427,6 +428,45 @@ sub compute {
> return $data;
> }
>
> +# ......................................................................
> +# interface methods dealing with whole namespace
> +
> +# $cache->clear();
> +#
> +# Remove all entries from the namespace.
> +# Namespace must be defined and not empty.
> +sub clear {
> + my $self = shift;
> +
> + return unless $self->get_namespace();
> +
> + my $namespace_dir = $self->path_to_namespace();
> + return if !-d $namespace_dir;
> +
> + my $renamed_dir = mktemp($namespace_dir . '.XXXX');
> + rename($namespace_dir, $renamed_dir);
> + rmtree($renamed_dir);
> + die "Couldn't remove '$renamed_dir' directory"
> + if -d $renamed_dir;
> +}
> +
> +# $size = $cache->size();
> +#
> +# Size of whole names (or whole cache if namespace empty)
> +sub size {
> + my $self = shift;
> +
> + my $namespace_dir = $self->path_to_namespace();
> + return if !-d $namespace_dir;
> +
> + my $total_size = 0;
> + my $add_size = sub { $total_size += -s $_ };
> +
> + File::Find::find({ wanted => $add_size, no_chdir => 1 }, $namespace_dir);
> +
> + return $total_size;
> +}
> +
> 1;
This looks good to me.
Acked-by: Thomas Adam <thomas@xteddy.org>
-- Thomas Adam
next prev parent reply other threads:[~2010-10-06 22:57 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-10-06 22:01 [PATCHv5 00/17] gitweb: Simple file based output caching Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 01/17] t/test-lib.sh: Export also GIT_BUILD_DIR in test_external Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 02/17] gitweb: Prepare for splitting gitweb Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 03/17] gitweb/lib - Very simple file based cache Jakub Narebski
2010-10-06 22:41 ` Thomas Adam
2010-10-06 22:44 ` Ævar Arnfjörð Bjarmason
2010-10-06 22:46 ` Thomas Adam
2010-10-06 22:47 ` Ævar Arnfjörð Bjarmason
2010-10-06 23:00 ` Jakub Narebski
2010-10-06 23:12 ` Thomas Adam
2010-10-06 23:32 ` Jakub Narebski
2010-10-06 22:57 ` Ævar Arnfjörð Bjarmason
2010-10-06 23:46 ` Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 04/17] gitweb/lib - Stat-based cache expiration Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 05/17] gitweb/lib - Regenerate entry if the cache file has size of 0 Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 06/17] gitweb/lib - Simple select(FH) based output capture Jakub Narebski
2010-10-06 22:52 ` Thomas Adam
2010-10-06 23:22 ` Jakub Narebski
2010-10-06 23:03 ` Ævar Arnfjörð Bjarmason
2010-10-06 23:26 ` Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 07/17] gitweb/lib - Cache captured output (using get/set) Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 08/17] gitweb: Add optional output caching Jakub Narebski
2010-10-06 22:46 ` Ævar Arnfjörð Bjarmason
2010-10-06 23:06 ` Jakub Narebski
2010-10-06 23:16 ` Ævar Arnfjörð Bjarmason
2010-10-06 22:01 ` [PATCHv5 09/17] gitweb/lib - Adaptive cache expiration time Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 10/21] gitweb/lib - Use CHI compatibile (compute method) caching interface Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 11/17] gitweb/lib - Use locking to avoid 'cache miss stampede' problem Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 12/17] gitweb/lib - No need for File::Temp when locking Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 13/17] gitweb/lib - Serve stale data when waiting for filling cache Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 14/17] gitweb/lib - Regenerate (refresh) cache in background Jakub Narebski
2010-10-06 22:02 ` [PATCHv5 15/17] gitweb: Introduce %actions_info, gathering information about actions Jakub Narebski
2010-10-06 22:02 ` [PATCHv5/RFC 16/17] gitweb: Show appropriate "Generating..." page when regenerating cache Jakub Narebski
2010-10-06 22:02 ` [PATCHv5/RFC 17/17] gitweb: Add startup delay to activity indicator for cache Jakub Narebski
2010-10-06 22:02 ` [RFC/PATCHv5 18/17] gitweb/lib - Add clear() and size() methods to caching interface Jakub Narebski
2010-10-06 22:56 ` Thomas Adam [this message]
2010-10-06 22:02 ` [RFC PATCHv5 19/17] gitweb: Add beginnings of cache administration page Jakub Narebski
2010-10-06 22:02 ` [PoC PATCHv5 20/17] gitweb/lib - Benchmarking GitwebCache::SimpleFileCache (in t/9603/) Jakub Narebski
2010-10-06 22:02 ` [PoC PATCHv5 21/17] gitweb/lib - Alternate ways of capturing output Jakub Narebski
2010-10-10 20:32 ` [RFD] Possible improvements for output caching in gitweb Jakub Narebski
2010-10-24 21:34 ` [PATCHv5 00/17] gitweb: Simple file based output caching J.H.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=AANLkTimpXJT-R5vgdS4SvpuvhVOL-HCJxELMbfRcuqx5@mail.gmail.com \
--to=thomas@xteddy.org \
--cc=admin@repo.or.cz \
--cc=git@vger.kernel.org \
--cc=jnareb@gmail.com \
--cc=pasky@ucw.cz \
--cc=warthog9@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).