git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Thomas Adam <thomas@xteddy.org>
To: Jakub Narebski <jnareb@gmail.com>
Cc: git@vger.kernel.org,
	"John 'Warthog9' Hawley" <warthog9@kernel.org>,
	Petr Baudis <pasky@ucw.cz>,
	admin@repo.or.cz
Subject: Re: [RFC/PATCHv5 18/17] gitweb/lib - Add clear() and size() methods to caching interface
Date: Wed, 6 Oct 2010 23:56:42 +0100	[thread overview]
Message-ID: <AANLkTimpXJT-R5vgdS4SvpuvhVOL-HCJxELMbfRcuqx5@mail.gmail.com> (raw)
In-Reply-To: <1286402526-13143-19-git-send-email-jnareb@gmail.com>

On 6 October 2010 23:02, Jakub Narebski <jnareb@gmail.com> wrote:
> Add ->size() method, which following Cache::Cache interface returns
> estimated total size of all entries in whole cache (in the namsepace
> assiciated with give cache instance).  Note that ->get_size($key)
> returns size of a single entry!
>
> Add ->clear() method, which removes all entries from the namespace
> associated with given cache instance.  For safety it requires
> namespace to be set to true value, which means that it cannot be
> empty; therefore default namespace is changed to 'gitweb'.
>
> The ->clear() method should be fairly safe, because it first renames
> directory (which should be atomic), and only then removes it
> (following code from CGI::Driver::File).
>
> Signed-off-by: Jakub Narebski <jnareb@gmail.com>
> ---
> The only difference from previous version is that a comment about some
> method was moved to earlier commit that introduced it, where it
> belongs.
>
> This patch (implementing "backend") together with next patch
> (implementing "interface") are meant to provide nice web interface to
> safely cleaning (pruning) cache, as described in comments and
> documentation (how to do this manually).
>
>  gitweb/lib/GitwebCache/SimpleFileCache.pm |   48 ++++++++++++++++++++++++++--
>  t/t9503/test_cache_interface.pl           |    1 -
>  2 files changed, 44 insertions(+), 5 deletions(-)
>
> diff --git a/gitweb/lib/GitwebCache/SimpleFileCache.pm b/gitweb/lib/GitwebCache/SimpleFileCache.pm
> index 2a3d9cf..8f172cb 100644
> --- a/gitweb/lib/GitwebCache/SimpleFileCache.pm
> +++ b/gitweb/lib/GitwebCache/SimpleFileCache.pm
> @@ -20,8 +20,9 @@ package GitwebCache::SimpleFileCache;
>  use strict;
>  use warnings;
>
> -use File::Path qw(mkpath);
> -use File::Temp qw(tempfile);
> +use File::Path qw(mkpath rmtree);
> +use File::Temp qw(tempfile mktemp);
> +use File::Find qw(find);
>  use Digest::MD5 qw(md5_hex);
>
>  # by default, the cache nests all entries on the filesystem single
> @@ -37,7 +38,7 @@ our $DEFAULT_CACHE_ROOT = "cache";
>  # by default we don't use cache namespace (empty namespace);
>  # empty namespace does not allow for simple implementation of clear() method.
>  #
> -our $DEFAULT_NAMESPACE = '';
> +our $DEFAULT_NAMESPACE = "gitweb";
>
>  # ......................................................................
>  # constructor
> @@ -334,7 +335,7 @@ sub get_size {
>  }
>
>  # ......................................................................
> -# interface methods
> +# interface methods dealing with single item
>
>  # Removing and expiring
>
> @@ -427,6 +428,45 @@ sub compute {
>        return $data;
>  }
>
> +# ......................................................................
> +# interface methods dealing with whole namespace
> +
> +# $cache->clear();
> +#
> +# Remove all entries from the namespace.
> +# Namespace must be defined and not empty.
> +sub clear {
> +       my $self = shift;
> +
> +       return unless $self->get_namespace();
> +
> +       my $namespace_dir = $self->path_to_namespace();
> +       return if !-d $namespace_dir;
> +
> +       my $renamed_dir = mktemp($namespace_dir . '.XXXX');
> +       rename($namespace_dir, $renamed_dir);
> +       rmtree($renamed_dir);
> +       die "Couldn't remove '$renamed_dir' directory"
> +               if -d $renamed_dir;
> +}
> +
> +# $size = $cache->size();
> +#
> +# Size of whole names (or whole cache if namespace empty)
> +sub size {
> +       my $self = shift;
> +
> +       my $namespace_dir = $self->path_to_namespace();
> +       return if !-d $namespace_dir;
> +
> +       my $total_size = 0;
> +       my $add_size = sub { $total_size += -s $_ };
> +
> +       File::Find::find({ wanted => $add_size, no_chdir => 1 }, $namespace_dir);
> +
> +       return $total_size;
> +}
> +
>  1;

This looks good to me.

Acked-by:  Thomas Adam <thomas@xteddy.org>

-- Thomas Adam

  reply	other threads:[~2010-10-06 22:57 UTC|newest]

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-10-06 22:01 [PATCHv5 00/17] gitweb: Simple file based output caching Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 01/17] t/test-lib.sh: Export also GIT_BUILD_DIR in test_external Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 02/17] gitweb: Prepare for splitting gitweb Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 03/17] gitweb/lib - Very simple file based cache Jakub Narebski
2010-10-06 22:41   ` Thomas Adam
2010-10-06 22:44     ` Ævar Arnfjörð Bjarmason
2010-10-06 22:46       ` Thomas Adam
2010-10-06 22:47         ` Ævar Arnfjörð Bjarmason
2010-10-06 23:00     ` Jakub Narebski
2010-10-06 23:12       ` Thomas Adam
2010-10-06 23:32         ` Jakub Narebski
2010-10-06 22:57   ` Ævar Arnfjörð Bjarmason
2010-10-06 23:46     ` Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 04/17] gitweb/lib - Stat-based cache expiration Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 05/17] gitweb/lib - Regenerate entry if the cache file has size of 0 Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 06/17] gitweb/lib - Simple select(FH) based output capture Jakub Narebski
2010-10-06 22:52   ` Thomas Adam
2010-10-06 23:22     ` Jakub Narebski
2010-10-06 23:03   ` Ævar Arnfjörð Bjarmason
2010-10-06 23:26     ` Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 07/17] gitweb/lib - Cache captured output (using get/set) Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 08/17] gitweb: Add optional output caching Jakub Narebski
2010-10-06 22:46   ` Ævar Arnfjörð Bjarmason
2010-10-06 23:06     ` Jakub Narebski
2010-10-06 23:16       ` Ævar Arnfjörð Bjarmason
2010-10-06 22:01 ` [PATCHv5 09/17] gitweb/lib - Adaptive cache expiration time Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 10/21] gitweb/lib - Use CHI compatibile (compute method) caching interface Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 11/17] gitweb/lib - Use locking to avoid 'cache miss stampede' problem Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 12/17] gitweb/lib - No need for File::Temp when locking Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 13/17] gitweb/lib - Serve stale data when waiting for filling cache Jakub Narebski
2010-10-06 22:01 ` [PATCHv5 14/17] gitweb/lib - Regenerate (refresh) cache in background Jakub Narebski
2010-10-06 22:02 ` [PATCHv5 15/17] gitweb: Introduce %actions_info, gathering information about actions Jakub Narebski
2010-10-06 22:02 ` [PATCHv5/RFC 16/17] gitweb: Show appropriate "Generating..." page when regenerating cache Jakub Narebski
2010-10-06 22:02 ` [PATCHv5/RFC 17/17] gitweb: Add startup delay to activity indicator for cache Jakub Narebski
2010-10-06 22:02 ` [RFC/PATCHv5 18/17] gitweb/lib - Add clear() and size() methods to caching interface Jakub Narebski
2010-10-06 22:56   ` Thomas Adam [this message]
2010-10-06 22:02 ` [RFC PATCHv5 19/17] gitweb: Add beginnings of cache administration page Jakub Narebski
2010-10-06 22:02 ` [PoC PATCHv5 20/17] gitweb/lib - Benchmarking GitwebCache::SimpleFileCache (in t/9603/) Jakub Narebski
2010-10-06 22:02 ` [PoC PATCHv5 21/17] gitweb/lib - Alternate ways of capturing output Jakub Narebski
2010-10-10 20:32 ` [RFD] Possible improvements for output caching in gitweb Jakub Narebski
2010-10-24 21:34 ` [PATCHv5 00/17] gitweb: Simple file based output caching J.H.

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=AANLkTimpXJT-R5vgdS4SvpuvhVOL-HCJxELMbfRcuqx5@mail.gmail.com \
    --to=thomas@xteddy.org \
    --cc=admin@repo.or.cz \
    --cc=git@vger.kernel.org \
    --cc=jnareb@gmail.com \
    --cc=pasky@ucw.cz \
    --cc=warthog9@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).