From: Eric Wong <e@80x24.org> To: meta@public-inbox.org Subject: [PATCH 03/64] git: *_async: support nested callback invocations Date: Fri, 16 Oct 2020 06:59:33 +0000 Message-ID: <20201016070034.22204-3-e@80x24.org> (raw) In-Reply-To: <20201016070034.22204-1-e@80x24.org> For external indices, we'll need to support nested cat_async invocations to deduplicate cross-posted messages. Thus we need to ensure we do not clobber the {inflight*} queues while stepping through and ensure {cat_rbuf} is stored before invoking callbacks. This fixes the ->cat_async-only case, but does not yet account for the mix of ->check_async interspersed with ->cat_async calls, yet. More work will be needed on that front at a later date. --- lib/PublicInbox/Git.pm | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm index 449223ec..eb5de159 100644 --- a/lib/PublicInbox/Git.pm +++ b/lib/PublicInbox/Git.pm @@ -204,14 +204,14 @@ sub cat_async_step ($$) { } else { $self->fail("Unexpected result from async git cat-file: $head"); } - eval { $cb->($bref, $oid, $type, $size, $arg) }; $self->{cat_rbuf} = $rbuf if $$rbuf ne ''; + eval { $cb->($bref, $oid, $type, $size, $arg) }; warn "E: $oid: $@\n" if $@; } sub cat_async_wait ($) { my ($self) = @_; - my $inflight = delete $self->{inflight} or return; + my $inflight = $self->{inflight} or return; while (scalar(@$inflight)) { cat_async_step($self, $inflight); } @@ -251,14 +251,14 @@ sub check_async_step ($$) { my $ret = my_read($self->{in_c}, $rbuf, $type + 1); fail($self, defined($ret) ? 'read EOF' : "read: $!") if !$ret; } + $self->{chk_rbuf} = $rbuf if $$rbuf ne ''; eval { $cb->($hex, $type, $size, $arg, $self) }; warn "E: check($req) $@\n" if $@; - $self->{chk_rbuf} = $rbuf if $$rbuf ne ''; } sub check_async_wait ($) { my ($self) = @_; - my $inflight_c = delete $self->{inflight_c} or return; + my $inflight_c = $self->{inflight_c} or return; while (scalar(@$inflight_c)) { check_async_step($self, $inflight_c); } @@ -318,13 +318,15 @@ sub _destroy { sub cat_async_abort ($) { my ($self) = @_; - if (my $inflight = delete $self->{inflight}) { + if (my $inflight = $self->{inflight}) { while (@$inflight) { my ($req, $cb, $arg) = splice(@$inflight, 0, 3); $req =~ s/ .*//; # drop git_dir for Gcf2Client eval { $cb->(undef, $req, undef, undef, $arg) }; warn "E: $req: $@ (in abort)\n" if $@; } + delete $self->{cat_rbuf}; + delete $self->{inflight}; } cleanup($self); } @@ -357,6 +359,8 @@ sub cleanup { delete $self->{async_cat}; check_async_wait($self); cat_async_wait($self); + delete $self->{inflight}; + delete $self->{inflight_c}; _destroy($self, qw(cat_rbuf in out pid)); _destroy($self, qw(chk_rbuf in_c out_c pid_c err_c)); !!($self->{pid} || $self->{pid_c});
next prev parent reply other threads:[~2020-10-16 7:00 UTC|newest] Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-10-16 6:59 [PATCH 01/64] inbox: add uidvalidity method Eric Wong 2020-10-16 6:59 ` [PATCH 02/64] git: ensure ->destroy clobbers check_async read buffer Eric Wong 2020-10-16 6:59 ` Eric Wong [this message] 2020-10-16 6:59 ` [PATCH 04/64] git: async: loop inflight checks for nested callbacks Eric Wong 2020-10-16 7:02 ` oops, :x was supposed to be 1/3 for git: stuff Eric Wong 2020-10-16 7:03 ` [PATCH 01/64] inbox: add uidvalidity method Eric Wong
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: http://public-inbox.org/README * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20201016070034.22204-3-e@80x24.org \ --to=e@80x24.org \ --cc=meta@public-inbox.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
user/dev discussion of public-inbox itself This inbox may be cloned and mirrored by anyone: git clone --mirror http://public-inbox.org/meta git clone --mirror http://czquwvybam4bgbro.onion/meta git clone --mirror http://hjrcffqmbrq6wope.onion/meta git clone --mirror http://ou63pmih66umazou.onion/meta # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V1 meta meta/ http://public-inbox.org/meta \ meta@public-inbox.org public-inbox-index meta Example config snippet for mirrors. Newsgroups are available over NNTP: nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta nntp://news.gmane.io/gmane.mail.public-inbox.general note: .onion URLs require Tor: https://www.torproject.org/ code repositories for the project(s) associated with this inbox: https://80x24.org/public-inbox.git AGPL code for this site: git clone https://public-inbox.org/public-inbox.git