* [PATCH 14/15] lei_mail_sync: mv_src: use transaction, check UNIQUE
2021-10-21 21:10 6% [PATCH 00/15] use RENAME_NOREPLACE on Linux 3.15+ Eric Wong
@ 2021-10-21 21:10 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2021-10-21 21:10 UTC (permalink / raw)
To: meta
We need a transaction across two SQL statements so readers
(which don't use flock) will see the result as atomic.
This may help against some occasional test failures I'm seeing
from t/lei-auto-watch.t and t/lei-watch.t, or make the problem
more apparent.
---
lib/PublicInbox/LeiMailSync.pm | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/lib/PublicInbox/LeiMailSync.pm b/lib/PublicInbox/LeiMailSync.pm
index e70cb5de2b6b..124eb96948c5 100644
--- a/lib/PublicInbox/LeiMailSync.pm
+++ b/lib/PublicInbox/LeiMailSync.pm
@@ -182,16 +182,20 @@ sub mv_src {
my ($self, $folder, $oidbin, $id, $newbn) = @_;
my $lk = $self->lock_for_scope;
my $fid = $self->{fmap}->{$folder} //= fid_for($self, $folder, 1);
+ $self->{dbh}->begin_work;
my $sth = $self->{dbh}->prepare_cached(<<'');
UPDATE blob2name SET name = ? WHERE fid = ? AND oidbin = ? AND name = ?
- my $nr = $sth->execute($newbn, $fid, $oidbin, $$id);
- if ($nr == 0) { # may race with a clear_src, ensure new value exists
+ # eval since unique constraint may fail due to race
+ my $nr = eval { $sth->execute($newbn, $fid, $oidbin, $$id) };
+ if (!defined($nr) || $nr == 0) { # $nr may be `0E0'
+ # may race with a clear_src, ensure new value exists
$sth = $self->{dbh}->prepare_cached(<<'');
INSERT OR IGNORE INTO blob2name (oidbin, fid, name) VALUES (?, ?, ?)
$sth->execute($oidbin, $fid, $newbn);
}
+ $self->{dbh}->commit;
}
# read-only, iterates every oidbin + UID or name for a given folder
^ permalink raw reply related [relevance 7%]
* [PATCH 00/15] use RENAME_NOREPLACE on Linux 3.15+
@ 2021-10-21 21:10 6% Eric Wong
2021-10-21 21:10 7% ` [PATCH 14/15] lei_mail_sync: mv_src: use transaction, check UNIQUE Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2021-10-21 21:10 UTC (permalink / raw)
To: meta
What started off as a trivial, "something-to-do-before-bedtime"
change several days ago (15/15) ended up forcing me to fix
various hard-to-reproduce race conditions around Maildirs.
Eventually, `schedtool -a 0x1 -e ...' helped me reproduce
some races more easily (but still not with 100% reliability).
t/lei-{watch,auto-watch,export-kw}.t should all be more
reliable, now.
There's also several other minor fixes I found along the way...
Eric Wong (15):
t/lei-{auto-watch,export-kw}: extra diagnostics on failure
t/lei-import-maildir: rename fix (SR -> RS)
t/lei-p2q: extra diagnostics
lei/store: check for any unexpected process death
lei note-event: drop unnecessary eval guard
lei note-event: wq_io_do => wq_do
lei_search: try harder to associate "lei index"-ed messages
watch: check for {quit} before IDLE
watch: remove redundant signal mask manipulation
doc: lei-overview: add CAVEATS section
lei note-event: clear_src on ENOENT
dir_idle: treat IN_MOVED_FROM as a gone event
lei: no Perl FileHandle for `undef' w/ ECONNRESET
lei_mail_sync: mv_src: use transaction, check UNIQUE
lei: use RENAME_NOREPLACE on Linux 3.15+
Documentation/lei-overview.pod | 5 ++++
MANIFEST | 1 +
devel/syscall-list | 8 +++++-
lib/PublicInbox/DirIdle.pm | 3 +-
lib/PublicInbox/FakeInotify.pm | 3 ++
lib/PublicInbox/LEI.pm | 3 +-
lib/PublicInbox/LeiExportKw.pm | 19 ++++---------
lib/PublicInbox/LeiMailSync.pm | 8 ++++--
lib/PublicInbox/LeiNoteEvent.pm | 13 +++++----
lib/PublicInbox/LeiSearch.pm | 13 ++++++++-
lib/PublicInbox/LeiStore.pm | 16 +++++++----
lib/PublicInbox/LeiToMail.pm | 7 ++---
lib/PublicInbox/Syscall.pm | 49 +++++++++++++++++++++++++++++++--
lib/PublicInbox/Watch.pm | 13 +++------
t/lei-auto-watch.t | 3 +-
t/lei-export-kw.t | 39 ++++++++++++++++++--------
t/lei-import-maildir.t | 2 +-
t/lei-p2q.t | 2 +-
t/rename_noreplace.t | 26 +++++++++++++++++
19 files changed, 173 insertions(+), 60 deletions(-)
create mode 100644 t/rename_noreplace.t
^ permalink raw reply [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2021-10-21 21:10 6% [PATCH 00/15] use RENAME_NOREPLACE on Linux 3.15+ Eric Wong
2021-10-21 21:10 7% ` [PATCH 14/15] lei_mail_sync: mv_src: use transaction, check UNIQUE Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).