user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 02/36] lei_store: use per-machine refname as git HEAD
Date: Thu, 31 Dec 2020 13:51:20 +0000	[thread overview]
Message-ID: <20201231135154.6070-3-e@80x24.org> (raw)
In-Reply-To: <20201231135154.6070-1-e@80x24.org>

It may be helpful to identify the source of messages
and perhaps avoid conflicting history.

On the other hand, this may be a terrible idea for users who
move portable storage (e.g. USB sticks) across computers...
---
 lib/PublicInbox/Import.pm   | 10 ++++++----
 lib/PublicInbox/LeiStore.pm | 21 ++++++++++++++++++++-
 2 files changed, 26 insertions(+), 5 deletions(-)

diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm
index 7258e848..60cff9c2 100644
--- a/lib/PublicInbox/Import.pm
+++ b/lib/PublicInbox/Import.pm
@@ -463,16 +463,18 @@ EOD
 EOC
 
 sub init_bare {
-	my ($dir) = @_; # or self
+	my ($dir, $head) = @_; # or self
 	$dir = $dir->{git}->{git_dir} if ref($dir);
 	require File::Path;
 	File::Path::mkpath([ map { "$dir/$_" } qw(objects/info refs/heads) ]);
 	$INIT_FILES[1] //= 'ref: '.default_branch."\n";
-	for (my $i = 0; $i < @INIT_FILES; $i++) {
-		my $f = $dir.'/'.$INIT_FILES[$i++];
+	my @fn_contents = @INIT_FILES;
+	$fn_contents[1] = "ref: refs/heads/$head\n" if defined $head;
+	while (my ($fn, $contents) = splice(@fn_contents, 0, 2)) {
+		my $f = $dir.'/'.$fn;
 		next if -f $f;
 		open my $fh, '>', $f or die "open $f: $!";
-		print $fh $INIT_FILES[$i] or die "print $f: $!";
+		print $fh $contents or die "print $f: $!";
 		close $fh or die "close $f: $!";
 	}
 }
diff --git a/lib/PublicInbox/LeiStore.pm b/lib/PublicInbox/LeiStore.pm
index 553adbc8..a17c7bab 100644
--- a/lib/PublicInbox/LeiStore.pm
+++ b/lib/PublicInbox/LeiStore.pm
@@ -60,6 +60,24 @@ sub git_ident ($) {
 		('lei user', 'x@example.com')
 }
 
+# We will support users combining storage across multiple machines
+# somehow.  Use per-machine refnames to make it easy-to-identify
+# where a message came from
+sub host_head () {
+	state $h = do {
+		my $x = PublicInbox::ExtSearchIdx::host_ident;
+		# Similar rules found in git.git/remote.c::valid_remote_nick
+		# and git.git/refs.c::check_refname_component
+		$x =~ s!(?:\.lock|/)+\z!!gs; # must not end with ".lock" or "/"
+		$x =~ tr/././s; # no dot-dot, collapse them
+		$x =~ s/@\{/\@-/gs;
+		$x =~ s/\A\./-/s;
+		# no "*", ":", "?", "[", "\", "^", "~", SP, TAB; "]" is OK
+		$x =~ tr^a-zA-Z0-9!"#$%&'()+,\-.;<=>@]_`{|}^-^c;
+		$x
+	};
+}
+
 sub importer {
 	my ($self) = @_;
 	my $max;
@@ -78,8 +96,8 @@ sub importer {
 	while (1) {
 		my $latest = "$pfx/$max.git";
 		my $old = -e $latest;
+		PublicInbox::Import::init_bare($latest, host_head);
 		my $git = PublicInbox::Git->new($latest);
-		PublicInbox::Import::init_bare({ git => $git });
 		$git->qx(qw(config core.sharedRepository 0600)) if !$old;
 		my $packed_bytes = $git->packed_bytes;
 		my $unpacked_bytes = $packed_bytes / $self->packing_factor;
@@ -92,6 +110,7 @@ sub importer {
 		$im->{bytes_added} = int($packed_bytes / $self->packing_factor);
 		$im->{lock_path} = undef;
 		$im->{path_type} = 'v2';
+		$im->{'ref'} = host_head;
 		return $im;
 	}
 }

  parent reply	other threads:[~2020-12-31 13:51 UTC|newest]

Thread overview: 37+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-31 13:51 [PATCH 00/36] another round of lei stuff Eric Wong
2020-12-31 13:51 ` [PATCH 01/36] import: respect init.defaultBranch Eric Wong
2020-12-31 13:51 ` Eric Wong [this message]
2020-12-31 13:51 ` [PATCH 03/36] revert "lei_store: use per-machine refname as git HEAD" Eric Wong
2020-12-31 13:51 ` [PATCH 04/36] lei_to_mail: initial implementation for writing mbox formats Eric Wong
2020-12-31 13:51 ` [PATCH 05/36] sharedkv: fork()-friendly key-value store Eric Wong
2020-12-31 13:51 ` [PATCH 06/36] sharedkv: split out index_values Eric Wong
2020-12-31 13:51 ` [PATCH 07/36] lei_to_mail: start atomic and compressed mbox writing Eric Wong
2020-12-31 13:51 ` [PATCH 08/36] mboxreader: new class for reading various mbox formats Eric Wong
2020-12-31 13:51 ` [PATCH 09/36] lei_to_mail: start --augment, dedupe, bz2 and xz Eric Wong
2020-12-31 13:51 ` [PATCH 10/36] lei: implement various deduplication strategies Eric Wong
2020-12-31 13:51 ` [PATCH 11/36] lei_to_mail: lazy-require LeiDedupe Eric Wong
2020-12-31 13:51 ` [PATCH 12/36] lei_to_mail: support for non-seekable outputs Eric Wong
2020-12-31 13:51 ` [PATCH 13/36] lei_to_mail: support Maildir, fix+test --augment Eric Wong
2020-12-31 13:51 ` [PATCH 14/36] ipc: generic IPC dispatch based on Storable Eric Wong
2020-12-31 13:51 ` [PATCH 15/36] ipc: support Sereal Eric Wong
2020-12-31 13:51 ` [PATCH 16/36] lei_store: add ->set_eml, ->add_eml can return smsg Eric Wong
2020-12-31 13:51 ` [PATCH 17/36] lei: rename "extinbox" => "external" Eric Wong
2020-12-31 13:51 ` [PATCH 18/36] mid: use defined-or with `push' for uniqueness check Eric Wong
2020-12-31 13:51 ` [PATCH 19/36] mid: hoist out mids_in sub Eric Wong
2020-12-31 13:51 ` [PATCH 20/36] lei_store: handle messages without Message-ID at all Eric Wong
2020-12-31 13:51 ` [PATCH 21/36] ipc: use shutdown(2), base atfork* callback Eric Wong
2020-12-31 13:51 ` [PATCH 22/36] lei_to_mail: unlink mboxes if not augmenting Eric Wong
2020-12-31 13:51 ` [PATCH 23/36] lei: add --mfolder as an --output alias Eric Wong
2020-12-31 13:51 ` [PATCH 24/36] spawn: move run_die here from PublicInbox::Import Eric Wong
2020-12-31 13:51 ` [PATCH 25/36] init: remove embedded UnlinkMe package Eric Wong
2020-12-31 13:51 ` [PATCH 26/36] t/run: avoid uninitialized var on incomplete test Eric Wong
2020-12-31 13:51 ` [PATCH 27/36] gcf2client: reap process on DESTROY Eric Wong
2020-12-31 13:51 ` [PATCH 28/36] lei_to_mail: open FIFOs O_WRONLY so we block Eric Wong
2020-12-31 13:51 ` [PATCH 29/36] searchidxshard: call DS->Reset at worker start Eric Wong
2020-12-31 13:51 ` [PATCH 30/36] t/ipc.t: test for references via `die' Eric Wong
2020-12-31 13:51 ` [PATCH 31/36] use PublicInbox::DS for dwaitpid Eric Wong
2020-12-31 13:51 ` [PATCH 32/36] syscall: SFD_NONBLOCK can be a constant, again Eric Wong
2020-12-31 13:51 ` [PATCH 33/36] lei: avoid Spawn package when starting daemon Eric Wong
2020-12-31 13:51 ` [PATCH 34/36] avoid calling waitpid from children in DESTROY Eric Wong
2020-12-31 13:51 ` [PATCH 35/36] ds: clobber $in_loop first at reset Eric Wong
2020-12-31 13:51 ` [PATCH 36/36] on_destroy: support PID owner guard Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201231135154.6070-3-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).