user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Kyle Meyer <kyle@kyleam.com>
To: meta@public-inbox.org
Cc: piem@inbox.kyleam.com
Subject: lei q: importing messages when specifying '-f *json*'?
Date: Thu, 30 Dec 2021 18:04:23 -0500	[thread overview]
Message-ID: <87k0fly9s8.fsf@kyleam.com> (raw)

The lei-q interface I'm working on for piem [1] consumes JSON output to
display search results.  From there, an individual message can be shown.
If the query is against a remote external, that involves two curl calls
to the remote, one for the search and one for the display.

I'd like to import hits from the first step into the local store so that
I can avoid the second curl call.  However, I haven't been able to
figure out a way to do this when requesting JSON output.

For example [2], say I don't have meta's 20211124154539.350522-1-e@80x24.org
locally:

  $ lei daemon-kill
  $ export HOME=$(mktemp -d "${TMPDIR:-/tmp}"/pi-testing-XXXXXXX)
  $ lei q m:20211124154539.350522-1-e@80x24.org
  # /tmp/pi-testing-3JaSz2K/.config/lei/config created
  [null]

If I search for that message against https://public-inbox.org/meta/ and
request JSON output

  $ lei q -I https://public-inbox.org/meta/ -f ldjson \
    m:20211124154539.350522-1-e@80x24.org
  # /usr/bin/curl -Sf -s -d '' https://public-inbox.org/meta/?x=m&q=m%3A20211124154539.350522-1-e%4080x24.org
  {"blob":"a8754283bd9e985d6e1156215071be59aa2b5a53",...}

then no message ends up in the local store:

  $ lei q m:20211124154539.350522-1-e@80x24.org
  [null]

In contrast, if I request mboxrd output

  $ lei q -I https://public-inbox.org/meta/ -f mboxrd \
    m:20211124154539.350522-1-e@80x24.org >/dev/null
  # /usr/bin/curl -Sf -s -d '' https://public-inbox.org/meta/?x=m&q=m%3A20211124154539.350522-1-e%4080x24.org

the message is imported to the local store:

  $ lei q -f ldjson m:20211124154539.350522-1-e@80x24.org
  {"blob":"a8754283bd9e985d6e1156215071be59aa2b5a53",...}
  $ git -C $HOME/.local/share/lei/store/local/0.git/ log --oneline
  ca12a1b (HEAD -> master) [PATCH] eliminate some unused subs

I was hoping that --import-remote might do the trick, but that doesn't
seem to be the case:

  $ lei daemon-kill
  $ export HOME=$(mktemp -d "${TMPDIR:-/tmp}"/pi-testing-XXXXXXX)
  $ lei q --import-remote -I https://public-inbox.org/meta/ \
    -f ldjson m:20211124154539.350522-1-e@80x24.org
  # /tmp/pi-testing-Ny7KDcB/.config/lei/config created
  # /usr/bin/curl -Sf -s -d '' https://public-inbox.org/meta/?x=m&q=m%3A20211124154539.350522-1-e%4080x24.org
  {"blob":"a8754283bd9e985d6e1156215071be59aa2b5a53",...}
  $ lei q m:20211124154539.350522-1-e@80x24.org
  [null]

Should --import-remote trigger an import in the case above?


[1] https://git.kyleam.com/piem/tree/piem-lei.el
[2] These are with public-inbox's current master (07cd8973baf).

                 reply	other threads:[~2021-12-30 23:04 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87k0fly9s8.fsf@kyleam.com \
    --to=kyle@kyleam.com \
    --cc=meta@public-inbox.org \
    --cc=piem@inbox.kyleam.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).