git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Jeff Hostetler via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: Jeff Hostetler <jeffhost@microsoft.com>,
	Jeff Hostetler <jeffhost@microsoft.com>
Subject: [PATCH 8/8] simple-ipc: update design documentation with more details
Date: Thu, 04 Mar 2021 20:17:27 +0000	[thread overview]
Message-ID: <eadde22c735e45e114776bc6e2e3affecb14cdcb.1614889047.git.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.893.git.1614889047.gitgitgadget@gmail.com>

From: Jeff Hostetler <jeffhost@microsoft.com>

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
---
 Documentation/technical/api-simple-ipc.txt | 131 ++++++++++++++++-----
 1 file changed, 101 insertions(+), 30 deletions(-)

diff --git a/Documentation/technical/api-simple-ipc.txt b/Documentation/technical/api-simple-ipc.txt
index 670a5c163e39..d79ad323e675 100644
--- a/Documentation/technical/api-simple-ipc.txt
+++ b/Documentation/technical/api-simple-ipc.txt
@@ -1,34 +1,105 @@
-simple-ipc API
+Simple-IPC API
 ==============
 
-The simple-ipc API is used to send an IPC message and response between
-a (presumably) foreground Git client process to a background server or
-daemon process.  The server process must already be running.  Multiple
-client processes can simultaneously communicate with the server
-process.
+The Simple-IPC API is a collection of `ipc_` prefixed library routines
+and a basic communication protocol that allow an IPC-client process to
+send an application-specific IPC-request message to an IPC-server
+process and receive an application-specific IPC-response message.
 
 Communication occurs over a named pipe on Windows and a Unix domain
-socket on other platforms.  Clients and the server rendezvous at a
-previously agreed-to application-specific pathname (which is outside
-the scope of this design).
-
-This IPC mechanism differs from the existing `sub-process.c` model
-(Documentation/technical/long-running-process-protocol.txt) and used
-by applications like Git-LFS.  In the simple-ipc model the server is
-assumed to be a very long-running system service.  In contrast, in the
-LFS-style sub-process model the helper is started with the foreground
-process and exits when the foreground process terminates.
-
-How the simple-ipc server is started is also outside the scope of the
-IPC mechanism.  For example, the server might be started during
-maintenance operations.
-
-The IPC protocol consists of a single request message from the client and
-an optional request message from the server.  For simplicity, pkt-line
-routines are used to hide chunking and buffering concerns.  Each side
-terminates their message with a flush packet.
-(Documentation/technical/protocol-common.txt)
-
-The actual format of the client and server messages is application
-specific.  The IPC layer transmits and receives an opaque buffer without
-any concern for the content within.
+socket on other platforms.  IPC-clients and IPC-servers rendezvous at
+a previously agreed-to application-specific pathname (which is outside
+the scope of this design) that is local to the computer system.
+
+The IPC-server routines within the server application process create a
+thread pool to listen for connections and receive request messages
+from multiple concurrent IPC-clients.  When received, these messages
+are dispatched up to the server application callbacks for handling.
+IPC-server routines then incrementally relay responses back to the
+IPC-client.
+
+The IPC-client routines within a client application process connect
+to the IPC-server and send a request message and wait for a response.
+When received, the response is returned back the caller.
+
+For example, the `fsmonitor--daemon` feature will be built as a server
+application on top of the IPC-server library routines.  It will have
+threads watching for file system events and a thread pool waiting for
+client connections.  Clients, such as `git status` will request a list
+of file system events since a point in time and the server will
+respond with a list of changed files and directories.  The formats of
+the request and response are application-specific; the IPC-client and
+IPC-server routines treat them as opaque byte streams.
+
+
+Comparison with sub-process model
+---------------------------------
+
+The Simple-IPC mechanism differs from the existing `sub-process.c`
+model (Documentation/technical/long-running-process-protocol.txt) and
+used by applications like Git-LFS.  In the LFS-style sub-process model
+the helper is started by the foreground process, communication happens
+via a pair of file descriptors bound to the stdin/stdout of the
+sub-process, the sub-process only serves the current foreground
+process, and the sub-process exits when the foreground process
+terminates.
+
+In the Simple-IPC model the server is a very long-running service.  It
+can service many clients at the same time and has a private socket or
+named pipe connection to each active client.  It might be started
+(on-demand) by the current client process or it might have been
+started by a previous client or by the OS at boot time.  The server
+process is not associated with a terminal and it persists after
+clients terminate.  Clients do not have access to the stdin/stdout of
+the server process and therefore must communicate over sockets or
+named pipes.
+
+
+Server startup and shutdown
+---------------------------
+
+How an application server based upon IPC-server is started is also
+outside the scope of the Simple-IPC design and is a property of the
+application using it.  For example, the server might be started or
+restarted during routine maintenance operations, or it might be
+started as a system service during the system boot-up sequence, or it
+might be started on-demand by a foreground Git command when needed.
+
+Similarly, server shutdown is a property of the application using
+the simple-ipc routines.  For example, the server might decide to
+shutdown when idle or only upon explicit request.
+
+
+Simple-IPC protocol
+-------------------
+
+The Simple-IPC protocol consists of a single request message from the
+client and an optional response message from the server.  Both the
+client and server messages are unlimited in length and are terminated
+with a flush packet.
+
+The pkt-line routines (Documentation/technical/protocol-common.txt)
+are used to simplify buffer management during message generation,
+transmission, and reception.  A flush packet is used to mark the end
+of the message.  This allows the sender to incrementally generate and
+transmit the message.  It allows the receiver to incrementally receive
+the message in chunks and to know when they have received the entire
+message.
+
+The actual byte format of the client request and server response
+messages are application specific.  The IPC layer transmits and
+receives them as opaque byte buffers without any concern for the
+content within.  It is the job of the calling application layer to
+understand the contents of the request and response messages.
+
+
+Summary
+-------
+
+Conceptually, the Simple-IPC protocol is similar to an HTTP REST
+request.  Clients connect, make an application-specific and
+stateless request, receive an application-specific
+response, and disconnect.  It is a one round trip facility for
+querying the server.  The Simple-IPC routines hide the socket,
+named pipe, and thread pool details and allow the application
+layer to focus on the application at hand.
-- 
gitgitgadget

  parent reply	other threads:[~2021-03-04 20:21 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-04 20:17 [PATCH 0/8] Simple IPC Cleanups Jeff Hostetler via GitGitGadget
2021-03-04 20:17 ` [PATCH 1/8] pkt-line: remove buffer arg from write_packetized_from_fd_no_flush() Jeff Hostetler via GitGitGadget
2021-03-04 22:55   ` Junio C Hamano
2021-03-04 20:17 ` [PATCH 2/8] unix-socket: simplify initialization of unix_stream_listen_opts Jeff Hostetler via GitGitGadget
2021-03-04 23:12   ` Junio C Hamano
2021-03-04 20:17 ` [PATCH 3/8] unix-stream-server: create unix-stream-server.c Jeff Hostetler via GitGitGadget
2021-03-04 20:17 ` [PATCH 4/8] simple-ipc: move error handling up a level Jeff Hostetler via GitGitGadget
2021-03-04 20:17 ` [PATCH 5/8] unix-stream-server: add st_dev and st_mode to socket stolen checks Jeff Hostetler via GitGitGadget
2021-03-06 11:42   ` René Scharfe
2021-03-08 14:14     ` Jeff Hostetler
2021-03-04 20:17 ` [PATCH 6/8] test-simple-ipc: refactor command line option processing in helper Jeff Hostetler via GitGitGadget
2021-03-04 20:17 ` [PATCH 7/8] test-simple-ipc: add --token=<token> string option Jeff Hostetler via GitGitGadget
2021-03-04 20:17 ` Jeff Hostetler via GitGitGadget [this message]
2021-03-05  0:24 ` [PATCH 0/8] Simple IPC Cleanups Junio C Hamano
2021-03-05 21:34   ` Jeff Hostetler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=eadde22c735e45e114776bc6e2e3affecb14cdcb.1614889047.git.gitgitgadget@gmail.com \
    --to=gitgitgadget@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=jeffhost@microsoft.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).