git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Pratyush Yadav <me@yadavpratyush.com>
To: Farhan Khan <farhan@farhan.codes>
Cc: Git <git@vger.kernel.org>
Subject: Re: How to determine when to stop receiving pack content
Date: Mon, 12 Aug 2019 05:52:44 +0530	[thread overview]
Message-ID: <20190812002244.lvsokkxcpfshjiyq@localhost.localdomain> (raw)
In-Reply-To: <53515e0fb56928114a63bfe87e24c743@farhan.codes>

On 11/08/19 11:31PM, Farhan Khan wrote:
> August 11, 2019 11:04 AM, "Pratyush Yadav" <me@yadavpratyush.com> wrote:
> 
> > On 10/08/19 11:47PM, Farhan Khan wrote:
> > 
> >> Hi,
> >> 
> >> I am trying to write an implementation of git clone over ssh and am a little confused how to
> >> determine a server response has ended. Specifically, after a client sends its requested 'want', the
> >> server sends the pack content over. However, how does the client know to stop reading data? If I
> >> run a simple read() of the file descriptor:
> >> 
> >> A. If I use reading blocking, the client will wait until new data is available, potentially
> >> forever.
> >> B. If I use non-blocking, the client might terminate reading for new data, when in reality new data
> >> is in transit.
> >> 
> >> I do not see a mechanism to specify the size or to indicate the end of the pack content. Am I
> >> missing something?
> > 
> > Well, I am not very familiar with git-clone internals, but I did some
> > digging around, and I think I know what answer to your problem is.
> > 
> > Looking at Documentation/technical/protocol-v2.txt:34, the flush packet
> > indicates the end of a message. Looking in the output section of the
> > fetch command (protocol-v2.txt:342), it sends you some optional
> > sections, and then the packfile and then sends a flush packet.
> > 
> > So your read should stop reading data when it sees the flush packet.
> > 
> > Another way would be to look at the packfile contents. Looking at
> > Documentation/technical/pack-format.txt, the packfile contains the
> > number of objects in the packfile, and each object entry has the object
> > size. So you can stop reading after you have received the last object in
> > the packfile (plus the 20-byte trailer).
> > 
> > I don't know which is the better way, but the former seems like a better
> > choice to me.
> > 
> > --
> > Regards,
> > Pratyush Yadav
> 
> Hi Pratyush,
> 
> Thanks for your reply!
> 
> Unless I am mistaken, a pack file does not end in a flush_pkt. I ran some tests and did not see the stream end in "0000". Is there is a mistake somewhere on my end?
 
Hm, turns out I was on the pu branch, not master when I looked at 
protocol-v2.txt. The file was updated about 3 days ago (not in master 
yet) (7ee4ab7e8c3310fc28d9dd7d47da26e497e73556), where it seems to imply 
that flush-pkt will be sent after the packfile (see excerpt below).

--- protocol-v2.txt ---
    output = acknowledgements flush-pkt |
	     [acknowledgments delim-pkt] [shallow-info delim-pkt]
	     [wanted-refs delim-pkt] [packfile-uris delim-pkt]
	     packfile flush-pkt
---

So either something changed in the protocol with that merge, or there is 
a discrepancy in the documentation, because the above output format 
seems to imply the packfile will be followed by a flush packet. I 
haven't looked at the full contents of the merge, but none of the commit 
messages mention changing this behaviour.

Either way, you can probably parse the packfile to know how many objects 
you will get, and stop after the last object. Or like Junio said, just 
wait for an EOF.

Sorry for the wrong information.

-- 
Regards,
Pratyush Yadav

      reply	other threads:[~2019-08-12  0:22 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-10 23:47 How to determine when to stop receiving pack content Farhan Khan
2019-08-11 15:04 ` Pratyush Yadav
2019-08-11 22:38 ` Junio C Hamano
2019-08-11 23:31 ` Farhan Khan
2019-08-12  0:22   ` Pratyush Yadav [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190812002244.lvsokkxcpfshjiyq@localhost.localdomain \
    --to=me@yadavpratyush.com \
    --cc=farhan@farhan.codes \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).