From tokuhirom@gmail.com

When I'm implementing a http client, I need to read 1 packet data without expecting size. In socket programming context, this case is generic requirement.

In rakudo's src/core/IO/Socket.pm. It calls nqp​::readfh.

  method recv (Cool $chars = Inf, :$bin? = False) {
  fail('Socket not available') unless $!PIO;
  if $bin {
  nqp​::readfh($!PIO, nqp​::decont(buf8.new),
  $chars == Inf ?? 1048576 !! $chars.Int);
  }
  else {
  nqp​::p6box_s(nqp​::readcharsfh($!PIO,
  $chars == Inf ?? 1048576 !! $chars.Int));
  }
  }

It's mapped to read_fhb on MoarVM.

  QAST​::MASTOperations.add_core_moarop_mapping('readfh', 'read_fhb', 1);

see src/vm/moar/QAST/QASTOperationsMAST.nqp.

And then, it calls MVM_io_syncstream_read_bytes(scrc/io/syncstream.c). If there's no enough bytes in the buffer, MoarVM calls read_to_buffer, that calls uv_read_start -> uv_read_stop async. Main thread goes to MVM_string_decodestream_bytes_to_buf(src/strings/decode_stream.c). It will do a busy loop to read bytes to buffering the requested bytes.

I request to implement a way to read 1 packet.

From @geekosaur

On Fri, Oct 9, 2015 at 7​:51 PM, Tokuhiro Matsuno <
perl6-bugs-followup@​perl.org> wrote​:

I need to read 1 packet data without expecting size.

In TCP there is no such thing as 1 packet; TCP is a stream, you *cannot*
see the packet(s) underlying it unless you use a raw socket and implement
the TCP layer yourself. What are you actually looking for?

--
brandon s allbery kf8nh sine nomine associates
allbery.b@​gmail.com ballbery@​sinenomine.net
unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net

The RT System itself - Status changed from 'new' to 'open'

From @moritz

On Fri Oct 09 18​:41​:11 2015, allbery.b@​gmail.com wrote​:

On Fri, Oct 9, 2015 at 7​:51 PM, Tokuhiro Matsuno <
perl6-bugs-followup@​perl.org> wrote​:

I need to read 1 packet data without expecting size.

In TCP there is no such thing as 1 packet; TCP is a stream, you *cannot*
see the packet(s) underlying it unless you use a raw socket and implement
the TCP layer yourself. What are you actually looking for?

Probably "read everything that's in the buffer right now, and if nothing is in there, wait until something is". Which is what recv should actually do, IMHO. And the limit passed to recv is traditionally an upper limit, not an "at least".

From @jnthn

On Fri Oct 09 16​:51​:39 2015, tokuhirom wrote​:

When I'm implementing a http client, I need to read 1 packet data
without expecting size. In socket programming context, this case is
generic requirement.

This is what recv does.

In rakudo's src/core/IO/Socket.pm. It calls nqp​::readfh.

method recv (Cool $chars = Inf, :$bin? = False) {
fail('Socket not available') unless $!PIO;
if $bin {
nqp​::readfh($!PIO, nqp​::decont(buf8.new),
$chars == Inf ?? 1048576 !! $chars.Int);
}
else {
nqp​::p6box_s(nqp​::readcharsfh($!PIO,
$chars == Inf ?? 1048576 !! $chars.Int));
}
}

Correct so far. Note that unlike the implementation of read, there is no loop here.

It's mapped to read_fhb on MoarVM.

QAST​::MASTOperations.add_core_moarop_mapping('readfh', 'read_fhb', 1);

see src/vm/moar/QAST/QASTOperationsMAST.nqp.

And then, it calls
MVM_io_syncstream_read_bytes(scrc/io/syncstream.c). If there's no
enough bytes in the buffer, MoarVM calls read_to_buffer, that calls
uv_read_start -> uv_read_stop async. Main thread goes to
MVM_string_decodestream_bytes_to_buf(src/strings/decode_stream.c).
The code in question is this​:

  /* See if we've already enough; if not, try and grab more. */
  if (!MVM_string_decodestream_have_bytes(tc, data->ds, bytes))
  read_to_buffer(tc, data, bytes > CHUNK_SIZE ? bytes : CHUNK_SIZE);

  /* Read as many as we can, up to the limit. */
  return MVM_string_decodestream_bytes_to_buf(tc, data->ds, buf, bytes);

Note that there is no loop here. At most we issue one read_to_buffer, which does not contain a loop either. We wait until libuv gives us data exactly once.

It will do a busy loop to read bytes to buffering the requested bytes.

Structurally, a decode stream of itself cannot cause another I/O request to be made. It doesn't hold any I/O handles, just a linked list of byte buffers it can decode if it needs to. The loop inside of the decode stream you refer to looks like this​:

  while (taken < bytes && ds->bytes_head) {

The first part of the condition is about not handing back data beyond the limit. The second is making sure we terminate the loop when we've consumed all of the byte buffers already in the decode stream. Further, we can see the only calls in this function are to memcpy and MVM_free, so it certainly can't be causing more I/O.

In summary, I think recv is already doing what you want. To be sure, I wrote tests in S32-io/socket-recv-vs-read.t to make sure recv does not block, which also pass.

Thanks,

/jnthn

@jnthn - Status changed from 'open' to 'resolved'