Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

methods for accessing binary data in Buf objects #3561

Open
p6rt opened this issue Oct 20, 2014 · 4 comments
Open

methods for accessing binary data in Buf objects #3561

p6rt opened this issue Oct 20, 2014 · 4 comments

Comments

@p6rt
Copy link

p6rt commented Oct 20, 2014

Migrated from rt.perl.org#123015 (status was 'open')

Searchable as RT123015$

@p6rt
Copy link
Author

p6rt commented Oct 20, 2014

From alex.hartmaier@gmail.com

As discussed on IRC mainly with moritz I'd need a way to get the number of bytes, not elements, of a Buf object so it can be looped and each byte accessed with $buf.[$idx].
I'm requiring this in DBDish​::Oracle for passing UTF-16 encoded values and their byte-length to the OCI C library using NativeCall.

@p6rt
Copy link
Author

p6rt commented Oct 20, 2014

From @moritz

On Mon Oct 20 06​:54​:16 2014, abraxxa wrote​:

As discussed on IRC mainly with moritz I'd need a way to get the
number of bytes, not elements, of a Buf object so it can be looped and
each byte accessed with $buf.[$idx].
I'm requiring this in DBDish​::Oracle for passing UTF-16 encoded values
and their byte-length to the OCI C library using NativeCall.

Another use case​: cryptography, which typically works on the byte level, even for multi-byte encodings.

@p6rt
Copy link
Author

p6rt commented Sep 2, 2017

From @skids

On Mon, 20 Oct 2014 07​:01​:07 -0700, moritz wrote​:

On Mon Oct 20 06​:54​:16 2014, abraxxa wrote​:

As discussed on IRC mainly with moritz I'd need a way to get the
number of bytes, not elements, of a Buf object so it can be looped
and
each byte accessed with $buf.[$idx].
I'm requiring this in DBDish​::Oracle for passing UTF-16 encoded
values
and their byte-length to the OCI C library using NativeCall.

Another use case​: cryptography, which typically works on the byte
level, even for multi-byte encodings.

A lot of time has passed and we have has the first part for quite some time.

$ perl6 -e '.bytes.say for buf8.new(1,2,3), buf16.new(1,2,3), buf32.new(1,2,3), buf64.new(1,2,3)'
3
6
12
24

...which is both specced and already tested. Also as a Container type
you get .of​:

$ perl6 -e '.of.say for Buf[int8].new(1,2,3), Buf[uint16].new(1,2,3), Buf[int32], Buf[uint64]'
(int8)
(uint16)
(int32)
(uint64)

...and the native types can be nativesizeof'd

$ perl6 -e 'use NativeCall; nativesizeof(.of).say for Buf[int8].new(1,2,3), Buf[uint16].new(1,2,3), Buf[int32], Buf[uint64]'
1
2
4
8

Also there are ways to finagle the second need with NativeCall​:

$ perl6 -e 'use NativeCall; my $b16 = buf16.new(1,2,3); my $b8 = nativecast(CArray[uint8], $b16); $b8[^6].say'
(1 0 2 0 3 0)

...though internally this relies on some GC manipulations and the
details on exactly if/when it becomes fragile aren't documented.

Granted a lot more could be done to make this, and endianness conversions easier,
e.g. safe and optimized indexing adverbs like buf32.new(1,2)[^8]​:swab8 might be
nice, as well as having types that carry their endianness information around with them,
but I think that falls in the 6.d-and-later design discussion category rather than RT.

So I'd vote to 'resolve' this ticket, and maybe if the next person who picks
it up agrees, they should do so.

@p6rt
Copy link
Author

p6rt commented Sep 2, 2017

The RT System itself - Status changed from 'new' to 'open'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant