From @haukex

Hi all,

On Windows, doing `PerlIO​::get_layers($filehandle)` will always report
the :crlf layer, even when one has done `binmode $filehandle` and the
:crlf layer isn't active. The attached script demonstrates this; note I
also did a hexdump of the file to make sure that it indeed only contains
an LF, not CRLF.

The PerlIO documentation does say that :crlf is part of the "default
stack" on Windows, but I don't see an indication in the documentation of
why PerlIO​::get_layers() would *always* report that layer, even when
it's not active.

I tested this on Strawberry Perl 5.8.9 through 5.28 on Windows 7. There
is one minor difference​: on Perls before 5.14, using the :raw
pseudolayer directly in the `open` actually does cause the :crlf layer
to no longer be reported by PerlIO​::get_layers(). perl5140delta does
mention several changes to PerlIO, but AFAICT nothing related to the
aforementioned issue.

This definitely feels like a bug to me, but I don't know enough about
the details of PerlIO on Windows to say more at the moment.

Thanks,
-- Hauke D

From @haukex

winbug.pl

From @Leont

On Sat, Sep 28, 2019 at 11​:05 AM Hauke D (via RT)
<perlbug-followup@​perl.org> wrote​:

Hi all,

On Windows, doing `PerlIO​::get_layers($filehandle)` will always report
the :crlf layer, even when one has done `binmode $filehandle` and the
:crlf layer isn't active. The attached script demonstrates this; note I
also did a hexdump of the file to make sure that it indeed only contains
an LF, not CRLF.

The PerlIO documentation does say that :crlf is part of the "default
stack" on Windows, but I don't see an indication in the documentation of
why PerlIO​::get_layers() would *always* report that layer, even when
it's not active.

Yeah, this is confusing behavior.

:crlf is not just a translation layer on top of the buffering system,
it's an alternative buffering system that will optionally do the
translation. Calling binmode will disable that translation but not the
layer.

For extra confusion, on Linux binmode will pop off the crlf layer,
because it's not presumed to be the only buffering layer.

I presume this is implemented the way it is because of performance reasons.

I tested this on Strawberry Perl 5.8.9 through 5.28 on Windows 7. There
is one minor difference​: on Perls before 5.14, using the :raw
pseudolayer directly in the `open` actually does cause the :crlf layer
to no longer be reported by PerlIO​::get_layers(). perl5140delta does
mention several changes to PerlIO, but AFAICT nothing related to the
aforementioned issue.

Yeah, that was a serious bug because that meant no buffering would
happen anymore (which completely kills readline performance as without
buffering it will have to read byte by byte).

This definitely feels like a bug to me, but I don't know enough about
the details of PerlIO on Windows to say more at the moment.

It's mainly terribly confusing, but AFAIK it does do the right thing.
Though I do think it's possible to fix this.

Leon

The RT System itself - Status changed from 'new' to 'open'

From @haukex

Hi,

Thanks for the quick reply and the explanation!

It's mainly terribly confusing, but AFAIK it does do the right thing.
Though I do think it's possible to fix this.

I think that would be nice. AFAICT from the documentation, :utf8 is also just a flag, not really a layer, and in any case, seeing :crlf in the layer list is confusing, and I don't know of any other way to ask PerlIO whether CRLF translation is turned on or off?

:crlf is not just a translation layer on top of the buffering system,
it's an alternative buffering system that will optionally do the
translation. Calling binmode will disable that translation but not the
layer.

If it's important that this buffering system be reported by get_layers(), perhaps it could be reported as a separate pseudolayer? I.e. get_layers() returns either `("unix","winbuf","crlf")` or `("unix","winbuf")` depending on whether CRLF translation is actually turned on or off (there's probably a better name than "winbuf").

Best,
-- Hauke D

On Sat, 28 Sep 2019 02​:50​:45 -0700, LeonT wrote​:

On Sat, Sep 28, 2019 at 11​:05 AM Hauke D (via RT)
<perlbug-followup@​perl.org> wrote​:

Hi all,

On Windows, doing `PerlIO​::get_layers($filehandle)` will always report
the :crlf layer, even when one has done `binmode $filehandle` and the
:crlf layer isn't active. The attached script demonstrates this; note I
also did a hexdump of the file to make sure that it indeed only contains
an LF, not CRLF.

The PerlIO documentation does say that :crlf is part of the "default
stack" on Windows, but I don't see an indication in the documentation of
why PerlIO​::get_layers() would *always* report that layer, even when
it's not active.

Yeah, this is confusing behavior.

:crlf is not just a translation layer on top of the buffering system,
it's an alternative buffering system that will optionally do the
translation. Calling binmode will disable that translation but not the
layer.

For extra confusion, on Linux binmode will pop off the crlf layer,
because it's not presumed to be the only buffering layer.

I presume this is implemented the way it is because of performance reasons.

I tested this on Strawberry Perl 5.8.9 through 5.28 on Windows 7. There
is one minor difference​: on Perls before 5.14, using the :raw
pseudolayer directly in the `open` actually does cause the :crlf layer
to no longer be reported by PerlIO​::get_layers(). perl5140delta does
mention several changes to PerlIO, but AFAICT nothing related to the
aforementioned issue.

Yeah, that was a serious bug because that meant no buffering would
happen anymore (which completely kills readline performance as without
buffering it will have to read byte by byte).

This definitely feels like a bug to me, but I don't know enough about
the details of PerlIO on Windows to say more at the moment.

It's mainly terribly confusing, but AFAIK it does do the right thing.
Though I do think it's possible to fix this.

Leon