The :crlf PerlIO layer doesn't like the :encoding layer. #8325
Comments
From ciaran@tnauk.org.ukCreated by ciaran@tnauk.org.uk(note: This report was compiled with perlbug, then sent manually with There seems to be a problem in the way that the ":crlf" PerlIO layer is * Perl v5.8.7, Cygwin (prepackaged) Here is a test case: ===CUT HERE=== open(FILE, ">:encoding(UTF-16):crlf", "test-file"); This should generate a valid UTF-16 file , containing "Test On a standard version of Perl, what actually happens is that I get a
and the pound sign is replaced with a null character. Using a literal One workaround seems to be to install PerlIO::eol from CPAN and replace I have not mentioned ActiveState Perl in this report as I assume it is Thank you. Following is the output of "perl5.9.1 -V": ===CUT HERE=== Characteristics of this binary (from libperl): Perl Info |
From @LeontI've attached a possible fix for this bug. The patch is relative to my Currently, if a «:crlf» layer is given it will first try to (re)enable Testing it would be very welcome, in particular on Windows (not my habitat). Leon |
From @Leont0001-binmode-FH-crlf-only-modifies-top-crlf-layer.patchFrom d65510200bc7d39254ff62c8d4240280dd988276 Mon Sep 17 00:00:00 2001
From: Leon Timmermans <fawaka@gmail.com>
Date: Thu, 20 Jan 2011 23:32:28 +0100
Subject: [PATCH] binmode FH, ":crlf" only modifies top crlf layer
When pushed on top of the stack, crlf will no longer enable crlf layers
lower in the stack. This will prevent unexpected results.
---
perlio.c | 5 ++---
t/io/layers.t | 18 +++++++++++++++---
2 files changed, 17 insertions(+), 6 deletions(-)
diff --git a/perlio.c b/perlio.c
index 3ce31d1..6ce142e 100644
--- a/perlio.c
+++ b/perlio.c
@@ -4513,7 +4513,7 @@ PerlIOCrlf_pushed(pTHX_ PerlIO *f, const char *mode, SV *arg, PerlIO_funcs *tab)
* any given moment at most one CRLF-capable layer being enabled
* in the whole layer stack. */
PerlIO *g = PerlIONext(f);
- while (PerlIOValid(g)) {
+ if (PerlIOValid(g)) {
PerlIOl *b = PerlIOBase(g);
if (b && b->tab == &PerlIO_crlf) {
if (!(b->flags & PERLIO_F_CRLF))
@@ -4521,8 +4521,7 @@ PerlIOCrlf_pushed(pTHX_ PerlIO *f, const char *mode, SV *arg, PerlIO_funcs *tab)
S_inherit_utf8_flag(g);
PerlIO_pop(aTHX_ f);
return code;
- }
- g = PerlIONext(g);
+ }
}
}
S_inherit_utf8_flag(f);
diff --git a/t/io/layers.t b/t/io/layers.t
index dea3d09..b0bcf1e 100644
--- a/t/io/layers.t
+++ b/t/io/layers.t
@@ -43,7 +43,7 @@ if (${^UNICODE} & 1) {
} else {
$UTF8_STDIN = 0;
}
-my $NTEST = 45 - (($DOSISH || !$FASTSTDIO) ? 7 : 0) - ($DOSISH ? 5 : 0)
+my $NTEST = 55 - (($DOSISH || !$FASTSTDIO) ? 7 : 0) - ($DOSISH ? 7 : 0)
+ $UTF8_STDIN;
sub PerlIO::F_UTF8 () { 0x00008000 } # from perliol.h
@@ -105,7 +105,7 @@ SKIP: {
# 5 tests potentially skipped because
# DOSISH systems already have a CRLF layer
# which will make new ones not stick.
- @$expected = grep { $_ ne 'crlf' } @$expected;
+ splice @$expected, 1, 1 if $expected->[1] eq 'crlf';
}
my $n = scalar @$expected;
is(scalar @$result, $n, "$id - layers == $n");
@@ -132,13 +132,25 @@ SKIP: {
[ qw(stdio crlf) ],
"open :crlf");
+ binmode(F, ":crlf");
+
+ check([ PerlIO::get_layers(F) ],
+ [ qw(stdio crlf) ],
+ "binmode :crlf");
+
binmode(F, ":encoding(cp1047)");
check([ PerlIO::get_layers(F) ],
[ qw[stdio crlf encoding(cp1047) utf8] ],
":encoding(cp1047)");
+
+ binmode(F, ":crlf");
+
+ check([ PerlIO::get_layers(F) ],
+ [ qw[stdio crlf encoding(cp1047) utf8 crlf utf8] ],
+ ":encoding(cp1047):crlf");
- binmode(F, ":pop");
+ binmode(F, ":pop:pop");
check([ PerlIO::get_layers(F) ],
[ qw(stdio crlf) ],
--
1.7.1
|
|
The RT System itself - Status changed from 'new' to 'open' |
From @cpansproutOn Thu Jan 20 15:04:30 2011, LeonT wrote:
Thank you. I have just applied this as 7826b36. Let’s see whether it |
|
@cpansprout - Status changed from 'open' to 'resolved' |
From @LeontOn Fri, Jan 28, 2011 at 6:43 AM, Father Chrysostomos via RT
Comment and documentation updates with regard to the previous patch. Leon |
From @Leont0001-Update-comment-for-PerlIOCrlf_pushed-wrt-7826b36.patchFrom 27e349d88293e66ca5dcaaa21f258e31fc51cc15 Mon Sep 17 00:00:00 2001
From: Leon Timmermans <fawaka@gmail.com>
Date: Fri, 25 Mar 2011 12:55:46 +0100
Subject: [PATCH 1/2] Update comment for PerlIOCrlf_pushed wrt 7826b36
7826b36fbbf24cfa659558ee5af3de424faa2d5a changed the behavior of
PerlIOCrlf_pushed but its comment wasn't updated along with it.
---
perlio.c | 6 ++----
1 files changed, 2 insertions(+), 4 deletions(-)
diff --git a/perlio.c b/perlio.c
index 42bdb84..f2f8729 100644
--- a/perlio.c
+++ b/perlio.c
@@ -4544,10 +4544,8 @@ PerlIOCrlf_pushed(pTHX_ PerlIO *f, const char *mode, SV *arg, PerlIO_funcs *tab)
PerlIOBase(f)->flags);
#endif
{
- /* Enable the first CRLF capable layer you can find, but if none
- * found, the one we just pushed is fine. This results in at
- * any given moment at most one CRLF-capable layer being enabled
- * in the whole layer stack. */
+ /* If the old top layer is a CRLF layer, reactivate it (if
+ * necessary) and remove this new layer from the stack */
PerlIO *g = PerlIONext(f);
if (PerlIOValid(g)) {
PerlIOl *b = PerlIOBase(g);
--
1.7.1
|
From @Leont0002-Update-PerlIO-docs-for-crlf-wrt-7826b36f.patchFrom aaa228e7b6721a6ebb009c0486665f6cec624e6e Mon Sep 17 00:00:00 2001
From: Leon Timmermans <fawaka@gmail.com>
Date: Fri, 25 Mar 2011 12:58:51 +0100
Subject: [PATCH 2/2] Update PerlIO docs for :crlf wrt 7826b36f
7826b36fbbf24cfa659558ee5af3de424faa2d5a changed the behavior of
PerlIOCrlf_pushed but :crlf's documentation wasn't updated along with
it.
---
lib/PerlIO.pm | 16 ++--------------
1 files changed, 2 insertions(+), 14 deletions(-)
diff --git a/lib/PerlIO.pm b/lib/PerlIO.pm
index f4a0197..e76e0b4 100644
--- a/lib/PerlIO.pm
+++ b/lib/PerlIO.pm
@@ -85,24 +85,12 @@ C<:perlio> will insert a C<:unix> layer below itself to do low level IO.
A layer that implements DOS/Windows like CRLF line endings. On read
converts pairs of CR,LF to a single "\n" newline character. On write
-converts each "\n" to a CR,LF pair. Note that this layer likes to be
-one of its kind: it silently ignores attempts to be pushed into the
-layer stack more than once.
+converts each "\n" to a CR,LF pair. Note that this layer will silently
+refuse to be pushed on top of itself.
It currently does I<not> mimic MS-DOS as far as treating of Control-Z
as being an end-of-file marker.
-(Gory details follow) To be more exact what happens is this: after
-pushing itself to the stack, the C<:crlf> layer checks all the layers
-below itself to find the first layer that is capable of being a CRLF
-layer but is not yet enabled to be a CRLF layer. If it finds such a
-layer, it enables the CRLFness of that other deeper layer, and then
-pops itself off the stack. If not, fine, use the one we just pushed.
-
-The end result is that a C<:crlf> means "please enable the first CRLF
-layer you can find, and if you can't find one, here would be a good
-spot to place a new one."
-
Based on the C<:perlio> layer.
=item :mmap
--
1.7.1
|
From @cpansproutOn Fri Mar 25 05:20:25 2011, LeonT wrote:
|
From [Unknown Contact. See original ticket]On Fri Mar 25 05:20:25 2011, LeonT wrote:
|
Migrated from rt.perl.org#38456 (status was 'resolved')
Searchable as RT38456$
The text was updated successfully, but these errors were encountered: