New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The :crlf PerlIO layer doesn't like the :encoding layer. #8325
Comments
From ciaran@tnauk.org.ukCreated by ciaran@tnauk.org.uk(note: This report was compiled with perlbug, then sent manually with There seems to be a problem in the way that the ":crlf" PerlIO layer is * Perl v5.8.7, Cygwin (prepackaged) Here is a test case: ===CUT HERE=== open(FILE, ">:encoding(UTF-16):crlf", "test-file"); This should generate a valid UTF-16 file , containing "Test On a standard version of Perl, what actually happens is that I get a
and the pound sign is replaced with a null character. Using a literal One workaround seems to be to install PerlIO::eol from CPAN and replace I have not mentioned ActiveState Perl in this report as I assume it is Thank you. Following is the output of "perl5.9.1 -V": ===CUT HERE=== Characteristics of this binary (from libperl): Perl Info
|
From @LeontI've attached a possible fix for this bug. The patch is relative to my Currently, if a «:crlf» layer is given it will first try to (re)enable Testing it would be very welcome, in particular on Windows (not my habitat). Leon |
From @Leont0001-binmode-FH-crlf-only-modifies-top-crlf-layer.patchFrom d65510200bc7d39254ff62c8d4240280dd988276 Mon Sep 17 00:00:00 2001
From: Leon Timmermans <fawaka@gmail.com>
Date: Thu, 20 Jan 2011 23:32:28 +0100
Subject: [PATCH] binmode FH, ":crlf" only modifies top crlf layer
When pushed on top of the stack, crlf will no longer enable crlf layers
lower in the stack. This will prevent unexpected results.
---
perlio.c | 5 ++---
t/io/layers.t | 18 +++++++++++++++---
2 files changed, 17 insertions(+), 6 deletions(-)
diff --git a/perlio.c b/perlio.c
index 3ce31d1..6ce142e 100644
--- a/perlio.c
+++ b/perlio.c
@@ -4513,7 +4513,7 @@ PerlIOCrlf_pushed(pTHX_ PerlIO *f, const char *mode, SV *arg, PerlIO_funcs *tab)
* any given moment at most one CRLF-capable layer being enabled
* in the whole layer stack. */
PerlIO *g = PerlIONext(f);
- while (PerlIOValid(g)) {
+ if (PerlIOValid(g)) {
PerlIOl *b = PerlIOBase(g);
if (b && b->tab == &PerlIO_crlf) {
if (!(b->flags & PERLIO_F_CRLF))
@@ -4521,8 +4521,7 @@ PerlIOCrlf_pushed(pTHX_ PerlIO *f, const char *mode, SV *arg, PerlIO_funcs *tab)
S_inherit_utf8_flag(g);
PerlIO_pop(aTHX_ f);
return code;
- }
- g = PerlIONext(g);
+ }
}
}
S_inherit_utf8_flag(f);
diff --git a/t/io/layers.t b/t/io/layers.t
index dea3d09..b0bcf1e 100644
--- a/t/io/layers.t
+++ b/t/io/layers.t
@@ -43,7 +43,7 @@ if (${^UNICODE} & 1) {
} else {
$UTF8_STDIN = 0;
}
-my $NTEST = 45 - (($DOSISH || !$FASTSTDIO) ? 7 : 0) - ($DOSISH ? 5 : 0)
+my $NTEST = 55 - (($DOSISH || !$FASTSTDIO) ? 7 : 0) - ($DOSISH ? 7 : 0)
+ $UTF8_STDIN;
sub PerlIO::F_UTF8 () { 0x00008000 } # from perliol.h
@@ -105,7 +105,7 @@ SKIP: {
# 5 tests potentially skipped because
# DOSISH systems already have a CRLF layer
# which will make new ones not stick.
- @$expected = grep { $_ ne 'crlf' } @$expected;
+ splice @$expected, 1, 1 if $expected->[1] eq 'crlf';
}
my $n = scalar @$expected;
is(scalar @$result, $n, "$id - layers == $n");
@@ -132,13 +132,25 @@ SKIP: {
[ qw(stdio crlf) ],
"open :crlf");
+ binmode(F, ":crlf");
+
+ check([ PerlIO::get_layers(F) ],
+ [ qw(stdio crlf) ],
+ "binmode :crlf");
+
binmode(F, ":encoding(cp1047)");
check([ PerlIO::get_layers(F) ],
[ qw[stdio crlf encoding(cp1047) utf8] ],
":encoding(cp1047)");
+
+ binmode(F, ":crlf");
+
+ check([ PerlIO::get_layers(F) ],
+ [ qw[stdio crlf encoding(cp1047) utf8 crlf utf8] ],
+ ":encoding(cp1047):crlf");
- binmode(F, ":pop");
+ binmode(F, ":pop:pop");
check([ PerlIO::get_layers(F) ],
[ qw(stdio crlf) ],
--
1.7.1
|
The RT System itself - Status changed from 'new' to 'open' |
From @cpansproutOn Thu Jan 20 15:04:30 2011, LeonT wrote:
Thank you. I have just applied this as 7826b36. Let’s see whether it |
@cpansprout - Status changed from 'open' to 'resolved' |
From @LeontOn Fri, Jan 28, 2011 at 6:43 AM, Father Chrysostomos via RT
Comment and documentation updates with regard to the previous patch. Leon |
From @Leont0001-Update-comment-for-PerlIOCrlf_pushed-wrt-7826b36.patchFrom 27e349d88293e66ca5dcaaa21f258e31fc51cc15 Mon Sep 17 00:00:00 2001
From: Leon Timmermans <fawaka@gmail.com>
Date: Fri, 25 Mar 2011 12:55:46 +0100
Subject: [PATCH 1/2] Update comment for PerlIOCrlf_pushed wrt 7826b36
7826b36fbbf24cfa659558ee5af3de424faa2d5a changed the behavior of
PerlIOCrlf_pushed but its comment wasn't updated along with it.
---
perlio.c | 6 ++----
1 files changed, 2 insertions(+), 4 deletions(-)
diff --git a/perlio.c b/perlio.c
index 42bdb84..f2f8729 100644
--- a/perlio.c
+++ b/perlio.c
@@ -4544,10 +4544,8 @@ PerlIOCrlf_pushed(pTHX_ PerlIO *f, const char *mode, SV *arg, PerlIO_funcs *tab)
PerlIOBase(f)->flags);
#endif
{
- /* Enable the first CRLF capable layer you can find, but if none
- * found, the one we just pushed is fine. This results in at
- * any given moment at most one CRLF-capable layer being enabled
- * in the whole layer stack. */
+ /* If the old top layer is a CRLF layer, reactivate it (if
+ * necessary) and remove this new layer from the stack */
PerlIO *g = PerlIONext(f);
if (PerlIOValid(g)) {
PerlIOl *b = PerlIOBase(g);
--
1.7.1
|
From @Leont0002-Update-PerlIO-docs-for-crlf-wrt-7826b36f.patchFrom aaa228e7b6721a6ebb009c0486665f6cec624e6e Mon Sep 17 00:00:00 2001
From: Leon Timmermans <fawaka@gmail.com>
Date: Fri, 25 Mar 2011 12:58:51 +0100
Subject: [PATCH 2/2] Update PerlIO docs for :crlf wrt 7826b36f
7826b36fbbf24cfa659558ee5af3de424faa2d5a changed the behavior of
PerlIOCrlf_pushed but :crlf's documentation wasn't updated along with
it.
---
lib/PerlIO.pm | 16 ++--------------
1 files changed, 2 insertions(+), 14 deletions(-)
diff --git a/lib/PerlIO.pm b/lib/PerlIO.pm
index f4a0197..e76e0b4 100644
--- a/lib/PerlIO.pm
+++ b/lib/PerlIO.pm
@@ -85,24 +85,12 @@ C<:perlio> will insert a C<:unix> layer below itself to do low level IO.
A layer that implements DOS/Windows like CRLF line endings. On read
converts pairs of CR,LF to a single "\n" newline character. On write
-converts each "\n" to a CR,LF pair. Note that this layer likes to be
-one of its kind: it silently ignores attempts to be pushed into the
-layer stack more than once.
+converts each "\n" to a CR,LF pair. Note that this layer will silently
+refuse to be pushed on top of itself.
It currently does I<not> mimic MS-DOS as far as treating of Control-Z
as being an end-of-file marker.
-(Gory details follow) To be more exact what happens is this: after
-pushing itself to the stack, the C<:crlf> layer checks all the layers
-below itself to find the first layer that is capable of being a CRLF
-layer but is not yet enabled to be a CRLF layer. If it finds such a
-layer, it enables the CRLFness of that other deeper layer, and then
-pops itself off the stack. If not, fine, use the one we just pushed.
-
-The end result is that a C<:crlf> means "please enable the first CRLF
-layer you can find, and if you can't find one, here would be a good
-spot to place a new one."
-
Based on the C<:perlio> layer.
=item :mmap
--
1.7.1
|
From @cpansproutOn Fri Mar 25 05:20:25 2011, LeonT wrote:
|
From [Unknown Contact. See original ticket]On Fri Mar 25 05:20:25 2011, LeonT wrote:
|
Migrated from rt.perl.org#38456 (status was 'resolved')
Searchable as RT38456$
The text was updated successfully, but these errors were encountered: