Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reading from DATA on win32 with perlio behaves oddly #7207

Closed
p5pRT opened this issue Mar 31, 2004 · 31 comments
Closed

reading from DATA on win32 with perlio behaves oddly #7207

p5pRT opened this issue Mar 31, 2004 · 31 comments

Comments

@p5pRT
Copy link

p5pRT commented Mar 31, 2004

Migrated from rt.perl.org#28106 (status was 'resolved')

Searchable as RT28106$

@p5pRT
Copy link
Author

p5pRT commented Mar 24, 2004

From mjcarman@mchsi.com

Created by mjcarman@mchsi.com

This is a bug report for perl from mjcarman@​mchsi.com,
generated with the help of perlbug 1.34 running under perl v5.8.3.

-----------------------------------------------------------------
I've run across an odd bug showing an unexpected interaction between
while() and either system() or backticks.

Take the following script​:

  #!perl -w
  use strict;
 
  while (<DATA>) {
  chomp;
  print "$.​: '$_'\n";
  system(); # or ``;
  }
 
  __DATA__
  1
  2
  3

Expected output​:
  1​: '1'
  2​: '2'
  3​: '3'

Actual output​:
  1​: '1'
  2​: ''
  3​: '2'
  4​: ''
  5​: '3'

The chomp() and print() don't affect the behavior; they are only used to
show where the unexpected behavior is occuring.

This works as expected in 5.6.1 but is broken in 5.8.0 and 5.8.3 (and
presumably the versions in between).

If the source file is in *nix format (not normal, as I'm on WinXP) then
it works as expected.

If I change 'while' to 'foreach' it works as expected.

-mjc

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl v5.8.3:

Configured by ActiveState at Tue Feb  3 00:28:38 2004.

Summary of my perl5 (revision 5 version 8 subversion 3) configuration:
  Platform:
    osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    usethreads=undef use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cl', ccflags ='-nologo -Gf -W3 -MD -Zi -DNDEBUG -O1 -DWIN32 -D_CONSOLE
-DNO_STRICT -DHAVE_DES_FCRYPT  -DNO_HASH_SEED -DPERL_IMPLICIT_CONTEXT
-DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX',
    optimize='-MD -Zi -DNDEBUG -O1',
    cppflags='-DWIN32'
    ccversion='', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf 
-libpath:"C:\Perl\lib\CORE"  -machine:x86'
    libpth=C:\PROGRA~1\MICROS~3\VC98\lib
    libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib 
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib
uuid.lib wsock32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib
msvcrt.lib
    perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib 
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib
uuid.lib wsock32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib
msvcrt.lib
    libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib
    gnulibc_version='undef'
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf 
-libpath:"C:\Perl\lib\CORE"  -machine:x86'

Locally applied patches:
    ACTIVEPERL_LOCAL_PATCHES_ENTRY
    22218 Remove the caveat about detached threads crashing on Windows
    22201 Avoid threads+win32 crash by freeing Perl interpreter slightly later
    22169 Display 'out of memeory' errors using low-level I/O
    22159 Upgrade to Time::Hires 1.55
    22120 Make 'Configure -Dcf_by=...' work
    22051 Upgrade to Time::HiRes 1.54
    21540 Fix backward-compatibility issues in if.pm


@INC for perl v5.8.3:
    C:/Perl/lib
    C:/Perl/site/lib
    .


Environment for perl v5.8.3:
    HOME=U:\
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
   
PATH=C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT\bin;C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT\bin\x11;C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT\mksnt;C:\Perl\bin\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program
Files\Dazel\Output Envoy\bin\;C:\lotus\notes\data;C:\Program Files\Rational
Software\ClearCase\bin;C:\Program
Files\Hummingbird\Connectivity\7.11\Accessories\;C:\Program
Files\Rational\common;C:\Program Files\Rational\ClearQuest;C:\Program
Files\Rational\Rose\TopLink\;C:\Program Files\Rational\Rational
Test;C:\MinGW\bin;C:\Tools
    PERLDOC_PAGER=C:\WINDOWS\SYSTEM32\LESS.EXE<
    PERL_BADLANG (unset)
    SHELL (unset)

@p5pRT
Copy link
Author

p5pRT commented Mar 31, 2004

From jeremyd713@hotmail.com

Created by jeremyd713@hotmail.com

This test illustrates the problem​:
-- begin test.pl --
my $line1 = <DATA>;
`echo foo`; # or any sort of piped read
my $line2 = <DATA>;

if ($line1 eq "one\n") { print "ok 1\n" } else { print "not ok 1\n" }
if ($line2 eq "two\n") { print "ok 2\n" } else { print "not ok 2\n" }

__DATA__
one
two
-- end test.pl --

On a Windows machine with all 5.8 perls that I've tested the second check
will fail. I've found 3 ways to make the problem go away...

1) Comment out the `echo foo` line (and don't mix any sort of piped reads in
with the <DATA> reads)

2) Set PERLIO=stdio

3) Apply this change to toke.c​:
--- begin patch ---

Inline Patch
--- toke.c.orig Tue Mar 30 23:19:45 2004
+++ toke.c      Tue Mar 30 23:19:45 2004
@@ -4153,33 +4153,15 @@
#if defined(WIN32) && !defined(PERL_TEXTMODE_SCRIPTS)   /\* if the script was opened in binmode\, we need to revert   \* it to text mode for compatibility; but only iff it has CRs   \* XXX this is a questionable hack at best\. \*/   if \(PL\_bufend\-PL\_bufptr > 2   && PL\_bufend\[\-1\] == '\\n' && PL\_bufend\[\-2\] == '\\r'\)   \{ \- Off\_t loc = 0; \- if \(IoTYPE\(GvIOp\(gv\)\) == IoTYPE\_RDONLY\) \{ \- loc = PerlIO\_tell\(PL\_rsfp\); \- \(void\)PerlIO\_seek\(PL\_rsfp\, 0L\, 0\); \- \} \-\#ifdef NETWARE \- if \(PerlLIO\_setmode\(PL\_rsfp\, O\_TEXT\) \!= \-1\) \{ \-\#else \- if \(PerlLIO\_setmode\(PerlIO\_fileno\(PL\_rsfp\)\, O\_TEXT\) \!= \-1\) \{ \-\#endif /\* NETWARE \*/ \-\#ifdef PERLIO\_IS\_STDIO /\* really? \*/ \-\# if defined\(\_\_BORLANDC\_\_\) \- /\* XXX see note in do\_binmode\(\) \*/ \- \(\(FILE\*\)PL\_rsfp\)\->flags &= ~\_F\_BIN; \-\# endif \-\#endif \- if \(loc > 0\) \- PerlIO\_seek\(PL\_rsfp\, loc\, 0\); \- \} \+ PerlIO\_apply\_layers\(aTHX\_ PL\_rsfp\, NULL\, "​:crlf"\);   \} \#endif \#ifdef PERLIO\_LAYERS   if \(\!IN\_BYTES\) \{   if \(UTF\)   PerlIO\_apply\_layers\(aTHX\_ PL\_rsfp\, NULL\, "​:utf8"\);   else if \(PL\_encoding\) \{ \-\-\- end patch \-\-\-
Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl v5.8.3:

Configured by ActiveState at Tue Feb  3 00:28:38 2004.

Summary of my perl5 (revision 5 version 8 subversion 3) configuration:
  Platform:
    osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread
    uname=''
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    usethreads=undef use5005threads=undef useithreads=define 
usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cl', ccflags ='-nologo -Gf -W3 -MD -Zi -DNDEBUG -O1 -DWIN32 
-D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT  -DNO_HASH_SEED 
-DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO 
-DPERL_MSVCRT_READFIX',
    optimize='-MD -Zi -DNDEBUG -O1',
    cppflags='-DWIN32'
    ccversion='', gccversion='', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', 
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf  
-libpath:"C:\perl-5.8.3-AS809\lib\CORE"  -machine:x86'
    libpth=C:\PROGRA~1\MICROS~3\VC98\lib
    libs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib 
uuid.lib wsock32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib 
msvcrt.lib
    perllibs=  oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib  
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib  netapi32.lib 
uuid.lib wsock32.lib mpr.lib winmm.lib  version.lib odbc32.lib odbccp32.lib 
msvcrt.lib
    libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib
    gnulibc_version='undef'
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug 
-opt:ref,icf  -libpath:"C:\perl-5.8.3-AS809\lib\CORE"  -machine:x86'

Locally applied patches:
    ACTIVEPERL_LOCAL_PATCHES_ENTRY
    22218 Remove the caveat about detached threads crashing on Windows
    22201 Avoid threads+win32 crash by freeing Perl interpreter slightly 
later
    22169 Display 'out of memeory' errors using low-level I/O
    22159 Upgrade to Time::Hires 1.55
    22120 Make 'Configure -Dcf_by=...' work
    22051 Upgrade to Time::HiRes 1.54
    21540 Fix backward-compatibility issues in if.pm


@INC for perl v5.8.3:
    C:/perl-5.8.3-AS809/lib
    C:/perl-5.8.3-AS809/site/lib
    .


Environment for perl v5.8.3:
    HOME (unset)
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=C:\Program Files\Microsoft Visual Studio 
.NET\Common7\IDE;C:\Program Files\Microsoft Visual Studio 
.NET\VC7\BIN;C:\Program Files\Microsoft Visual Studio 
.NET\Common7\Tools;C:\Program Files\Microsoft Visual Studio 
.NET\Common7\Tools\bin\prerelease;C:\Program Files\Microsoft Visual Studio 
.NET\Common7\Tools\bin;C:\Program Files\Microsoft Visual Studio 
.NET\FrameworkSDK\bin;C:\WINDOWS\Microsoft.NET\Framework\v1.0.3705;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program 
Files\SecureCRT 3.0;
    PERL_BADLANG (unset)
    SHELL (unset)

_________________________________________________________________
Find a broadband plan that fits. Great local deals on high-speed Internet 
access. 
https://broadband.msn.com/?pgmarket=en-us/go/onm00200360ave/direct/01/


@p5pRT
Copy link
Author

p5pRT commented Apr 7, 2004

From jeremyd713@hotmail.com

I included a patch with my bug report for #28106. Has anyone more familiar
with the guts of perl had a chance to examine the patch for sanity? I'm not
entirely sure I understand the code but the patch seems to fix the bug and
'make test' keeps working.

I realize this is still a fairly fresh bug report but it's been causing me
headaches. It seems I've already missed the 5.8.4 train...

Thanks,
Jeremy

_________________________________________________________________
Check out MSN PC Safety & Security to help ensure your PC is protected and
safe. http​://specials.msn.com/msn/security.asp

@p5pRT
Copy link
Author

p5pRT commented Dec 15, 2004

From @smpeters

[mjcarman@​mchsi.com - Wed Mar 24 11​:49​:59 2004]​:

This is a bug report for perl from mjcarman@​mchsi.com,
generated with the help of perlbug 1.34 running under perl v5.8.3.

-----------------------------------------------------------------
I've run across an odd bug showing an unexpected interaction between
while() and either system() or backticks.

Take the following script​:

\#\!perl \-w
use strict;

while \(\<DATA>\) \{
    chomp;
    print "$\.&#8203;: '$\_'\\n";
    system\(\); \# or \`\`;
\}

\_\_DATA\_\_
1
2
3

Expected output​:
1​: '1'
2​: '2'
3​: '3'

Actual output​:
1​: '1'
2​: ''
3​: '2'
4​: ''
5​: '3'

The chomp() and print() don't affect the behavior; they are only used
to
show where the unexpected behavior is occuring.

This works as expected in 5.6.1 but is broken in 5.8.0 and 5.8.3 (and
presumably the versions in between).

If the source file is in *nix format (not normal, as I'm on WinXP)
then
it works as expected.

If I change 'while' to 'foreach' it works as expected.

-mjc

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=medium
---
Site configuration information for perl v5.8.3​:

Configured by ActiveState at Tue Feb 3 00​:28​:38 2004.

Summary of my perl5 (revision 5 version 8 subversion 3) configuration​:
Platform​:
osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread
uname=''
config_args='undef'
hint=recommended, useposix=true, d_sigaction=undef
usethreads=undef use5005threads=undef useithreads=define
usemultiplicity=define
useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler​:
cc='cl', ccflags ='-nologo -Gf -W3 -MD -Zi -DNDEBUG -O1 -DWIN32
-D_CONSOLE
-DNO_STRICT -DHAVE_DES_FCRYPT -DNO_HASH_SEED -DPERL_IMPLICIT_CONTEXT
-DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX',
optimize='-MD -Zi -DNDEBUG -O1',
cppflags='-DWIN32'
ccversion='', gccversion='', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
ivtype='long', ivsize=4, nvtype='double', nvsize=8,
Off_t='__int64', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries​:
ld='link', ldflags ='-nologo -nodefaultlib -debug -opt​:ref,icf
-libpath​:"C​:\Perl\lib\CORE" -machine​:x86'
libpth=C​:\PROGRA1\MICROS3\VC98\lib
libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
netapi32.lib
uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib
odbccp32.lib
msvcrt.lib
perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib
winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
netapi32.lib
uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib
odbccp32.lib
msvcrt.lib
libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib
gnulibc_version='undef'
Dynamic Linking​:
dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug
-opt​:ref,icf
-libpath​:"C​:\Perl\lib\CORE" -machine​:x86'

Locally applied patches​:
ACTIVEPERL_LOCAL_PATCHES_ENTRY
22218 Remove the caveat about detached threads crashing on Windows
22201 Avoid threads+win32 crash by freeing Perl interpreter
slightly later
22169 Display 'out of memeory' errors using low-level I/O
22159 Upgrade to Time​::Hires 1.55
22120 Make 'Configure -Dcf_by=...' work
22051 Upgrade to Time​::HiRes 1.54
21540 Fix backward-compatibility issues in if.pm

---
@​INC for perl v5.8.3​:
C​:/Perl/lib
C​:/Perl/site/lib
.

I could not replicate the above on a OpenBSD or Mac OS X, but could
replicate on Windows. I replicated on both ActiveState 5.8.4 Build 810
and a freshly compiled Perl 5.8.6.

@p5pRT
Copy link
Author

p5pRT commented Dec 15, 2004

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Dec 15, 2004

From @smpeters

[mjcarman@​mchsi.com - Wed Mar 24 11​:49​:59 2004]​:

This is a bug report for perl from mjcarman@​mchsi.com,
generated with the help of perlbug 1.34 running under perl v5.8.3.

-----------------------------------------------------------------
I've run across an odd bug showing an unexpected interaction between
while() and either system() or backticks.

Take the following script​:

\#\!perl \-w
use strict;

while \(\<DATA>\) \{
    chomp;
    print "$\.&#8203;: '$\_'\\n";
    system\(\); \# or \`\`;
\}

\_\_DATA\_\_
1
2
3

Expected output​:
1​: '1'
2​: '2'
3​: '3'

Actual output​:
1​: '1'
2​: ''
3​: '2'
4​: ''
5​: '3'

The chomp() and print() don't affect the behavior; they are only used
to
show where the unexpected behavior is occuring.

This works as expected in 5.6.1 but is broken in 5.8.0 and 5.8.3 (and
presumably the versions in between).

If the source file is in *nix format (not normal, as I'm on WinXP)
then
it works as expected.

If I change 'while' to 'foreach' it works as expected.

-mjc

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=medium
---
Site configuration information for perl v5.8.3​:

Configured by ActiveState at Tue Feb 3 00​:28​:38 2004.

Summary of my perl5 (revision 5 version 8 subversion 3) configuration​:
Platform​:
osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread
uname=''
config_args='undef'
hint=recommended, useposix=true, d_sigaction=undef
usethreads=undef use5005threads=undef useithreads=define
usemultiplicity=define
useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler​:
cc='cl', ccflags ='-nologo -Gf -W3 -MD -Zi -DNDEBUG -O1 -DWIN32
-D_CONSOLE
-DNO_STRICT -DHAVE_DES_FCRYPT -DNO_HASH_SEED -DPERL_IMPLICIT_CONTEXT
-DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX',
optimize='-MD -Zi -DNDEBUG -O1',
cppflags='-DWIN32'
ccversion='', gccversion='', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
ivtype='long', ivsize=4, nvtype='double', nvsize=8,
Off_t='__int64', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries​:
ld='link', ldflags ='-nologo -nodefaultlib -debug -opt​:ref,icf
-libpath​:"C​:\Perl\lib\CORE" -machine​:x86'
libpth=C​:\PROGRA1\MICROS3\VC98\lib
libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
netapi32.lib
uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib
odbccp32.lib
msvcrt.lib
perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib
winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
netapi32.lib
uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib
odbccp32.lib
msvcrt.lib
libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib
gnulibc_version='undef'
Dynamic Linking​:
dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug
-opt​:ref,icf
-libpath​:"C​:\Perl\lib\CORE" -machine​:x86'

Locally applied patches​:
ACTIVEPERL_LOCAL_PATCHES_ENTRY
22218 Remove the caveat about detached threads crashing on Windows
22201 Avoid threads+win32 crash by freeing Perl interpreter
slightly later
22169 Display 'out of memeory' errors using low-level I/O
22159 Upgrade to Time​::Hires 1.55
22120 Make 'Configure -Dcf_by=...' work
22051 Upgrade to Time​::HiRes 1.54
21540 Fix backward-compatibility issues in if.pm

---
@​INC for perl v5.8.3​:
C​:/Perl/lib
C​:/Perl/site/lib
.

I could not replicate the above on a OpenBSD or Mac OS X, but could
replicate on Windows. I replicated on both ActiveState 5.8.4 Build 810
and a freshly compiled Perl 5.8.6.

@p5pRT
Copy link
Author

p5pRT commented Dec 15, 2004

From @smpeters

[weezel - Tue Mar 30 23​:45​:53 2004]​:

This is a bug report for perl from jeremyd713@​hotmail.com,
generated with the help of perlbug 1.34 running under perl v5.8.3.

-----------------------------------------------------------------
[Please enter your report here]

This test illustrates the problem​:
-- begin test.pl --
my $line1 = <DATA>;
`echo foo`; # or any sort of piped read
my $line2 = <DATA>;

if ($line1 eq "one\n") { print "ok 1\n" } else { print "not ok 1\n" }
if ($line2 eq "two\n") { print "ok 2\n" } else { print "not ok 2\n" }

__DATA__
one
two
-- end test.pl --

On a Windows machine with all 5.8 perls that I've tested the second
check
will fail. I've found 3 ways to make the problem go away...

1) Comment out the `echo foo` line (and don't mix any sort of piped
reads in
with the <DATA> reads)

2) Set PERLIO=stdio

3) Apply this change to toke.c​:
--- begin patch ---

--- toke.c.orig Tue Mar 30 23​:19​:45 2004
+++ toke.c Tue Mar 30 23​:19​:45 2004
@​@​ -4153,33 +4153,15 @​@​
#if defined(WIN32) && !defined(PERL_TEXTMODE_SCRIPTS)
/* if the script was opened in binmode, we need to
revert
* it to text mode for compatibility; but only iff it
has
CRs
* XXX this is a questionable hack at best. */
if (PL_bufend-PL_bufptr > 2
&& PL_bufend[-1] == '\n' && PL_bufend[-2] == '\r')
{
- Off_t loc = 0;
- if (IoTYPE(GvIOp(gv)) == IoTYPE_RDONLY) {
- loc = PerlIO_tell(PL_rsfp);
- (void)PerlIO_seek(PL_rsfp, 0L, 0);
- }
-#ifdef NETWARE
- if (PerlLIO_setmode(PL_rsfp, O_TEXT) != -1) {
-#else
- if (PerlLIO_setmode(PerlIO_fileno(PL_rsfp),
O_TEXT) !=
-1) {
-#endif /* NETWARE */
-#ifdef PERLIO_IS_STDIO /* really? */
-# if defined(__BORLANDC__)
- /* XXX see note in do_binmode() */
- ((FILE*)PL_rsfp)->flags &= ~_F_BIN;
-# endif
-#endif
- if (loc > 0)
- PerlIO_seek(PL_rsfp, loc, 0);
- }
+ PerlIO_apply_layers(aTHX_ PL_rsfp, NULL, "​:crlf");
}
#endif
#ifdef PERLIO_LAYERS
if (!IN_BYTES) {
if (UTF)
PerlIO_apply_layers(aTHX_ PL_rsfp, NULL,
"​:utf8");
else if (PL_encoding) {
--- end patch ---

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=medium
---
Site configuration information for perl v5.8.3​:

Configured by ActiveState at Tue Feb 3 00​:28​:38 2004.

Summary of my perl5 (revision 5 version 8 subversion 3) configuration​:
Platform​:
osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread
uname=''
config_args='undef'
hint=recommended, useposix=true, d_sigaction=undef
usethreads=undef use5005threads=undef useithreads=define
usemultiplicity=define
useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler​:
cc='cl', ccflags ='-nologo -Gf -W3 -MD -Zi -DNDEBUG -O1 -DWIN32
-D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DNO_HASH_SEED
-DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO
-DPERL_MSVCRT_READFIX',
optimize='-MD -Zi -DNDEBUG -O1',
cppflags='-DWIN32'
ccversion='', gccversion='', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
ivtype='long', ivsize=4, nvtype='double', nvsize=8,
Off_t='__int64',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries​:
ld='link', ldflags ='-nologo -nodefaultlib -debug -opt​:ref,icf
-libpath​:"C​:\perl-5.8.3-AS809\lib\CORE" -machine​:x86'
libpth=C​:\PROGRA1\MICROS3\VC98\lib
libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
netapi32.lib
uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib
odbccp32.lib
msvcrt.lib
perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib
winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
netapi32.lib
uuid.lib wsock32.lib mpr.lib winmm.lib version.lib odbc32.lib
odbccp32.lib
msvcrt.lib
libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib
gnulibc_version='undef'
Dynamic Linking​:
dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug
-opt​:ref,icf -libpath​:"C​:\perl-5.8.3-AS809\lib\CORE" -machine​:x86'

Locally applied patches​:
ACTIVEPERL_LOCAL_PATCHES_ENTRY
22218 Remove the caveat about detached threads crashing on Windows
22201 Avoid threads+win32 crash by freeing Perl interpreter
slightly
later
22169 Display 'out of memeory' errors using low-level I/O
22159 Upgrade to Time​::Hires 1.55
22120 Make 'Configure -Dcf_by=...' work
22051 Upgrade to Time​::HiRes 1.54
21540 Fix backward-compatibility issues in if.pm

---
@​INC for perl v5.8.3​:
C​:/perl-5.8.3-AS809/lib
C​:/perl-5.8.3-AS809/site/lib
.

---
Environment for perl v5.8.3​:
HOME (unset)
LANG (unset)
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=C​:\Program Files\Microsoft Visual Studio
.NET\Common7\IDE;C​:\Program Files\Microsoft Visual Studio
.NET\VC7\BIN;C​:\Program Files\Microsoft Visual Studio
.NET\Common7\Tools;C​:\Program Files\Microsoft Visual Studio
.NET\Common7\Tools\bin\prerelease;C​:\Program Files\Microsoft Visual
Studio
.NET\Common7\Tools\bin;C​:\Program Files\Microsoft Visual Studio

.NET\FrameworkSDK\bin;C​:\WINDOWS\Microsoft.NET\Framework\v1.0.3705;C​:\WINDOWS\system32;C​:\WINDOWS;C​:\WINDOWS\System32\Wbem;C​:\Program

Files\SecureCRT 3.0;
PERL_BADLANG (unset)
SHELL (unset)

_________________________________________________________________
Find a broadband plan that fits. Great local deals on high-speed
Internet
access.
https://broadband.msn.com/?pgmarket=en-us/go/onm00200360ave/direct/01/

After looking at ticket #27918, I ran into this ticket. This one is the
actual cause. Attempting to read from <DATA> while performing system()
or `` commands will lead to unreliable results.

Setting PERLIO to stdio does solve the problem as a workaround. I'm
interested in the patch, but, unfortunately, can't get to it this
evening. My concern is for Borland and Netware based system. I'm
guessing this patch could do harm to those systems, but, frankly, I
don't know. Can someone with some more Win32 experience check out the
above patch, particularly if you know something about Netware or Borland?

Also, just re-writing the patch to leave Borland and Netware alone would
probably work fine as well.

@p5pRT
Copy link
Author

p5pRT commented Dec 15, 2004

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Dec 16, 2006

From @smpeters

After looking at ticket #27918, I ran into this ticket. This one is
the
actual cause. Attempting to read from <DATA> while performing
system()
or `` commands will lead to unreliable results.

Setting PERLIO to stdio does solve the problem as a workaround. I'm
interested in the patch, but, unfortunately, can't get to it this
evening. My concern is for Borland and Netware based system. I'm
guessing this patch could do harm to those systems, but, frankly, I
don't know. Can someone with some more Win32 experience check out the
above patch, particularly if you know something about Netware or
Borland?

Also, just re-writing the patch to leave Borland and Netware alone
would
probably work fine as well.

I can no longer replicate this bug with either bleadperl (compiled with
VC++ 8.0 or borland), or with ActiveState Perl 5.8.8 .

@p5pRT
Copy link
Author

p5pRT commented Dec 16, 2006

@smpeters - Status changed from 'open' to 'resolved'

@p5pRT
Copy link
Author

p5pRT commented Dec 16, 2006

From @smpeters

On Tue Dec 14 19​:01​:05 2004, stmpeters wrote​:

[mjcarman@​mchsi.com - Wed Mar 24 11​:49​:59 2004]​:

I could not replicate the above on a OpenBSD or Mac OS X, but could
replicate on Windows. I replicated on both ActiveState 5.8.4 Build 810
and a freshly compiled Perl 5.8.6.

This now works with a current bleadperl and with ActiveState 5.8.8 build
819.

@p5pRT
Copy link
Author

p5pRT commented Dec 16, 2006

@smpeters - Status changed from 'open' to 'resolved'

@p5pRT
Copy link
Author

p5pRT commented Jul 5, 2007

From @jandubois

On Sat, 16 Dec 2006, Steve Peters via RT wrote​:

After looking at ticket #27918, I ran into this ticket. This one is
the actual cause. Attempting to read from <DATA> while performing
system() or `` commands will lead to unreliable results.

Setting PERLIO to stdio does solve the problem as a workaround. I'm
interested in the patch, but, unfortunately, can't get to it this
evening. My concern is for Borland and Netware based system. I'm
guessing this patch could do harm to those systems, but, frankly, I
don't know. Can someone with some more Win32 experience check out
the above patch, particularly if you know something about Netware or
Borland?

Also, just re-writing the patch to leave Borland and Netware alone
would probably work fine as well.

I can no longer replicate this bug with either bleadperl (compiled
with VC++ 8.0 or borland), or with ActiveState Perl 5.8.8 .

This problem is not fixed; it still happens with 5.8.8 and blead. It only
happens when you store your script with CRLF line endings, so maybe that
is the reason you couldn't reproduce it anymore.

You'll see this error with the t/op/readline.t test in blead too, if you
convert the bleadperl sources to Windows line-endings first.

The backticks are triggering the issue because they will flush all
filehandles. You'll see that the internal fileposition for the DATA
handle is wrong; you can replace the backticks with the following line
to achieve the same effect​:

  seek(DATA, tell DATA, 0);

which should have been a no-op.

I was thinking about another way to fix this​: Defining the
PERL_TEXTMODE_SCRIPTS preprocessor variable to always open scripts in
text mode. This breaks ByteLoader, but that doesn't matter because
ByteLoader is no more. However, it might also break source filters
storing binary data in the __DATA__ section, so it may still not be safe
for blead. It does not break any tests in blead though. :)

Cheers,
-Jan

@p5pRT
Copy link
Author

p5pRT commented Nov 7, 2007

@rgs - Status changed from 'resolved' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Dec 20, 2007

From @demerphq

This is a bug report for perl from demerphq@​fresh,
generated with the help of perlbug 1.36 running under perl 5.10.0.


I downloaded the sources from

  http​://cekirdek.pardus.org.tr/~ismail/dist/perl-5.10.0.tar.gz

and for some silly reason unpacked them with WinZip, which apparently did
file ending conversion on the text files in the distribution, which lead to
test 16-18 failing in op/readline.t. Trying to duplicate one of the errors on
the command line using

  perl -e"$x=bless []; $x.=<>; print $x"

did not show a problem. When I noticed the file was in Win32 line endings
I converted it back to unix and the tests pass. Obviously rcatline is not
working properly with win32 line endings, possibly only when operating
on the DATA filehandle.

This is possibly related to perl #38631​: tied variables don't work with .= <>,
however I believe Jan Dubois has encountered other errors from similar causes
(win32 source files).

demerphq



Flags​:
  category=core
  severity=medium


Site configuration information for perl 5.10.0​:

Configured by demerphq at Thu Dec 20 21​:27​:09 2007.

Summary of my perl5 (revision 5 version 10 subversion 0) configuration​:
  Platform​:
  osname=MSWin32, osvers=5.00, archname=MSWin32-x86-multi-thread
  uname=''
  config_args='undef'
  hint=recommended, useposix=true, d_sigaction=undef
  useithreads=define, usemultiplicity=define
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=undef, use64bitall=undef, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cl', ccflags ='-nologo -GF -W3 -MD -Zi -DNDEBUG -O1 -DWIN32
-D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DPERL_IMPLICIT_CONTEXT
-DPERL_IMPLICIT_SYS -DUSE_PERLIO -DPERL_MSVCRT_READFIX',
  optimize='-MD -Zi -DNDEBUG -O1',
  cppflags='-DWIN32'
  ccversion='13.00.9466', gccversion='', gccosandvers=''
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
  d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
  ivtype='long', ivsize=4, nvtype='double', nvsize=8,
Off_t='__int64', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='link', ldflags ='-nologo -nodefaultlib -debug -opt​:ref,icf
-libpath​:"c​:\perl\lib\CORE" -machine​:x86'
  libpth=C​:\dotNet\Vc7\lib
  libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib
odbc32.lib odbccp32.lib msvcrt.lib
  perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib
winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib
oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib
version.lib odbc32.lib odbccp32.lib msvcrt.lib
  libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl510.lib
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
  cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug
-opt​:ref,icf -libpath​:"c​:\perl\lib\CORE" -machine​:x86'

Locally applied patches​:


@​INC for perl 5.10.0​:
  C​:/perl/lib
  C​:/perl/site/lib
  .


Environment for perl 5.10.0​:
  CYGWIN=ntea
  HOME (unset)
  LANG (unset)
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=C​:\Program Files\X-Chat
2\lib;d​:\ruby\bin;C​:\Util\UEdit\UC;C​:\Python24;C​:\dotNet\Common7\IDE;C​:\dotNet\VC7\BIN;C​:\dotNet\Common7\Tools;C​:\dotNet\Common7\Tools\bin\prerelease;C​:\dotNet\Common7\Tools\bin;C​:\dotNet\FrameworkSDK\bin;C​:\WINNT\Microsoft.NET\Framework\v1.0.3705;C​:\Util\UEdit;c​:\imagemagick-6.2.1-q16;D​:\Py\Python24;D​:\ASPerl\811\bin;D​:\ASPerl\638\bin;C​:\WINNT\system32;C​:\WINNT;C​:\WINNT\system32\wbem;C​:\Util\totalcmd;C​:\Util\7-Zip;D​:\svn\Tortoise\bin;D​:\BerkeleyDB\bin;D​:\BerkeleyDB\bin\debug;D​:\svn\1.4.2\bin;D​:\svn\BDB-4.3.29\bin;D​:\svn\BDB-4.3.29\bin\debug;C​:\cygwin\bin;E​:\Perforce;C​:\Program
Files\QuickTime\QTSystem\;c​:\util\sysinternals;e​:\bzr
  PERL_BADLANG (unset)
  SHELL (unset)

@p5pRT
Copy link
Author

p5pRT commented Dec 20, 2007

From @jandubois

On Thu, 20 Dec 2007, yves orton (via RT) wrote​:

I downloaded the sources from

http&#8203;://cekirdek\.pardus\.org\.tr/~ismail/dist/perl\-5\.10\.0\.tar\.gz

and for some silly reason unpacked them with WinZip, which apparently did
file ending conversion on the text files in the distribution, which lead to
test 16-18 failing in op/readline.t. Trying to duplicate one of the errors on
the command line using

perl \-e"$x=bless \[\]; $x\.=\<>; print $x"

did not show a problem. When I noticed the file was in Win32 line endings
I converted it back to unix and the tests pass. Obviously rcatline is not
working properly with win32 line endings, possibly only when operating
on the DATA filehandle.

This is possibly related to perl #38631​: tied variables don't work with .= <>,
however I believe Jan Dubois has encountered other errors from similar causes
(win32 source files).

No, this error has nothing to do with rcatline; the problem is
fresh_perl_is() screws up the DATA file position because of some
auto- flushing.

This bug is a duplicate of http​://rt.perl.org/rt3//Public/Bug/Display.html?id=28106.

I thought I asked for the bug to be re-opened, but it is still marked as
closed. Looking again, I actually asked for another duplicate of the same bug
to be re-opened​:

http​://groups.google.com/group/perl.perl5.porters/msg/bc743771de29a268

So in summary, I think #27918, #28106 and #48959 are all the same
problem.

Cheers,
-Jan

PS​: Is there a way to get write access to the bug database so I can
  update the status myself?

@p5pRT
Copy link
Author

p5pRT commented Dec 20, 2007

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Dec 21, 2007

@demerphq - Status changed from 'resolved' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Dec 21, 2007

From @demerphq

On 21/12/2007, Jan Dubois <jand@​activestate.com> wrote​:

On Thu, 20 Dec 2007, yves orton (via RT) wrote​:

I downloaded the sources from

  http&#8203;://cekirdek\.pardus\.org\.tr/~ismail/dist/perl\-5\.10\.0\.tar\.gz

and for some silly reason unpacked them with WinZip, which apparently did
file ending conversion on the text files in the distribution, which lead to
test 16-18 failing in op/readline.t. Trying to duplicate one of the errors on
the command line using

  perl \-e"$x=bless \[\]; $x\.=\<>; print $x"

did not show a problem. When I noticed the file was in Win32 line endings
I converted it back to unix and the tests pass. Obviously rcatline is not
working properly with win32 line endings, possibly only when operating
on the DATA filehandle.

This is possibly related to perl #38631​: tied variables don't work with .= <>,
however I believe Jan Dubois has encountered other errors from similar causes
(win32 source files).

No, this error has nothing to do with rcatline; the problem is
fresh_perl_is() screws up the DATA file position because of some
auto- flushing.

This bug is a duplicate of http​://rt.perl.org/rt3//Public/Bug/Display.html?id=28106.

I thought I asked for the bug to be re-opened, but it is still marked as
closed. Looking again, I actually asked for another duplicate of the same bug
to be re-opened​:

http​://groups.google.com/group/perl.perl5.porters/msg/bc743771de29a268

So in summary, I think #27918, #28106 and #48959 are all the same
problem.

Thanks for the explanation. I merged the three bugs together into
28106. As for access, I think you have to ask Robert Spier. (cc'ed)

Cheers,
Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

p5pRT commented Mar 15, 2011

From brent.shields@ieee.org

Hi,
The report generated by perlbug is attached. Thanks.

 

@p5pRT
Copy link
Author

p5pRT commented Mar 15, 2011

From brent.shields@ieee.org

Created by brent.shields@ieee.org

When I use <DATA> to read part of the DATA file and then "tell DATA", the value returned by "tell DATA" is off by the number of newlines that
have been read.

This has been discussed at perlmonks and another experienced programmer agreed that it is likely a bug. I tested it on Knoppix Linux and it does not happen there.
Another user checked it on ActivePerl 5.12.1 with the code below and it occurred there also.

Thanks,
Brent Shields

http​://www.perlmonks.com/?node_id=892784

#!/usr/bin/perl
use strict;
use warnings;

my @​data_positions = tell(DATA);
while (<DATA>){
  if (/^__DATA__$/) {
  push @​data_positions, tell(DATA);
  }
}

my @​fh_positions;
open(my $fh, '<', $0) or die;
while (<$fh>){
  if (/^__DATA__$/) {
  push @​fh_positions, tell($fh);
  }
}

print("@​data_positions\n");
print("@​fh_positions\n");

__DATA__
ab
__DATA__
ab

__DATA__
ab
__DATA__
lotsa junk
nothing

*****************************************************
Output​:
389 401 414 426
389 403 419 433

Perl Info
---
Flags:
    category=core
    severity=medium
---
Site configuration information for perl 5.12.0:

Configured by 1 at Thu May  6 16:22:52 2010.

Summary of my perl5 (revision 5 version 12 subversion 0) configuration:
   
  Platform:
    osname=MSWin32, osvers=5.1, archname=MSWin32-x86-multi-thread
    uname='Win32 strawberryperl 5.12.0.1 #1 Thu May  6 16:09:27 2010 i386'
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags =' -s -O2 -DWIN32 -DHAVE_DES_FCRYPT  -DUSE_SITECUSTOMIZE -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -fno-strict-aliasing -mms-bitfields -DPERL_MSVCRT_READFIX',
    optimize='-s -O2',
    cppflags='-DWIN32'
    ccversion='', gccversion='4.4.3', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='long long', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='g++', ldflags ='-s -L"C:\strawberry\perl\lib\CORE" -L"C:\strawberry\c\lib"'
    libpth=C:\strawberry\c\lib C:\strawberry\c\i686-w64-mingw32\lib
    libs=-lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    perllibs=-lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    libc=, so=dll, useshrplib=true, libperl=libperl512.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-mdll -s -L"C:\strawberry\perl\lib\CORE" -L"C:\strawberry\c\lib"'

Locally applied patches:
    

---
@INC for perl 5.12.0:
    C:/strawberry/perl/site/lib
    C:/strawberry/perl/vendor/lib
    C:/strawberry/perl/lib
    .

---
Environment for perl 5.12.0:
    HOME (unset)
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=C:\WINNT;C:\WINNT\system32;C:\WINNT\System32\Wbem;C:\Program Files\Intel\DMIX;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Ulead Systems\MPEG;C:\Program Files\ThinkPad\ConnectUtilities;C:\Program Files\Common Files\Lenovo;C:\SQLLIB\BIN;C:\SQLLIB\FUNCTION;C:\Program Files\Utimaco\SafeGuard Easy\;C:\strawberry\c\bin;C:\strawberry\perl\site\bin;C:\strawberry\perl\bin;C:\Program Files\VanDyke Software\SecureFX\;C:\Program Files\GnuWin32\bin\;c:\b\;c:\usr\bin\;C:\Eterra\e_terrabrowser\
    PERL_BADLANG (unset)
    SHELL (unset)

@p5pRT
Copy link
Author

p5pRT commented Mar 16, 2011

From @jandubois

On Mon, 14 Mar 2011, via RT wrote​:

When I use <DATA> to read part of the DATA file and then "tell DATA",
the value returned by "tell DATA" is off by the number of newlines
that have been read.

This has been discussed at perlmonks and another experienced
programmer agreed that it is likely a bug. I tested it on Knoppix
Linux and it does not happen there. Another user checked it on
ActivePerl 5.12.1 with the code below and it occurred there also.

This is an issue on Perl for Windows only. The problem is that
Perl source code is read in binary mode, and once the tokenizer
finds the __DATA__ token the filehandle is incompletely changed
into text mode. Search toke.c for PERL_TEXTMODE_SCRIPTS to locate
the questionable code.

The reason for opening the scripts in binmode (on Windows only) is that
using textmode breaks ByteLoader. Given that ByteLoader has been
abandoned, the easiest approach to fixing this bug might be to switch
reading scripts in text mode.

Cheers,
-Jan

@p5pRT
Copy link
Author

p5pRT commented Mar 16, 2011

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Mar 16, 2011

From @jandubois

On Tue, 15 Mar 2011, Jan Dubois wrote​:

The reason for opening the scripts in binmode (on Windows only) is that
using textmode breaks ByteLoader. Given that ByteLoader has been
abandoned, the easiest approach to fixing this bug might be to switch
reading scripts in text mode.

Building Perl on Windows with -DPERL_TEXTMODE_SCRIPTS passes all tests
and also provides correct behavior for tell(DATA)​:

c​:\tmp>\git\perl\perl data.pl
389 403 419 433
389 403 419 433

I'll make -DPERL_TEXTMODE_SCRIPTS the default in win32/Makefile and
win32/makefile.mk, unless someone on p5p knows of a good reason not
to do this. I think a working tell(DATA) is much more important
than ByteLoader support.

Cheers,
-Jan

@p5pRT
Copy link
Author

p5pRT commented Mar 16, 2011

From @rurban

2011/3/16 Jan Dubois <jand@​activestate.com>​:

On Mon, 14 Mar 2011, via RT wrote​:

When I use <DATA> to read part of the DATA file and then "tell DATA",
the value returned by "tell DATA" is off by the number of newlines
that have been read.

This has been discussed at perlmonks and another experienced
programmer agreed that it is likely a bug. I tested it on Knoppix
Linux and it does not happen there. Another user checked it on
ActivePerl 5.12.1 with the code below and it occurred there also.

This is an issue on Perl for Windows only.  The problem is that
Perl source code is read in binary mode, and once the tokenizer
finds the __DATA__ token the filehandle is incompletely changed
into text mode.  Search toke.c for PERL_TEXTMODE_SCRIPTS to locate
the questionable code.

The reason for opening the scripts in binmode (on Windows only) is that
using textmode breaks ByteLoader. Given that ByteLoader has been
abandoned, the easiest approach to fixing this bug might be to switch
reading scripts in text mode.

As maintainer of ByteLoader I'm of course against core changes which
break ByteLoader and other source filters on Windows.

And I never heard so far that ByteLoader has been abandoned, just
removed from CORE with 5.10. If so it would be nice to tell me
officially that I should I switch over to do other things in my free
time.

--
Reini Urban
http​://www.perl-compiler.org/

@p5pRT
Copy link
Author

p5pRT commented Mar 16, 2011

From @jandubois

On Wed, 16 Mar 2011, Reini Urban wrote​:

As maintainer of ByteLoader I'm of course against core changes which
break ByteLoader and other source filters on Windows.

I'm not aware of any other source filters that are affected by this.

I'm also (so far) just talking about changing the default build
options on Windows to read source files in text mode (just as
on all other platforms). Users could still compile their own
Perl that will read scripts in binary mode. However, given that
the code is broken, and unlikely to be fixed, I will argue in
the future that it should be removed completely.

And I never heard so far that ByteLoader has been abandoned, just
removed from CORE with 5.10. If so it would be nice to tell me
officially that I should I switch over to do other things in my free
time.

So far I'm not aware of any useful application of ByteLoader. AFAIK
loading the bytecode is slower than actually compiling the source
again. Who is using ByteLoader, and to what end?

If ByteLoader is indeed important, than I would suggest that ByteLoader
should be fixed to work with scripts being read in text mode. I would
expect that adding something like this to the ByteLoader prolog should
make it work again​:

  seek(DATA, 0, 0);
  binmode(DATA);
  while (<DATA>) {
  last if /^__DATA__\r?$/;
  }

Yes, I know it currently doesn't use an explicit marker, like __DATA__,
to indicate the start of the binary data, but that could easily be
fixed as well.

Cheers,
-Jan

@p5pRT
Copy link
Author

p5pRT commented Mar 17, 2011

From @jandubois

On Wed, 16 Mar 2011, Jan Dubois wrote​:

I'm also (so far) just talking about changing the default build
options on Windows to read source files in text mode (just as
on all other platforms). Users could still compile their own
Perl that will read scripts in binary mode. However, given that
the code is broken, and unlikely to be fixed, I will argue in
the future that it should be removed completely.

I've merged this bug with

  http​://rt.perl.org/rt3//Public/Bug/Display.html?id=28106

which already contained the merged history of 3 related bug reports,
all based on the same fundamental issue.

I have now changed the default build options for Windows, and added
all the test scripts from the various bugs as regression tests​:

  http​://perl5.git.perl.org/perl.git/commitdiff/270ca148c

As you can see from those tests, the failures aren't restricted to
tell(DATA) returning incorrect values; even just reading from
DATA after Perl internals have flushed all file handles to prepare
for a fork() (or system() call etc) could cause data corruption.

Reading Perl source code in text mode makes all the tests PASS.

Cheers,
-Jan

@p5pRT
Copy link
Author

p5pRT commented May 16, 2011

From @iabyn

On Wed, Mar 16, 2011 at 06​:59​:11PM -0700, Jan Dubois wrote​:

On Wed, 16 Mar 2011, Jan Dubois wrote​:

I'm also (so far) just talking about changing the default build
options on Windows to read source files in text mode (just as
on all other platforms). Users could still compile their own
Perl that will read scripts in binary mode. However, given that
the code is broken, and unlikely to be fixed, I will argue in
the future that it should be removed completely.

I've merged this bug with

http&#8203;://rt\.perl\.org/rt3//Public/Bug/Display\.html?id=28106

which already contained the merged history of 3 related bug reports,
all based on the same fundamental issue.

I have now changed the default build options for Windows, and added
all the test scripts from the various bugs as regression tests​:

http&#8203;://perl5\.git\.perl\.org/perl\.git/commitdiff/270ca148c

As you can see from those tests, the failures aren't restricted to
tell(DATA) returning incorrect values; even just reading from
DATA after Perl internals have flushed all file handles to prepare
for a fork() (or system() call etc) could cause data corruption.

Reading Perl source code in text mode makes all the tests PASS.

Is it ok to close this ticket?

--
"I do not resent criticism, even when, for the sake of emphasis,
it parts for the time with reality".
  -- Winston Churchill, House of Commons, 22nd Jan 1941.

@p5pRT
Copy link
Author

p5pRT commented May 16, 2011

From @jandubois

I consider this "fixed" now (and intend to remove the broken "read
scripts in binary mode" parts when blead re-opens for 5.15 development).

@p5pRT
Copy link
Author

p5pRT commented May 16, 2011

@jandubois - Status changed from 'open' to 'resolved'

@p5pRT p5pRT closed this as completed May 16, 2011
@p5pRT
Copy link
Author

p5pRT commented May 16, 2011

From @jandubois

On Mon, 16 May 2011, Dave Mitchell wrote​:

Is it ok to close this ticket?

I just "resolved" it.

Cheers,
-Jan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant