Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

utf-8 + void map problem in perl >= 5.8.3 #7695

Closed
p5pRT opened this issue Dec 8, 2004 · 6 comments
Closed

utf-8 + void map problem in perl >= 5.8.3 #7695

p5pRT opened this issue Dec 8, 2004 · 6 comments

Comments

@p5pRT
Copy link

p5pRT commented Dec 8, 2004

Migrated from rt.perl.org#32964 (status was 'rejected')

Searchable as RT32964$

@p5pRT
Copy link
Author

p5pRT commented Dec 8, 2004

From davekam@pobox.com

Created by davekam@ext5.eva.mpg.de

OK, after a long discussion on #perl on freenode we've narrowed this
bug down.
It occurs only in 5.8.3 or later, on various architectures (tried on
darwin, x86,
opteron, etc.). It happens only with use encoding 'utf8', NOT with use
utf8. It
doesn't happen at all if neither are specified. The '<​:utf8' in open is
irrelevant.
Go to http​://www.plover.com/~mjd/misc/perl/ningu/ to see the original
script and
input file -- it returns 58018 lines instead of 58019. The crucial line
is map.
If you change it to foreach, it works properly.

The following oneliners (contributed by Somni on #perl) also reproduce
the problem​:

perl -Mencoding=utf8 -e 'sub i { "" } map { i join("", ""), "" }
1..2042; "$i";'
perl -Mencoding=utf8 -e 'map { join("", ""), "" } 1..2042;'
perl -Mencoding=utf8 -e 'map { join("", "") } 1..2042;'

These scripts typically produce segfault, memory wrap, or attempt to
deallocate
unreferenced scalar errors. Different archs seem to have different
magic numbers
required to trigger it, but typically a number somewhere in the 2040 to
2043 range
will do it, and anything higher. But there are also some weird things
where
sometimes higher numbers will work fine, or certain lower numbers will
cause the
error.

Good luck tracking this one down!
Regards,
Dave (ningu)

Perl Info

Flags:
     category=core
     severity=medium

Site configuration information for perl v5.8.5:

Configured by davekam at Thu Nov 18 18:00:57 CET 2004.

Summary of my perl5 (revision 5 version 8 subversion 5) configuration:
   Platform:
     osname=darwin, osvers=7.6.0, archname=darwin-2level
     uname='darwin ext5.eva.mpg.de 7.6.0 darwin kernel version 7.6.0:  
sun oct 10 12:05:27 pdt 2004; root:xnuxnu-517.9.4.obj~1release_ppc  
power macintosh powerpc '
     config_args='-des -Dprefix=/opt -Dccflags=-I'/opt/include'  
-Dldflags=-L/opt/lib -Dvendorprefix=/opt -Duseshrplib'
     hint=recommended, useposix=true, d_sigaction=define
     usethreads=undef use5005threads=undef useithreads=undef  
usemultiplicity=undef
     useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
     use64bitint=undef use64bitall=undef uselongdouble=undef
     usemymalloc=n, bincompat5005=undef
   Compiler:
     cc='cc', ccflags ='-I/opt/include -fno-common -DPERL_DARWIN  
-no-cpp-precomp -fno-strict-aliasing -pipe -I/usr/local/include',
     optimize='-Os',
     cppflags='-no-cpp-precomp -I/opt/include -fno-common -DPERL_DARWIN  
-no-cpp-precomp -fno-strict-aliasing -pipe -I/usr/local/include'
     ccversion='', gccversion='3.3 20030304 (Apple Computer, Inc. build  
1666)', gccosandvers=''
     intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321
     d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=8
     ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',  
lseeksize=8
     alignbytes=8, prototype=define
   Linker and Libraries:
     ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags ='-L/opt/lib  
-L/usr/local/lib'
     libpth=/usr/local/lib /usr/lib
     libs=-ldbm -ldl -lm -lc
     perllibs=-ldl -lm -lc
     libc=/usr/lib/libc.dylib, so=dylib, useshrplib=true,  
libperl=libperl.dylib
     gnulibc_version=''
   Dynamic Linking:
     dlsrc=dl_dyld.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
     cccdlflags=' ', lddlflags='-L/opt/lib -bundle -undefined  
dynamic_lookup -L/usr/local/lib'

Locally applied patches:



@INC for perl v5.8.5:
     /opt/lib/perl5/5.8.5/darwin-2level
     /opt/lib/perl5/5.8.5
     /opt/lib/perl5/site_perl/5.8.5/darwin-2level
     /opt/lib/perl5/site_perl/5.8.5
     /opt/lib/perl5/site_perl
     /opt/lib/perl5/vendor_perl/5.8.5/darwin-2level
     /opt/lib/perl5/vendor_perl/5.8.5
     /opt/lib/perl5/vendor_perl
     .


Environment for perl v5.8.5:
     DYLD_LIBRARY_PATH (unset)
     HOME=/Users/davekam
     LANG (unset)
     LANGUAGE (unset)
     LD_LIBRARY_PATH (unset)
     LOGDIR (unset)
      
PATH=/sw/bin:/opt/bin:/opt/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/bin: 
/sbin:/usr/X11R6/bin:/usr/local/teTeX/bin/powerpc-apple-darwin-current
     PERL_BADLANG (unset)
     SHELL=/bin/bash


@p5pRT
Copy link
Author

p5pRT commented Dec 13, 2004

From @schwern

I can verify this problem exists in 5.8.5, 5.8.6 and bleadperl but not
5.8.1 RC3.

@p5pRT
Copy link
Author

p5pRT commented Dec 13, 2004

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jun 4, 2015

From @jkeenan

On Wed Dec 08 10​:51​:54 2004, ningu wrote​:

This is a bug report for perl from davekam@​ext5.eva.mpg.de,
generated with the help of perlbug 1.35 running under perl v5.8.5.

-----------------------------------------------------------------
[Please enter your report here]

OK, after a long discussion on #perl on freenode we've narrowed this
bug down.
It occurs only in 5.8.3 or later, on various architectures (tried on
darwin, x86,
opteron, etc.). It happens only with use encoding 'utf8', NOT with use
utf8. It
doesn't happen at all if neither are specified. The '<​:utf8' in open is
irrelevant.
Go to http​://www.plover.com/~mjd/misc/perl/ningu/ to see the original
script and
input file -- it returns 58018 lines instead of 58019. The crucial line
is map.
If you change it to foreach, it works properly.

The following oneliners (contributed by Somni on #perl) also reproduce
the problem​:

perl -Mencoding=utf8 -e 'sub i { "" } map { i join("", ""), "" }
1..2042; "$i";'
perl -Mencoding=utf8 -e 'map { join("", ""), "" } 1..2042;'
perl -Mencoding=utf8 -e 'map { join("", "") } 1..2042;'

These scripts typically produce segfault, memory wrap, or attempt to
deallocate
unreferenced scalar errors. Different archs seem to have different
magic numbers
required to trigger it, but typically a number somewhere in the 2040 to
2043 range
will do it, and anything higher. But there are also some weird things
where
sometimes higher numbers will work fine, or certain lower numbers will
cause the
error.

Good luck tracking this one down!
Regards,
Dave (ningu)

I cannot reproduce this problem with any of the 3 one-liners on Perl 5.22.0 on Ubuntu Linux 14.04-LTS.

#####
[] 28 $ perlbrew switch perl-5.22.0
[
] 29 $ perl -Mencoding=utf8 -e 'map { join("", "") } 1..2042;'
Use of the encoding pragma is deprecated at -e line 0.
[] 30 $ perl -Mencoding=utf8 -e 'map { join("", ""), "" } 1..2042;'
Use of the encoding pragma is deprecated at -e line 0.
[
] 31 $ perl -Mencoding=utf8 -e 'sub i { "" } map { i join("", ""), "" } 1..2042; "$i";'
Use of the encoding pragma is deprecated at -e line 0.
#####

The link to plover.com cited in the original post 11 years ago is no longer active, so I can't consult that source.

Can anyone else generate these errors with perl-5.22.0? If not, then I think we can close this ticket.

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Dec 9, 2015

From @jkeenan

On Wed Jun 03 18​:45​:33 2015, jkeenan wrote​:

On Wed Dec 08 10​:51​:54 2004, ningu wrote​:

This is a bug report for perl from davekam@​ext5.eva.mpg.de,
generated with the help of perlbug 1.35 running under perl v5.8.5.

-----------------------------------------------------------------
[Please enter your report here]

OK, after a long discussion on #perl on freenode we've narrowed this
bug down.
It occurs only in 5.8.3 or later, on various architectures (tried on
darwin, x86,
opteron, etc.). It happens only with use encoding 'utf8', NOT with
use
utf8. It
doesn't happen at all if neither are specified. The '<​:utf8' in open
is
irrelevant.
Go to http​://www.plover.com/~mjd/misc/perl/ningu/ to see the original
script and
input file -- it returns 58018 lines instead of 58019. The crucial
line
is map.
If you change it to foreach, it works properly.

The following oneliners (contributed by Somni on #perl) also
reproduce
the problem​:

perl -Mencoding=utf8 -e 'sub i { "" } map { i join("", ""), "" }
1..2042; "$i";'
perl -Mencoding=utf8 -e 'map { join("", ""), "" } 1..2042;'
perl -Mencoding=utf8 -e 'map { join("", "") } 1..2042;'

These scripts typically produce segfault, memory wrap, or attempt to
deallocate
unreferenced scalar errors. Different archs seem to have different
magic numbers
required to trigger it, but typically a number somewhere in the 2040
to
2043 range
will do it, and anything higher. But there are also some weird things
where
sometimes higher numbers will work fine, or certain lower numbers
will
cause the
error.

Good luck tracking this one down!
Regards,
Dave (ningu)

I cannot reproduce this problem with any of the 3 one-liners on Perl
5.22.0 on Ubuntu Linux 14.04-LTS.

#####
[] 28 $ perlbrew switch perl-5.22.0
[
] 29 $ perl -Mencoding=utf8 -e 'map { join("", "") } 1..2042;'
Use of the encoding pragma is deprecated at -e line 0.
[] 30 $ perl -Mencoding=utf8 -e 'map { join("", ""), "" } 1..2042;'
Use of the encoding pragma is deprecated at -e line 0.
[
] 31 $ perl -Mencoding=utf8 -e 'sub i { "" } map { i join("", ""),
"" } 1..2042; "$i";'
Use of the encoding pragma is deprecated at -e line 0.
#####

The link to plover.com cited in the original post 11 years ago is no
longer active, so I can't consult that source.

Can anyone else generate these errors with perl-5.22.0? If not, then
I think we can close this ticket.

Thank you very much.

No response in a year and a day. Closing ticket.

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Dec 9, 2015

@jkeenan - Status changed from 'open' to 'rejected'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant