Skip Menu |

To: perlbug [...] perl.org
Date: Sat, 21 Oct 2017 00:51:50 +0200
From: Harald.Joerg [...] arcor.de (Harald Jörg)
Subject: In Perl 5.26 and above, the /o modifier has side effects on split
This is a bug report for perl from Harald.Joerg@arcor.de, generated with the help of perlbug 1.40 running under perl 5.26.0. #!/usr/bin/perl -w # The following test program works fine with Perl 5.24, but fails with # perlbrew-installed Perl 5.26.0, 5.26.1 and 5.27.1. # # No modules beyond Test::More are required. # # In the test case, a split procedure takes the separator and the text # as parameters. As a special case, a separator of '0' stands for # 'split on an empty string'. In the test data we have 'ab', to be # split on the empty string, and 'a;b', to be split on the # semicolon. In both cases the result should be the array ('a','b'). # Perl versions up to 5.24 behave as expected, 5.26 and 5.27 fail with # the second split. use Test::More; my @records = ( { separator => '0', effective => '', text => 'ab' }, { separator => ';', effective => ';', text => 'a;b' }, ); for (@records) { my ($separator, $effective, $text) = @$_{qw(separator effective text)}; # The next statement is supposed to replace a separator of '0' by # an empty string. For whatever reason, this substution carries # the /o modifier which should be both useless and harmless # because there are no variables in the pattern. $separator =~ s/0//o; # <---- Easy workaround: remove the /o modifier # Just verifying what we're going to pass to the split function: is($separator,$effective,"Going to split '$text' with '$separator'"); my @result = split($separator,$text); is_deeply(\@result,['a','b'],"Resulting in ('a','b')"); # Yet, with Perl 5.26 the split in the second test case behaves if # the separator is still the empty string: The semicolon is an # element in @result. } done_testing; __END__ This is a breakdown from the TWiki (http://twiki.org) test suite. I have eliminated as much as I managed from the code. The bug only manifests itself *after* a split on an empty string. --- Flags: category=core severity=low --- Site configuration information for perl 5.26.0: Configured by haj at Fri Aug 11 22:05:56 CEST 2017. Summary of my perl5 (revision 5 version 26 subversion 0) configuration: Platform: osname=linux osvers=4.9.0-3-amd64 archname=x86_64-linux uname='linux hajdesktop 4.9.0-3-amd64 #1 smp debian 4.9.30-2+deb9u3 (2017-08-06) x86_64 gnulinux ' config_args='-de -Dprefix=/home/haj/perl5/perlbrew/perls/perl-5.26.0 -Aeval:scriptdir=/home/haj/perl5/perlbrew/perls/perl-5.26.0/bin' hint=recommended useposix=true d_sigaction=define useithreads=undef usemultiplicity=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='cc' ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' optimize='-O2' cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='' gccversion='6.3.0 20170516' gccosandvers='' intsize=4 longsize=8 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long' ivsize=8 nvtype='double' nvsize=8 Off_t='off_t' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='cc' ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/6/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.24.so so=so useshrplib=false libperl=libperl.a gnulibc_version='2.24' Dynamic Linking: dlsrc=dl_dlopen.xs dlext=so d_dlsymun=undef ccdlflags='-Wl,-E' cccdlflags='-fPIC' lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong' Locally applied patches: Devel::PatchPerl 1.46 --- @INC for perl 5.26.0: /home/haj/perl5/perlbrew/perls/perl-5.26.0/lib/site_perl/5.26.0/x86_64-linux /home/haj/perl5/perlbrew/perls/perl-5.26.0/lib/site_perl/5.26.0 /home/haj/perl5/perlbrew/perls/perl-5.26.0/lib/5.26.0/x86_64-linux /home/haj/perl5/perlbrew/perls/perl-5.26.0/lib/5.26.0 --- Environment for perl 5.26.0: HOME=/home/haj LANG=en_US.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/haj/perl5/perlbrew/bin:/home/haj/perl5/perlbrew/perls/perl-5.26.0/bin:/home/haj/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/home/haj/devel/twiki/trunk/TWikiDevContrib/tools PERL5LIB= PERLBREW_LIB= PERLBREW_MANPATH=/home/haj/perl5/perlbrew/perls/perl-5.26.0/man PERLBREW_PATH=/home/haj/perl5/perlbrew/bin:/home/haj/perl5/perlbrew/perls/perl-5.26.0/bin PERLBREW_PERL=perl-5.26.0 PERLBREW_ROOT=/home/haj/perl5/perlbrew PERLBREW_SKIP_INIT=1 PERLBREW_VERSION=0.78 PERL_BADLANG (unset) PERL_LOCAL_LIB_ROOT= SHELL=/bin/bash
RT-Send-CC: perl5-porters [...] perl.org
On Fri, 20 Oct 2017 22:59:52 GMT, Harald.Joerg@arcor.de wrote: Show quoted text
> This is a bug report for perl from Harald.Joerg@arcor.de, > generated with the help of perlbug 1.40 running under perl 5.26.0. > > #!/usr/bin/perl -w > # The following test program works fine with Perl 5.24, but fails with > # perlbrew-installed Perl 5.26.0, 5.26.1 and 5.27.1. > # > # No modules beyond Test::More are required. > # > # In the test case, a split procedure takes the separator and the text > # as parameters. As a special case, a separator of '0' stands for > # 'split on an empty string'. In the test data we have 'ab', to be > # split on the empty string, and 'a;b', to be split on the > # semicolon. In both cases the result should be the array ('a','b'). > # Perl versions up to 5.24 behave as expected, 5.26 and 5.27 fail with > # the second split. > > use Test::More; > > my @records = ( > { separator => '0', effective => '', text => 'ab' }, > { separator => ';', effective => ';', text => 'a;b' }, > ); > > for (@records) { > my ($separator, $effective, $text) = @$_{qw(separator effective > text)}; > > # The next statement is supposed to replace a separator of '0' by > # an empty string. For whatever reason, this substution carries > # the /o modifier which should be both useless and harmless > # because there are no variables in the pattern. > $separator =~ s/0//o; # <---- Easy workaround: remove the /o modifier > > # Just verifying what we're going to pass to the split function: > is($separator,$effective,"Going to split '$text' with '$separator'"); > > my @result = split($separator,$text); > > is_deeply(\@result,['a','b'],"Resulting in ('a','b')"); > # Yet, with Perl 5.26 the split in the second test case behaves if > # the separator is still the empty string: The semicolon is an > # element in @result. > } > > done_testing; > __END__ > > This is a breakdown from the TWiki (http://twiki.org) test suite. I > have eliminated as much as I managed from the code. The bug only > manifests itself *after* a split on an empty string. > ---
I cannot verify your claim that tests were passing with perl-5.24.0 and only started to fail with 5.26.0. In fact, using various versions of perl installed via perlbrew, I got the same results as you going back as far as perl-5.8.9. ##### [p5p] 530 $ perlbrew use perl-5.8.9 [p5p] 531 $ which perl /home/jkeenan/perl5/perlbrew/perls/perl-5.8.9/bin/perl [p5p] 532 $ prove -v 132334-split.t 132334-split....1..4 ok 1 - Going to split 'ab' with '' ok 2 - Resulting in ('a','b') ok 3 - Going to split 'a;b' with ';' not ok 4 - Resulting in ('a','b') # Failed test 'Resulting in ('a','b')' # at 132334-split.t line 25. # Structures begin differing at: # $got->[1] = ';' # $expected->[1] = 'b' # Looks like you failed 1 test of 4. dubious Test returned status 1 (wstat 256, 0x100) DIED. FAILED test 4 Failed 1/4 tests, 75.00% okay Failed Test Stat Wstat Total Fail List of Failed ------------------------------------------------------------------------------- 132334-split.t 1 256 4 1 4 Failed 1/1 test scripts. 1/4 subtests failed. Files=1, Tests=4, 0 wallclock secs ( 0.01 cusr + 0.00 csys = 0.01 CPU) Failed 1/1 test programs. 1/4 subtests failed. ##### In fact, it was only when I went back to perl-5.6.1 that I was able to get all tests to PASS. (To use perl-5.6.1, I had to provide an explicit plan for Test::More, as 'done_testing' did not exist nineteen years ago.) ##### $ perlbrew use perl-5.6.2 $ which perl /home/jkeenan/perl5/perlbrew/perls/perl-5.6.2/bin/perl $ prove -v 132334-split.t 132334-split.t .. 1..4 ok 1 - Going to split 'ab' with '' ok 2 - Resulting in ('a','b') ok 3 - Going to split 'a;b' with ';' ok 4 - Resulting in ('a','b') ok All tests successful. Files=1, Tests=4, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.01 cusr 0.00 csys = 0.03 CPU) Result: PASS ##### In any event, the '/o' modifier is described in 'perldoc perlop' as "largely obsolete." So I would recommend purging it from use in your test suite. Thank you very much. -- James E Keenan (jkeenan@cpan.org)
Subject: 132334-split.t
Download 132334-split.t
text/plain 1k
use strict; use warnings; use Test::More (tests => 4); my @records = ( { separator => '0', effective => '', text => 'ab' }, { separator => ';', effective => ';', text => 'a;b' }, ); for (@records) { my ($separator, $effective, $text) = @$_{qw(separator effective text)}; # The next statement is supposed to replace a separator of '0' by # an empty string. For whatever reason, this substution carries # the /o modifier which should be both useless and harmless # because there are no variables in the pattern. $separator =~ s/0//o; # <---- Easy workaround: remove the /o modifier # Just verifying what we're going to pass to the split function: is($separator,$effective,"Going to split '$text' with '$separator'"); my @result = split($separator,$text); is_deeply(\@result,['a','b'],"Resulting in ('a','b')"); # Yet, with Perl 5.26 the split in the second test case behaves if # the separator is still the empty string: The semicolon is an # element in @result. } #done_testing; __END__
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 2.1k
On Fri, 20 Oct 2017 16:36:47 -0700, jkeenan wrote: Show quoted text
> I cannot verify your claim that tests were passing with perl-5.24.0 > and only started to fail with 5.26.0. In fact, using various versions > of perl installed via perlbrew, I got the same results as you going > back as far as perl-5.8.9. > > ##### > [p5p] 530 $ perlbrew use perl-5.8.9 > [p5p] 531 $ which perl > /home/jkeenan/perl5/perlbrew/perls/perl-5.8.9/bin/perl > [p5p] 532 $ prove -v 132334-split.t > 132334-split....1..4 > ok 1 - Going to split 'ab' with '' > ok 2 - Resulting in ('a','b') > ok 3 - Going to split 'a;b' with ';' > > not ok 4 - Resulting in ('a','b') > # Failed test 'Resulting in ('a','b')' > # at 132334-split.t line 25. > # Structures begin differing at: > # $got->[1] = ';' > # $expected->[1] = 'b' > # Looks like you failed 1 test of 4. > dubious > Test returned status 1 (wstat 256, 0x100) > DIED. FAILED test 4 > Failed 1/4 tests, 75.00% okay > Failed Test Stat Wstat Total Fail List of Failed
... Show quoted text
> In fact, it was only when I went back to perl-5.6.1 that I was able to > get all tests to PASS.
I get that same failure with 5.24.2 and 5.22.4, but I get a different failure with 5.20.3: not ok 4 - Resulting in ('a','b') # Failed test 'Resulting in ('a','b')' # at - line 22. # Structures begin differing at: # $got->[0] = ';' # $expected->[0] = 'a' 1..4 (This time it fails on the first element of the array.) 5.18.3 gives the same results as 5.20.3. All of those are non-threaded builds. My 5.14.4 and 5.24.1 installations are threaded and give the following: $ pbpaste|perl5.14.4 ok 1 - Going to split 'ab' with '' ok 2 - Resulting in ('a','b') ok 3 - Going to split 'a;b' with ';' ok 4 - Resulting in ('a','b') 1..4 So it seems we have a discrepancy in behaviour between threaded and non-threaded builds, which is definitely a bug. Show quoted text
> In any event, the '/o' modifier is described in 'perldoc perlop' as > "largely obsolete."
Which is a bit of an exaggeration, in my opinion. Show quoted text
> So I would recommend purging it from use in your > test suite.
Nevertheless, it is a bug, and should be fixed, since /o is not deprecated. -- Father Chrysostomos
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 2.7k
On Sat, 21 Oct 2017 00:12:58 GMT, sprout wrote: Show quoted text
> On Fri, 20 Oct 2017 16:36:47 -0700, jkeenan wrote:
> > I cannot verify your claim that tests were passing with perl-5.24.0 > > and only started to fail with 5.26.0. In fact, using various > > versions > > of perl installed via perlbrew, I got the same results as you going > > back as far as perl-5.8.9. > > > > ##### > > [p5p] 530 $ perlbrew use perl-5.8.9 > > [p5p] 531 $ which perl > > /home/jkeenan/perl5/perlbrew/perls/perl-5.8.9/bin/perl > > [p5p] 532 $ prove -v 132334-split.t > > 132334-split....1..4 > > ok 1 - Going to split 'ab' with '' > > ok 2 - Resulting in ('a','b') > > ok 3 - Going to split 'a;b' with ';' > > > > not ok 4 - Resulting in ('a','b') > > # Failed test 'Resulting in ('a','b')' > > # at 132334-split.t line 25. > > # Structures begin differing at: > > # $got->[1] = ';' > > # $expected->[1] = 'b' > > # Looks like you failed 1 test of 4. > > dubious > > Test returned status 1 (wstat 256, 0x100) > > DIED. FAILED test 4 > > Failed 1/4 tests, 75.00% okay > > Failed Test Stat Wstat Total Fail List of Failed
> ...
> > In fact, it was only when I went back to perl-5.6.1 that I was able > > to > > get all tests to PASS.
> > I get that same failure with 5.24.2 and 5.22.4, but I get a different > failure with 5.20.3: > > not ok 4 - Resulting in ('a','b') > # Failed test 'Resulting in ('a','b')' > # at - line 22. > # Structures begin differing at: > # $got->[0] = ';' > # $expected->[0] = 'a' > 1..4 > > (This time it fails on the first element of the array.) > > 5.18.3 gives the same results as 5.20.3. > > All of those are non-threaded builds. My 5.14.4 and 5.24.1 > installations are threaded and give the following: > > $ pbpaste|perl5.14.4 > ok 1 - Going to split 'ab' with '' > ok 2 - Resulting in ('a','b') > ok 3 - Going to split 'a;b' with ';' > ok 4 - Resulting in ('a','b') > 1..4 > > So it seems we have a discrepancy in behaviour between threaded and > non-threaded builds, which is definitely a bug. >
Agreed. Building a threaded perl at tag v5.24.0 and running that test file, I get all tests PASSing. My unthreaded perl-5.24.1 had a FAIL on the 4th test. So, is the original poster's test setup "correct" -- in the sense that having all 4 tests pass is the expected behavior? Harald.Joerg: Do you know which of your perl builds were threaded and which were not? Show quoted text
> > In any event, the '/o' modifier is described in 'perldoc perlop' as > > "largely obsolete."
> > Which is a bit of an exaggeration, in my opinion. >
> > So I would recommend purging it from use in your > > test suite.
> > Nevertheless, it is a bug, and should be fixed, since /o is not > deprecated.
-- James E Keenan (jkeenan@cpan.org)
Subject: Re: [perl #132334] In Perl 5.26 and above, the /o modifier has side effects on split
From: Dave Mitchell <davem [...] iabyn.com>
To: Father Chrysostomos via RT <perlbug-followup [...] perl.org>
CC: perl5-porters [...] perl.org
Date: Sat, 21 Oct 2017 10:23:09 +0100
Download (untitled) / with headers
text/plain 817b
On Fri, Oct 20, 2017 at 05:12:58PM -0700, Father Chrysostomos via RT wrote: Show quoted text
> So it seems we have a discrepancy in behaviour between threaded and > non-threaded builds, which is definitely a bug.
I haven't looked too closely, but this looks like another #129158 "null ptr deref, segfault in Perl_pp_split () at pp.c:5738" It turns out that this is another variant of RT #124368: on non-threaded builds, the s///o combines: modifying the op tree under /o to avoid recompilation on subsequent iterations; and m// using the last successful pattern. The combination of the two makes /o end up modifying some random other part of the op tree. -- print+qq&$}$"$/$s$,$a$d$g$s$@$.$q$,$:$.$q$^$,$@$a$~$;$.$q$m&if+map{m,^\d{0\,},,${$::{$'}}=chr($"+=$&||1)}q&10m22,42}6:17a2~2.3@3;^2dg3q/s"&=~m*\d\*.*g
To: "James E Keenan via RT" <perlbug-followup [...] perl.org>
Subject: Re: [perl #132334] In Perl 5.26 and above, the /o modifier has side effects on split
From: Harald.Joerg [...] arcor.de (Harald Jörg)
Date: Sat, 21 Oct 2017 13:03:31 +0200
Download (untitled) / with headers
text/plain 1.1k
"James E Keenan via RT" <perlbug-followup@perl.org> writes: Show quoted text
> [...] > Agreed. Building a threaded perl at tag v5.24.0 and running that test > file, I get all tests PASSing. My unthreaded perl-5.24.1 had a FAIL > on the 4th test. > > So, is the original poster's test setup "correct" -- in the sense that > having all 4 tests pass is the expected behavior? > > Harald.Joerg: Do you know which of your perl builds were threaded and > which were not?
Sure! I have to apologize: *with a non-threaded 5.24.3, the fourth test fails as well*. I had run the 5.24 tests with Debian's system perl, which is threaded. So, this isn't about the perl version, but about threaded/unthreaded builds. Sorry for the confusion. I did some more perlbrew installs and tests: The test succeeds with: (v5.26.1) built for x86_64-linux-thread-multi (v5.24.1) built for x86_64-linux-gnu-thread-multi (v5.20.2) built for x86_64-linux-gnu-thread-multi The test fails with: (v5.27.1) built for x86_64-linux (v5.26.1) built for x86_64-linux (v5.24.3) built for x86_64-linux The "gnu-thread" Perls are what's distributed with Debian 8/9, all the others are perlbrewed. Thanks for the clarification! -- Cheers, haj
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.3k
On Sat, 21 Oct 2017 09:54:22 -0700, Harald.Joerg@arcor.de wrote: Show quoted text
> "James E Keenan via RT" <perlbug-followup@perl.org> writes: >
> > [...] > > Agreed. Building a threaded perl at tag v5.24.0 and running that test > > file, I get all tests PASSing. My unthreaded perl-5.24.1 had a FAIL > > on the 4th test. > > > > So, is the original poster's test setup "correct" -- in the sense that > > having all 4 tests pass is the expected behavior? > > > > Harald.Joerg: Do you know which of your perl builds were threaded and > > which were not?
> > Sure! I have to apologize: *with a non-threaded 5.24.3, the fourth test > fails as well*. I had run the 5.24 tests with Debian's system perl, > which is threaded. So, this isn't about the perl version, but about > threaded/unthreaded builds. Sorry for the confusion. I did some more > perlbrew installs and tests: > > The test succeeds with: > > (v5.26.1) built for x86_64-linux-thread-multi > (v5.24.1) built for x86_64-linux-gnu-thread-multi > (v5.20.2) built for x86_64-linux-gnu-thread-multi > > The test fails with: > (v5.27.1) built for x86_64-linux > (v5.26.1) built for x86_64-linux > (v5.24.3) built for x86_64-linux > > The "gnu-thread" Perls are what's distributed with Debian 8/9, all the > others are perlbrewed. Thanks for the clarification!
This might have been fixed by 3cb4cde3dd4d2af2f5065053905708bffa5168f9 Tony
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 2.9k
On Sat, 28 Oct 2017 08:10:56 GMT, tonyc wrote: Show quoted text
> On Sat, 21 Oct 2017 09:54:22 -0700, Harald.Joerg@arcor.de wrote:
> > "James E Keenan via RT" <perlbug-followup@perl.org> writes: > >
> > > [...] > > > Agreed. Building a threaded perl at tag v5.24.0 and running that test > > > file, I get all tests PASSing. My unthreaded perl-5.24.1 had a FAIL > > > on the 4th test. > > > > > > So, is the original poster's test setup "correct" -- in the sense that > > > having all 4 tests pass is the expected behavior? > > > > > > Harald.Joerg: Do you know which of your perl builds were threaded and > > > which were not?
> > > > Sure! I have to apologize: *with a non-threaded 5.24.3, the fourth test > > fails as well*. I had run the 5.24 tests with Debian's system perl, > > which is threaded. So, this isn't about the perl version, but about > > threaded/unthreaded builds. Sorry for the confusion. I did some more > > perlbrew installs and tests: > > > > The test succeeds with: > > > > (v5.26.1) built for x86_64-linux-thread-multi > > (v5.24.1) built for x86_64-linux-gnu-thread-multi > > (v5.20.2) built for x86_64-linux-gnu-thread-multi > > > > The test fails with: > > (v5.27.1) built for x86_64-linux > > (v5.26.1) built for x86_64-linux > > (v5.24.3) built for x86_64-linux > > > > The "gnu-thread" Perls are what's distributed with Debian 8/9, all the > > others are perlbrewed. Thanks for the clarification!
> > This might have been fixed by 3cb4cde3dd4d2af2f5065053905708bffa5168f9 > > Tony
I believe it has been fixed. Here are the results of running test file at commit eecd4d1156438b131ac47c72b020788f66b6d178 on non-threaded, then threaded builds on Linux. ##### [perl] 519 $ ./perl -Ilib -V | grep thread useithreads=undef libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc [perl] 520 $ ./perl -Ilib ~/learn/perl/p5p/132334-split.t 1..4 ok 1 - Going to split 'ab' with '' ok 2 - Resulting in ('a','b') ok 3 - Going to split 'a;b' with ';' ok 4 - Resulting in ('a','b') ##### [perl] 523 $ ./perl -Ilib -V | grep thread archname=x86_64-linux-thread-multi config_args='-des -Dusedevel -Duseithreads' useithreads=define libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc /usr/local/lib/perl5/site_perl/5.27.6/x86_64-linux-thread-multi /usr/local/lib/perl5/5.27.6/x86_64-linux-thread-multi [perl] 524 $ ./perl -Ilib ~/learn/perl/p5p/132334-split.t 1..4 ok 1 - Going to split 'ab' with '' ok 2 - Resulting in ('a','b') ok 3 - Going to split 'a;b' with ';' ok 4 - Resulting in ('a','b') ##### Questions: 1. Are the tests added as part of 3cb4cde3dd4d2af2f5065053905708bffa5168f9 sufficient to demonstrate that the problem has been solved? Or should we add a variant of the test file above to the test suite? 2. Should this be backported to 5.24 and 5.26? Thank you very much. -- James E Keenan (jkeenan@cpan.org)
Date: Sun, 29 Oct 2017 11:36:02 +1100
Subject: Re: [perl #132334] In Perl 5.26 and above, the /o modifier has side effects on split
From: Tony Cook <tony [...] develop-help.com>
To: James E Keenan via RT <perlbug-followup [...] perl.org>
CC: ;, perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 3.2k
On Sat, Oct 28, 2017 at 06:35:12AM -0700, James E Keenan via RT wrote: Show quoted text
> On Sat, 28 Oct 2017 08:10:56 GMT, tonyc wrote:
> > On Sat, 21 Oct 2017 09:54:22 -0700, Harald.Joerg@arcor.de wrote:
> > > "James E Keenan via RT" <perlbug-followup@perl.org> writes: > > >
> > > > [...] > > > > Agreed. Building a threaded perl at tag v5.24.0 and running that test > > > > file, I get all tests PASSing. My unthreaded perl-5.24.1 had a FAIL > > > > on the 4th test. > > > > > > > > So, is the original poster's test setup "correct" -- in the sense that > > > > having all 4 tests pass is the expected behavior? > > > > > > > > Harald.Joerg: Do you know which of your perl builds were threaded and > > > > which were not?
> > > > > > Sure! I have to apologize: *with a non-threaded 5.24.3, the fourth test > > > fails as well*. I had run the 5.24 tests with Debian's system perl, > > > which is threaded. So, this isn't about the perl version, but about > > > threaded/unthreaded builds. Sorry for the confusion. I did some more > > > perlbrew installs and tests: > > > > > > The test succeeds with: > > > > > > (v5.26.1) built for x86_64-linux-thread-multi > > > (v5.24.1) built for x86_64-linux-gnu-thread-multi > > > (v5.20.2) built for x86_64-linux-gnu-thread-multi > > > > > > The test fails with: > > > (v5.27.1) built for x86_64-linux > > > (v5.26.1) built for x86_64-linux > > > (v5.24.3) built for x86_64-linux > > > > > > The "gnu-thread" Perls are what's distributed with Debian 8/9, all the > > > others are perlbrewed. Thanks for the clarification!
> > > > This might have been fixed by 3cb4cde3dd4d2af2f5065053905708bffa5168f9 > > > > Tony
> > I believe it has been fixed. Here are the results of running test file at commit eecd4d1156438b131ac47c72b020788f66b6d178 on non-threaded, then threaded builds on Linux. > > ##### > [perl] 519 $ ./perl -Ilib -V | grep thread > useithreads=undef > libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc > perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc > [perl] 520 $ ./perl -Ilib ~/learn/perl/p5p/132334-split.t > 1..4 > ok 1 - Going to split 'ab' with '' > ok 2 - Resulting in ('a','b') > ok 3 - Going to split 'a;b' with ';' > ok 4 - Resulting in ('a','b') > > > ##### > [perl] 523 $ ./perl -Ilib -V | grep thread > archname=x86_64-linux-thread-multi > config_args='-des -Dusedevel -Duseithreads' > useithreads=define > libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc > perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc > /usr/local/lib/perl5/site_perl/5.27.6/x86_64-linux-thread-multi > /usr/local/lib/perl5/5.27.6/x86_64-linux-thread-multi > [perl] 524 $ ./perl -Ilib ~/learn/perl/p5p/132334-split.t > 1..4 > ok 1 - Going to split 'ab' with '' > ok 2 - Resulting in ('a','b') > ok 3 - Going to split 'a;b' with ';' > ok 4 - Resulting in ('a','b') > ##### > > Questions: > > 1. Are the tests added as part of 3cb4cde3dd4d2af2f5065053905708bffa5168f9 sufficient to demonstrate that the problem has been solved? Or should we add a variant of the test file above to the test suite?
A test specific this case might be useful. Show quoted text
> 2. Should this be backported to 5.24 and 5.26?
No, it changes behaviour for non-threaded builds (when it doesn't crash.) Tony
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 3.6k
On Sun, 29 Oct 2017 00:36:22 GMT, tonyc wrote: Show quoted text
> On Sat, Oct 28, 2017 at 06:35:12AM -0700, James E Keenan via RT wrote:
> > On Sat, 28 Oct 2017 08:10:56 GMT, tonyc wrote:
> > > On Sat, 21 Oct 2017 09:54:22 -0700, Harald.Joerg@arcor.de wrote:
> > > > "James E Keenan via RT" <perlbug-followup@perl.org> writes: > > > >
> > > > > [...] > > > > > Agreed. Building a threaded perl at tag v5.24.0 and running > > > > > that test > > > > > file, I get all tests PASSing. My unthreaded perl-5.24.1 had a > > > > > FAIL > > > > > on the 4th test. > > > > > > > > > > So, is the original poster's test setup "correct" -- in the > > > > > sense that > > > > > having all 4 tests pass is the expected behavior? > > > > > > > > > > Harald.Joerg: Do you know which of your perl builds were > > > > > threaded and > > > > > which were not?
> > > > > > > > Sure! I have to apologize: *with a non-threaded 5.24.3, the > > > > fourth test > > > > fails as well*. I had run the 5.24 tests with Debian's system > > > > perl, > > > > which is threaded. So, this isn't about the perl version, but > > > > about > > > > threaded/unthreaded builds. Sorry for the confusion. I did some > > > > more > > > > perlbrew installs and tests: > > > > > > > > The test succeeds with: > > > > > > > > (v5.26.1) built for x86_64-linux-thread-multi > > > > (v5.24.1) built for x86_64-linux-gnu-thread-multi > > > > (v5.20.2) built for x86_64-linux-gnu-thread-multi > > > > > > > > The test fails with: > > > > (v5.27.1) built for x86_64-linux > > > > (v5.26.1) built for x86_64-linux > > > > (v5.24.3) built for x86_64-linux > > > > > > > > The "gnu-thread" Perls are what's distributed with Debian 8/9, > > > > all the > > > > others are perlbrewed. Thanks for the clarification!
> > > > > > This might have been fixed by > > > 3cb4cde3dd4d2af2f5065053905708bffa5168f9 > > > > > > Tony
> > > > I believe it has been fixed. Here are the results of running test > > file at commit eecd4d1156438b131ac47c72b020788f66b6d178 on non- > > threaded, then threaded builds on Linux. > > > > ##### > > [perl] 519 $ ./perl -Ilib -V | grep thread > > useithreads=undef > > libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc > > perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc > > [perl] 520 $ ./perl -Ilib ~/learn/perl/p5p/132334-split.t > > 1..4 > > ok 1 - Going to split 'ab' with '' > > ok 2 - Resulting in ('a','b') > > ok 3 - Going to split 'a;b' with ';' > > ok 4 - Resulting in ('a','b') > > > > > > ##### > > [perl] 523 $ ./perl -Ilib -V | grep thread > > archname=x86_64-linux-thread-multi > > config_args='-des -Dusedevel -Duseithreads' > > useithreads=define > > libs=-lpthread -lnsl -ldb -ldl -lm -lcrypt -lutil -lc > > perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc > > /usr/local/lib/perl5/site_perl/5.27.6/x86_64-linux-thread-multi > > /usr/local/lib/perl5/5.27.6/x86_64-linux-thread-multi > > [perl] 524 $ ./perl -Ilib ~/learn/perl/p5p/132334-split.t > > 1..4 > > ok 1 - Going to split 'ab' with '' > > ok 2 - Resulting in ('a','b') > > ok 3 - Going to split 'a;b' with ';' > > ok 4 - Resulting in ('a','b') > > ##### > > > > Questions: > > > > 1. Are the tests added as part of > > 3cb4cde3dd4d2af2f5065053905708bffa5168f9 sufficient to demonstrate > > that the problem has been solved? Or should we add a variant of the > > test file above to the test suite?
> > A test specific this case might be useful. >
Test adapted from original poster's code added in commit 6e8135a4380966ca56b30d95854b54dc8d9c08d4. Show quoted text
> > 2. Should this be backported to 5.24 and 5.26?
> > No, it changes behaviour for non-threaded builds (when it doesn't > crash.) > > Tony
Marking ticket Resolved. Thank you very much. -- James E Keenan (jkeenan@cpan.org)


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org