m?? doesn't only once match on -Duseithreads #12449

p5pRT · 2012-09-29T06:25:15Z

Migrated from rt.perl.org#115080 (status was 'resolved')

Searchable as RT115080$

p5pRT · 2012-09-29T06:25:15Z

From ina.cpan@gmail.com

Created by ina@cpan.org

m?? doesn't only once match on -Duseithreads

On perl 5.17.x

Without -Duseithreads
test1 = (1 0 0) m?AAA$var? $ver=qr// # ok
test2 = (1 0 0) m?AAA$var? $ver=qr/(?{1})/ # ok
test3 = (1 0 0) m?AAA$var?o $ver=qr/(?{1})/ # ok

With -Duseithreads
test1 = (1 0 0) m?AAA$var? $ver=qr// # ok
test2 = (1 1 1) m?AAA$var? $ver=qr/(?{1})/ # not only once!
test3 = (1 0 0) m?AAA$var?o $ver=qr/(?{1})/ # ok

# match_once.pl

$var = qr//;
@m_once = ();
for (qw(AAA AAA AAA)) {
if (m?AAA$var?) {
push @m_once, 1;
}
else {
push @m_once, 0;
}
}
print qq{test1 = (@m_once) m?AAA¥$var? ¥$ver=qr//¥n};

$var = qr/(?{1})/;
@m_once = ();
for (qw(AAA AAA AAA)) {
if (m?AAA$var?) {
push @m_once, 1;
}
else {
push @m_once, 0;
}
}
print qq{test2 = (@m_once) m?AAA¥$var? ¥$ver=qr/(?{1})/¥n};

$var = qr/(?{1})/;
@m_once = ();
for (qw(AAA AAA AAA)) {
if (m?AAA$var?o) { # with /o
push @m_once, 1;
}
else {
push @m_once, 0;
}
}
print qq{test3 = (@m_once) m?AAA¥$var?o ¥$ver=qr/(?{1})/¥n};

__END__

Perl Info


Flags:
    category=core
    severity=medium

Site configuration information for perl 5.17.3:

Configured by hinaba at Sat Sep 29 13:36:48 JST 2012.

Summary of my perl5 (revision 5 version 17 subversion 3) configuration:

  Platform:
    osname=linux, osvers=2.6.27-7-generic, archname=i686-linux-thread-multi
    uname='linux hinaba-desktop 2.6.27-7-generic #1 smp thu oct 30
04:18:38 utc 2008 i686 gnulinux '
    config_args='-de -Duserelocatableinc
-Dprefix=/home/hinaba/perl5/perlbrew/perls/perl-5.17.3 -Duseithreads
-Dusedevel -Aeval:scriptdir=/home/hinaba/perl5/perlbrew/perls/perl-5.17.3/bin'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.3.2', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/../lib /usr/lib/../lib /lib /usr/lib
    libs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.8.90.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.8.90'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib
-fstack-protector'

Locally applied patches:



@INC for perl 5.17.3:
    /home/hinaba/perl5/perlbrew/perls/perl-5.17.3/lib/site_perl/5.17.3/i686-linux-thread-multi
    /home/hinaba/perl5/perlbrew/perls/perl-5.17.3/lib/site_perl/5.17.3
    /home/hinaba/perl5/perlbrew/perls/perl-5.17.3/lib/5.17.3/i686-linux-thread-multi
    /home/hinaba/perl5/perlbrew/perls/perl-5.17.3/lib/5.17.3
    .


Environment for perl 5.17.3:
    HOME=/home/hinaba
    LANG=ja_JP.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
    PERL_BADLANG (unset)
    SHELL=/bin/bash

p5pRT · 2012-09-29T07:04:34Z

From @cpansprout

On Fri Sep 28 23:25:15 2012, ina.cpan@gmail.com wrote:

Cc: ina@cpan.org
Subject: m?? doesn't only once match on -Duseithreads
Message-Id: <5.17.3_27618_1348898317@hinaba-desktop>
Reply-To: ina@cpan.org
To: perlbug@perl.org
From: ina@cpan.org

This is a bug report for perl from ina@cpan.org,
generated with the help of perlbug 1.39 running under perl 5.17.3.

-----------------------------------------------------------------
[Please describe your issue here]

m?? doesn't only once match on -Duseithreads

On perl 5.17.x

Without -Duseithreads
test1 = (1 0 0) m?AAA$var? $ver=qr// # ok
test2 = (1 0 0) m?AAA$var? $ver=qr/(?{1})/ # ok
test3 = (1 0 0) m?AAA$var?o $ver=qr/(?{1})/ # ok

With -Duseithreads
test1 = (1 0 0) m?AAA$var? $ver=qr// # ok
test2 = (1 1 1) m?AAA$var? $ver=qr/(?{1})/ # not only once!
test3 = (1 0 0) m?AAA$var?o $ver=qr/(?{1})/ # ok

# match_once.pl

$var = qr//;
@m_once = ();
for (qw(AAA AAA AAA)) {
if (m?AAA$var?) {
push @m_once, 1;
}
else {
push @m_once, 0;
}
}
print qq{test1 = (@m_once) m?AAA�$var? �$ver=qr//�n};

$var = qr/(?{1})/;
@m_once = ();
for (qw(AAA AAA AAA)) {
if (m?AAA$var?) {
push @m_once, 1;
}
else {
push @m_once, 0;
}
}
print qq{test2 = (@m_once) m?AAA�$var? �$ver=qr/(?{1})/�n};

$var = qr/(?{1})/;
@m_once = ();
for (qw(AAA AAA AAA)) {
if (m?AAA$var?o) { # with /o
push @m_once, 1;
}
else {
push @m_once, 0;
}
}
print qq{test3 = (@m_once) m?AAA�$var?o �$ver=qr/(?{1})/�n};

__END__

629cd4f is the first bad commit
commit 629cd4f
Author: David Mitchell <davem@iabyn.com>
Date: Mon Dec 19 11:33:07 2011 +0000

force recompiling of regex where closures matter

There are some cases where on the second run of a run-time regex, the
text of the pattern hasn't changed, but we should still recompile to
ensure that closure behaviour is correct.

These cases are:

1) run-time code:

my $code = '(??{$x})';
for my $x (1..3) {
$x =~ /$code/; # recompile to see fresh value of $x
}

2) embedded regexes with code:

for my $x (1..3) {
my $r = qr/(??{$x})/;
"A$x" =~ /A$r/; # recompile to see new $r
}

With this fix, all the TODO tests in re/pat_re_eval.t now pass.
(Note that
a couple of those TODO tests were actually broken and are fixed in this
commit)

:100644 100644 021e14e3f1b3ecf706e654fe6040133ff1adfc0d
ebff883ed11db84e2bed3478ebcad9ea67988778 M regcomp.c
:040000 040000 d0e087a68c5fe6fef3d684ffcd65f679efa724c6
e93aee8600b4dfe7ffbd2eda44dae4acaaa2ed1e M t
bisect run success
That took 458 seconds

--

Father Chrysostomos

p5pRT · 2012-09-29T07:04:35Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2012-09-29T20:10:38Z

From @demerphq

On 29 September 2012 08:25, ina cpan <perlbug-followup@perl.org> wrote:

# New Ticket Created by ina cpan
# Please include the string: [perl #115080]
# in the subject line of all future correspondence about this issue.
# <URL: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=115080 >

Cc: ina@cpan.org
Subject: m?? doesn't only once match on -Duseithreads
Message-Id: <5.17.3_27618_1348898317@hinaba-desktop>
Reply-To: ina@cpan.org
To: perlbug@perl.org
From: ina@cpan.org

This is a bug report for perl from ina@cpan.org,
generated with the help of perlbug 1.39 running under perl 5.17.3.

-----------------------------------------------------------------
[Please describe your issue here]

m?? doesn't only once match on -Duseithreads

On perl 5.17.x

Without -Duseithreads
test1 = (1 0 0) m?AAA$var? $ver=qr// # ok
test2 = (1 0 0) m?AAA$var? $ver=qr/(?{1})/ # ok
test3 = (1 0 0) m?AAA$var?o $ver=qr/(?{1})/ # ok

With -Duseithreads
test1 = (1 0 0) m?AAA$var? $ver=qr// # ok
test2 = (1 1 1) m?AAA$var? $ver=qr/(?{1})/ # not only once!
test3 = (1 0 0) m?AAA$var?o $ver=qr/(?{1})/ # ok

# match_once.pl

$var = qr//;
@m_once = ();
for (qw(AAA AAA AAA)) {
if (m?AAA$var?) {
push @m_once, 1;
}
else {
push @m_once, 0;
}
}
print qq{test1 = (@m_once) m?AAA¥$var? ¥$ver=qr//¥n};

$var = qr/(?{1})/;
@m_once = ();
for (qw(AAA AAA AAA)) {
if (m?AAA$var?) {
push @m_once, 1;
}
else {
push @m_once, 0;
}
}
print qq{test2 = (@m_once) m?AAA¥$var? ¥$ver=qr/(?{1})/¥n};

$var = qr/(?{1})/;
@m_once = ();
for (qw(AAA AAA AAA)) {
if (m?AAA$var?o) { # with /o
push @m_once, 1;
}
else {
push @m_once, 0;
}
}
print qq{test3 = (@m_once) m?AAA¥$var?o ¥$ver=qr/(?{1})/¥n};

Im really curious what you use this for. It is rarely seen in modules,
and IMO rarely used at all, and something that from time to time I
think about proposing for removal. Can you explain what you use it
for?

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

p5pRT · 2012-10-01T04:12:33Z

From ina.cpan@gmail.com

With this fix\, all the TODO tests in re/pat\_re\_eval\.t now pass\.
(Note that
a couple of those TODO tests were actually broken and are fixed in this
commit)

Can perl 5.17.5 pass match_once.pl too?

INABA Hitoshi

p5pRT · 2012-10-01T04:12:49Z

From ina.cpan@gmail.com

2012/9/30 yves orton via RT <perlbug-followup@perl.org>:

Im really curious what you use this for. It is rarely seen in modules,
and IMO rarely used at all, and something that from time to time I
think about proposing for removal. Can you explain what you use it
for?

Yves

I have the same question with you. Because I am making source
code filters of Perl. Source code filter must handle m?? similar to
Perl. But only "thread-multi-ld" and "thread-multi" can't pass tests.

Char-EUCJP-0.83:
- x86_64-linux-thread-multi-ld / 5.17.1:
- FAIL http://www.cpantesters.org/cpan/report/abad4454-09c1-11e2-ab76-3e541f03dc99

Char-GBK-0.83:
- x86_64-linux-thread-multi / 5.17.5:
- FAIL http://www.cpantesters.org/cpan/report/6777fd48-09a7-11e2-8a94-b1761f03dc99
- x86_64-linux-thread-multi-ld / 5.17.2:
- FAIL http://www.cpantesters.org/cpan/report/f6ed22c8-09a7-11e2-a8b5-191b1f03dc99
- x86_64-linux-thread-multi-ld / 5.17.5:
- FAIL http://www.cpantesters.org/cpan/report/8bac6302-09a7-11e2-b685-f2fe1e03dc99

Char-UHC-0.83:
- x86_64-linux-thread-multi-ld / 5.17.1:
- FAIL http://www.cpantesters.org/cpan/report/5380f764-09c4-11e2-b515-bf601f03dc99
- x86_64-linux-thread-multi-ld / 5.17.2:
- FAIL http://www.cpantesters.org/cpan/report/2e9e7db8-09c4-11e2-8fa2-5f581f03dc99

Thanks your curious,
INABA Hitoshi

p5pRT · 2012-10-01T05:19:58Z

From @cpansprout

On Sun Sep 30 21:12:33 2012, ina.cpan@gmail.com wrote:

With this fix\, all the TODO tests in re/pat\_re\_eval\.t now pass\.
(Note that
a couple of those TODO tests were actually broken and are fixed
in this
commit)
Can perl 5.17.5 pass match_once.pl too?

No. What I was quoting was the commit that stopped it from working.
(Finding that out is the first step in trying to fix it.)

--

Father Chrysostomos

p5pRT · 2013-01-04T16:29:12Z

From @iabyn

On Fri, Sep 28, 2012 at 11:25:15PM -0700, ina cpan wrote:

m?? doesn't only once match on -Duseithreads

Now fixed in blead with the following commit:

commit d234778
Author: David Mitchell <davem@iabyn.com>
AuthorDate: Fri Jan 4 16:10:53 2013 +0000
Commit: David Mitchell <davem@iabyn.com>
CommitDate: Fri Jan 4 16:23:32 2013 +0000

make m?$pat? match only once under ithreads

[perl #115080]

m?...? is only supposed to match once, until reset. Normally this is done
by setting the PMf_USED flag on the PMOP. Under ithreads we can't modify
ops, so instead we indicate by setting the regex's SV to readonly. (This
is a bit of a hack: the flag should be associated with the PMOP, not the
regex).

This breaks with run-time regexes when the pattern gets recompiled; for
example:

for my $c (qw(a b c)) {
print "matched $c\n" if $c =~ m?^$c$?;
}

outputs

matched a

on unthreaded, but

matched a
matched b
matched c

on threaded.

The re_eval jumbo fix made this more noticeable by sometimes recompiling
even when the pattern text hasn't changed (to make closures work ok).

The quick fix is to propagate the readonlyness of the old re to the new
re. (The proper fix would be to store the flag state in a pad slot
associated with the PMOP).
Needless to say, I've gone for the quick fix.

--
Justice is when you get what you deserve.
Law is when you get what you pay for.

p5pRT · 2013-01-04T16:30:57Z

@iabyn - Status changed from 'open' to 'resolved'

p5pRT · 2013-01-07T00:17:18Z

From ina.cpan@gmail.com

I'm glad this ticket has been resolved. Thank you!

ina

2013/1/5 Dave Mitchell via RT <perlbug-followup@perl.org>:

According to our records, your request regarding
"m?? doesn't only once match on -Duseithreads"
has been resolved.

If you have any further questions or concerns, please respond to this message.
For other topics, please create a new ticket.
<URL: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=115080 >

p5pRT closed this as completed Jan 4, 2013

p5pRT added Severity Medium distro-Linux type-core type-regex labels Oct 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

m?? doesn't only once match on -Duseithreads #12449

m?? doesn't only once match on -Duseithreads #12449

p5pRT commented Sep 29, 2012

p5pRT commented Sep 29, 2012

p5pRT commented Sep 29, 2012

p5pRT commented Sep 29, 2012

p5pRT commented Sep 29, 2012

p5pRT commented Oct 1, 2012

p5pRT commented Oct 1, 2012

p5pRT commented Oct 1, 2012

p5pRT commented Jan 4, 2013

p5pRT commented Jan 4, 2013

p5pRT commented Jan 7, 2013

m?? doesn't only once match on -Duseithreads #12449

m?? doesn't only once match on -Duseithreads #12449

Comments

p5pRT commented Sep 29, 2012

p5pRT commented Sep 29, 2012

From ina.cpan@gmail.com

Created by ina@cpan.org

p5pRT commented Sep 29, 2012

From @cpansprout

p5pRT commented Sep 29, 2012

p5pRT commented Sep 29, 2012

From @demerphq

p5pRT commented Oct 1, 2012

From ina.cpan@gmail.com

p5pRT commented Oct 1, 2012

From ina.cpan@gmail.com

p5pRT commented Oct 1, 2012

From @cpansprout

p5pRT commented Jan 4, 2013

From @iabyn

p5pRT commented Jan 4, 2013

p5pRT commented Jan 7, 2013

From ina.cpan@gmail.com