From @iabyn

Created by @iabyn

This code prints 'a' in 5.16.x, undef in 5.18.0​:

  my $pat = qr/a/;
  'aaaa' =~ /$pat/gp or die;
  print "MATCH=[${^MATCH}]\n";

A bisect shows I broke it​:

commit 2c7b5d7
Author​: David Mitchell <davem@​iabyn.com>
Date​: Thu Jul 26 15​:35​:39 2012 +0100

  Separate handling of ${^PREMATCH} from $` etc
 
  Currently the handling of getting the value, length etc of ${^PREMATCH}
  etc is identical to that of $` etc.
 
  Handle them separately, by adding RX_BUFF_IDX_CARET_PREMATCH etc
  constants to the existing RX_BUFF_IDX_PREMATCH set.
 
  This allows, when retrieving them, to always return undef if the current
  match didn't use //p. Previously the result depended on stuff such
  as whether the (non-//p) pattern included captures or not.
 
  The documentation for ${^PREMATCH} etc states that it's only guaranteed to
  return a defined value when the last pattern was //p.
 
  As well as making things more consistent, this is a necessary
  prerequisite for the following commit, which may not always copy the
  whole string during a non-//p match.

First spotted on perlmonks​:

  http​://www.perlmonks.org/?node_id=1035503

(I intend to work on this ticket)

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.18.0:

Configured by davem at Sat May 18 22:48:32 BST 2013.

Summary of my perl5 (revision 5 version 18 subversion 0) configuration:
   
  Platform:
    osname=linux, osvers=3.8.11-200.fc18.x86_64, archname=x86_64-linux
    uname='linux robin 3.8.11-200.fc18.x86_64 #1 smp wed may 1 19:44:27 utc 2013 x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Dprefix=/home/davem/perl5/a/perl-5.18.0.out -Uinstallusrbinperl -Doptimize=-g -Accflags=-DDEBUGGING -ggdb'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-DDEBUGGING -ggdb -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-g',
    cppflags='-DDEBUGGING -ggdb -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.7.2 20121109 (Red Hat 4.7.2-8)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/../lib64 /usr/lib/../lib64 /lib /usr/lib /lib64 /usr/lib64 /usr/local/lib64
    libs=-lnsl -ldb -ldl -lm -lcrypt -lutil -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
    libc=/lib/libc-2.16.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.16'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -g -L/usr/local/lib -fstack-protector'

Locally applied patches:
    


@INC for perl 5.18.0:
    /home/davem/perl5/a/perl-5.18.0.out/lib/site_perl/5.18.0/x86_64-linux
    /home/davem/perl5/a/perl-5.18.0.out/lib/site_perl/5.18.0
    /home/davem/perl5/a/perl-5.18.0.out/lib/5.18.0/x86_64-linux
    /home/davem/perl5/a/perl-5.18.0.out/lib/5.18.0
    .


Environment for perl 5.18.0:
    HOME=/home/davem
    LANG=en_GB.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/lib64/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/home/davem/bin:/home/davem/bin-perl
    PERL_BADLANG (unset)
    SHELL=/bin/bash

From @cpansprout

On Tue May 28 03​:45​:45 2013, davem wrote​:

This is a bug report for perl from davem@​iabyn.com,
generated with the help of perlbug 1.39 running under perl 5.18.0.

-----------------------------------------------------------------
[Please describe your issue here]

This code prints 'a' in 5.16.x, undef in 5.18.0​:

my $pat = qr/a/;
'aaaa'  =~ /$pat/gp or die;
print "MATCH=\[$\{^MATCH\}\]\\n";

This variation​:

my $pat = qr/a/;
'aaaa' =~ /$pat/gp or die;
print "MATCH=[${^MATCH}-$&]\n";

prints this under blead​:

MATCH=[-a]

It’s this bit in regcomp.c​:Perl_reg_numbered_buff_fetch​:

  if ( ( n == RX_BUFF_IDX_CARET_PREMATCH
  || n == RX_BUFF_IDX_CARET_FULLMATCH
  || n == RX_BUFF_IDX_CARET_POSTMATCH
  )
  && !(rx->extflags & RXf_PMf_KEEPCOPY)
  )
  goto ret_undef;

Instead of checking the KEEPCOPY flag, is it possible to check whether
the offsets are recorded?

I know this flag makes no difference under COW and we can probably
eliminate that whole if(). But do we need to fix this bug under
PERL_NO_COW as well?

--

Father Chrysostomos

The RT System itself - Status changed from 'new' to 'open'

From @iabyn

On Sat, Jul 06, 2013 at 10​:48​:57PM -0700, Father Chrysostomos via RT wrote​:

This code prints 'a' in 5.16.x, undef in 5.18.0​:

my $pat = qr/a/;
'aaaa'  =~ /$pat/gp or die;
print "MATCH=\[$\{^MATCH\}\]\\n";

This variation​:

my $pat = qr/a/;
'aaaa' =~ /$pat/gp or die;
print "MATCH=[${^MATCH}-$&]\n";

prints this under blead​:

MATCH=[-a]

It’s this bit in regcomp.c​:Perl_reg_numbered_buff_fetch​:

if \( \(    n == RX\_BUFF\_IDX\_CARET\_PREMATCH
       || n == RX\_BUFF\_IDX\_CARET\_FULLMATCH
       || n == RX\_BUFF\_IDX\_CARET\_POSTMATCH
     \)
     && \!\(rx\->extflags & RXf\_PMf\_KEEPCOPY\)
\)
    goto ret\_undef;

Instead of checking the KEEPCOPY flag, is it possible to check whether
the offsets are recorded?

I know this flag makes no difference under COW and we can probably
eliminate that whole if(). But do we need to fix this bug under
PERL_NO_COW as well?

Now fixed by 5b0e71e, with a 2-commit maint-5.18 branch available
as davem/maint-5.18-118213.

--
"There's something wrong with our bloody ships today, Chatfield."
  -- Admiral Beatty at the Battle of Jutland, 31st May 1916.

From @cpansprout

On Tue Jul 30 12​:37​:25 2013, davem wrote​:

On Sat, Jul 06, 2013 at 10​:48​:57PM -0700, Father Chrysostomos via RT
wrote​:

This code prints 'a' in 5.16.x, undef in 5.18.0​:

my $pat = qr/a/;
'aaaa'  =~ /$pat/gp or die;
print "MATCH=\[$\{^MATCH\}\]\\n";

This variation​:

my $pat = qr/a/;
'aaaa' =~ /$pat/gp or die;
print "MATCH=[${^MATCH}-$&]\n";

prints this under blead​:

MATCH=[-a]

It’s this bit in regcomp.c​:Perl_reg_numbered_buff_fetch​:

if \( \(    n == RX\_BUFF\_IDX\_CARET\_PREMATCH
       || n == RX\_BUFF\_IDX\_CARET\_FULLMATCH
       || n == RX\_BUFF\_IDX\_CARET\_POSTMATCH
     \)
     && \!\(rx\->extflags & RXf\_PMf\_KEEPCOPY\)
\)
    goto ret\_undef;

Instead of checking the KEEPCOPY flag, is it possible to check whether
the offsets are recorded?

I know this flag makes no difference under COW and we can probably
eliminate that whole if(). But do we need to fix this bug under
PERL_NO_COW as well?

Now fixed by 5b0e71e, with a 2-commit maint-5.18 branch available
as davem/maint-5.18-118213.

+1 for 5.18.whatever

--

Father Chrysostomos

From @rjbs

These were applied as b4d6b02 and d126b76.
--
rjbs

@rjbs - Status changed from 'open' to 'resolved'