From Mark.Martinec@ijs.si

Created by Mark.Martinec@ijs.si

Came across a small handful of complex regular expressions
(in SpamAssassin rules), which seem to unwarrantedly hit an
error​: "Variable length lookbehind not implemented in regex"
under perl 5.16.0 (on FreeBSD).

Here is a distilled-down sample case​:

good​:
  $ perl -e 'm/(?<!abc|cde)/i'

good​:
  $ perl -e 'm/(?<!abc|css)/'

incorrect diagnostics​:
  $ perl -e 'm/(?<!abc|css)/i'
  Variable length lookbehind not implemented in regex m/(?<!abc|css)/ at -e line 1.

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.16.0:

Configured by mark at Mon May 28 17:29:38 CEST 2012.

Summary of my perl5 (revision 5 version 16 subversion 0) configuration:
   
  Platform:
    osname=freebsd, osvers=7.2-release-p2, archname=amd64-freebsd
    uname='freebsd dorothy.ijs.si 7.2-release-p2 freebsd 7.2-release-p2 #0: wed jul 15 15:45:26 cest 2009 
lesi@dorothy.ijs.si:usrobjusrsrcsysdorothy amd64 '
    config_args='-sde -Dprefix=/usr/local -Darchlib=/usr/local/lib/perl5/5.16.0/mach -Dprivlib=/usr/local/lib/perl5/5.16.0 -
Dman3dir=/usr/local/lib/perl5/5.16.0/perl/man/man3 -Dman1dir=/usr/local/man/man1 -
Dsitearch=/usr/local/lib/perl5/site_perl/5.16.0/mach -Dsitelib=/usr/local/lib/perl5/site_perl/5.16.0 -Dscriptdir=/usr/local/bin -
Dsiteman3dir=/usr/local/lib/perl5/5.16.0/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Ui_malloc -Ui_iconv -Uinstallusrbinperl -
Dcc=cc -Duseshrplib -Dinc_version_list=none -Dccflags=-DAPPLLIB_EXP="/usr/local/lib/perl5/5.16.0/BSDPAN" -Doptimize=-O2 -fno-
strict-aliasing -pipe -Ui_gdbm -Dusethreads=n -Dusemymalloc=n -Duse64bitint'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-DAPPLLIB_EXP="/usr/local/lib/perl5/5.16.0/BSDPAN" -DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-
aliasing -pipe -fstack-protector -I/usr/local/include',
    optimize='-O2 -fno-strict-aliasing -pipe',
    cppflags='-DAPPLLIB_EXP="/usr/local/lib/perl5/5.16.0/BSDPAN" -DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe 
-fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.2.1 20070719  [FreeBSD]', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags ='-pthread -Wl,-E  -fstack-protector -L/usr/local/lib'
    libpth=/usr/lib /usr/local/lib
    libs=-lgdbm -lm -lcrypt -lutil
    perllibs=-lm -lcrypt -lutil
    libc=, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='  -Wl,-R/usr/local/lib/perl5/5.16.0/mach/CORE'
    cccdlflags='-DPIC -fPIC', lddlflags='-shared  -L/usr/local/lib -fstack-protector'

Locally applied patches:
    


@INC for perl 5.16.0:
    /usr/local/lib/perl5/5.16.0/BSDPAN
    /usr/local/lib/perl5/site_perl/5.16.0/mach
    /usr/local/lib/perl5/site_perl/5.16.0
    /usr/local/lib/perl5/5.16.0/mach
    /usr/local/lib/perl5/5.16.0
    .


Environment for perl 5.16.0:
    HOME=/root
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin
    PERL_BADLANG (unset)
    SHELL=/usr/local/bin/bash

From @doy

On Mon, Jun 04, 2012 at 09​:50​:44AM -0700, Mark Martinec wrote​:

Came across a small handful of complex regular expressions
(in SpamAssassin rules), which seem to unwarrantedly hit an
error​: "Variable length lookbehind not implemented in regex"
under perl 5.16.0 (on FreeBSD).

Here is a distilled-down sample case​:

good​:
$ perl -e 'm/(?<!abc|cde)/i'

good​:
$ perl -e 'm/(?<!abc|css)/'

incorrect diagnostics​:
$ perl -e 'm/(?<!abc|css)/i'
Variable length lookbehind not implemented in regex m/(?<!abc|css)/ at -e line 1.

This is likely because in 5.16, /ss/i compiles to something along the
lines of /(?​:ss|ß)/. You could possibly use \K instead of lookbehind
(depending on the pattern), or use the /a regex modifier to enforce
ASCII semantics.

-doy

The RT System itself - Status changed from 'new' to 'open'

From @cpansprout

On Mon Jun 04 09​:59​:07 2012, doy@​tozt.net wrote​:

On Mon, Jun 04, 2012 at 09​:50​:44AM -0700, Mark Martinec wrote​:

Came across a small handful of complex regular expressions
(in SpamAssassin rules), which seem to unwarrantedly hit an
error​: "Variable length lookbehind not implemented in regex"
under perl 5.16.0 (on FreeBSD).

Here is a distilled-down sample case​:

good​:
$ perl -e 'm/(?<!abc|cde)/i'

good​:
$ perl -e 'm/(?<!abc|css)/'

incorrect diagnostics​:
$ perl -e 'm/(?<!abc|css)/i'
Variable length lookbehind not implemented in regex
m/(?<!abc|css)/ at -e line 1.

This is likely because in 5.16, /ss/i compiles to something along the
lines of /(?​:ss|�)/. You could possibly use \K instead of lookbehind
(depending on the pattern), or use the /a regex modifier to enforce
ASCII semantics.

The /aa modifier is what you need here.

--

Father Chrysostomos

From @ikegami

On Mon, Jun 4, 2012 at 12​:58 PM, Jesse Luehrs <doy@​tozt.net> wrote​:

On Mon, Jun 04, 2012 at 09​:50​:44AM -0700, Mark Martinec wrote​:

Came across a small handful of complex regular expressions
(in SpamAssassin rules), which seem to unwarrantedly hit an
error​: "Variable length lookbehind not implemented in regex"
under perl 5.16.0 (on FreeBSD).

Here is a distilled-down sample case​:

good​:
$ perl -e 'm/(?<!abc|cde)/i'

good​:
$ perl -e 'm/(?<!abc|css)/'

incorrect diagnostics​:
$ perl -e 'm/(?<!abc|css)/i'
Variable length lookbehind not implemented in regex m/(?<!abc|css)/ at
-e line 1.

This is likely because in 5.16, /ss/i compiles to something along the
lines of /(?​:ss|ß)/. You could possibly use \K instead of lookbehind
(depending on the pattern), or use the /a regex modifier to enforce
ASCII semantics.

This works too​:

$ perl -e 'm/(?<!abc|cs[s])/i'

From Mark.Martinec@ijs.si

This is likely because in 5.16, /ss/i compiles to
something along the lines of /(?​:ss|�)/.
The /aa modifier is what you need here.

Thank you for the explanation!

Looks like we're onto some more surprises in that area.

As far as I'm concerned, this ticket can be closed.

  Mark

@rjbs - Status changed from 'open' to 'resolved'

@ikegami - Status changed from 'resolved' to 'rejected'