Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

regex compilation can leak memory #16724

Open
p5pRT opened this issue Oct 14, 2018 · 4 comments
Open

regex compilation can leak memory #16724

p5pRT opened this issue Oct 14, 2018 · 4 comments

Comments

@p5pRT
Copy link

p5pRT commented Oct 14, 2018

Migrated from rt.perl.org#133589 (status was 'open')

Searchable as RT133589$

@p5pRT
Copy link
Author

p5pRT commented Oct 14, 2018

From @khwilliamson

This is a bug report for perl from khw@​cpan.org,
generated with the help of perlbug 1.41 running under perl 5.29.4.


In looking at the code in regcomp.c, I noticed that one way it tries to
avoid leaks when raising errors is that it mortalizes an SV before
dying. This works, but the scalar doesn't get mortalized if warnings
are generated, so if those are fatalized there would be a leak. Further,
other functions are called which can output warnings or errors, and
those functions don't mortalize the SV. As an example, grok_hex() can
be called with options that allow it to output warnings. If those are
fatal, a leak will occur. User-defined \p{} properties call perl code
which can die or warn; again leaks can occur.



Flags​:
  category=core
  severity=low


Site configuration information for perl 5.29.4​:

Configured by khw at Sun Oct 14 10​:24​:54 MDT 2018.

Summary of my perl5 (revision 5 version 29 subversion 4) configuration​:
  Commit id​: 7bf15fe
  Platform​:
  osname=linux
  osvers=4.15.0-36-generic
  archname=x86_64-linux-thread-multi-ld
  uname='linux khw-xps-8930 4.15.0-36-generic #39-ubuntu smp mon sep
24 16​:19​:09 utc 2018 x86_64 x86_64 x86_64 gnulinux '
  config_args='-des -Uversiononly -Dprefix=/home/khw/blead -Dusedevel
-A'optimize=-ggdb3' -A'optimize=-O0' -Accflags='-Wno-deprecated'
-Accflags='-DPERL_EXTERNAL_GLOB' -Dman1dir=none -Dman3dir=none -Dcc=g++
-DDEBUGGING -Dusemorebits -Dusecbacktrace -Dusethreads'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=define
  usemultiplicity=define
  use64bitint=define
  use64bitall=define
  uselongdouble=define
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='g++'
  ccflags ='-D_REENTRANT -D_GNU_SOURCE -Wno-deprecated
-DPERL_EXTERNAL_GLOB -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include -DUSE_C_BACKTRACE -g
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
  optimize='-O2 -ggdb3 -O0'
  cppflags='-D_REENTRANT -D_GNU_SOURCE -Wno-deprecated
-DPERL_EXTERNAL_GLOB -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='7.3.0'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='long double'
  nvsize=16
  Off_t='off_t'
  lseeksize=8
  alignbytes=16
  prototype=define
  Linker and Libraries​:
  ld='g++'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/include/c++/7 /usr/include/x86_64-linux-gnu/c++/7
/usr/include/c++/7/backward /usr/local/lib
/usr/lib/gcc/x86_64-linux-gnu/7/include-fixed
/usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.27.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.27'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -O2 -ggdb3 -O0 -L/usr/local/lib
-fstack-protector-strong'


@​INC for perl 5.29.4​:
  /home/khw/perl/blead/lib
  /home/khw/perl/blead/t
  /home/khw/blead/lib/perl5/site_perl/5.29.4/x86_64-linux-thread-multi-ld
  /home/khw/blead/lib/perl5/site_perl/5.29.4
  /home/khw/blead/lib/perl5/5.29.4/x86_64-linux-thread-multi-ld
  /home/khw/blead/lib/perl5/5.29.4
  /home/khw/blead/lib/perl5/site_perl/5.29.3
  /home/khw/blead/lib/perl5/site_perl/5.29.2
  /home/khw/blead/lib/perl5/site_perl/5.29.1
  /home/khw/blead/lib/perl5/site_perl/5.29.0
  /home/khw/blead/lib/perl5/site_perl/5.28.0
  /home/khw/blead/lib/perl5/site_perl/5.27.11
  /home/khw/blead/lib/perl5/site_perl/5.27.10
  /home/khw/blead/lib/perl5/site_perl/5.27.9
  /home/khw/blead/lib/perl5/site_perl/5.27.8
  /home/khw/blead/lib/perl5/site_perl/5.27.7
  /home/khw/blead/lib/perl5/site_perl/5.27.6
  /home/khw/blead/lib/perl5/site_perl/5.27.5
  /home/khw/blead/lib/perl5/site_perl/5.27.4
  /home/khw/blead/lib/perl5/site_perl/5.27.3
  /home/khw/blead/lib/perl5/site_perl/5.27.2
  /home/khw/blead/lib/perl5/site_perl/5.27.1
  /home/khw/blead/lib/perl5/site_perl/5.27.0
  /home/khw/blead/lib/perl5/site_perl/5.26.0
  /home/khw/blead/lib/perl5/site_perl/5.25.12
  /home/khw/blead/lib/perl5/site_perl/5.25.11
  /home/khw/blead/lib/perl5/site_perl/5.25.10
  /home/khw/blead/lib/perl5/site_perl/5.25.9
  /home/khw/blead/lib/perl5/site_perl/5.25.8
  /home/khw/blead/lib/perl5/site_perl/5.25.7
  /home/khw/blead/lib/perl5/site_perl/5.25.6
  /home/khw/blead/lib/perl5/site_perl/5.25.5
  /home/khw/blead/lib/perl5/site_perl/5.25.4
  /home/khw/blead/lib/perl5/site_perl/5.25.3
  /home/khw/blead/lib/perl5/site_perl/5.25.2
  /home/khw/blead/lib/perl5/site_perl/5.25.1
  /home/khw/blead/lib/perl5/site_perl/5.24.0
  /home/khw/blead/lib/perl5/site_perl/5.23.10
  /home/khw/blead/lib/perl5/site_perl/5.23.9
  /home/khw/blead/lib/perl5/site_perl/5.23.8
  /home/khw/blead/lib/perl5/site_perl/5.23.7
  /home/khw/blead/lib/perl5/site_perl/5.23.6
  /home/khw/blead/lib/perl5/site_perl/5.23.5
  /home/khw/blead/lib/perl5/site_perl/5.23.4
  /home/khw/blead/lib/perl5/site_perl/5.23.3
  /home/khw/blead/lib/perl5/site_perl/5.23.2
  /home/khw/blead/lib/perl5/site_perl/5.23.1
  /home/khw/blead/lib/perl5/site_perl/5.23.0
  /home/khw/blead/lib/perl5/site_perl/5.22.0
  /home/khw/blead/lib/perl5/site_perl/5.21.12
  /home/khw/blead/lib/perl5/site_perl/5.21.11
  /home/khw/blead/lib/perl5/site_perl/5.21.10
  /home/khw/blead/lib/perl5/site_perl/5.21.9
  /home/khw/blead/lib/perl5/site_perl/5.21.8
  /home/khw/blead/lib/perl5/site_perl/5.21.7
  /home/khw/blead/lib/perl5/site_perl/5.21.6
  /home/khw/blead/lib/perl5/site_perl/5.21.5
  /home/khw/blead/lib/perl5/site_perl/5.21.4
  /home/khw/blead/lib/perl5/site_perl/5.21.3
  /home/khw/blead/lib/perl5/site_perl/5.21.2
  /home/khw/blead/lib/perl5/site_perl/5.21.1
  /home/khw/blead/lib/perl5/site_perl/5.20.0
  /home/khw/blead/lib/perl5/site_perl/5.19.12
  /home/khw/blead/lib/perl5/site_perl/5.19.11
  /home/khw/blead/lib/perl5/site_perl/5.19.10
  /home/khw/blead/lib/perl5/site_perl


Environment for perl 5.29.4​:
  HOME=/home/khw
  LANG=en_US.UTF-8
  LANGUAGE=en_US
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)

PATH=/usr/lib/ccache​:/home/khw/bin​:/home/khw/perl5/perlbrew/bin​:/home/khw/print/bin​:/bin​:/usr/local/sbin​:/usr/local/bin​:/usr/sbin​:/usr/bin​:/sbin​:/usr/games​:/usr/local/games​:/snap/bin​:/home/khw/iands/www​:/home/khw/cxoffice/bin
  PERL5OPT=-w
  PERL_BADLANG (unset)
  PERL_DIFF_TOOL=wgdiff
  PERL_POD_PEDANTIC=1
  SHELL=/bin/ksh

@p5pRT
Copy link
Author

p5pRT commented Dec 6, 2018

From @jkeenan

On Sun, 14 Oct 2018 18​:56​:50 GMT, public@​khwilliamson.com wrote​:

This is a bug report for perl from khw@​cpan.org,
generated with the help of perlbug 1.41 running under perl 5.29.4.

-----------------------------------------------------------------
In looking at the code in regcomp.c, I noticed that one way it tries
to
avoid leaks when raising errors is that it mortalizes an SV before
dying. This works, but the scalar doesn't get mortalized if warnings
are generated, so if those are fatalized there would be a leak.
Further,
other functions are called which can output warnings or errors, and
those functions don't mortalize the SV. As an example, grok_hex() can
be called with options that allow it to output warnings. If those are
fatal, a leak will occur. User-defined \p{} properties call perl code
which can die or warn; again leaks can occur.

Could a TODO test be written for this?

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Dec 6, 2018

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Dec 9, 2018

From @khwilliamson

On 12/6/18 1​:26 PM, James E Keenan via RT wrote​:

On Sun, 14 Oct 2018 18​:56​:50 GMT, public@​khwilliamson.com wrote​:

This is a bug report for perl from khw@​cpan.org,
generated with the help of perlbug 1.41 running under perl 5.29.4.

-----------------------------------------------------------------
In looking at the code in regcomp.c, I noticed that one way it tries
to
avoid leaks when raising errors is that it mortalizes an SV before
dying. This works, but the scalar doesn't get mortalized if warnings
are generated, so if those are fatalized there would be a leak.
Further,
other functions are called which can output warnings or errors, and
those functions don't mortalize the SV. As an example, grok_hex() can
be called with options that allow it to output warnings. If those are
fatal, a leak will occur. User-defined \p{} properties call perl code
which can die or warn; again leaks can occur.

Could a TODO test be written for this?

Thank you very much.

Certainly. I know of various cases where it happens, but it would be
best to fix it in the general case, and I don't know all the places it
could happen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants