Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loading Data::Dumper corrupts PL_sv_yes, or changes semantics of !0 at the least #12397

Closed
p5pRT opened this issue Sep 11, 2012 · 7 comments
Closed

Comments

@p5pRT
Copy link

p5pRT commented Sep 11, 2012

Migrated from rt.perl.org#114838 (status was 'resolved')

Searchable as RT114838$

@p5pRT
Copy link
Author

p5pRT commented Sep 11, 2012

From @demerphq

Created by @demerphq

This is a bug report for perl from demerphq@​gmail.com,
generated with the help of perlbug 1.39 running under perl 5.17.4.

-----------------------------------------------------------------
Loading Data​::Dumper on thread enabled perls seems to change the meaning
of !0, possibly indicating that it corrupts PL_sv_yes.

The aritificially high REFCNT demonstrates that we get PL_sv_yes and
PL_sv_no as expected​:

$ ~/bleadperl/bin/perl5.17.4 -MDevel​::Peek -le'Dump(sub{ \@​_ }->(!1,
!0))' 2>&1 | grep "REFCNT"
  REFCNT = 1
  REFCNT = 1
  REFCNT = 2147483648
  REFCNT = 2147483645

Now we load Data​::Dumper first, and we see that the !0 case magically changes​:

$ ~/bleadperl/bin/perl5.17.4 -MDevel​::Peek -MData​::Dumper
-le'Dump(sub{ \@​_ }->(!1, !0))' 2>&1 | grep "REFCNT"
  REFCNT = 1
  REFCNT = 1
  REFCNT = 2147483648
  REFCNT = 2

A closer inspection of both​:
  REFCNT = 2147483645
  FLAGS = (PADTMP,IOK,NOK,POK,READONLY,pIOK,pNOK,pPOK)
vs
  REFCNT = 2
  FLAGS = (PADTMP,IOK,NOK,POK,READONLY,pIOK,pNOK,pPOK)

Shows that other than the refcount and the fact that it is no longer
PL_sv_yes the
returned value is as expected. See
Sereal/Sereal@21a209e
for the test fail fix in Sereal which lead to this, inside Sereal we test for

sv == &PL_sv_yes

which stops happening if DD is loaded early.

Full Devel​::Peek output is here​:

$ ~/bleadperl/bin/perl5.17.4 -MDevel​::Peek -le'Dump(sub{ \@​_ }->(!1, !0))'
SV = IV(0x1126e80) at 0x1126e90
  REFCNT = 1
  FLAGS = (TEMP,ROK)
  RV = 0x1146ce8
  SV = PVAV(0x1128c58) at 0x1146ce8
  REFCNT = 1
  FLAGS = ()
  ARRAY = 0x113ebc8
  FILL = 1
  MAX = 1
  ARYLEN = 0x0
  FLAGS = (REAL)
  Elt No. 0
  SV = PVNV(0x1125cb8) at 0x1124830
  REFCNT = 2147483648
  FLAGS = (PADTMP,IOK,NOK,POK,READONLY,pIOK,pNOK,pPOK)
  IV = 0
  NV = 0
  PV = 0x7f4105 ""
  CUR = 0
  LEN = 0
  Elt No. 1
  SV = PVNV(0x1125cd8) at 0x1124848
  REFCNT = 2147483645
  FLAGS = (PADTMP,IOK,NOK,POK,READONLY,pIOK,pNOK,pPOK)
  IV = 1
  NV = 1
  PV = 0x7f4103 "1"
  CUR = 1
  LEN = 0
$ ~/bleadperl/bin/perl5.17.4 -MDevel​::Peek -MData​::Dumper
-le'Dump(sub{ \@​_ }->(!1, !0))'
SV = IV(0x17d1e98) at 0x17d1ea8
  REFCNT = 1
  FLAGS = (TEMP,ROK)
  RV = 0x17d1da0
  SV = PVAV(0x17d3c58) at 0x17d1da0
  REFCNT = 1
  FLAGS = ()
  ARRAY = 0x17e9c08
  FILL = 1
  MAX = 1
  ARYLEN = 0x0
  FLAGS = (REAL)
  Elt No. 0
  SV = PVNV(0x17d0cb8) at 0x17cf830
  REFCNT = 2147483648
  FLAGS = (PADTMP,IOK,NOK,POK,READONLY,pIOK,pNOK,pPOK)
  IV = 0
  NV = 0
  PV = 0x7f4105 ""
  CUR = 0
  LEN = 0
  Elt No. 1
  SV = PVNV(0x1a42478) at 0x17fff00
  REFCNT = 2
  FLAGS = (PADTMP,IOK,NOK,POK,READONLY,pIOK,pNOK,pPOK)
  IV = 1
  NV = 1
  PV = 0x19f3bc8 "1"\0
  CUR = 1
  LEN = 16

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.17.4:

Configured by yorton at Tue Sep 11 10:59:05 CEST 2012.

Summary of my perl5 (revision 5 version 17 subversion 4) configuration:
  Commit id: 3eebda9adef018d9dd8bb23c07514da7a3e65b97
  Platform:
    osname=linux, osvers=3.0.0-12-generic, archname=x86_64-linux-thread-multi
    uname='linux yam 3.0.0-12-generic #20-ubuntu smp fri oct 7
14:56:25 utc 2011 x86_64 x86_64 x86_64 gnulinux '
    config_args='-Doptimize=-g -d -Dusedevel -Dusethreads
-Dprefix=~/bleadperl -Dcc=ccache gcc -Dld=gcc -DDEBUGGING'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='ccache gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBUGGING
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBUGGING
-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.6.1', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
    libs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.13'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -g -L/usr/local/lib
-fstack-protector'

Locally applied patches:



@INC for perl 5.17.4:
    /home/yorton/bleadperl/lib/site_perl/5.17.4/x86_64-linux-thread-multi
    /home/yorton/bleadperl/lib/site_perl/5.17.4
    /home/yorton/bleadperl/lib/5.17.4/x86_64-linux-thread-multi
    /home/yorton/bleadperl/lib/5.17.4
    .


Environment for perl 5.17.4:
    HOME=/home/yorton
    LANG=en_US.UTF-8
    LANGUAGE=en
    LC_COLLATE=en_US.UTF-8
    LC_CTYPE=en_US.UTF-8
    LC_MESSAGES=en_US.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/yorton/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Sep 11, 2012

From @demerphq

On 11 September 2012 12​:09, yves orton <perlbug-followup@​perl.org> wrote​:

# New Ticket Created by yves orton
# Please include the string​: [perl #114838]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=114838 >

This is a bug report for perl from demerphq@​gmail.com,
generated with the help of perlbug 1.39 running under perl 5.17.4.

-----------------------------------------------------------------
Loading Data​::Dumper on thread enabled perls seems to change the meaning
of !0, possibly indicating that it corrupts PL_sv_yes.

The aritificially high REFCNT demonstrates that we get PL_sv_yes and
PL_sv_no as expected​:

$ ~/bleadperl/bin/perl5.17.4 -MDevel​::Peek -le'Dump(sub{ \@​_ }->(!1,
!0))' 2>&1 | grep "REFCNT"
REFCNT = 1
REFCNT = 1
REFCNT = 2147483648
REFCNT = 2147483645

Now we load Data​::Dumper first, and we see that the !0 case magically changes​:

$ ~/bleadperl/bin/perl5.17.4 -MDevel​::Peek -MData​::Dumper
-le'Dump(sub{ \@​_ }->(!1, !0))' 2>&1 | grep "REFCNT"
REFCNT = 1
REFCNT = 1
REFCNT = 2147483648
REFCNT = 2

A closer inspection of both​:
REFCNT = 2147483645
FLAGS = (PADTMP,IOK,NOK,POK,READONLY,pIOK,pNOK,pPOK)
vs
REFCNT = 2
FLAGS = (PADTMP,IOK,NOK,POK,READONLY,pIOK,pNOK,pPOK)

Shows that other than the refcount and the fact that it is no longer
PL_sv_yes the
returned value is as expected. See
Sereal/Sereal@21a209e
for the test fail fix in Sereal which lead to this, inside Sereal we test for

sv == &PL_sv_yes

which stops happening if DD is loaded early.

Full Devel​::Peek output is here​:

$ ~/bleadperl/bin/perl5.17.4 -MDevel​::Peek -le'Dump(sub{ \@​_ }->(!1, !0))'
SV = IV(0x1126e80) at 0x1126e90
REFCNT = 1
FLAGS = (TEMP,ROK)
RV = 0x1146ce8
SV = PVAV(0x1128c58) at 0x1146ce8
REFCNT = 1
FLAGS = ()
ARRAY = 0x113ebc8
FILL = 1
MAX = 1
ARYLEN = 0x0
FLAGS = (REAL)
Elt No. 0
SV = PVNV(0x1125cb8) at 0x1124830
REFCNT = 2147483648
FLAGS = (PADTMP,IOK,NOK,POK,READONLY,pIOK,pNOK,pPOK)
IV = 0
NV = 0
PV = 0x7f4105 ""
CUR = 0
LEN = 0
Elt No. 1
SV = PVNV(0x1125cd8) at 0x1124848
REFCNT = 2147483645
FLAGS = (PADTMP,IOK,NOK,POK,READONLY,pIOK,pNOK,pPOK)
IV = 1
NV = 1
PV = 0x7f4103 "1"
CUR = 1
LEN = 0
$ ~/bleadperl/bin/perl5.17.4 -MDevel​::Peek -MData​::Dumper
-le'Dump(sub{ \@​_ }->(!1, !0))'
SV = IV(0x17d1e98) at 0x17d1ea8
REFCNT = 1
FLAGS = (TEMP,ROK)
RV = 0x17d1da0
SV = PVAV(0x17d3c58) at 0x17d1da0
REFCNT = 1
FLAGS = ()
ARRAY = 0x17e9c08
FILL = 1
MAX = 1
ARYLEN = 0x0
FLAGS = (REAL)
Elt No. 0
SV = PVNV(0x17d0cb8) at 0x17cf830
REFCNT = 2147483648
FLAGS = (PADTMP,IOK,NOK,POK,READONLY,pIOK,pNOK,pPOK)
IV = 0
NV = 0
PV = 0x7f4105 ""
CUR = 0
LEN = 0
Elt No. 1
SV = PVNV(0x1a42478) at 0x17fff00
REFCNT = 2
FLAGS = (PADTMP,IOK,NOK,POK,READONLY,pIOK,pNOK,pPOK)
IV = 1
NV = 1
PV = 0x19f3bc8 "1"\0
CUR = 1
LEN = 16

I can reduce this down further and eliminate Data​::Dumper​:

$ ~/bleadperl/bin/perl5.17.4 -MDevel​::Peek -le'BEGIN { Dump(sub{ \@​_
}->(!1, !0));} Dump(sub{ \@​_ }->(!1, !0));' 2>&1 | grep "REFCNT\|0x"
SV = IV(0x2870f10) at 0x2870f20
  REFCNT = 1
  RV = 0x2870080
  SV = PVAV(0x2844d20) at 0x2870080
  REFCNT = 1
  ARRAY = 0x288c068
  ARYLEN = 0x0
  SV = PVNV(0x2841cb8) at 0x2840830
  REFCNT = 2147483648
  PV = 0x7f4105 ""
  SV = PVNV(0x2841cd8) at 0x2840848
  REFCNT = 2147483645
  PV = 0x7f4103 "1"
SV = IV(0x2842e80) at 0x2842e90
  REFCNT = 1
  RV = 0x2870038
  SV = PVAV(0x2845810) at 0x2870038
  REFCNT = 1
  ARRAY = 0x285abf8
  ARYLEN = 0x0
  SV = PVNV(0x2842218) at 0x2862cb8
  REFCNT = 2
  PV = 0x2878e58 ""\0
  SV = PVNV(0x2842238) at 0x2842da0
  REFCNT = 2
  PV = 0x28779d8 "1"\0

I have no idea what is going on. This only happens under threads.

Cheers,
Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

p5pRT commented Oct 9, 2012

From @cpansprout

On Tue Sep 11 03​:25​:41 2012, demerphq wrote​:

I can reduce this down further and eliminate Data​::Dumper​:

$ ~/bleadperl/bin/perl5.17.4 -MDevel​::Peek -le'BEGIN { Dump(sub{ \@​_
}->(!1, !0));} Dump(sub{ \@​_ }->(!1, !0));' 2>&1 | grep "REFCNT\|0x"
SV = IV(0x2870f10) at 0x2870f20
REFCNT = 1
RV = 0x2870080
SV = PVAV(0x2844d20) at 0x2870080
REFCNT = 1
ARRAY = 0x288c068
ARYLEN = 0x0
SV = PVNV(0x2841cb8) at 0x2840830
REFCNT = 2147483648
PV = 0x7f4105 ""
SV = PVNV(0x2841cd8) at 0x2840848
REFCNT = 2147483645
PV = 0x7f4103 "1"
SV = IV(0x2842e80) at 0x2842e90
REFCNT = 1
RV = 0x2870038
SV = PVAV(0x2845810) at 0x2870038
REFCNT = 1
ARRAY = 0x285abf8
ARYLEN = 0x0
SV = PVNV(0x2842218) at 0x2862cb8
REFCNT = 2
PV = 0x2878e58 ""\0
SV = PVNV(0x2842238) at 0x2842da0
REFCNT = 2
PV = 0x28779d8 "1"\0

I have no idea what is going on. This only happens under threads.

If you show the full Devel​::Peek output, you will see that the immortals
have the PADTMP flag set.

There is this awful piece of code in op.c​:S_finalize_op​:

  case OP_CONST​:
  if (cSVOPo->op_private & OPpCONST_STRICT)
  no_bareword_allowed(o);
  /* FALLTHROUGH */
#ifdef USE_ITHREADS
  case OP_HINTSEVAL​:
  case OP_METHOD_NAMED​:
  /* Relocate sv to the pad for thread safety.
  * Despite being a "constant", the SV is written to,
  * for reference counts, sv_upgrade() etc. */
  if (cSVOPo->op_sv) {
  const PADOFFSET ix = pad_alloc(OP_CONST, SVs_PADTMP);
  if (o->op_type != OP_METHOD_NAMED &&
  (SvPADTMP(cSVOPo->op_sv) || SvPADMY(cSVOPo->op_sv)))
  {
  /* If op_sv is already a PADTMP/MY then it is being used by
  * some pad, so make a copy. */
  sv_setsv(PAD_SVl(ix),cSVOPo->op_sv);
  SvREADONLY_on(PAD_SVl(ix));
  SvREFCNT_dec(cSVOPo->op_sv);
  }

See also
<https://rt-archive.perl.org/perl5/Ticket/Display.html?id=105906#txn-1048294>.
It’s basically the same bug.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Oct 9, 2012

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jul 26, 2013

From @cpansprout

Fixed in 82b84d0.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jul 26, 2013

From [Unknown Contact. See original ticket]

Fixed in 82b84d0.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jul 26, 2013

@cpansprout - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant