Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

length($@)>0 for empty $@ if utf8 is in use #9238

Closed
p5pRT opened this issue Mar 3, 2008 · 19 comments
Closed

length($@)>0 for empty $@ if utf8 is in use #9238

p5pRT opened this issue Mar 3, 2008 · 19 comments

Comments

@p5pRT
Copy link

p5pRT commented Mar 3, 2008

Migrated from rt.perl.org#51370 (status was 'resolved')

Searchable as RT51370$

@p5pRT
Copy link
Author

p5pRT commented Mar 3, 2008

From pajas@ufal.mff.cuni.cz

Created by pajas@ufal.mff.cuni.cz

Hi,

Perl seems to cache character length of scalars, but in case of $@​
it does not reset it after a successfull eval. Here is an example.

The following is reproducible with 5.10.0 and also on 5.8.8 (didn't
try other releases)​:

perl -MCarp -e 'use utf8; eval { die "\x{10d}"}; length($@​); print
$@​; eval { 1 }; print "\$@​ is q($@​), length(\$@​)
is ".length($@​)."\n" '

The first eval dies and spits a non-ascii character. Then the
character length of $@​ is asked (we are in the scope of the utf8
pragma). The returned value (16 in this case) seems to be
remembered,
since after a subsequent eval{1} which resets $@​ to q(), the
function
length($@​) still returns 16 (but 0 if asked in the scope of 'use
byte').

-- Petr

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.10.0:

Configured by pajas at Tue Jan  8 20:45:59 CET 2008.

Summary of my perl5 (revision 5 version 10 subversion 0) 
configuration:
  Platform:
    osname=linux, osvers=2.6.22.13-0.3-default, 
archname=x86_64-linux
    uname='linux stain 2.6.22.13-0.3-default #1 smp 20071119 
15:02:58 utc x86_64 x86_64 x86_64 gnulinux '
    config_args='-de'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    useperlio=define, d_sfio=undef, uselargefiles=define, 
usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags 
='-fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='4.2.1 (SUSE Linux)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, 
byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, 
longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, 
Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    
libpth=/usr/local/lib /lib /usr/lib /lib64 /usr/lib64 /usr/local/lib64
    libs=-lnsl -lndbm -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
    libc=/lib/libc-2.6.1.so, so=so, useshrplib=false, 
libperl=libperl.a
    gnulibc_version='2.6.1'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, 
ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib'

Locally applied patches:
    


@INC for perl 5.10.0:
    /usr/local/lib/perl5/5.10.0/x86_64-linux
    /usr/local/lib/perl5/5.10.0
    /usr/local/lib/perl5/site_perl/5.10.0/x86_64-linux
    /usr/local/lib/perl5/site_perl/5.10.0
    .


Environment for perl 5.10.0:
    HOME=/home/pajas
    LANG=cs_CZ.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    
PATH=/home/pajas/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/kde3/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/usr/local/texlive/2007/bin/i386-linux
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Apr 26, 2008

From p5p@spam.wizbit.be

On Mon Mar 03 06​:41​:54 2008, pajas@​ufal.mff.cuni.cz wrote​:

Perl seems to cache character length of scalars, but in case of $@​
it does not reset it after a successfull eval. Here is an example.

...

Hello,

I took a look at this bug-report and the magic assoicated with $@​ does
not get cleared.

I did some testing on how to turn it off but I'm not sure what 'the
correct way' is. (Therefor there is no patch)
Can someone offer some guidance?

The test case​:

#!/usr/bin/perl -l

use utf8;
use Devel​::Peek
eval { 1 };
Dump($@​);
eval { die "\x{a10d}"};
$_ = length $@​;
eval { 1 };
Dump($@​);

The current output (on blead)​:

SV = PV(0x8130f04) at 0x812ffd8
  REFCNT = 1
  FLAGS = (POK,pPOK)
  PV = 0x813d8d0 ""\0
  CUR = 0
  LEN = 240
SV = PVMG(0x813d390) at 0x812ffd8
  REFCNT = 1
  FLAGS = (SMG,POK,pPOK,UTF8)
  IV = 0
  NV = 0
  PV = 0x813d8d0 ""\0 [UTF8 ""]
  CUR = 0
  LEN = 240
  MAGIC = 0x8145e38
  MG_VIRTUAL = &PL_vtbl_utf8
  MG_TYPE = PERL_MAGIC_utf8(w)
  MG_LEN = 23

Which shows that the flags SMG and UTF8 are (still) set and the
IV,NV,MAGIC is also set.

In pp_ctl.c, Perl_create_eval_scope I modified this code​:

  if (flags & G_KEEPERR)
  PL_in_eval |= EVAL_KEEPERR;
  else
  sv_setpvn_mg(ERRSV,"",0);

Into​:

  if (flags & G_KEEPERR)
  PL_in_eval |= EVAL_KEEPERR;
  else {
  sv_setpvn_mg(ERRSV,"",0);
  }

Which turned the output into​:

SV = PV(0x814e3a8) at 0x81504f8
  REFCNT = 1
  FLAGS = (POK,pPOK)
  PV = 0x815cbc0 ""\0
  CUR = 0
  LEN = 240
SV = PVMG(0x815c870) at 0x81504d8
  REFCNT = 1
  FLAGS = (SMG,POK,pPOK,UTF8)
  IV = 0
  NV = 0
  PV = 0x815cba0 ""\0 [UTF8 ""]
  CUR = 0
  LEN = 240
  MAGIC = 0x8167638
  MG_VIRTUAL = &PL_vtbl_utf8
  MG_TYPE = PERL_MAGIC_utf8(w)
  MG_LEN = -1

Which makes the length 0 but it stil lhas all the extra flags attached
to it.

Next​: I changed it into

  if (flags & G_KEEPERR)
  PL_in_eval |= EVAL_KEEPERR;
  else {
  sv_setpvn_mg(ERRSV,"",0);
  if (SvMAGICAL(ERRSV)) {
  mg_clear(ERRSV);
  }
  }

Output​:

SV = PV(0x814e3a8) at 0x81504f8
  REFCNT = 1
  FLAGS = (POK,pPOK)
  PV = 0x815cbc0 ""\0
  CUR = 0
  LEN = 240
SV = PVMG(0x815c890) at 0x81504f8
  REFCNT = 1
  FLAGS = (SMG,POK,pPOK,UTF8)
  IV = 0
  NV = 0
  PV = 0x815cbc0 ""\0 [UTF8 ""]
  CUR = 0
  LEN = 240
  MAGIC = 0x8167658
  MG_VIRTUAL = &PL_vtbl_utf8
  MG_TYPE = PERL_MAGIC_utf8(w)
  MG_LEN = -1

Which is the same as the previous output.

Next​:

  if (flags & G_KEEPERR)
  PL_in_eval |= EVAL_KEEPERR;
  else {
  sv_setpvn_mg(ERRSV,"",0);
  if (SvMAGICAL(ERRSV)) {
  mg_free(ERRSV);
  }
  }

Output​:

SV = PV(0x814e3a8) at 0x81504f8
  REFCNT = 1
  FLAGS = (POK,pPOK)
  PV = 0x815cbc0 ""\0
  CUR = 0
  LEN = 240
SV = PVMG(0x815c890) at 0x81504f8
  REFCNT = 1
  FLAGS = (SMG,POK,pPOK,UTF8)
  IV = 0
  NV = 0
  PV = 0x815cbc0 ""\0 [UTF8 ""]
  CUR = 0
  LEN = 240

Now the MAGIC is gone, but the UTF8 and the flags are still set.

Next​:

  if (flags & G_KEEPERR)
  PL_in_eval |= EVAL_KEEPERR;
  else {
  sv_setpvn_mg(ERRSV,"",0);
  if (SvMAGICAL(ERRSV)) {
  mg_free(ERRSV);
  }
  SvPOK_only(ERRSV);
  }

Output​:

SV = PV(0x814e3c8) at 0x8150518
  REFCNT = 1
  FLAGS = (POK,pPOK)
  PV = 0x815cbe0 ""\0
  CUR = 0
  LEN = 240
SV = PVMG(0x815c8b0) at 0x8150518
  REFCNT = 1
  FLAGS = (SMG,POK,pPOK)
  IV = 0
  NV = 0
  PV = 0x815cbe0 ""\0
  CUR = 0
  LEN = 240

Now the MAGIC and the UTF8 are gone but the SMG flag is still there.

Next​:

  if (flags & G_KEEPERR)
  PL_in_eval |= EVAL_KEEPERR;
  else {
  sv_setpvn_mg(ERRSV,"",0);
  if (SvMAGICAL(ERRSV)) {
  mg_free(ERRSV);
  mg_clear(ERRSV);
  }
  SvPOK_only(ERRSV);
  }

Output​:

SV = PV(0x814e3e8) at 0x8150538
  REFCNT = 1
  FLAGS = (POK,pPOK)
  PV = 0x815cc00 ""\0
  CUR = 0
  LEN = 240
SV = PVMG(0x815c8d0) at 0x8150538
  REFCNT = 1
  FLAGS = (POK,pPOK)
  IV = 0
  NV = 0
  PV = 0x815cc00 ""\0
  CUR = 0
  LEN = 240

The MAGIC is gone, the UTF8 is gone, the SMG flag is gone.

Next​: (back to sv_setpvn instead of sv_setpvn_mg)

  if (flags & G_KEEPERR)
  PL_in_eval |= EVAL_KEEPERR;
  else {
  sv_setpvn(ERRSV,"",0);
  if (SvMAGICAL(ERRSV)) {
  mg_free(ERRSV);
  mg_clear(ERRSV);
  }
  SvPOK_only(ERRSV);
  }

Output​:

SV = PV(0x814e3e8) at 0x8150538
  REFCNT = 1
  FLAGS = (POK,pPOK)
  PV = 0x815cc00 ""\0
  CUR = 0
  LEN = 240
SV = PVMG(0x815c8d0) at 0x8150538
  REFCNT = 1
  FLAGS = (POK,pPOK)
  IV = 0
  NV = 0
  PV = 0x815cc00 ""\0
  CUR = 0
  LEN = 240

So, is this the correct way?

And​: should this be changed in every place where sv_setpvn(ERRSV,"",0)
is used?

grep -r 'sv_setpvn(ERRSV,"",0);' * # shows that it is used in 9 other
places

And​: if it has to change everywhere then what would the best approach
be?
Placing it in a separate function? (If so, in what file, what
name, ...)

(If I know the correct way I'll create a patch with a test case for it)

Kind regards,

Bram

@p5pRT
Copy link
Author

p5pRT commented Apr 26, 2008

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Apr 27, 2008

From @rgs

2008/4/26 Animator via RT <perlbug-followup@​perl.org>​:

I took a look at this bug-report and the magic assoicated with $@​ does
not get cleared.

I did some testing on how to turn it off but I'm not sure what 'the
correct way' is. (Therefor there is no patch)
Can someone offer some guidance?

The test case​:

#!/usr/bin/perl -l

use utf8;
use Devel​::Peek
eval { 1 };
Dump($@​);
eval { die "\x{a10d}"};
$_ = length $@​;
eval { 1 };
Dump($@​);

The current output (on blead)​:

SV = PV(0x8130f04) at 0x812ffd8
REFCNT = 1
FLAGS = (POK,pPOK)
PV = 0x813d8d0 ""\0
CUR = 0
LEN = 240
SV = PVMG(0x813d390) at 0x812ffd8
REFCNT = 1
FLAGS = (SMG,POK,pPOK,UTF8)
IV = 0
NV = 0
PV = 0x813d8d0 ""\0 [UTF8 ""]
CUR = 0
LEN = 240
MAGIC = 0x8145e38
MG_VIRTUAL = &PL_vtbl_utf8
MG_TYPE = PERL_MAGIC_utf8(w)
MG_LEN = 23

Which shows that the flags SMG and UTF8 are (still) set and the
IV,NV,MAGIC is also set.

In pp_ctl.c, Perl_create_eval_scope I modified this code​:

if \(flags & G\_KEEPERR\)
    PL\_in\_eval |= EVAL\_KEEPERR;
else
    sv\_setpvn\_mg\(ERRSV\,""\,0\);

Into​:

if \(flags & G\_KEEPERR\)
    PL\_in\_eval |= EVAL\_KEEPERR;
else \{
    sv\_setpvn\_mg\(ERRSV\,""\,0\);
\}

The first one should be sv_setpvn actually.

Which turned the output into​:

SV = PV(0x814e3a8) at 0x81504f8
REFCNT = 1
FLAGS = (POK,pPOK)
PV = 0x815cbc0 ""\0
CUR = 0
LEN = 240
SV = PVMG(0x815c870) at 0x81504d8
REFCNT = 1
FLAGS = (SMG,POK,pPOK,UTF8)
IV = 0
NV = 0
PV = 0x815cba0 ""\0 [UTF8 ""]
CUR = 0
LEN = 240
MAGIC = 0x8167638
MG_VIRTUAL = &PL_vtbl_utf8
MG_TYPE = PERL_MAGIC_utf8(w)
MG_LEN = -1

Which makes the length 0 but it stil lhas all the extra flags attached
to it.

Next​: I changed it into

if \(flags & G\_KEEPERR\)
    PL\_in\_eval |= EVAL\_KEEPERR;
else \{
    sv\_setpvn\_mg\(ERRSV\,""\,0\);
    if \(SvMAGICAL\(ERRSV\)\) \{
        mg\_clear\(ERRSV\);
    \}
\}

Output​:

SV = PV(0x814e3a8) at 0x81504f8
REFCNT = 1
FLAGS = (POK,pPOK)
PV = 0x815cbc0 ""\0
CUR = 0
LEN = 240
SV = PVMG(0x815c890) at 0x81504f8
REFCNT = 1
FLAGS = (SMG,POK,pPOK,UTF8)
IV = 0
NV = 0
PV = 0x815cbc0 ""\0 [UTF8 ""]
CUR = 0
LEN = 240
MAGIC = 0x8167658
MG_VIRTUAL = &PL_vtbl_utf8
MG_TYPE = PERL_MAGIC_utf8(w)
MG_LEN = -1

Which is the same as the previous output.

Next​:

if \(flags & G\_KEEPERR\)
    PL\_in\_eval |= EVAL\_KEEPERR;
else \{
    sv\_setpvn\_mg\(ERRSV\,""\,0\);
    if \(SvMAGICAL\(ERRSV\)\) \{
        mg\_free\(ERRSV\);
    \}
\}

Output​:

SV = PV(0x814e3a8) at 0x81504f8
REFCNT = 1
FLAGS = (POK,pPOK)
PV = 0x815cbc0 ""\0
CUR = 0
LEN = 240
SV = PVMG(0x815c890) at 0x81504f8
REFCNT = 1
FLAGS = (SMG,POK,pPOK,UTF8)
IV = 0
NV = 0
PV = 0x815cbc0 ""\0 [UTF8 ""]
CUR = 0
LEN = 240

Now the MAGIC is gone, but the UTF8 and the flags are still set.

Next​:

if \(flags & G\_KEEPERR\)
    PL\_in\_eval |= EVAL\_KEEPERR;
else \{
    sv\_setpvn\_mg\(ERRSV\,""\,0\);
    if \(SvMAGICAL\(ERRSV\)\) \{
        mg\_free\(ERRSV\);
    \}
    SvPOK\_only\(ERRSV\);
\}

Output​:

SV = PV(0x814e3c8) at 0x8150518
REFCNT = 1
FLAGS = (POK,pPOK)
PV = 0x815cbe0 ""\0
CUR = 0
LEN = 240
SV = PVMG(0x815c8b0) at 0x8150518
REFCNT = 1
FLAGS = (SMG,POK,pPOK)
IV = 0
NV = 0
PV = 0x815cbe0 ""\0
CUR = 0
LEN = 240

Now the MAGIC and the UTF8 are gone but the SMG flag is still there.

Next​:

if \(flags & G\_KEEPERR\)
    PL\_in\_eval |= EVAL\_KEEPERR;
else \{
    sv\_setpvn\_mg\(ERRSV\,""\,0\);
    if \(SvMAGICAL\(ERRSV\)\) \{
        mg\_free\(ERRSV\);
        mg\_clear\(ERRSV\);
    \}
    SvPOK\_only\(ERRSV\);
\}

Output​:

SV = PV(0x814e3e8) at 0x8150538
REFCNT = 1
FLAGS = (POK,pPOK)
PV = 0x815cc00 ""\0
CUR = 0
LEN = 240
SV = PVMG(0x815c8d0) at 0x8150538
REFCNT = 1
FLAGS = (POK,pPOK)
IV = 0
NV = 0
PV = 0x815cc00 ""\0
CUR = 0
LEN = 240

The MAGIC is gone, the UTF8 is gone, the SMG flag is gone.

Next​: (back to sv_setpvn instead of sv_setpvn_mg)

if \(flags & G\_KEEPERR\)
    PL\_in\_eval |= EVAL\_KEEPERR;
else \{
    sv\_setpvn\(ERRSV\,""\,0\);
    if \(SvMAGICAL\(ERRSV\)\) \{
        mg\_free\(ERRSV\);
        mg\_clear\(ERRSV\);
    \}
    SvPOK\_only\(ERRSV\);
\}

Output​:

SV = PV(0x814e3e8) at 0x8150538
REFCNT = 1
FLAGS = (POK,pPOK)
PV = 0x815cc00 ""\0
CUR = 0
LEN = 240
SV = PVMG(0x815c8d0) at 0x8150538
REFCNT = 1
FLAGS = (POK,pPOK)
IV = 0
NV = 0
PV = 0x815cc00 ""\0
CUR = 0
LEN = 240

So, is this the correct way?

Might be correct. Do all tests pass including the new ones you'll
probably have written for this bug ? :)
However there might be a more concise way to achieve more or less the
same thing, maybe along the macro SvPVbyte_force or another similar one
(sorry, I always have to check the code when I dabble in the SV
internals, this is not the part of the core I know the most.)

And​: should this be changed in every place where sv_setpvn(ERRSV,"",0)
is used?

Probably, yes.

grep -r 'sv_setpvn(ERRSV,"",0);' * # shows that it is used in 9 other
places

And​: if it has to change everywhere then what would the best approach
be?
Placing it in a separate function? (If so, in what file, what
name, ...)

(If I know the correct way I'll create a patch with a test case for it)

@p5pRT
Copy link
Author

p5pRT commented Apr 27, 2008

From p5p@perl.wizbit.be

On Sun, 27 Apr 2008 23​:38​:30 +0200, "Rafael Garcia-Suarez"
<rgarciasuarez@​gmail.com> wrote​:

Might be correct. Do all tests pass including the new ones you'll
probably have written for this bug ? :)

Writing tests is another issue...

What should be tested? That length $@​ is empty? (which was the problem in
the bug report)
Or would a proper test be to check that only the SvPV flag is set (and
that there is no magic, no UTF8, ...) at the start of an eval/after a
successfull eval? (If so - how does one tests the flags of the SV in the
test scripts? any examples?)

Kind regards,

Bram

@p5pRT
Copy link
Author

p5pRT commented Apr 28, 2008

From chromatic@wgz.org

On Sunday 27 April 2008 15​:56​:41 Bram wrote​:

(If so - how does one tests the flags of the SV in the  
test scripts? any examples?)

Scraping Devel​::Peek output?

-- c

@p5pRT
Copy link
Author

p5pRT commented Apr 30, 2008

From p5p@perl.wizbit.be

And​: if it has to change everywhere then what would the best approach
be?
Placing it in a separate function? (If so, in what file, what
name, ...)

My current plan​:

Adding Perl_sv_setpvn_clearmg in sv.c which basically contains​:
  sv_setpvn(sv,ptr,len);
  if (SvMAGICAL(sv)) {
  mg_free(sv);
  mg_clear(sv);
  }
  SvPOK_only(sv);

Defining sv_setpvn_clearmg in embed.h
#define sv_setpvn_clearmg Perl_sv_setpvn_clearmg

Changing​: sv_setpvn(ERRSV,"",0) into sv_setpvn_clearmg(ERRSV,"",0);

Testing-A​:

eval { 1 };
eval { die "\x{a10d};"; }
$_ = length $@​;
eval { 1 };
print "ok" if (not $@​ and not length $@​);

Testing-B​:

use Devel​::Peek;
eval { 1 };
{ no warnings; $_ = $@​ + 0 }
my $eval_1 = _get_output_of_Dump($@​);

eval { die "\x{a10d};"; }
$_ = length $@​;

eval { 1 };
my $eval_2 = _get_output_of_Dump($@​);

print "ok" if $eval_1 eq $eval_2;

The $@​ + 0 is needed because the Dump of the second eval always includes​:
  IV = 0
  NV = 0

Which the first does not. (Or is there another way to get rid of it?)

Can someone comment on this approach?

Kind regards,

Bram

@p5pRT
Copy link
Author

p5pRT commented May 3, 2008

From @nwc10

On Wed, Apr 30, 2008 at 01​:36​:40PM +0200, Bram wrote​:

And​: if it has to change everywhere then what would the best approach
be?
Placing it in a separate function? (If so, in what file, what
name, ...)

My current plan​:

Adding Perl_sv_setpvn_clearmg in sv.c which basically contains​:
sv_setpvn(sv,ptr,len);
if (SvMAGICAL(sv)) {
mg_free(sv);
mg_clear(sv);
}
SvPOK_only(sv);

In general, are you sure you want that order? Wouldn't it be conceptually more
logical to free up any magic, then do the assignment to the SV?

Also, your order mg_free() then mg_clear() is backwards, as mg_clear() will
have to be a no-op this way round, because mg_free will have just freed all
the magic.

Defining sv_setpvn_clearmg in embed.h
#define sv_setpvn_clearmg Perl_sv_setpvn_clearmg

Changing​: sv_setpvn(ERRSV,"",0) into sv_setpvn_clearmg(ERRSV,"",0);

How many of these are there to change?

I don't feel that comfortable with adding another "class" macro of macro for
that one particular operation, because it will start to feel like we need
*_clearmg variants for everything other one. It seems better to find a tight
way of doing something like the mg_clear() then mg_free() then SvOK(off)
implied by the above, and replace sv_setpvn(ERRSV,"",0) with
this_clearing_thing(ERRSV); sv_setpvn(ERRSV,"",0);

Testing-A​:

eval { 1 };
eval { die "\x{a10d};"; }
$_ = length $@​;
eval { 1 };
print "ok" if (not $@​ and not length $@​);

Well, not literally that as there isn't ; after the second eval.

But, um, that example is interesting​:

$ ./perl -e 'eval { 1 }; eval { die "\x{a10d};"; }; $_ = length $@​; eval { 1 }; length $@​'
panic​: sv_len_utf8 cache 17 real 0 for at -e line 1.

I don't have time to investigate *that* right now.

The $@​ + 0 is needed because the Dump of the second eval always includes​:
IV = 0
NV = 0

Which the first does not. (Or is there another way to get rid of it?)

Assign something that is PVMG to $@​ before starting, to ensure that it has
already been upgraded?

$ perl -MDevel​::Peek -e '$@​ = $!; eval "die"; Dump $@​'
SV = PVMG(0x80ba60) at 0x800138
  REFCNT = 1
  FLAGS = (POK,pPOK)
  IV = 0
  NV = 0
  PV = 0x201290 "Died at (eval 2) line 1.\n"\0
  CUR = 25
  LEN = 240

It's a bit of a hack, but it feels less hacky than a bonus eval.

Nicholas Clark

@p5pRT
Copy link
Author

p5pRT commented May 3, 2008

From p5p@perl.wizbit.be

Quoting Nicholas Clark <nick@​ccl4.org>​:

On Wed, Apr 30, 2008 at 01​:36​:40PM +0200, Bram wrote​:

And​: if it has to change everywhere then what would the best approach
be?
Placing it in a separate function? (If so, in what file, what
name, ...)

My current plan​:

Adding Perl_sv_setpvn_clearmg in sv.c which basically contains​:
sv_setpvn(sv,ptr,len);
if (SvMAGICAL(sv)) {
mg_free(sv);
mg_clear(sv);
}
SvPOK_only(sv);

In general, are you sure you want that order? Wouldn't it be
conceptually more
logical to free up any magic, then do the assignment to the SV?

I'm not sure about it at all...
Hence the questions. :)

Also, your order mg_free() then mg_clear() is backwards, as mg_clear() will
have to be a no-op this way round, because mg_free will have just freed all
the magic.

Copy from an earlier​:

if (flags & G_KEEPERR)
  PL_in_eval |= EVAL_KEEPERR;
else {
  sv_setpvn_mg(ERRSV,"",0);
  if (SvMAGICAL(ERRSV)) {
  mg_free(ERRSV);
  }
  SvPOK_only(ERRSV);
}

Output​:

SV = PV(0x814e3c8) at 0x8150518
  REFCNT = 1
  FLAGS = (POK,pPOK)
  PV = 0x815cbe0 ""\0
  CUR = 0
  LEN = 240
SV = PVMG(0x815c8b0) at 0x8150518
  REFCNT = 1
  FLAGS = (SMG,POK,pPOK)
  IV = 0
  NV = 0
  PV = 0x815cbe0 ""\0
  CUR = 0
  LEN = 240

With the mg_clear added after the mg_free​:

if (flags & G_KEEPERR)
  PL_in_eval |= EVAL_KEEPERR;
else {
  sv_setpvn_mg(ERRSV,"",0);
  if (SvMAGICAL(ERRSV)) {
  mg_free(ERRSV);
  mg_clear(ERRSV);
  }
  SvPOK_only(ERRSV);
}

SV = PV(0x814e3e8) at 0x8150538
  REFCNT = 1
  FLAGS = (POK,pPOK)
  PV = 0x815cc00 ""\0
  CUR = 0
  LEN = 240
SV = PVMG(0x815c8d0) at 0x8150538
  REFCNT = 1
  FLAGS = (POK,pPOK)
  IV = 0
  NV = 0
  PV = 0x815cc00 ""\0
  CUR = 0
  LEN = 240

Whichs shows that it is not a no-op (the SMG flag is gone).
Or would that be a bug in mg_clear?

Defining sv_setpvn_clearmg in embed.h
#define sv_setpvn_clearmg Perl_sv_setpvn_clearmg

Changing​: sv_setpvn(ERRSV,"",0) into sv_setpvn_clearmg(ERRSV,"",0);

How many of these are there to change?

There are 9.

I don't feel that comfortable with adding another "class" macro of macro for
that one particular operation, because it will start to feel like we need
*_clearmg variants for everything other one. It seems better to find a tight
way of doing something like the mg_clear() then mg_free() then SvOK(off)
implied by the above, and replace sv_setpvn(ERRSV,"",0) with
this_clearing_thing(ERRSV); sv_setpvn(ERRSV,"",0);

Suggestions for the name of this_clearing_thing?

Testing-A​:

eval { 1 };
eval { die "\x{a10d};"; }
$_ = length $@​;
eval { 1 };
print "ok" if (not $@​ and not length $@​);

Well, not literally that as there isn't ; after the second eval.

But, um, that example is interesting​:

$ ./perl -e 'eval { 1 }; eval { die "\x{a10d};"; }; $_ = length $@​;
eval { 1 }; length $@​'
panic​: sv_len_utf8 cache 17 real 0 for at -e line 1.

I don't have time to investigate *that* right now.

What version/revision is that? (It never happend with me.)

But note, this is what would be fixed by clearing the magic...
The problem is that length still uses the old utf8 length and not the new one.
So either a new panic was added for it or something else is different...

The $@​ + 0 is needed because the Dump of the second eval always includes​:
IV = 0
NV = 0

Which the first does not. (Or is there another way to get rid of it?)

Assign something that is PVMG to $@​ before starting, to ensure that it has
already been upgraded?

$ perl -MDevel​::Peek -e '$@​ = $!; eval "die"; Dump $@​'
SV = PVMG(0x80ba60) at 0x800138
REFCNT = 1
FLAGS = (POK,pPOK)
IV = 0
NV = 0
PV = 0x201290 "Died at (eval 2) line 1.\n"\0
CUR = 25
LEN = 240

It's a bit of a hack, but it feels less hacky than a bonus eval.

I see.

Kind regards,

Bram

@p5pRT
Copy link
Author

p5pRT commented May 4, 2008

From @nwc10

On Sat, May 03, 2008 at 10​:05​:10PM +0200, Bram wrote​:

Quoting Nicholas Clark <nick@​ccl4.org>​:

On Wed, Apr 30, 2008 at 01​:36​:40PM +0200, Bram wrote​:

And​: if it has to change everywhere then what would the best approach
be?
Placing it in a separate function? (If so, in what file, what
name, ...)

My current plan​:

Adding Perl_sv_setpvn_clearmg in sv.c which basically contains​:
sv_setpvn(sv,ptr,len);
if (SvMAGICAL(sv)) {
mg_free(sv);
mg_clear(sv);
}
SvPOK_only(sv);

In general, are you sure you want that order? Wouldn't it be
conceptually more
logical to free up any magic, then do the assignment to the SV?

I'm not sure about it at all...
Hence the questions. :)

Well, actually nor am I, so it comes to detective work...

Also, your order mg_free() then mg_clear() is backwards, as mg_clear() will
have to be a no-op this way round, because mg_free will have just freed all
the magic.

Copy from an earlier​:

if (flags & G_KEEPERR)
PL_in_eval |= EVAL_KEEPERR;
else {
sv_setpvn_mg(ERRSV,"",0);
if (SvMAGICAL(ERRSV)) {
mg_free(ERRSV);
}
SvPOK_only(ERRSV);
}

Output​:

SV = PV(0x814e3c8) at 0x8150518
REFCNT = 1
FLAGS = (POK,pPOK)
PV = 0x815cbe0 ""\0
CUR = 0
LEN = 240
SV = PVMG(0x815c8b0) at 0x8150518
REFCNT = 1
FLAGS = (SMG,POK,pPOK)
IV = 0
NV = 0
PV = 0x815cbe0 ""\0
CUR = 0
LEN = 240

With the mg_clear added after the mg_free​:

if (flags & G_KEEPERR)
PL_in_eval |= EVAL_KEEPERR;
else {xs
sv_setpvn_mg(ERRSV,"",0);
if (SvMAGICAL(ERRSV)) {
mg_free(ERRSV);
mg_clear(ERRSV);
}
SvPOK_only(ERRSV);
}

SV = PV(0x814e3e8) at 0x8150538
REFCNT = 1
FLAGS = (POK,pPOK)
PV = 0x815cc00 ""\0
CUR = 0
LEN = 240
SV = PVMG(0x815c8d0) at 0x8150538
REFCNT = 1
FLAGS = (POK,pPOK)
IV = 0
NV = 0
PV = 0x815cc00 ""\0
CUR = 0
LEN = 240

Whichs shows that it is not a no-op (the SMG flag is gone).
Or would that be a bug in mg_clear?

Well, I don't know, but I grep'ed the core for all uses of mg_free() and
mg_clear() and a had a think. And I think the bug would be in mg_free(),
which currently reads like this​:

int
Perl_mg_free(pTHX_ SV *sv)
{
  MAGIC* mg;
  MAGIC* moremagic;

  PERL_ARGS_ASSERT_MG_FREE;

  for (mg = SvMAGIC(sv); mg; mg = moremagic) {
  const MGVTBL* const vtbl = mg->mg_virtual;
  moremagic = mg->mg_moremagic;
  if (vtbl && vtbl->svt_free)
  CALL_FPTR(vtbl->svt_free)(aTHX_ sv, mg);
  if (mg->mg_ptr && mg->mg_type != PERL_MAGIC_regex_global) {
  if (mg->mg_len > 0 || mg->mg_type == PERL_MAGIC_utf8)
  Safefree(mg->mg_ptr);
  else if (mg->mg_len == HEf_SVKEY)
  SvREFCNT_dec((SV*)mg->mg_ptr);
  }
  if (mg->mg_flags & MGf_REFCOUNTED)
  SvREFCNT_dec(mg->mg_obj);
  Safefree(mg);
  SvMAGIC_set(sv, moremagic);
  }
  SvMAGIC_set(sv, NULL);
  return 0;
}

and probably should have a line added at the end to reset all the SvFLAGS
relating to magic. (which mg_clear() must do somehow, but after 5 minutes of
trying to trace the code, I can't work out where. Also, I'm now not sure
which Perl code "1" liner you're using to test this)

And then fix up any callers of mg_free() that clear all the SV magic flags
already, now that mg_free() does it for them. (I found at least one plausible
place.)

Defining sv_setpvn_clearmg in embed.h
#define sv_setpvn_clearmg Perl_sv_setpvn_clearmg

Changing​: sv_setpvn(ERRSV,"",0) into sv_setpvn_clearmg(ERRSV,"",0);

How many of these are there to change?

There are 9.

Ah right. I'm not convinced that it's worth it as a one liner function.
A macro for resetting ERRSV might seem more appropriate.

Suggestions for the name of this_clearing_thing?

#define clear_errsv() STMT_START { ... do stuff ... } STMT_END

?

What version/revision is that? (It never happend with me.)

blead, built with -g and therefore -DDEBUGGING and I think by default UTF-8
cache checking.

But note, this is what would be fixed by clearing the magic...

I'm a bit confused. mg_clear() clearing the magic?

The problem is that length still uses the old utf8 length and not the new
one.
So either a new panic was added for it or something else is different...

$ ./perl -e '${^UTF8CACHE} = 0; eval { 1 }; eval { die "\x{a10d};"; }; $_ = length $@​; eval { 1 }; warn length $@​'
0 at -e line 1.
$ ./perl -e '${^UTF8CACHE} = 1; eval { 1 }; eval { die "\x{a10d};"; }; $_ = length $@​; eval { 1 }; warn length $@​'
17 at -e line 1.
$ ./perl -e '${^UTF8CACHE} = -1; eval { 1 }; eval { die "\x{a10d};"; }; $_ = length $@​; eval { 1 }; warn length $@​'
panic​: sv_len_utf8 cache 17 real 0 for at -e line 1.

(0 disables the caching, 1 is caching mode (the non -DDEBUGGING default), -1
is caching assertion mode, where the cache store code is enabled, but the
answers cross checked every time.)

Mmm, in the end, reading the documentation and the source, should every
sv_setpvn(ERRSV, ...); become sv_setpvn_mg(ERRSV, ...), and
sv_catsv(ERRSV, ...) become sv_catsv_mg(ERRSV, ...) ?
Will that solve it?

Nicholas Clark

@p5pRT
Copy link
Author

p5pRT commented May 4, 2008

From @nwc10

On Sun, May 04, 2008 at 06​:03​:02PM +0100, Nicholas Clark wrote​:

Well, actually nor am I, so it comes to detective work...

I failed to note, writing this e-mail took half an hour. It adds up.

Nicholas Clark

@p5pRT
Copy link
Author

p5pRT commented May 5, 2008

From p5p@perl.wizbit.be

Quoting Nicholas Clark <nick@​ccl4.org>​:

Also, your order mg_free() then mg_clear() is backwards, as mg_clear() will
have to be a no-op this way round, because mg_free will have just freed all
the magic.

Copy from an earlier​:

if (flags & G_KEEPERR)
PL_in_eval |= EVAL_KEEPERR;
else {
sv_setpvn_mg(ERRSV,"",0);
if (SvMAGICAL(ERRSV)) {
mg_free(ERRSV);
}
SvPOK_only(ERRSV);
}

Output​:

SV = PV(0x814e3c8) at 0x8150518
REFCNT = 1
FLAGS = (POK,pPOK)
PV = 0x815cbe0 ""\0
CUR = 0
LEN = 240
SV = PVMG(0x815c8b0) at 0x8150518
REFCNT = 1
FLAGS = (SMG,POK,pPOK)
IV = 0
NV = 0
PV = 0x815cbe0 ""\0
CUR = 0
LEN = 240

With the mg_clear added after the mg_free​:

if (flags & G_KEEPERR)
PL_in_eval |= EVAL_KEEPERR;
else {xs
sv_setpvn_mg(ERRSV,"",0);
if (SvMAGICAL(ERRSV)) {
mg_free(ERRSV);
mg_clear(ERRSV);
}
SvPOK_only(ERRSV);
}

SV = PV(0x814e3e8) at 0x8150538
REFCNT = 1
FLAGS = (POK,pPOK)
PV = 0x815cc00 ""\0
CUR = 0
LEN = 240
SV = PVMG(0x815c8d0) at 0x8150538
REFCNT = 1
FLAGS = (POK,pPOK)
IV = 0
NV = 0
PV = 0x815cc00 ""\0
CUR = 0
LEN = 240

Whichs shows that it is not a no-op (the SMG flag is gone).
Or would that be a bug in mg_clear?

Well, I don't know, but I grep'ed the core for all uses of mg_free() and
mg_clear() and a had a think. And I think the bug would be in mg_free(),
which currently reads like this​:

int
Perl_mg_free(pTHX_ SV *sv)
{
MAGIC* mg;
MAGIC* moremagic;

PERL\_ARGS\_ASSERT\_MG\_FREE;

for \(mg = SvMAGIC\(sv\); mg; mg = moremagic\) \{
    const MGVTBL\* const vtbl = mg\->mg\_virtual;
moremagic = mg\->mg\_moremagic;
if \(vtbl && vtbl\->svt\_free\)
    CALL\_FPTR\(vtbl\->svt\_free\)\(aTHX\_ sv\, mg\);
if \(mg\->mg\_ptr && mg\->mg\_type \!= PERL\_MAGIC\_regex\_global\) \{
    if \(mg\->mg\_len > 0 || mg\->mg\_type == PERL\_MAGIC\_utf8\)
    Safefree\(mg\->mg\_ptr\);
    else if \(mg\->mg\_len == HEf\_SVKEY\)
    SvREFCNT\_dec\(\(SV\*\)mg\->mg\_ptr\);
\}
if \(mg\->mg\_flags & MGf\_REFCOUNTED\)
    SvREFCNT\_dec\(mg\->mg\_obj\);
Safefree\(mg\);
SvMAGIC\_set\(sv\, moremagic\);
\}
SvMAGIC\_set\(sv\, NULL\);
return 0;

}

and probably should have a line added at the end to reset all the SvFLAGS
relating to magic. (which mg_clear() must do somehow, but after 5 minutes of
trying to trace the code, I can't work out where.

Could this be it?​: (mg_clear calls save_magic which calls SvMAGICAL_off)

int
Perl_mg_clear(pTHX_ SV *sv)
{
  const I32 mgs_ix = SSNEW(sizeof(MGS));
  MAGIC* mg;

  PERL_ARGS_ASSERT_MG_CLEAR;

  save_magic(mgs_ix, sv);

  for (mg = SvMAGIC(sv); mg; mg = mg->mg_moremagic) {
  const MGVTBL* const vtbl = mg->mg_virtual;
  /* omit GSKIP -- never set here */

  if (vtbl && vtbl->svt_clear)
  CALL_FPTR(vtbl->svt_clear)(aTHX_ sv, mg);
  }

  restore_magic(INT2PTR(void*, (IV)mgs_ix));
  return 0;
}

S_save_magic(pTHX_ I32 mgs_ix, SV *sv)
{
  dVAR;
  MGS* mgs;

  PERL_ARGS_ASSERT_SAVE_MAGIC;

  assert(SvMAGICAL(sv));
  /* Turning READONLY off for a copy-on-write scalar (including shared
  hash keys) is a bad idea. */
  if (SvIsCOW(sv))
  sv_force_normal_flags(sv, 0);

  SAVEDESTRUCTOR_X(S_restore_magic, INT2PTR(void*, (IV)mgs_ix));

  mgs = SSPTR(mgs_ix, MGS*);
  mgs->mgs_sv = sv;
  mgs->mgs_flags = SvMAGICAL(sv) | SvREADONLY(sv);
  mgs->mgs_ss_ix = PL_savestack_ix; /* points after the saved
destructor */

  SvMAGICAL_off(sv);
  SvREADONLY_off(sv);
  if (!(SvFLAGS(sv) & (SVf_IOK|SVf_NOK|SVf_POK))) {
  /* No public flags are set, so promote any private flags to
public. */
  SvFLAGS(sv) |= (SvFLAGS(sv) & (SVp_IOK|SVp_NOK|SVp_POK)) >> PRIVSHIFT;
  }
}

#define SvMAGICAL_off(sv) (SvFLAGS(sv) &= ~(SVs_GMG|SVs_SMG|SVs_RMG))

(Curently building and testing with a SvMAGICAL_off(sv); after
SvMAGIC_set(sv, NULL); )

Also, I'm now not sure
which Perl code "1" liner you're using to test this)

See the previous mail/the ticket on RT.

http​://rt.perl.org/rt3/Public/Bug/Display.html?id=51370

And then fix up any callers of mg_free() that clear all the SV magic flags
already, now that mg_free() does it for them. (I found at least one plausible
place.)

Defining sv_setpvn_clearmg in embed.h
#define sv_setpvn_clearmg Perl_sv_setpvn_clearmg

Changing​: sv_setpvn(ERRSV,"",0) into sv_setpvn_clearmg(ERRSV,"",0);

How many of these are there to change?

There are 9.

Ah right. I'm not convinced that it's worth it as a one liner function.
A macro for resetting ERRSV might seem more appropriate.

Suggestions for the name of this_clearing_thing?

#define clear_errsv() STMT_START { ... do stuff ... } STMT_END

?

Ok.

What version/revision is that? (It never happend with me.)

blead, built with -g and therefore -DDEBUGGING and I think by default UTF-8
cache checking.

Rebuilding blead with -g to see if I can get a panic.

But note, this is what would be fixed by clearing the magic...

I'm a bit confused. mg_clear() clearing the magic?

The panic would be fixed by clearing the (utf8-)magic that is attached to $@​.

The problem is that length still uses the old utf8 length and not the new
one.
So either a new panic was added for it or something else is different...

$ ./perl -e '${^UTF8CACHE} = 0; eval { 1 }; eval { die "\x{a10d};";
}; $_ = length $@​; eval { 1 }; warn length $@​'
0 at -e line 1.
$ ./perl -e '${^UTF8CACHE} = 1; eval { 1 }; eval { die "\x{a10d};";
}; $_ = length $@​; eval { 1 }; warn length $@​'
17 at -e line 1.
$ ./perl -e '${^UTF8CACHE} = -1; eval { 1 }; eval { die
"\x{a10d};"; }; $_ = length $@​; eval { 1 }; warn length $@​'
panic​: sv_len_utf8 cache 17 real 0 for at -e line 1.

(0 disables the caching, 1 is caching mode (the non -DDEBUGGING default), -1
is caching assertion mode, where the cache store code is enabled, but the
answers cross checked every time.)

Mmm, in the end, reading the documentation and the source, should every
sv_setpvn(ERRSV, ...); become sv_setpvn_mg(ERRSV, ...), and
sv_catsv(ERRSV, ...) become sv_catsv_mg(ERRSV, ...) ?
Will that solve it?

As noted in my original messages​: that will solve the length issue but
then $@​ still has the extra flags/magic set.

I'm completly clueless if that would be a problem or not.
(Rafael replied to that that it actually should be sv_setpvn)

Kind regards,

Bram

@p5pRT
Copy link
Author

p5pRT commented May 5, 2008

From @nwc10

On Mon, May 05, 2008 at 09​:17​:35AM +0200, Bram wrote​:

Quoting Nicholas Clark <nick@​ccl4.org>​:

Mmm, in the end, reading the documentation and the source, should every
sv_setpvn(ERRSV, ...); become sv_setpvn_mg(ERRSV, ...), and
sv_catsv(ERRSV, ...) become sv_catsv_mg(ERRSV, ...) ?
Will that solve it?

As noted in my original messages​: that will solve the length issue but
then $@​ still has the extra flags/magic set.

It doesn't matter if it has the UTF-8 length caching magic still in place,
when the value is not UTF-8, providing the magic has had its "set" method
called correctly, because then the cache will be marked as void.

There wasn't any other magic involved, was there?

I'm completly clueless if that would be a problem or not.
(Rafael replied to that that it actually should be sv_setpvn)

Yes, and I'm now not sure why. To my mind, making them all *_mg seems right,
as it would trigger any set magic on ERRSV.

Although (I think) that this would allow rather sick things like tie $@​ to
work.

Nicholas Clark

@p5pRT
Copy link
Author

p5pRT commented May 5, 2008

From @davidnicol

On Sat, May 3, 2008 at 3​:05 PM, Bram <p5p@​perl.wizbit.be> wrote​:

Suggestions for the name of this_clearing_thing?

mg_bono_busboy /* is free, clears tables */

mg_paidoff /* "free and clear" */

@p5pRT
Copy link
Author

p5pRT commented Jun 11, 2008

From p5p@spam.wizbit.be

On Mon Mar 03 06​:41​:54 2008, pajas@​ufal.mff.cuni.cz wrote​:

Perl seems to cache character length of scalars, but in case of $@​
it does not reset it after a successfull eval. Here is an example.

The following is reproducible with 5.10.0 and also on 5.8.8 (didn't
try other releases)​:

perl -MCarp -e 'use utf8; eval { die "\x{10d}"}; length($@​); print
$@​; eval { 1 }; print "\$@​ is q($@​), length(\$@​)
is ".length($@​)."\n" '

The first eval dies and spits a non-ascii character. Then the
character length of $@​ is asked (we are in the scope of the utf8
pragma). The returned value (16 in this case) seems to be
remembered,
since after a subsequent eval{1} which resets $@​ to q(), the
function
length($@​) still returns 16 (but 0 if asked in the scope of 'use
byte').

-----------------------------------------------------------------

Attached are two patches​:

errsv_1.txt​:
- this patch clears all the magic associated with $@​,
- has the test of the bug report
- scrapes the output of Devel​::Peek​::Dump at the start and compares it
again after two evals (one that fails and one that succeeds).

errsv_2.txt​:

In the thread (Clearing magic) Nicholas said​:
<quote>
To my mind, making them all *_mg seems right, as it would trigger any
set magic on ERRSV.

Although (I think) that this would allow rather sick things like tie $@​
to work.
</quote>

- this patch changes most of them (not sure if I got them all) into _mg
- has the test of the bug report
- does a tie on $@​ and checks if STORE gets called.

Only one of the two patches has to be applied...

Kind regards,

Bram

@p5pRT
Copy link
Author

p5pRT commented Jun 11, 2008

From p5p@spam.wizbit.be

Inline Patch
diff -Naur old/perl-current/op.c new/perl-current/op.c
--- old/perl-current/op.c	2008-06-07 16:05:37.000000000 +0200
+++ new/perl-current/op.c	2008-06-07 18:43:01.000000000 +0200
@@ -2521,7 +2521,7 @@
     case 3:
 	/* Something tried to die.  Abandon constant folding.  */
 	/* Pretend the error never happened.  */
-	sv_setpvn(ERRSV,"",0);
+        clear_errsv();
 	o->op_next = old_next;
 	break;
     default:
diff -Naur old/perl-current/perl.c new/perl-current/perl.c
--- old/perl-current/perl.c	2008-06-07 16:05:41.000000000 +0200
+++ new/perl-current/perl.c	2008-06-07 18:43:04.000000000 +0200
@@ -2679,8 +2679,9 @@
  redo_body:
 	    CALL_BODY_SUB((OP*)&myop);
 	    retval = PL_stack_sp - (PL_stack_base + oldmark);
-	    if (!(flags & G_KEEPERR))
-		sv_setpvn(ERRSV,"",0);
+	    if (!(flags & G_KEEPERR)) {
+		clear_errsv();
+	    }
 	    break;
 	case 1:
 	    STATUS_ALL_FAILURE;
@@ -2780,8 +2781,9 @@
  redo_body:
 	CALL_BODY_EVAL((OP*)&myop);
 	retval = PL_stack_sp - (PL_stack_base + oldmark);
-	if (!(flags & G_KEEPERR))
-	    sv_setpvn(ERRSV,"",0);
+	if (!(flags & G_KEEPERR)) {
+	    clear_errsv();
+	}
 	break;
     case 1:
 	STATUS_ALL_FAILURE;
@@ -3559,7 +3561,7 @@
     gv_SVadd(PL_errgv);
 #endif
     sv_grow(ERRSV, 240);	/* Preallocate - for immediate signals. */
-    sv_setpvn(ERRSV, "", 0);
+    clear_errsv();
     PL_curstash = PL_defstash;
     CopSTASH_set(&PL_compiling, PL_defstash);
     PL_debstash = GvHV(gv_fetchpvs("DB::", GV_ADDMULTI, SVt_PVHV));
diff -Naur old/perl-current/perl.h new/perl-current/perl.h
--- old/perl-current/perl.h	2008-06-07 16:05:37.000000000 +0200
+++ new/perl-current/perl.h	2008-06-07 18:43:05.000000000 +0200
@@ -6008,6 +6008,8 @@
 
 #endif /* Include guard */
 
+#define clear_errsv() STMT_START { sv_setpvn(ERRSV,"",0); if (SvMAGICAL(ERRSV)) { mg_free(ERRSV); } SvPOK_only(ERRSV); } STMT_END
+
 /*
  * Local variables:
  * c-indentation-style: bsd
diff -Naur old/perl-current/pp_ctl.c new/perl-current/pp_ctl.c
--- old/perl-current/pp_ctl.c	2008-06-07 16:05:34.000000000 +0200
+++ new/perl-current/pp_ctl.c	2008-06-07 18:42:59.000000000 +0200
@@ -2148,8 +2148,9 @@
     PL_curpm = newpm;	/* ... and pop $1 et al */
 
     LEAVESUB(sv);
-    if (clear_errsv)
-	sv_setpvn(ERRSV,"",0);
+    if (clear_errsv) {
+	clear_errsv();
+    }
     return retop;
 }
 
@@ -3000,8 +3001,9 @@
     CopARYBASE_set(PL_curcop, 0);
     if (saveop && (saveop->op_type != OP_REQUIRE) && (saveop->op_flags & OPf_SPECIAL))
 	PL_in_eval |= EVAL_KEEPERR;
-    else
-	sv_setpvn(ERRSV,"",0);
+    else {
+	clear_errsv();
+    }
     if (yyparse() || PL_parser->error_count || !PL_eval_root) {
 	SV **newsp;			/* Used by POPBLOCK. */
 	PERL_CONTEXT *cx = &cxstack[cxstack_ix];
@@ -3772,8 +3774,9 @@
     }
     else {
 	LEAVE;
-	if (!(save_flags & OPf_SPECIAL))
-	    sv_setpvn(ERRSV,"",0);
+	if (!(save_flags & OPf_SPECIAL)) {
+	    clear_errsv();
+	}
     }
 
     RETURNOP(retop);
@@ -3816,8 +3819,9 @@
     PL_in_eval = EVAL_INEVAL;
     if (flags & G_KEEPERR)
 	PL_in_eval |= EVAL_KEEPERR;
-    else
-	sv_setpvn(ERRSV,"",0);
+    else {
+	clear_errsv();
+    }
     if (flags & G_FAKINGEVAL) {
 	PL_eval_root = PL_op; /* Only needed so that goto works right. */
     }
@@ -3876,7 +3880,7 @@
     PL_curpm = newpm;	/* Don't pop $1 et al till now */
 
     LEAVE;
-    sv_setpvn(ERRSV,"",0);
+    clear_errsv();
     RETURN;
 }
 
diff -Naur old/perl-current/t/op/eval.t new/perl-current/t/op/eval.t
--- old/perl-current/t/op/eval.t	2008-06-07 16:05:35.000000000 +0200
+++ new/perl-current/t/op/eval.t	2008-06-07 18:42:49.000000000 +0200
@@ -5,7 +5,7 @@
     @INC = '../lib';
 }
 
-print "1..95\n";
+print "1..98\n";
 
 eval 'print "ok 1\n";';
 
@@ -485,4 +485,63 @@
 }
 
 
+# [perl #51370] eval { die "\x{a10d}" } followed by eval { 1 } did not reset
+# length $@ 
+$@ = "";
+eval { die "\x{a10d}"; };
+$_ = length $@;
+eval { 1 };
+
+print "not " if ($@ ne "");
+print "ok $test # length of \$@ after eval\n"; $test++;
+
+print "not " if (length $@ != 0);
+print "ok $test # length of \$@ after eval\n"; $test++;
+
+# Check if eval { 1 }; compeltly resets $@
+if (eval "use Devel::Peek; 1;") {
+  
+  open PROG, ">", "peek_eval_$$.t" or die "Can't create test file";
+  print PROG <<'END_EVAL_TEST';
+    use Devel::Peek;
+    $! = 0;
+    $@ = $!;
+    my $ok = 0;
+    open(SAVERR, ">&STDERR") or die "Can't dup STDERR: $!";
+    if (open(OUT,">peek_eval$$")) {
+      open(STDERR, ">&OUT") or die "Can't dup OUT: $!";
+      Dump($@);
+      print STDERR "******\n";
+      eval { die "\x{a10d}"; };
+      $_ = length $@;
+      eval { 1 };
+      Dump($@);
+      open(STDERR, ">&SAVERR") or die "Can't restore STDERR: $!";
+      close(OUT);
+      if (open(IN, "peek_eval$$")) {
+        local $/;
+        my $in = <IN>;
+        my ($first, $second) = split (/\*\*\*\*\*\*\n/, $in, 2);
+        $first =~ s/,pNOK//;
+        $ok = 1 if ($first eq $second);
+      }
+    }
+
+    print $ok;
+    END {
+      1 while unlink("peek_eval$$");
+    }
+END_EVAL_TEST
+   close PROG;
+
+   my $ok = runperl(progfile => "peek_eval_$$.t");
+   print "not " unless $ok;
+   print "ok $test # eval { 1 } completly resets \$@\n";
+
+   $test++;
+   1 while unlink("peek_eval_$$.t");
+}
+else {
+  print "ok $test # skipped - eval { 1 } completly resets \$@";
+}
 

@p5pRT
Copy link
Author

p5pRT commented Jun 11, 2008

From p5p@spam.wizbit.be

Inline Patch
diff -Naur old/perl-current/op.c new/perl-current/op.c
--- old/perl-current/op.c	2008-06-07 16:05:37.000000000 +0200
+++ new/perl-current/op.c	2008-06-07 20:22:25.000000000 +0200
@@ -2521,7 +2521,7 @@
     case 3:
 	/* Something tried to die.  Abandon constant folding.  */
 	/* Pretend the error never happened.  */
-	sv_setpvn(ERRSV,"",0);
+	sv_setpvn_mg(ERRSV,"",0);
 	o->op_next = old_next;
 	break;
     default:
@@ -5719,7 +5719,7 @@
 		    Perl_croak(aTHX_ not_safe);
 		else {
 		    /* force display of errors found but not reported */
-		    sv_catpv(ERRSV, not_safe);
+		    sv_catpv_mg(ERRSV, not_safe);
 		    Perl_croak(aTHX_ "%"SVf, SVfARG(ERRSV));
 		}
 	    }
diff -Naur old/perl-current/perl.c new/perl-current/perl.c
--- old/perl-current/perl.c	2008-06-07 16:05:41.000000000 +0200
+++ new/perl-current/perl.c	2008-06-07 20:22:30.000000000 +0200
@@ -2680,7 +2680,7 @@
 	    CALL_BODY_SUB((OP*)&myop);
 	    retval = PL_stack_sp - (PL_stack_base + oldmark);
 	    if (!(flags & G_KEEPERR))
-		sv_setpvn(ERRSV,"",0);
+		sv_setpvn_mg(ERRSV,"",0);
 	    break;
 	case 1:
 	    STATUS_ALL_FAILURE;
@@ -2781,7 +2781,7 @@
 	CALL_BODY_EVAL((OP*)&myop);
 	retval = PL_stack_sp - (PL_stack_base + oldmark);
 	if (!(flags & G_KEEPERR))
-	    sv_setpvn(ERRSV,"",0);
+	    sv_setpvn_mg(ERRSV,"",0);
 	break;
     case 1:
 	STATUS_ALL_FAILURE;
@@ -3559,7 +3559,7 @@
     gv_SVadd(PL_errgv);
 #endif
     sv_grow(ERRSV, 240);	/* Preallocate - for immediate signals. */
-    sv_setpvn(ERRSV, "", 0);
+    sv_setpvn_mg(ERRSV, "", 0);
     PL_curstash = PL_defstash;
     CopSTASH_set(&PL_compiling, PL_defstash);
     PL_debstash = GvHV(gv_fetchpvs("DB::", GV_ADDMULTI, SVt_PVHV));
diff -Naur old/perl-current/pp_ctl.c new/perl-current/pp_ctl.c
--- old/perl-current/pp_ctl.c	2008-06-07 16:05:34.000000000 +0200
+++ new/perl-current/pp_ctl.c	2008-06-07 20:22:36.000000000 +0200
@@ -1488,7 +1488,7 @@
     PERL_ARGS_ASSERT_QERROR;
 
     if (PL_in_eval)
-	sv_catsv(ERRSV, err);
+	sv_catsv_mg(ERRSV, err);
     else if (PL_errors)
 	sv_catsv(PL_errors, err);
     else
@@ -1512,7 +1512,7 @@
 		SV * const err = ERRSV;
 		const char *e = NULL;
 		if (!SvPOK(err))
-		    sv_setpvn(err,"",0);
+		    sv_setpvn_mg(err,"",0);
 		else if (SvCUR(err) >= sizeof(prefix)+msglen-1) {
 		    STRLEN len;
 		    e = SvPV_const(err, len);
@@ -1522,8 +1522,8 @@
 		}
 		if (!e) {
 		    SvGROW(err, SvCUR(err)+sizeof(prefix)+msglen);
-		    sv_catpvn(err, prefix, sizeof(prefix)-1);
-		    sv_catpvn(err, message, msglen);
+		    sv_catpvn_mg(err, prefix, sizeof(prefix)-1);
+		    sv_catpvn_mg(err, message, msglen);
 		    if (ckWARN(WARN_MISC)) {
 			const STRLEN start = SvCUR(err)-msglen-sizeof(prefix)+1;
 			Perl_warner(aTHX_ packWARN(WARN_MISC), SvPVX_const(err)+start);
@@ -1531,7 +1531,7 @@
 		}
 	    }
 	    else {
-		sv_setpvn(ERRSV, message, msglen);
+		sv_setpvn_mg(ERRSV, message, msglen);
 	    }
 	}
 
@@ -2149,7 +2149,7 @@
 
     LEAVESUB(sv);
     if (clear_errsv)
-	sv_setpvn(ERRSV,"",0);
+	sv_setpvn_mg(ERRSV,"",0);
     return retop;
 }
 
@@ -3001,7 +3001,7 @@
     if (saveop && (saveop->op_type != OP_REQUIRE) && (saveop->op_flags & OPf_SPECIAL))
 	PL_in_eval |= EVAL_KEEPERR;
     else
-	sv_setpvn(ERRSV,"",0);
+	sv_setpvn_mg(ERRSV,"",0);
     if (yyparse() || PL_parser->error_count || !PL_eval_root) {
 	SV **newsp;			/* Used by POPBLOCK. */
 	PERL_CONTEXT *cx = &cxstack[cxstack_ix];
@@ -3037,7 +3037,7 @@
 	}
 	else {
 	    if (!*msg) {
-	        sv_setpvs(ERRSV, "Compilation error");
+	        sv_setpvs(ERRSV, "Compilation error"); /* Should this be _mg? sv_setpvs_mg doesn't exist? */ 
 	    }
 	}
 	PERL_UNUSED_VAR(newsp);
@@ -3773,7 +3773,7 @@
     else {
 	LEAVE;
 	if (!(save_flags & OPf_SPECIAL))
-	    sv_setpvn(ERRSV,"",0);
+	    sv_setpvn_mg(ERRSV,"",0);
     }
 
     RETURNOP(retop);
@@ -3817,7 +3817,7 @@
     if (flags & G_KEEPERR)
 	PL_in_eval |= EVAL_KEEPERR;
     else
-	sv_setpvn(ERRSV,"",0);
+	sv_setpvn_mg(ERRSV,"",0);
     if (flags & G_FAKINGEVAL) {
 	PL_eval_root = PL_op; /* Only needed so that goto works right. */
     }
@@ -3876,7 +3876,7 @@
     PL_curpm = newpm;	/* Don't pop $1 et al till now */
 
     LEAVE;
-    sv_setpvn(ERRSV,"",0);
+    sv_setpvn_mg(ERRSV,"",0);
     RETURN;
 }
 
diff -Naur old/perl-current/pp_sys.c new/perl-current/pp_sys.c
--- old/perl-current/pp_sys.c	2008-06-07 16:05:38.000000000 +0200
+++ new/perl-current/pp_sys.c	2008-06-07 20:22:39.000000000 +0200
@@ -432,7 +432,7 @@
   	SV * const error = ERRSV;
 	SvUPGRADE(error, SVt_PV);
 	if (SvPOK(error) && SvCUR(error))
-	    sv_catpvs(error, "\t...caught");
+	    sv_catpvs(error, "\t...caught"); /* Shuold this be _mg? sv_catpvs_mg doesn't exist? */
 	tmpsv = error;
 	tmps = SvPV_const(tmpsv, len);
     }
@@ -485,14 +485,14 @@
 		    PUTBACK;
 		    call_sv((SV*)GvCV(gv),
 			    G_SCALAR|G_EVAL|G_KEEPERR);
-		    sv_setsv(error,*PL_stack_sp--);
+		    sv_setsv_mg(error,*PL_stack_sp--);
 		}
 	    }
 	    DIE(aTHX_ NULL);
 	}
 	else {
 	    if (SvPOK(error) && SvCUR(error))
-		sv_catpvs(error, "\t...propagated");
+		sv_catpvs(error, "\t...propagated"); /* Should this be _mg? sv_catpvs_mg doesn't exist? */
 	    tmpsv = error;
 	    if (SvOK(tmpsv))
 		tmps = SvPV_const(tmpsv, len);
diff -Naur old/perl-current/t/op/eval.t new/perl-current/t/op/eval.t
--- old/perl-current/t/op/eval.t	2008-06-07 16:05:35.000000000 +0200
+++ new/perl-current/t/op/eval.t	2008-06-07 20:23:07.000000000 +0200
@@ -5,7 +5,7 @@
     @INC = '../lib';
 }
 
-print "1..95\n";
+print "1..101\n";
 
 eval 'print "ok 1\n";';
 
@@ -486,3 +486,44 @@
 
 
 
+# [perl #51370] eval { die "\x{a10d}" } followed by eval { 1 } did not reset
+# length $@ 
+$@ = "";
+eval { die "\x{a10d}"; };
+$_ = length $@;
+eval { 1 };
+
+print "not " if ($@ ne "");
+print "ok $test # length of \$@ after eval\n"; $test++;
+
+print "not " if (length $@ != 0);
+print "ok $test # length of \$@ after eval\n"; $test++;
+
+
+# In [perl #51370] Nicholas wondered wheter all calls with ERRSV should be changed
+# to _mg. This would fix the bug and allow tie'ing and stuff
+{
+    my $ok  = 0;
+    my $error = "foo";
+    package Eval2;
+    sub FETCH { $_[0]->[0]; }
+    sub STORE { $_[0]->[0] = $error = $_[1]; }
+    sub TIESCALAR { bless [] }
+
+    tie $@, "Eval2";
+    eval { 1 };
+    print "not " if ($@ ne "");
+    print "ok $test # test a tied \$@\n"; $test++;
+
+    print "not " if ($error ne "");
+    print "ok $test # test a tied \$@\n"; $test++;
+
+    eval { die "ttt\n" };
+    print "not " if ($@ ne "ttt\n");
+    print "ok $test # test a tied \$@\n"; $test++;
+
+    print "not " if ($error ne "ttt\n");
+    print "ok $test # test a tied \$@\n"; $test++;
+}
+
+
diff -Naur old/perl-current/toke.c new/perl-current/toke.c
--- old/perl-current/toke.c	2008-06-07 16:05:41.000000000 +0200
+++ new/perl-current/toke.c	2008-06-07 20:22:42.000000000 +0200
@@ -10652,7 +10652,7 @@
 
     /* Check the eval first */
     if (!PL_in_eval && SvTRUE(ERRSV)) {
- 	sv_catpvs(ERRSV, "Propagated");
+ 	sv_catpvs(ERRSV, "Propagated"); /* Should this be _mg? sv_catpvs_mg doesn't exist? */
 	yyerror(SvPV_nolen_const(ERRSV)); /* Duplicates the message inside eval */
 	(void)POPs;
 	res = SvREFCNT_inc_simple(sv);

@p5pRT
Copy link
Author

p5pRT commented Jun 17, 2008

From @rgs

2008/6/11 Bram via RT <perlbug-followup@​perl.org>​:

Attached are two patches​:

errsv_1.txt​:
- this patch clears all the magic associated with $@​,
- has the test of the bug report
- scrapes the output of Devel​::Peek​::Dump at the start and compares it
again after two evals (one that fails and one that succeeds).

errsv_2.txt​:

In the thread (Clearing magic) Nicholas said​:
<quote>
To my mind, making them all *_mg seems right, as it would trigger any
set magic on ERRSV.

Although (I think) that this would allow rather sick things like tie $@​
to work.
</quote>

- this patch changes most of them (not sure if I got them all) into _mg
- has the test of the bug report
- does a tie on $@​ and checks if STORE gets called.

Only one of the two patches has to be applied...

Thanks!
I applied the 1st patch as change #34068, and applied this on top of it​:

Change 34069 on 2008/06/17 by rgs@​scipion
  Rename the new macro clear_errsv() from last patch to CLEAR_ERRSV()

@p5pRT
Copy link
Author

p5pRT commented Jun 17, 2008

@rgs - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant