From rconover@infogears.com

Created by rconover@infogears.com

While attempting to do a split call in a Safe compartment
which normally handles utf8 data fine, a "panic​: POPSTACK"
occurs.

Code to reproduce crash​:

use Safe;
use Encode qw(decode);
my $c = new Safe("Bla");
# Put the data in the safe package
$Bla​::foo = decode("latin1", "x , yzäöü");
# Convert the variable to be marked as utf8
$Bla​::foo =~ /\s/;
 
$c->reval(q(split /[\s,\0]+/, $foo;
1)) || die $@​;

Code that executes as normal​:

use Safe;
use Encode qw(decode);
my $c = new Safe("Bla");
# Put the data in the safe package
$Bla​::foo = decode("latin1", "x , yzäöü");
# Convert the variable to be marked as utf8
$Bla​::foo =~ /\s/;
 
$c->reval(q($foo =~ /\s/;
1)) || die $@​;

I'm trying to autoload the utf8 stuff into the safe compartment
so that I can perform utf8 string ops on the data while executing
code via reval();

Any help greatly appreciated.

Thanks,

Rusty

Flags:
    category=core
    severity=critical

Site configuration information for perl v5.8.4:

Configured by root at Mon May 10 02:31:08 MDT 2004.

Summary of my perl5 (revision 5 version 8 subversion 4) configuration:
  Platform:
    osname=linux, osvers=2.4.25, archname=i686-linux
    uname='linux site-3.infogears.com 2.4.25 #1 tue apr 13 23:45:13 mdt 2004 i686 i686 i386 gnulinux '
    config_args=''
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-DDEBUGGING -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
    optimize='-g',
    cppflags='-DDEBUGGING -fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm'
    ccversion='', gccversion='3.2.2 20030222 (Red Hat Linux 3.2.2-5)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
    libc=/lib/libc-2.3.2.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.3.2'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.8.4:
    /usr/local/lib/perl5/5.8.4/i686-linux
    /usr/local/lib/perl5/5.8.4
    /usr/local/lib/perl5/site_perl/5.8.4/i686-linux
    /usr/local/lib/perl5/site_perl/5.8.4
    /usr/local/lib/perl5/site_perl/5.8.3/i686-linux
    /usr/local/lib/perl5/site_perl/5.8.3
    /usr/local/lib/perl5/site_perl/5.8.2/i686-linux
    /usr/local/lib/perl5/site_perl/5.8.2
    /usr/local/lib/perl5/site_perl
    .


Environment for perl v5.8.4:
    HOME=/home/rconover
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/local/BerkeleyDB.4.2/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/usr/local/pgsql/bin:/usr/local/ibolis-corporate:/usr/local/java/bin:/home/rconover/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

From @iabyn

On Sun, Jun 13, 2004 at 10​:27​:07PM -0000, Rusty Conover wrote​:

While attempting to do a split call in a Safe compartment
which normally handles utf8 data fine, a "panic​: POPSTACK"
occurs.

Code to reproduce crash​:

use Safe;
use Encode qw(decode);
my $c = new Safe("Bla");
# Put the data in the safe package
$Bla​::foo = decode("latin1", "x , yzäöü");
# Convert the variable to be marked as utf8
$Bla​::foo =~ /\s/;

$c->reval(q(split /[\s,\0]+/, $foo;
1)) || die $@​;

Thanks for the report. What's happening is as follows​:

Because the string being split is UTF8 encoded, exceuting the regex
causes a 'use utf8' to be executed behind the scenes. Since this involves
doing stuff not aloowed by Safe, it dies, and is caught by the eval.
However, when split() splits to an array, as an ompimisation it switches
thethe array and the current stack, so the split happens directly on the
array rather than copying stuff back off the stack. When the program dies,
the stack isn't switched back.

The patch below fixes it - it's been applied to the current development
version of Perl.

Nick - I'm not sure this should be integrated into 5.8.x - all this
stack switching is hair-raising stuff, and I'm not yet convinced that this
is the best fix for it.

Dave.

--
Never do today what you can put off till tomorrow.

Change 23023 by davem@​davem-percy on 2004/07/02 01​:49​:11

  [perl #30258] utf8 POPSTACK crash on split execution
  split() does a SWITCHSTACK to directly split to an array, but
  if it subsequently dies (eg the regex triggers a 'use utf8' which
  is then denied by Safe), then the switch doesn't get undone. Add
  a new save type to allow for this.

Affected files ...

... //depot/perl/pp.c#420 edit
... //depot/perl/scope.c#123 edit
... //depot/perl/scope.h#66 edit

Differences ...

==== //depot/perl/pp.c#420 (text) ====

@​@​ -4439,7 +4439,6 @​@​
  I32 origlimit = limit;
  I32 realarray = 0;
  I32 base;
- AV *oldstack = PL_curstack;
  I32 gimme = GIMME_V;
  I32 oldsave = PL_savestack_ix;
  I32 make_mortal = 1;
@​@​ -4488,8 +4487,7 @​@​
  AvARRAY(ary)[i] = &PL_sv_undef; /* don't free mere refs */
  }
  /* temporarily switch stacks */
- SWITCHSTACK(PL_curstack, ary);
- PL_curstackinfo->si_stack = ary;
+ SAVESWITCHSTACK(PL_curstack, ary);
  make_mortal = 0;
  }
  }
@​@​ -4658,7 +4656,6 @​@​
  }
  }

- LEAVE_SCOPE(oldsave);
  iters = (SP - PL_stack_base) - base;
  if (iters > maxiters)
  DIE(aTHX_ "Split loop");
@​@​ -4684,10 +4681,11 @​@​
  }
  }

+ PUTBACK;
+ LEAVE_SCOPE(oldsave); /* may undo an earlier SWITCHSTACK */
+ SPAGAIN;
  if (realarray) {
  if (!mg) {
- SWITCHSTACK(ary, oldstack);
- PL_curstackinfo->si_stack = oldstack;
  if (SvSMAGICAL(ary)) {
  PUTBACK;
  mg_set((SV*)ary);

==== //depot/perl/scope.c#123 (text) ====

@​@​ -1063,6 +1063,15 @​@​
  AvARRAY((PAD*)ptr)[off] = (SV*)SSPOPPTR;
  }
  break;
+ case SAVEt_SAVESWITCHSTACK​:
+ {
+ dSP;
+ AV* t = (AV*)SSPOPPTR;
+ AV* f = (AV*)SSPOPPTR;
+ SWITCHSTACK(t,f);
+ PL_curstackinfo->si_stack = f;
+ }
+ break;
  case SAVEt_SET_SVFLAGS​:
  {
  U32 val = (U32)SSPOPINT;

==== //depot/perl/scope.h#66 (text) ====

@​@​ -48,6 +48,7 @​@​
#define SAVEt_SHARED_PVREF 37
#define SAVEt_BOOL 38
#define SAVEt_SET_SVFLAGS 39
+#define SAVEt_SAVESWITCHSTACK 40

#ifndef SCOPE_SAVES_SIGNAL_MASK
#define SCOPE_SAVES_SIGNAL_MASK 0
@​@​ -169,6 +170,16 @​@​
  SSPUSHINT(SAVEt_COMPPAD); \
  } STMT_END

+#define SAVESWITCHSTACK(f,t) \
+ STMT_START { \
+ SSCHECK(3); \
+ SSPUSHPTR((SV*)(f)); \
+ SSPUSHPTR((SV*)(t)); \
+ SSPUSHINT(SAVEt_SAVESWITCHSTACK); \
+ SWITCHSTACK((f),(t)); \
+ PL_curstackinfo->si_stack = (t); \
+ } STMT_END
+
#ifdef USE_ITHREADS
# define SAVECOPSTASH(c) SAVEPPTR(CopSTASHPV(c))
# define SAVECOPSTASH_FREE(c) SAVESHAREDPV(CopSTASHPV(c))

The RT System itself - Status changed from 'new' to 'open'

@iabyn - Status changed from 'open' to 'resolved'

From @rgs

Dave Mitchell wrote​:

Because the string being split is UTF8 encoded, exceuting the regex
causes a 'use utf8' to be executed behind the scenes. Since this involves
doing stuff not aloowed by Safe, it dies, and is caught by the eval.
However, when split() splits to an array, as an ompimisation it switches
thethe array and the current stack, so the split happens directly on the
array rather than copying stuff back off the stack. When the program dies,
the stack isn't switched back.

The patch below fixes it - it's been applied to the current development
version of Perl.

Ooh, that's clever :) although perhaps a bit late for maint, I agree.

Change 23023 by davem@​davem-percy on 2004/07/02 01​:49​:11

\[perl \#30258\] utf8 POPSTACK crash on split execution
split\(\) does a SWITCHSTACK to directly split to an array\, but
if it subsequently dies \(eg the regex triggers a 'use utf8' which
is then denied by Safe\)\, then the switch doesn't get undone\. Add
a new save type to allow for this\.

From nick@ing-simmons.net

Rafael Garcia-Suarez <rgarciasuarez@​mandrakesoft.com> writes​:

Dave Mitchell wrote​:

Because the string being split is UTF8 encoded, exceuting the regex
causes a 'use utf8' to be executed behind the scenes. Since this involves
doing stuff not aloowed by Safe, it dies, and is caught by the eval.
However, when split() splits to an array, as an ompimisation it switches
thethe array and the current stack, so the split happens directly on the
array rather than copying stuff back off the stack. When the program dies,
the stack isn't switched back.

The patch below fixes it - it's been applied to the current development
version of Perl.

Ooh, that's clever :)

Seconded - a neat idea.

although perhaps a bit late for maint, I agree.

Maybe the next one once it has been tested a bit.

Change 23023 by davem@​davem-percy on 2004/07/02 01​:49​:11

\[perl \#30258\] utf8 POPSTACK crash on split execution
split\(\) does a SWITCHSTACK to directly split to an array\, but
if it subsequently dies \(eg the regex triggers a 'use utf8' which
is then denied by Safe\)\, then the switch doesn't get undone\. Add
a new save type to allow for this\.