Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*foo{SCALAR} autovivifies #10241

Open
p5pRT opened this issue Mar 20, 2010 · 14 comments
Open

*foo{SCALAR} autovivifies #10241

p5pRT opened this issue Mar 20, 2010 · 14 comments

Comments

@p5pRT
Copy link

p5pRT commented Mar 20, 2010

Migrated from rt.perl.org#73666 (status was 'open')

Searchable as RT73666$

@p5pRT
Copy link
Author

p5pRT commented Mar 20, 2010

From zefram@fysh.org

Created by zefram@fysh.org

$ perl -lwe 'sub foo; print *foo{SCALAR}'
SCALAR(0x804d830)
$ perl -lwe 'sub foo; print *foo{ARRAY}'
Use of uninitialized value in print at -e line 1.

$

*foo{SCALAR} is autovivifying the scalar slot of the GV, whereas
*foo{ARRAY} and the others do not. This interferes with package walking
implemented in pure Perl. The mechanism for this behaviour is that the
scalar case of pp_gelem() references GvSVn(gv) rather than plain GvSV(gv).
GvSVn() autovivifies; GvSV() does not.

It looks like this is historical crud from a time when the SV slot was
always vivified upon GV creation. This initial vivification no longer
occurs (well, it's actually controlled by a preprocessor variable that is
presumably never used any more). GvSVn() appears to exist to simulate
the old behaviour, by lazy initialisation, for the convenience of those
things that relied on it. *foo{SCALAR} is not an appropriate place to
perform the simulation.

Perl Info

Flags:
    category=core
    severity=low

Site configuration information for perl 5.11.5:

Configured by zefram at Sun Feb 21 21:31:04 GMT 2010.

Summary of my perl5 (revision 5 version 11 subversion 5) configuration:
   
  Platform:
    osname=linux, osvers=2.6.26-2-686, archname=i386-linux-thread-multi
    uname='linux vigo.rous.org 2.6.26-2-686 #1 smp wed feb 10 08:59:21 utc 2010 i686 gnulinux '
    config_args='-des -Darchname=i386-linux -Dcccdlflags=-fPIC -Dccdlflags=-rdynamic -Dprefix=/home/zefram/usr/perl/perl_install/perl-5.11.5-i32-f52 -Dman1ext=1 -Dman3ext=3perl -Duselargefiles -Dusethreads -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dusedevel -Ui_db'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.3.2', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lgdbm -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.7.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.7'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic -Wl,-rpath,/home/zefram/usr/perl/perl_install/perl-5.11.5-i32-f52/lib/5.11.5/i386-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'

Locally applied patches:
    


@INC for perl 5.11.5:
    /home/zefram/usr/perl/perl_install/perl-5.11.5-i32-f52/lib/site_perl/5.11.5/i386-linux-thread-multi
    /home/zefram/usr/perl/perl_install/perl-5.11.5-i32-f52/lib/site_perl/5.11.5
    /home/zefram/usr/perl/perl_install/perl-5.11.5-i32-f52/lib/5.11.5/i386-linux-thread-multi
    /home/zefram/usr/perl/perl_install/perl-5.11.5-i32-f52/lib/5.11.5
    .


Environment for perl 5.11.5:
    HOME=/home/zefram
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/zefram/usr/perl/perl_install/perl-5.11.5-i32-f52/bin:/home/zefram/usr/perl/util:/home/zefram/pub/i686-pc-linux-gnu/bin:/home/zefram/pub/common/bin:/usr/bin:/usr/X11R6/bin:/bin:/usr/local/bin:/usr/games
    PERL_BADLANG (unset)
    SHELL=/usr/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Nov 15, 2010

From @rafl

This patch does the suggested change and adjusts the tests for the
changed behaviour.

@p5pRT
Copy link
Author

p5pRT commented Nov 15, 2010

From @rafl

0001-Stop-foo-SCALAR-from-autovivifying-the-SV-slot.patch
From 53432edbd743f02af3d1b2b8ad7ad144b0cb1530 Mon Sep 17 00:00:00 2001
From: Florian Ragwitz <rafl@debian.org>
Date: Mon, 15 Nov 2010 05:10:17 +0100
Subject: [PATCH] Stop *foo{SCALAR} from autovivifying the SV slot

This gets things into sync with *foo{ARRAY} et.al.
---
 pp.c      |    2 +-
 t/op/gv.t |    4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/pp.c b/pp.c
index 9e762d5..af18a41 100644
--- a/pp.c
+++ b/pp.c
@@ -675,7 +675,7 @@ PP(pp_gelem)
 	    break;
 	case 'S':
 	    if (strEQ(second_letter, "CALAR"))
-		tmpRef = GvSVn(gv);
+		tmpRef = GvSV(gv);
 	    break;
 	}
     }
diff --git a/t/op/gv.t b/t/op/gv.t
index f04bda0..2150c3b 100644
--- a/t/op/gv.t
+++ b/t/op/gv.t
@@ -524,8 +524,8 @@ foreach my $value ([1,2,3], {1=>2}, *STDOUT{IO}, \&ok, *STDOUT{FORMAT}) {
 
     $g = \*vowm;
     $r = eval {use strict; ${*{$g}{SCALAR}}};
-    is ($@, '',
-	"PERL_DONT_CREATE_GVSV shouldn't affect thingy syntax under strict");
+    like ($@, qr/^Can't use an undefined value as a SCALAR reference/,
+	"PERL_DONT_CREATE_GVSV should affect thingy syntax under strict");
 }
 
 {
-- 
1.7.2.3

@p5pRT
Copy link
Author

p5pRT commented Nov 15, 2010

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Nov 15, 2010

From @cpansprout

On Sun Nov 14 20​:13​:28 2010, rafl wrote​:

This patch does the suggested change and adjusts the tests for the
changed behaviour.

Yes, please!

@p5pRT
Copy link
Author

p5pRT commented Nov 15, 2010

From zefram@fysh.org

Florian Ragwitz via RT wrote​:

This patch does the suggested change and adjusts the tests for the
changed behaviour.

I already posted a patch with slightly more extensive tests.
git​://lake.fysh.org/zefram/perl.git branch zefram/gvsv_empty.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Nov 15, 2010

From @rafl

Zefram <zefram@​fysh.org> writes​:

I already posted a patch with slightly more extensive tests.
git​://lake.fysh.org/zefram/perl.git branch zefram/gvsv_empty.

I wasn't aware of that. You're patch is obviously better. I'd love for
it to go in if there's no objections to changing this very odd
behaviour.

@p5pRT
Copy link
Author

p5pRT commented Nov 15, 2010

From zefram@fysh.org

Florian Ragwitz wrote​:

I wasn't aware of that. You're patch is obviously better. I'd love for
it to go in if there's no objections to changing this very odd
behaviour.

When I posted mine some months ago, no one objected, though there was some
expectation that it would break a small number of modules. They would
be easily fixed, of course, and it seems that no one thinks it enough
of a backcompat issue to stand in the way of the improved behaviour.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Nov 15, 2010

From @nwc10

On Mon, Nov 15, 2010 at 09​:40​:28AM +0000, Zefram wrote​:

Florian Ragwitz wrote​:

I wasn't aware of that. You're patch is obviously better. I'd love for
it to go in if there's no objections to changing this very odd
behaviour.

Very odd, but very old.

When I posted mine some months ago, no one objected, though there was some
expectation that it would break a small number of modules. They would
be easily fixed, of course, and it seems that no one thinks it enough
of a backcompat issue to stand in the way of the improved behaviour.

I'm not *comfortable* with the change. When I changed the internals, to not
allocate the SCALAR slot, I intentionally kept the Perl-space behaviour
consistent, because my hunch was that there would be obscure code out there
relying on it.

I was part-way through trying to work out what on CPAN references
*thingy{SCALAR}, and what would break. I think that it's this
[comments date from mid-May]​:

# PASS​: Acme-Pr0n/lib/Acme/Pr0n.pm
# Doesn't build, will break​: Acme-RPC/lib/Acme/RPC.pm
# PASS​: Acme-State/lib/Acme/State.pm
# PASS​: AnyDBM_File-Importer/lib/AnyDBM_File/Importer.pm
# PASS​: App-Context/lib/App.pm
# PASS (but needs Class​::ISA)​: App-SocialSKK/inc/Devel/Symdump.pm
# Doesn't build, won't break​:
  Archive-TarGzip/t/Archive/Data/Secs2.pm
# File​::Package bust since pre-5.10
# In comments​: CGI.pm/lib/CGI.pm
# Will BREAK, and breaks from Class​::MOP Catalyst-Runtime/lib/Catalyst/ClassData.pm
# Variable​::Magic bust post 5.12.0, different from Zefram's bug report.
# Moose hurt by $@​ change
# In C code​: Class-AutoDB/docs/Developer/Dumper.xs
# Class-AutoDB/lib/Class/AutoDB/Dumper.xs
# PASS​: Class-Load/lib/Class/Load.pm
# BREAKS, at compile time​: Class-MOP/lib/Class/MOP/Package.pm
# PASS​: Class-Rebless/lib/Class/Rebless.pm
# Bug prevents test pass on blead, will break​:
  Class-Tangram/bin/ct2pod.pl
# PASS​: DBIx-CGITables/CGITables.pm
# BREAK​: Data-Dump-Streamer/t/globtest.t
# CORE​: Data-Dumper/Dumper.xs
# Data-Dumper/t/dumper.t
# PASS​: Data-Page-FlickrLike/inc/YAML/Types.pm
# PASS​: Data-Peek/t/20_DPeek.t
# PASS​: Data-Rlist/Rlist.pm.html
# Data-Rlist/lib/Data/Rlist.pm
# will break​:
Data-Rmap/lib/Data/Rmap.pm
# Data​::Startup bust, won't break​:
  Data-Secs2/lib/Data/Secs2.pm
  Data-SecsPack/t/Data/Data/Secs2.pm
# won't break​:
  Data-Startup/t/Data/Data/Secs2.pm
# won't break​:
  Data-Str2Num/t/Data/Data/Secs2.pm
Data-Visitor/t/globs.t
# "PASS"​: Devel-Cover/tests/cond_or
Devel-RemoteTrace/lib/Devel/RemoteTrace.pm
# won't break​:
# PASS​: Devel-Symdump/lib/Devel/Symdump.pm
# won't break, only in POD
  Devel-TypeCheck/lib/Devel/TypeCheck/Type.pm
# won't break​:
# PASS​: ExportAbove/ExportAbove.pm
# won't break​:
  ExtUtils-SVDmaker/t/ExtUtils/SVDmaker/Data/Secs2.pm
  ExtUtils-SVDmaker/t/ExtUtils/SVDmaker/expected/Data/Secs2.pm
# infer won't break​:
  File-AnySpec/t/File/Data/Secs2.pm
# infer won't break​:
  File-Maker/t/File/Data/Secs2.pm
# infer won't break​:
  File-PM2File/t/File/Data/Secs2.pm
# infer won't break​:
  File-Package/t/File/Data/Secs2.pm
# infer won't break​:
  File-Revision/t/File/Data/Secs2.pm
# infer won't break​:
  File-SmartNL/t/File/Data/Secs2.pm
# infer won't break​:
  File-SubPM/t/File/Data/Secs2.pm
# infer won't break​:
  File-Where/t/File/Data/Secs2.pm
# PASS​: Getopt-Lazy/lib/Getopt/Lazy.pm
# Believe will break​: *AUTOLOAD = *main​::AUTOLOAD{SCALAR};
  Gtk2-GladeXML-OO/lib/Gtk2/GladeXML/OO.pm
# won't break​:
  HTML-Embperl/Embperl.pm
# PASS​: Inline-Files/lib/Inline/Files/Virtual.pm
# PASS​: Interpolation/lib/Interpolation.pm
# will break​:
  JOAP/JOAP/Proxy/Package/Class.pm
# will break​:
  JOAP/JOAP/Server/Object.pm
# will break​:
  Konstrukt/lib/Konstrukt/Lib.pm
# PASS​: Lexical-Import/lib/Lexical/Import.pm
# Lexical-Import/lib/Lexical/Import.xs
# BREAK​: Lexical-Var/t/glob.t
# will break​:
  Lib-Module/Module.pm
MRP/MRP/Introspection.pm
# will break​:
  Meta/Meta/Lang/Perl/Perl.pm
# PASS​: Module-Runtime/lib/Module/Runtime.pm
# PASS​: Mouse/lib/Mouse/PurePerl.pm
# PASS​: POD2-FR/FR/perlref.pod
# PASS​: POD2-IT/IT/perlref.pod
# PASS (but messy)​: Parse-PerlConfig/PerlConfig.pm
# won't break​:
  Perl6-Pugs/perl5/PIL2JS/lib/Class/Rebless.pm
# won't break, in documentation​:
Ruby/lib/Ruby/PerlObject.pod
# CORE​: Safe/t/safeops.t
# Won't break​: Scope-Upper/t/40-localize_delete-target.t
# PASS​: Scriptalicious/lib/Scriptalicious.pm
# BREAK​: Symbol-Util/lib/Symbol/Util.pm
# PASS​: Symbol-Values/lib/Symbol/Values.pm
# Symbol-Values/t/Symbol-Values.t
# infer won't break
  Test-STDmaker/t/Test/STDmaker/Data/Secs2.pm
# infer won't break
  Test-Tech/t/Test/Tech/Data/Secs2.pm
# infer won't break
  Text-Column/t/Text/Data/Secs2.pm
# infer won't break
  Text-Replace/t/Text/Data/Secs2.pm
# infer won't break
  Text-Scrub/t/Text/Data/Secs2.pm
# infer won't break
  Tie-Eudora/t/Tie/Data/Secs2.pm
# infer won't break
  Tie-Form/t/Tie/Data/Secs2.pm
# infer won't break
  Tie-FormA/t/Tie/Data/Secs2.pm
# infer won't break
  Tie-Gzip/t/Tie/Data/Secs2.pm
# infer won't break
  Tie-Layers/t/Tie/Data/Secs2.pm
# PASS​: Tk-Browser/Browser/LibModule.pm
# Tk-Browser/Browser.pm
# PASS​: WeakRef-Auto/t/01_autoweak.t
# won't break, in documentation​:
Win32-SharedFileOpen/lib/Win32/SharedFileOpen.pm
# PASS​: XML-SAX-SimpleDispatcher/inc/YAML/Types.pm
# PASS​: XML-Validator-Schema/t/lib/YAML.pm
# BREAKS​: YAML/lib/YAML/Types.pm
# BREAKS​: YAML-Old/lib/YAML/Old/Types.pm
# will break​:
  YATT/web/cgi-bin/yatt.lib/YATT/Util/VarExporter.pm
# won't break
  Zoidberg/lib/Zoidberg.pm
# PASS​: autodie/t/format-clobber.t
# CORE​: base/lib/base.pm
# PASS​: classes/lib/classes.pm
# BREAKS​: ensure/lib/ensure.pm
# ensure/t/tests.t
# infer will pass
forks/lib/forks/Devel/Symdump.pm
# PASS​: namespace-clean/t/06-other-types.t
perl/ext/Data-Dumper/Dumper.xs
perl/ext/Data-Dumper/t/dumper.t
perl/ext/Safe/t/safeops.t
perl/ext/XS-APItest/t/svpeek.t
perl/lib/CGI.pm
perl/lib/autodie/t/format-clobber.t
perl/lib/base.pm
perl/pod/perlref.pod
perl/regen_lib.pl
perl/t/op/gv.t
perl_mlb/CGI.pm
perl_mlb/base.pm
# PASS​: psh/lib/Psh/Builtins/Symbols.pm
# psh/lib/Psh/Completion.pm
# won't break
  tinyperl/lib/LibZip/ScanPack.pm

Moose
MooseX-Emulate-Class-Accessor-Fast

Which I count as 16 (/will break/i + /BREAKS/i)
But I'm not sure how many cascade failures it would cause, as my gut feeling
is that it's low-level code that uses this, and changing it could produce
weird failures, and action-at-a-distance failures. (Such as, "WTF isn't this
scalar here a reference?". Autovivification happening within subroutines when
it wasn't expected can have very strange side effects, such as updates not
happening in larger data structures)

Nicholas Clark

@p5pRT
Copy link
Author

p5pRT commented Nov 15, 2010

From @demerphq

On 15 November 2010 11​:00, Nicholas Clark <nick@​ccl4.org> wrote​:

On Mon, Nov 15, 2010 at 09​:40​:28AM +0000, Zefram wrote​:

Florian Ragwitz wrote​:

I wasn't aware of that. You're patch is obviously better. I'd love for
it to go in if there's no objections to changing this very odd
behaviour.

Very odd, but very old.

Indeed. See the comment from Sarathy in Data​::Dumper.

  next if $k eq "SCALAR" &amp;&amp; ! defined $$gval; # always there

Im not against this change, but i do think it has thepotential to break a lot.

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

p5pRT commented Nov 21, 2010

From @ap

* Nicholas Clark <nick@​ccl4.org> [2010-11-15 11​:05]​:

I'm not *comfortable* with the change. When I changed the
internals, to not allocate the SCALAR slot, I intentionally
kept the Perl-space behaviour consistent, because my hunch was
that there would be obscure code out there relying on it.

I was part-way through trying to work out what on CPAN
references *thingy{SCALAR}, and what would break. I think that
it's this [comments date from mid-May]​:

How about a deprecation cycle then?

I know I have personally written code that relied on this
behaviour.

I also know that all of the code I wrote was to work around,
along the lines of what Yves quoted from Data​::Dumper.

And I remember that this behaviour *prevented* me from doing
things I wished I could do because it means there is no way to
tell (from Perl space) previously extant package scalars.

I can’t really imagine any way in which this behaviour makes
anything *easier*.

I would gladly have this old code of mine broken and I’ll bet
most other people will agree.

If this can be achieved, even if it takes some work to get there,
I would be glad of it and would like to advocate for it.

I’m thinking of a deprecation warning when *foo{SCALAR} used as
an rvalue autovivifies the slot. Does that sound reasonable?

Regards,
--
Aristotle Pagaltzis // <http​://plasmasturm.org/>

@p5pRT
Copy link
Author

p5pRT commented Jul 4, 2012

From @doy

What needs to happen to move forward here? Could this perhaps get a full
CPAN smoke? I'm also in favor of this change.

-doy

@p5pRT
Copy link
Author

p5pRT commented Jul 6, 2012

From @rjbs

* Jesse Luehrs via RT <perlbug-followup@​perl.org> [2012-07-04T18​:32​:54]

What needs to happen to move forward here? Could this perhaps get a full
CPAN smoke? I'm also in favor of this change.

Want to give a go at using Steffen's machinery for that? I bet it could use a
second-party shake-down. :)

--
rjbs

@p5pRT
Copy link
Author

p5pRT commented Jul 3, 2013

From @cpansprout

On Mon Nov 15 02​:00​:48 2010, nicholas wrote​:

On Mon, Nov 15, 2010 at 09​:40​:28AM +0000, Zefram wrote​:

Florian Ragwitz wrote​:

I wasn't aware of that. You're patch is obviously better. I'd love
for
it to go in if there's no objections to changing this very odd
behaviour.

Very odd, but very old.

When I posted mine some months ago, no one objected, though there
was some
expectation that it would break a small number of modules. They
would
be easily fixed, of course, and it seems that no one thinks it
enough
of a backcompat issue to stand in the way of the improved behaviour.

I'm not *comfortable* with the change. When I changed the internals,
to not
allocate the SCALAR slot, I intentionally kept the Perl-space
behaviour
consistent, because my hunch was that there would be obscure code out
there
relying on it.

I was part-way through trying to work out what on CPAN references
*thingy{SCALAR}, and what would break. I think that it's this
[comments date from mid-May]​:
...

Which I count as 16 (/will break/i + /BREAKS/i)
But I'm not sure how many cascade failures it would cause, as my gut
feeling
is that it's low-level code that uses this, and changing it could
produce
weird failures, and action-at-a-distance failures. (Such as, "WTF
isn't this
scalar here a reference?". Autovivification happening within
subroutines when
it wasn't expected can have very strange side effects, such as updates
not
happening in larger data structures)

If we allow *foo{SCALAR} *assignment*, and deferred element magic too,
how does that change things?

(I would like to make glob elements assignable for other reasons, too.
E.g., there are CPAN modules with hideous workarounds for the fact that
individual slots cannot be undefined.)

--

Father Chrysostomos

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants