Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

referenced constant loses readonlyness #11929

Open
p5pRT opened this issue Feb 3, 2012 · 58 comments
Open

referenced constant loses readonlyness #11929

p5pRT opened this issue Feb 3, 2012 · 58 comments

Comments

@p5pRT
Copy link

p5pRT commented Feb 3, 2012

Migrated from rt.perl.org#109744 (status was 'open')

Searchable as RT109744$

@p5pRT
Copy link
Author

p5pRT commented Feb 3, 2012

From zefram@fysh.org

Created by zefram@fysh.org

On a threading Perl, a list-mode refgen applied to a constant will
copy the constant rather than reference it. An srefgen, however, will
reference it, and either will work correctly on a non-threading Perl​:

$ perl-single -MDevel​::Peek -lwe '$sref = \123; Dump $sref; ($lref,undef) = \(123,2); Dump $lref;'
SV = IV(0x80969a4) at 0x80969a8
  REFCNT = 1
  FLAGS = (ROK)
  RV = 0x8096988
  SV = IV(0x8096984) at 0x8096988
  REFCNT = 2
  FLAGS = (IOK,READONLY,pIOK)
  IV = 123
SV = IV(0x80969c4) at 0x80969c8
  REFCNT = 1
  FLAGS = (ROK)
  RV = 0x80969e8
  SV = IV(0x80969e4) at 0x80969e8
  REFCNT = 2
  FLAGS = (IOK,READONLY,pIOK)
  IV = 123
$ perl-threads -MDevel​::Peek -lwe '$sref = \123; Dump $sref; ($lref,undef) = \(123,2); Dump $lref;'
SV = IV(0x8617e50) at 0x8617e54
  REFCNT = 1
  FLAGS = (ROK)
  RV = 0x8617e34
  SV = IV(0x8617e30) at 0x8617e34
  REFCNT = 2
  FLAGS = (IOK,READONLY,pIOK)
  IV = 123
SV = IV(0x8617e70) at 0x8617e74
  REFCNT = 1
  FLAGS = (ROK)
  RV = 0x85fd7b4
  SV = IV(0x85fd7b0) at 0x85fd7b4
  REFCNT = 1
  FLAGS = (IOK,pIOK)
  IV = 123

I presume that this copy is being performed by S_refto() in pp.c, in
the SvPADTMP(sv) branch. This would occur because the constant goes
into the pad, and gets marked PADTMP, on a threading Perl​:

$ perl-single -MDevel​::Peek -lwe 'Dump 123'
SV = IV(0x97ae934) at 0x97ae938
  REFCNT = 1
  FLAGS = (IOK,READONLY,pIOK)
  IV = 123
$ perl-threads -MDevel​::Peek -lwe 'Dump 123'
SV = IV(0x831dde8) at 0x831ddec
  REFCNT = 1
  FLAGS = (PADTMP,IOK,READONLY,pIOK)
  IV = 123

The srefgen case behaves itself because it's constant-folded before the
constant moves to the pad. An srefgen that can't constant-fold will go
wrong in the same way as the list-mode refgen​:

$ perl-single -MDevel​::Peek -lwe 'sub dumpref($) { Dump \$_[0]; } dumpref 123'
SV = IV(0x9628074) at 0x9628078
  REFCNT = 1
  FLAGS = (TEMP,ROK)
  RV = 0x9637990
  SV = IV(0x963798c) at 0x9637990
  REFCNT = 2
  FLAGS = (IOK,READONLY,pIOK)
  IV = 123
$ perl-threads -MDevel​::Peek -lwe 'sub dumpref($) { Dump \$_[0]; } dumpref 123'
SV = IV(0x873f8f0) at 0x873f8f4
  REFCNT = 1
  FLAGS = (TEMP,ROK)
  RV = 0x873f7b4
  SV = IV(0x873f7b0) at 0x873f7b4
  REFCNT = 1
  FLAGS = (IOK,pIOK)
  IV = 123

It seems awfully wrong for S_refto() to ever copy the referent. I don't
know enough about the pad to see how it should be changed, though.

Perl Info

Flags:
    category=core
    severity=low

Site configuration information for perl 5.15.7:

Configured by zefram at Sat Jan 21 18:33:38 GMT 2012.

Summary of my perl5 (revision 5 version 15 subversion 7) configuration:
   
  Platform:
    osname=linux, osvers=2.6.32-5-686, archname=i386-linux-thread-multi
    uname='linux vigo.rous.org 2.6.32-5-686 #1 smp thu nov 3 04:23:54 utc 2011 i686 gnulinux '
    config_args='-des -Darchname=i386-linux -Dcccdlflags=-fPIC -Dccdlflags=-rdynamic -Dprefix=/home/zefram/usr/perl/perl_install/perl-5.15.7-i32-f52 -Dman1ext=1 -Dman3ext=3perl -Duselargefiles -Dusethreads -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dusedevel -Uversiononly -Ui_db'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.4.5', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/../lib /usr/lib/../lib /lib /usr/lib /usr/lib64
    libs=-lnsl -lgdbm -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.11.2.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.11.2'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic -Wl,-rpath,/home/zefram/usr/perl/perl_install/perl-5.15.7-i32-f52/lib/5.15.7/i386-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'

Locally applied patches:
    


@INC for perl 5.15.7:
    /home/zefram/usr/perl/perl_install/perl-5.15.7-i32-f52/lib/site_perl/5.15.7/i386-linux-thread-multi
    /home/zefram/usr/perl/perl_install/perl-5.15.7-i32-f52/lib/site_perl/5.15.7
    /home/zefram/usr/perl/perl_install/perl-5.15.7-i32-f52/lib/5.15.7/i386-linux-thread-multi
    /home/zefram/usr/perl/perl_install/perl-5.15.7-i32-f52/lib/5.15.7
    .


Environment for perl 5.15.7:
    HOME=/home/zefram
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/zefram/usr/perl/perl_install/perl-5.15.7-i32-f52/bin:/home/zefram/usr/perl/util:/home/zefram/pub/i686-pc-linux-gnu/bin:/home/zefram/pub/common/bin:/usr/bin:/bin:/usr/local/bin:/usr/games
    PERL_BADLANG (unset)
    SHELL=/usr/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Feb 3, 2012

From @cpansprout

On Fri Feb 03 08​:03​:52 2012, zefram@​fysh.org wrote​:

It seems awfully wrong for S_refto() to ever copy the referent. I
don't
know enough about the pad to see how it should be changed, though.

It’s necessary so that ops with TARG appear to return a different value
each time. The abstraction leaks badly, though. See #78194.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Feb 3, 2012

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Feb 3, 2012

From zefram@fysh.org

Father Chrysostomos via RT wrote​:

It's necessary so that ops with TARG appear to return a different value
each time. The abstraction leaks badly, though. See #78194.

It's tempting to write a module that replaces *every* const op with
my identity-preserving variant. But that'd break stuff that expects
(reasonably) to pull the SV directly out of const ops that it created.

I think the workaround that I need at this point is a function that
generates a fresh readonly scalar (with specified value), doesn't wave it
anywhere near a pad, and returns a ref to it. You could replace "\123"
with "readonly_scalar(123)" where needed to initialise a Lexical​::Var or
to assign to a glob. Thing is, I already figured I need such a function,
for situations where the desired value doesn't otherwise come in readonly
form (e.g., readonly_scalar([])).

-zefram

@p5pRT
Copy link
Author

p5pRT commented Feb 6, 2012

From zefram@fysh.org

I wrote​:

I think the workaround that I need at this point is a function that
generates a fresh readonly scalar (with specified value), doesn't wave it
anywhere near a pad, and returns a ref to it.

Now on CPAN as Scalar​::Construct​::constant().

-zefram

@p5pRT
Copy link
Author

p5pRT commented Jul 3, 2012

From @doy

Is this the same issue as #21979?

-doy

@p5pRT
Copy link
Author

p5pRT commented Jul 3, 2012

From @cpansprout

On Tue Jul 03 11​:19​:55 2012, doy wrote​:

Is this the same issue as #21979?

Pretty much. But fixing it is not as simple as ‘fixing’ it.

"$a$b" returns a new value each time, so there is no reason for it to be
read-only. Some would argue that "a" and 1..3 should return modifiable
values, since there’s no harm in that. But other want referential
identity to be preserved.

In the case of built-in operators (even const ops), I lean toward the
side of modifiability, though I know that referential identity is
important in many cases, too.

It’s hard to solve, really. How should constants behave? If a constant
is just a subroutine, then foo() should behave like a subroutine and
return a modifiable value (unless it has the :lvalue attribute). But
constants produced by ‘use constant’ are often used to inline literally
the same value.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jul 3, 2012

From [Unknown Contact. See original ticket]

On Tue Jul 03 11​:19​:55 2012, doy wrote​:

Is this the same issue as #21979?

Pretty much. But fixing it is not as simple as ‘fixing’ it.

"$a$b" returns a new value each time, so there is no reason for it to be
read-only. Some would argue that "a" and 1..3 should return modifiable
values, since there’s no harm in that. But other want referential
identity to be preserved.

In the case of built-in operators (even const ops), I lean toward the
side of modifiability, though I know that referential identity is
important in many cases, too.

It’s hard to solve, really. How should constants behave? If a constant
is just a subroutine, then foo() should behave like a subroutine and
return a modifiable value (unless it has the :lvalue attribute). But
constants produced by ‘use constant’ are often used to inline literally
the same value.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jul 3, 2012

From @rurban

On Tue, Jul 3, 2012 at 3​:18 PM, Father Chrysostomos via RT
<perlbug-comment@​perl.org> wrote​:

On Tue Jul 03 11​:19​:55 2012, doy wrote​:

Is this the same issue as #21979?

Pretty much. But fixing it is not as simple as ‘fixing’ it.

"$a$b" returns a new value each time, so there is no reason for it to be
read-only. Some would argue that "a" and 1..3 should return modifiable
values, since there’s no harm in that. But other want referential
identity to be preserved.

In the case of built-in operators (even const ops), I lean toward the
side of modifiability, though I know that referential identity is
important in many cases, too.

It’s hard to solve, really. How should constants behave? If a constant
is just a subroutine, then foo() should behave like a subroutine and
return a modifiable value (unless it has the :lvalue attribute). But
constants produced by ‘use constant’ are often used to inline literally
the same value.

FYI​: My branch typed/ro branch adds support for const pads which is missing
so far in core.
Create and preserve constness on pads, define the behavior (also for
function arguments and return values),
do compile-time optimizations. I'm not there yet.

F<https://github.com/rurban/perl/commits/typed/ro>
--
Reini Urban
http​://cpanel.net/ http​://www.perl-compiler.org/

@p5pRT
Copy link
Author

p5pRT commented Oct 7, 2012

From @cpansprout

On Tue Jul 03 13​:18​:19 2012, sprout wrote​:

On Tue Jul 03 11​:19​:55 2012, doy wrote​:

Is this the same issue as #21979?

Pretty much. But fixing it is not as simple as ‘fixing’ it.

"$a$b" returns a new value each time, so there is no reason for it to be
read-only. Some would argue that "a" and 1..3 should return modifiable
values, since there’s no harm in that. But other want referential
identity to be preserved.

In the case of built-in operators (even const ops), I lean toward the
side of modifiability, though I know that referential identity is
important in many cases, too.

It’s hard to solve, really. How should constants behave? If a constant
is just a subroutine, then foo() should behave like a subroutine and
return a modifiable value (unless it has the :lvalue attribute). But
constants produced by ‘use constant’ are often used to inline literally
the same value.

Interestingly, copy-on-write can affect whether a ‘constant’ is constant​:

use overload;
BEGIN { overload'constant"integer" =>=> sub { __PACKAGE__ } }
eval { ${\5} = "whatever" };
print $@​ || "no error\n";
BEGIN { overload'constant"integer" =>=> sub { "main" } }
eval { ${\5} = "whatever" };
print $@​ || "no error\n";
__END__

Ouput​:

no error
Modification of a read-only value attempted at - line 6.

And with ‘use constant’​:

no warnings 'redefine';
use constant foo => __PACKAGE__;
eval { ${\foo} = "whatever" };
print $@​ || "no error\n";
use constant foo => "main";
eval { ${\foo} = "whatever" };
print $@​ || "no error\n";

Simply turning on the read-only flag has no effect on copy-on-write
scalars (like that returned by __PACKAGE__).

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jun 11, 2013

From @cpansprout

On Tue Jul 03 13​:18​:19 2012, sprout wrote​:

On Tue Jul 03 11​:19​:55 2012, doy wrote​:

Is this the same issue as #21979?

Pretty much. But fixing it is not as simple as ‘fixing’ it.

"$a$b" returns a new value each time, so there is no reason for it to be
read-only. Some would argue that "a" and 1..3 should return modifiable
values, since there’s no harm in that. But other want referential
identity to be preserved.

In the case of built-in operators (even const ops), I lean toward the
side of modifiability, though I know that referential identity is
important in many cases, too.

It’s hard to solve, really. How should constants behave? If a constant
is just a subroutine, then foo() should behave like a subroutine and
return a modifiable value (unless it has the :lvalue attribute). But
constants produced by ‘use constant’ are often used to inline literally
the same value.

OK, so there are three arguments regarding whether values return by
potentially constant operators should be modifiable​:

1) It doesn’t cause any harm to make it modifiable, so why not? It’s
Perlish.
2) I was expecting it not to be modifiable, so that it is modifiable
must be a bug.
3) I need this thing to return exactly the same scalar, so I can compare
memory addresses.

I’m afraid the second one sounds stupid because I am biased against it
and know not how to express it convincingly. I have seen a similar
argument come up in bug reports, which stated what was expected, but not
why. I don’t find it convincing at all.

That leaves us with 1 and 3. I propose that we make threaded and
non-threaded perls the same, like this (a compromise between the two
current sets of behaviour, with a twist)​:

• Operators that return different values depending on parameters ($a+$b,
"$a$b") should continue to return modifiable values. This will include
1..3, since the values returned are scalars generated on the fly.
• For consistency (constant folding is just an optimisation, and should
not change behaviour), this will be extended to foldable operations, too
(1+2, "a"."b"), which were modifiable only under threads.
• Other literal strings and numbers will be read-only, even when passed
to subroutines. Currently under threads, it’s a bit of both, and the
implementation details leak like a sieve.
• Return values of ! and other operators returning booleans will
continue to be read-only, as they are not generated scalars, but two
specific scalars, &PL_sv_yes and &PL_sv_no.
• Inlined constant subs created by ‘use constant’ and ‘sub (){42}’ will
be *modifiable*, because sub calls copy their return values. In other
words, these two will behave the same way​:

  for ( sub(){ 42}->() ) { $_++ }
  for ( sub(){return 42}->() ) { $_++ }

Furthermore, ‘use constant’ usually copies the values anyway.
• Referential identity will still be preserved by XS modules that make
their own const ops. Whether the value is read-only will depend on
whether the module makes it read-only.

I’m going to go head and work on this. You’d better yell loudly if you
disagree. :-)

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jun 11, 2013

From @ikegami

On Mon, Jun 10, 2013 at 9​:14 PM, Father Chrysostomos via RT <
perlbug-followup@​perl.org> wrote​:

• Operators that return different values depending on parameters ($a+$b,
"$a$b") should continue to return modifiable values. This will include
1..3, since the values returned are scalars generated on the fly.

Not they're not. The list is flattened into an array of modifiable scalars
at compile time, and that's the cause of bug RT#3105.

  >perl -e"for (1..2) { for ((), 1..3) { print $_++; } print qq{\n}; }"
  123
  234

(The parens ensure that the ".." is a range operator rather than the part
of the counting loop syntax "for (EXPR..EXPR)".)

I tried to fix it, but there was no consensus as to how to fix it. IIRC,
Abigail wanted modifiable values and Zephram wanted constants. (Note​: Most
of the conversation about RT#3105 doesn't show up in the ticket.)

Modifiable was better for backwards compatibility if nothing else.

@p5pRT
Copy link
Author

p5pRT commented Jun 11, 2013

From @cpansprout

On Mon Jun 10 19​:15​:59 2013, ikegami@​adaelis.com wrote​:

On Mon, Jun 10, 2013 at 9​:14 PM, Father Chrysostomos via RT <
perlbug-followup@​perl.org> wrote​:

• Operators that return different values depending on parameters ($a+$b,
"$a$b") should continue to return modifiable values. This will include
1..3, since the values returned are scalars generated on the fly.

Not they're not. The list is flattened into an array of modifiable scalars
at compile time, and that's the cause of bug RT#3105.

I had forgotten the exact implementation. At least *conceptually* they
are generated on the fly.

>perl \-e"for \(1\.\.2\) \{ for \(\(\)\, 1\.\.3\) \{ print $\_\+\+; \} print qq\{\\n\}; \}"
123
234

(The parens ensure that the ".." is a range operator rather than the part
of the counting loop syntax "for (EXPR..EXPR)".)

I tried to fix it, but there was no consensus as to how to fix it. IIRC,
Abigail wanted modifiable values and Zephram wanted constants. (Note​: Most
of the conversation about RT#3105 doesn't show up in the ticket.)

Modifiable was better for backwards compatibility if nothing else.

I was trying to come up with a self-consistent overarching plan that
would hopefully overcome the lack of consensus. At least you don’t
appear to have explicitly disagreed with me. :-)

Keeping it modifiable while fixing the bug is simple​: Set the PADTMP
flag and make sure PADTMP thingies get copied not only by refgen, but
also by for loops, sub calls, etc. (see ticket #78194).

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jun 11, 2013

From @chipdude

On 6/10/2013 6​:14 PM, Father Chrysostomos via RT wrote​:

OK, so there are three arguments regarding whether values return by
potentially constant operators should be modifiable​:

1) It doesn’t cause any harm to make it modifiable, so why not? It’s
Perlish.
2) I was expecting it not to be modifiable, so that it is modifiable
must be a bug.
3) I need this thing to return exactly the same scalar, so I can compare
memory addresses.

I’m afraid the second one sounds stupid because I am biased against it
and know not how to express it convincingly.

It's an argument from parsimony​: Perl should not do unnecessary work.
Specifically, it should not create N SVs all with the same constant,
because it can avoid doing so by making one read-only SV. Perhaps that
doesn't sound so bad?

@p5pRT
Copy link
Author

p5pRT commented Jun 11, 2013

From @ikegami

On Mon, Jun 10, 2013 at 11​:38 PM, Father Chrysostomos via RT <
perlbug-followup@​perl.org> wrote​:

I was trying to come up with a self-consistent overarching plan that
would hopefully overcome the lack of consensus. At least you don’t
appear to have explicitly disagreed with me. :-)

Correct. I don't care much either way.

My analysis​:

Allowing "writable constants" allows more code like the following​:

  my @​a = map { s/^\s+//; s/\s+\z//; } <$fh>;

I'm not sure if that's a good thing or not. On the plus side, it saves you
from writing something like the following​: (Yes, I know of s///r)

  my @​a = map { my $s = $_; $s =~ s/^\s+//; $s =~ s/\s+\z//; $s } <$fh>;

On the con side, things get interesting if you change the input from a
constant to something else.

  my @​a = map { s/^\s+//; s/\s+\z//; } @​b; # @​b gets changed too.

Another possible downsides of having "writable constants" is that it might
delay error discovery. But I can't come up with an example, so it can't be
that much of a problem.

Does any of this matter to me? I doubt it will ever come up.

@p5pRT
Copy link
Author

p5pRT commented Jun 11, 2013

From @cpansprout

On Mon Jun 10 20​:38​:52 2013, rev.chip@​gmail.com wrote​:

On 6/10/2013 6​:14 PM, Father Chrysostomos via RT wrote​:

OK, so there are three arguments regarding whether values return by
potentially constant operators should be modifiable​:

1) It doesn’t cause any harm to make it modifiable, so why not? It’s
Perlish.
2) I was expecting it not to be modifiable, so that it is modifiable
must be a bug.
3) I need this thing to return exactly the same scalar, so I can compare
memory addresses.

I’m afraid the second one sounds stupid because I am biased against it
and know not how to express it convincingly.

It's an argument from parsimony​: Perl should not do unnecessary work.
Specifically, it should not create N SVs all with the same constant,
because it can avoid doing so by making one read-only SV. Perhaps that
doesn't sound so bad?

No, but it’s wrong. :-) A single read-only SV can suddenly turn into
multiple SVs when necessary. I.e., we can pretend there are multiple
SVs, but optimise it down to one behind the scenes. We already have the
mechanism for that; it’s just very buggy right now, and I know how to
fix it.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jun 11, 2013

From @nwc10

On Mon, Jun 10, 2013 at 11​:28​:09PM -0700, Father Chrysostomos via RT wrote​:

On Mon Jun 10 20​:38​:52 2013, rev.chip@​gmail.com wrote​:

On 6/10/2013 6​:14 PM, Father Chrysostomos via RT wrote​:

OK, so there are three arguments regarding whether values return by
potentially constant operators should be modifiable​:

1) It doesn't cause any harm to make it modifiable, so why not? It's
Perlish.
2) I was expecting it not to be modifiable, so that it is modifiable
must be a bug.
3) I need this thing to return exactly the same scalar, so I can compare
memory addresses.

I'm afraid the second one sounds stupid because I am biased against it
and know not how to express it convincingly.

I realise. But it's bugging me a lot that if the name is "constant" it should
be just that. Not a generator of mutable temporaries initialised to the value.

It's not "I was expecting it not to be modifiable" but more "the names used to
describe this thing are terms with semantics of 'not modifiable'. So why does
the implementation not enforce the semantics?"

It's an argument from parsimony​: Perl should not do unnecessary work.
Specifically, it should not create N SVs all with the same constant,
because it can avoid doing so by making one read-only SV. Perhaps that
doesn't sound so bad?

No, but it's wrong. :-) A single read-only SV can suddenly turn into
multiple SVs when necessary. I.e., we can pretend there are multiple
SVs, but optimise it down to one behind the scenes. We already have the
mechanism for that; it's just very buggy right now, and I know how to
fix it.

I'm still not comfortable that this is "fix", at a language design level.
Yes, the current implementation leaks through. But an alternative change is
to stop it leaking, and have things that are named "constant" be constant.

I think that "we can do this" is coming at it from the wrong end.

Start with "what should the semantics be?" and then make the implementation
deliver them.

Nicholas Clark

@p5pRT
Copy link
Author

p5pRT commented Jun 11, 2013

From @cpansprout

On Tue Jun 11 01​:20​:55 2013, nicholas wrote​:

On Mon, Jun 10, 2013 at 11​:28​:09PM -0700, Father Chrysostomos via RT
wrote​:

On Mon Jun 10 20​:38​:52 2013, rev.chip@​gmail.com wrote​:

On 6/10/2013 6​:14 PM, Father Chrysostomos via RT wrote​:

OK, so there are three arguments regarding whether values return
by
potentially constant operators should be modifiable​:

1) It doesn't cause any harm to make it modifiable, so why not?
It's
Perlish.
2) I was expecting it not to be modifiable, so that it is
modifiable
must be a bug.
3) I need this thing to return exactly the same scalar, so I can
compare
memory addresses.

I'm afraid the second one sounds stupid because I am biased
against it
and know not how to express it convincingly.

I realise. But it's bugging me a lot that if the name is "constant" it
should
be just that. Not a generator of mutable temporaries initialised to
the value.

It's not "I was expecting it not to be modifiable" but more "the names
used to
describe this thing are terms with semantics of 'not modifiable'. So
why does
the implementation not enforce the semantics?"

It's an argument from parsimony​: Perl should not do unnecessary
work.
Specifically, it should not create N SVs all with the same
constant,
because it can avoid doing so by making one read-only SV. Perhaps
that
doesn't sound so bad?

No, but it's wrong. :-) A single read-only SV can suddenly turn
into
multiple SVs when necessary. I.e., we can pretend there are
multiple
SVs, but optimise it down to one behind the scenes. We already have
the
mechanism for that; it's just very buggy right now, and I know how
to
fix it.

I'm still not comfortable that this is "fix", at a language design
level.
Yes, the current implementation leaks through. But an alternative
change is
to stop it leaking, and have things that are named "constant" be
constant.

I think that "we can do this" is coming at it from the wrong end.

Start with "what should the semantics be?"

That’s precisely what
<https://rt-archive.perl.org/perl5/Ticket/Display.html?id=109744#txn-1223457> is
meant to be!

and then make the
implementation
deliver them.

I’m trying to get to that, but we seem to have a disagreement above.

For me, the term ‘constant’ doesn’t necessarily have to be ‘read-only
SV’. If in an expression like ‘$x = PI’ the ‘PI’ returns exactly the
same value every time, then it is constant, is it not?

I feel strongly that sub(){42} and sub(){return 42} should behave the
same way.

If you feel equally strongly that ‘use constant foo=>42; ${\foo}++’
should die, then maybe the problem here is the equivalence of constants
and subroutines. Maybe the constants generated by constant.pm should
actually be constants (according to your definition), and should be
handled distinctly from sub foo(){42}, which I think *should* allow
${\foo}++, precisely because the variant with ‘return’ allows it.

I could implement it *that* way (inlinable subs are not necessarily
constant) if you prefer.

(BTW, the way that PADMY and PADTMP currently interact is completely
nonsensical, but you probably already knew that. :-)

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jun 14, 2013

From @nwc10

On Tue Jun 11 14​:53​:37 2013, sprout wrote​:

That’s precisely what
<https://rt-archive.perl.org/perl5/Ticket/Display.html?id=109744#txn-1223457> is
meant to be!

Sorry. I'd missed this.

and then make the
implementation
deliver them.

I’m trying to get to that, but we seem to have a disagreement above.

For me, the term ‘constant’ doesn’t necessarily have to be ‘read-only
SV’. If in an expression like ‘$x = PI’ the ‘PI’ returns exactly the
same value every time, then it is constant, is it not?

No. To my mind, strongly, constant means "constant" in the sense of
immutable. My mind doesn't see "constants" as anonymous value
generators.

(Although in that example, $x won't be immutable, as it's a copy of
the immutable value.)

I feel strongly that sub(){42} and sub(){return 42} should behave the
same way.

This is a good point, I don't have a strong view on that, but I can
see that it's ugly that they aren't.

If you feel equally strongly that ‘use constant foo=>42; ${\foo}++’
should die, then maybe the problem here is the equivalence of constants
and subroutines. Maybe the constants generated by constant.pm should
actually be constants (according to your definition), and should be
handled distinctly from sub foo(){42}, which I think *should* allow
${\foo}++, precisely because the variant with ‘return’ allows it.

I could implement it *that* way (inlinable subs are not necessarily
constant) if you prefer.

Yes, I think that decoupling the behaviour of sub foo(){42} from
the internals of how `use constant` makes constant is a much much
better plan. IIRC Dave has said that the logic in pad.c needed to
cause the closure cloning routine to see that

  sub foo() { $a; }

isn't to be treated as a closure is an utter hack.

Changing the implementation of constant.pm to always avoid subroutines,
would mean that that hack could be removed. As best I can tell from
looking at its code, it only actually *usefully* uses a ()
prototyped closure for a scalar constant where it already finds a
typeglob. I think that it could work round that by assigning to an
empty symbol table slot and then using typeglob assignment to get
the constant into the place where it's needed. I don't think that
these two are going to ever do anything "useful" (here, @​_ != 1)​:

  } elsif (@​_) {
  my @​list = @​_;
  *$full_name = sub () { @​list };
  } else {
  *$full_name = sub () { };
  }

(BTW, the way that PADMY and PADTMP currently interact is completely
nonsensical, but you probably already knew that. :-)

I'm not familiar enough with it to be able to cite why without
looking again at the source code. I remember that a lot of stuff gets
copied for ithreads. And that there's then a special case not to
copy PL_sv_undef. And that the copies of constants are in the pad
for each level of recursion for a recursive subroutine. (And have
NULL name slots)

My head seems to be overfull, not everything fits, and stuff that
used to be there has leaked out.

Nicholas Clark

@p5pRT
Copy link
Author

p5pRT commented Jun 14, 2013

From @cpansprout

On Fri Jun 14 06​:09​:56 2013, nicholas wrote​:

Yes, I think that decoupling the behaviour of sub foo(){42} from
the internals of how `use constant` makes constant is a much much
better plan.

We might not agree on the reasons, but at least here we agree on what
needs to be done. :-)

IIRC Dave has said that the logic in pad.c needed to
cause the closure cloning routine to see that

sub foo\(\) \{ $a; \}

isn't to be treated as a closure is an utter hack.

I want to kill that code. $a could change after \&foo has been referenced.

Changing the implementation of constant.pm to always avoid subroutines,
would mean that that hack could be removed. As best I can tell from
looking at its code, it only actually *usefully* uses a ()
prototyped closure for a scalar constant where it already finds a
typeglob. I think that it could work round that by assigning to an
empty symbol table slot and then using typeglob assignment to get
the constant into the place where it's needed.

Yes, that could work perfectly well.

I don't think that
these two are going to ever do anything "useful" (here, @​_ != 1)​:

    \} elsif \(@&#8203;\_\) \{
    my @&#8203;list = @&#8203;\_;
    \*$full\_name = sub \(\) \{ @&#8203;list \};
    \} else \{
    \*$full\_name = sub \(\) \{ \};
    \}

For a long time now, it has bothered me that list constants are not
inlined. I see no reason why they couldn’t be. The default storage
could be $​::{foo} = [], and then we may need a separate constlist op.

The way it is currently implemented, there is nothing constant about
these, according to your definition of constant, whether under threads
or no.

(BTW, the way that PADMY and PADTMP currently interact is completely
nonsensical, but you probably already knew that. :-)

I'm not familiar enough with it to be able to cite why without
looking again at the source code. I remember that a lot of stuff gets
copied for ithreads. And that there's then a special case not to
copy PL_sv_undef. And that the copies of constants are in the pad
for each level of recursion for a recursive subroutine. (And have
NULL name slots)

Because whether a pad slot is in use is stored in the value, no two pad
slots can share the same value, resulting in a dozen copies if your code
says PI a dozen times. :-(

I’m still discovering things, and each new discovery makes me want to
lift myself even higher by the hair out of my seat.

My head seems to be overfull, not everything fits, and stuff that
used to be there has leaked out.

I hope this bit hasn’t leaked out​:

commit 05d04d9
Author​: Nicholas Clark <nick@​ccl4.org>
Date​: Thu Feb 25 21​:35​:39 2010 +0000

  Don't clone the contents of lexicals in pads.
 
  This stops the values of lexicals in active stack frames in the
parent leaking
  into the lexicals in the child thread.
 
  With an exception for lexicals with a reference count of > 1, to
cope with the
  implementation of ?{{ ... }} blocks in regexps. :-(

...

Notice you check the refcount here​:

+ if(SvREFCNT(oldpad[ix]) > 1) {
+ pad1a[ix] = sv_dup_inc(oldpad[ix], param);
+ } else {
...

Further down​:

+ /* SvREFCNT(oldpad[ix]) != 1 for some code in threads.xs
+ FIXTHAT before merging this branch.
+ (And I know how to) */

It seems to me you simply forgot to delete the comment.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jun 15, 2013

From @chipdude

On 6/11/2013 2​:53 PM, Father Chrysostomos via RT wrote​:

I feel strongly that sub(){42} and sub(){return 42} should behave the
same way.

This is impossible unless what you mean is to make the second parse as
the first. The former is documented to be evaluated at compile-time and
e.g. participate in constant folding. The "not same" bridge was thus
already crossed.

@p5pRT
Copy link
Author

p5pRT commented Jun 15, 2013

From @chipdude

On Mon, Jun 10, 2013 at 11​:28​:09PM -0700, Father Chrysostomos via RT wrote​:

On Mon Jun 10 20​:38​:52 2013, rev.chip@​gmail.com wrote​:

On 6/10/2013 6​:14 PM, Father Chrysostomos via RT wrote​:

OK, so there are three arguments regarding whether values return by
potentially constant operators should be modifiable​:

1) It doesn’t cause any harm to make it modifiable, so why not? It’s
Perlish.
2) I was expecting it not to be modifiable, so that it is modifiable
must be a bug.
3) I need this thing to return exactly the same scalar, so I can compare
memory addresses.

I’m afraid the second one sounds stupid because I am biased against it
and know not how to express it convincingly.

It's an argument from parsimony​: Perl should not do unnecessary work.
Specifically, it should not create N SVs all with the same constant,
because it can avoid doing so by making one read-only SV. Perhaps that
doesn't sound so bad?

No, but it’s wrong. :-) A single read-only SV can suddenly turn into
multiple SVs when necessary.

You've missed the point, I think. For Perl to make a read-only SV is good
because it *permits* parsimony. That SV is read-only and thus safe; it does
not need to be copied to stay safe.

If that SV is the same as some other SV, then fine, no harm per se; and when
someone gets a tuit and figures out how to merge them, all the better.

Meanwhile, if copies of that SV are made due to some other necessity, fine;
those copies can be elided too, once someone gets a tuit to code the elision.

I don't care deeply about this, but I want the argument to be understood.

@p5pRT
Copy link
Author

p5pRT commented Jun 15, 2013

From @demerphq

On 11 June 2013 23​:53, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Tue Jun 11 01​:20​:55 2013, nicholas wrote​:

On Mon, Jun 10, 2013 at 11​:28​:09PM -0700, Father Chrysostomos via RT
That’s precisely what
<https://rt-archive.perl.org/perl5/Ticket/Display.html?id=109744#txn-1223457> is
meant to be!

and then make the
implementation
deliver them.

I’m trying to get to that, but we seem to have a disagreement above.

For me, the term ‘constant’ doesn’t necessarily have to be ‘read-only
SV’. If in an expression like ‘$x = PI’ the ‘PI’ returns exactly the
same value every time, then it is constant, is it not?

I feel strongly that sub(){42} and sub(){return 42} should behave the
same way.

*ahem* You also felt like that about split and IMO chose the worst way
possible to make them consistent, which I had to undo to resolve the
things it broke. See dbc200c.

If you feel equally strongly that ‘use constant foo=>42; ${\foo}++’
should die, then maybe the problem here is the equivalence of constants
and subroutines. Maybe the constants generated by constant.pm should
actually be constants (according to your definition), and should be
handled distinctly from sub foo(){42}, which I think *should* allow
${\foo}++, precisely because the variant with ‘return’ allows it.

Given above I think you will have to forgive me for questioning
whether you are making things consistent in the right way.

The opposite interpretation is just as and perhaps more reasonable
than the one you are proposing.

Yves

perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

p5pRT commented Jun 15, 2013

From @cpansprout

On Sat Jun 15 00​:31​:49 2013, demerphq wrote​:

On 11 June 2013 23​:53, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Tue Jun 11 01​:20​:55 2013, nicholas wrote​:

On Mon, Jun 10, 2013 at 11​:28​:09PM -0700, Father Chrysostomos via RT
That’s precisely what
<https://rt-archive.perl.org/perl5/Ticket/Display.html?id=109744#txn-1223457> is
meant to be!

and then make the
implementation
deliver them.

I’m trying to get to that, but we seem to have a disagreement above.

For me, the term ‘constant’ doesn’t necessarily have to be ‘read-only
SV’. If in an expression like ‘$x = PI’ the ‘PI’ returns exactly the
same value every time, then it is constant, is it not?

I feel strongly that sub(){42} and sub(){return 42} should behave the
same way.

*ahem* You also felt like that about split and IMO chose the worst way
possible to make them consistent, which I had to undo to resolve the
things it broke.

Well, it’s good that we are diligently keeping watch over each other. :-)

See dbc200c.

Yep, and that also broke a CPAN module by changing split "$space". When
fixing bugs, sometimes something has to give.

If you feel equally strongly that ‘use constant foo=>42; ${\foo}++’
should die, then maybe the problem here is the equivalence of constants
and subroutines. Maybe the constants generated by constant.pm should
actually be constants (according to your definition), and should be
handled distinctly from sub foo(){42}, which I think *should* allow
${\foo}++, precisely because the variant with ‘return’ allows it.

Given above I think you will have to forgive me for questioning
whether you are making things consistent in the right way.

The opposite interpretation is just as and perhaps more reasonable
than the one you are proposing.

That is why I am writing to the list. :-)

I have already tweaked my plans twice based on input from others.

I honestly don’t think I can fix everything without breaking some CPAN
modules (those that use experimental APIs), but SvPADTMP has got to stop
having two unrelated meanings.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jun 15, 2013

From @cpansprout

On Sat Jun 15 00​:31​:49 2013, demerphq wrote​:

Given above I think you will have to forgive me for questioning
whether you are making things consistent in the right way.

The opposite interpretation is just as and perhaps more reasonable
than the one you are proposing.

Are you saying that you disagree with my general approach (and that what
I should do is slightly different from what I had planned)? Or is this
just a word of caution?

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jun 15, 2013

From @Hugmeir

On Tue, Jun 11, 2013 at 6​:53 PM, Father Chrysostomos via RT <
perlbug-followup@​perl.org> wrote​:

On Tue Jun 11 01​:20​:55 2013, nicholas wrote​:

On Mon, Jun 10, 2013 at 11​:28​:09PM -0700, Father Chrysostomos via RT
wrote​:

On Mon Jun 10 20​:38​:52 2013, rev.chip@​gmail.com wrote​:

On 6/10/2013 6​:14 PM, Father Chrysostomos via RT wrote​:

OK, so there are three arguments regarding whether values return
by
potentially constant operators should be modifiable​:

1) It doesn't cause any harm to make it modifiable, so why not?
It's
Perlish.
2) I was expecting it not to be modifiable, so that it is
modifiable
must be a bug.
3) I need this thing to return exactly the same scalar, so I can
compare
memory addresses.

I'm afraid the second one sounds stupid because I am biased
against it
and know not how to express it convincingly.

I realise. But it's bugging me a lot that if the name is "constant" it
should
be just that. Not a generator of mutable temporaries initialised to
the value.

It's not "I was expecting it not to be modifiable" but more "the names
used to
describe this thing are terms with semantics of 'not modifiable'. So
why does
the implementation not enforce the semantics?"

It's an argument from parsimony​: Perl should not do unnecessary
work.
Specifically, it should not create N SVs all with the same
constant,
because it can avoid doing so by making one read-only SV. Perhaps
that
doesn't sound so bad?

No, but it's wrong. :-) A single read-only SV can suddenly turn
into
multiple SVs when necessary. I.e., we can pretend there are
multiple
SVs, but optimise it down to one behind the scenes. We already have
the
mechanism for that; it's just very buggy right now, and I know how
to
fix it.

I'm still not comfortable that this is "fix", at a language design
level.
Yes, the current implementation leaks through. But an alternative
change is
to stop it leaking, and have things that are named "constant" be
constant.

I think that "we can do this" is coming at it from the wrong end.

Start with "what should the semantics be?"

That’s precisely what
<https://rt-archive.perl.org/perl5/Ticket/Display.html?id=109744#txn-1223457> is
meant to be!

and then make the
implementation
deliver them.

I’m trying to get to that, but we seem to have a disagreement above.

For me, the term ‘constant’ doesn’t necessarily have to be ‘read-only
SV’. If in an expression like ‘$x = PI’ the ‘PI’ returns exactly the
same value every time, then it is constant, is it not?

I feel strongly that sub(){42} and sub(){return 42} should behave the
same way.

Could someone explain/link to what the argument against this is? Having
them work the same had always been my expectation.

@p5pRT
Copy link
Author

p5pRT commented Jun 15, 2013

From @cpansprout

On Sat Jun 15 08​:55​:02 2013, Hugmeir wrote​:

On Tue, Jun 11, 2013 at 6​:53 PM, Father Chrysostomos via RT <
perlbug-followup@​perl.org> wrote​:

I feel strongly that sub(){42} and sub(){return 42} should behave the
same way.

Could someone explain/link to what the argument against this is? Having
them work the same had always been my expectation.

I don’t think it’s so much an argument against it, as it is simply that
they don’t work the same way currently.

What Chip was referring to is the fact that sub(){42} is documented to
be inlinable, so that redefinition of the sub is ignored by code that
has already inlined a call to that sub as something other than a sub call.

I don’t have a problem with continued disparity in the case of sub
redefinition. What bothers me is that, in the absence of any sub
redefinition, the presence of a ‘return’ can change the behaviour at the
call site. In the context of this bug, I am referring specifically to
the fact that for(sub(){42}->()){$_++} will croak, but a ‘return’ will
stop it from croaking.

The other thing is that ‘use constant’ uses sub(){}, and Nicholas Clark
insists (and I’m sure others do, too, though I don’t mind either way)
that the values returned by its constants be constant.

So, again, there are not really arguments against having them behave the
same way, but merely technical obstacles.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jun 16, 2013

From @demerphq

On 15 June 2013 15​:49, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Sat Jun 15 00​:31​:49 2013, demerphq wrote​:

On 11 June 2013 23​:53, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Tue Jun 11 01​:20​:55 2013, nicholas wrote​:

On Mon, Jun 10, 2013 at 11​:28​:09PM -0700, Father Chrysostomos via RT
That’s precisely what
<https://rt-archive.perl.org/perl5/Ticket/Display.html?id=109744#txn-1223457> is
meant to be!

and then make the
implementation
deliver them.

I’m trying to get to that, but we seem to have a disagreement above.

For me, the term ‘constant’ doesn’t necessarily have to be ‘read-only
SV’. If in an expression like ‘$x = PI’ the ‘PI’ returns exactly the
same value every time, then it is constant, is it not?

I feel strongly that sub(){42} and sub(){return 42} should behave the
same way.

*ahem* You also felt like that about split and IMO chose the worst way
possible to make them consistent, which I had to undo to resolve the
things it broke.

Well, it’s good that we are diligently keeping watch over each other. :-)

Arguably in that case the problem was that I /wasn't/ watching over
you. :-) (And trust me, I dont watch over you, I have no time.)

I was a bit grumpy when I made that comment, and I apologize for being catty.

What I should have done is caution you against a rush to judgement and
encourage you to keep the big picture in mind.

Or something like that.

See dbc200c.

Yep, and that also broke a CPAN module by changing split "$space". When
fixing bugs, sometimes something has to give.

Indeed. I think however this was definitely in the end a change for the better.

If you feel equally strongly that ‘use constant foo=>42; ${\foo}++’
should die, then maybe the problem here is the equivalence of constants
and subroutines. Maybe the constants generated by constant.pm should
actually be constants (according to your definition), and should be
handled distinctly from sub foo(){42}, which I think *should* allow
${\foo}++, precisely because the variant with ‘return’ allows it.

Given above I think you will have to forgive me for questioning
whether you are making things consistent in the right way.

The opposite interpretation is just as and perhaps more reasonable
than the one you are proposing.

That is why I am writing to the list. :-)

I have already tweaked my plans twice based on input from others.

I honestly don’t think I can fix everything without breaking some CPAN
modules (those that use experimental APIs), but SvPADTMP has got to stop
having two unrelated meanings.

Just to be clear, I am firm believer that the old expression "you need
to break a few eggs to make an omelette" applies to Perl.

A bit of CPAN fallout for a good change to perl is acceptable damage in my book.

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

p5pRT commented Jun 16, 2013

From @demerphq

On 15 June 2013 15​:50, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Sat Jun 15 00​:31​:49 2013, demerphq wrote​:

Given above I think you will have to forgive me for questioning
whether you are making things consistent in the right way.

The opposite interpretation is just as and perhaps more reasonable
than the one you are proposing.

Are you saying that you disagree with my general approach (and that what
I should do is slightly different from what I had planned)? Or is this
just a word of caution?

Maybe both. Not sure. :-)

My understanding is that the issue at hand is that these two subs​:

sub foo(){ 42 }
sub foo(){ return 42 }

behave differently in the face of the code like​:

${\foo}++

$ perl -le'sub foo(){return 42} ${\foo}++; print "ok";'
ok
$ perl -le'sub foo(){ 42} ${\foo}++; print "ok";'
Modification of a read-only value attempted at -e line 1.

And the question is should do something about it and if so what. Which
gives us a) do nothing, b) make them both like the version with return
(not die/not readonly), or c) make them both like the version with no
return (die/readonly).

You seem to be taking the position that a) is off the table, and that
we should take b). I am not convinced that a) should be off the table,
nor convinced that b) is better than c)​: since c) would not change the
behavior of constants it seems to me it should be preferred if we must
choose between b) and c).

You also have suggested that we should separate subs from constants. I
think this is a fine idea in principle, but there is a lot of code out
there that manufactures constants from subs without using constant.pm,
so we have a backwards compatibility issue there that needs to be
addressed.

cheers
Yves

@p5pRT
Copy link
Author

p5pRT commented Jun 16, 2013

From @cpansprout

On Sun Jun 16 02​:44​:14 2013, demerphq wrote​:

On 15 June 2013 15​:50, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Sat Jun 15 00​:31​:49 2013, demerphq wrote​:

Given above I think you will have to forgive me for questioning
whether you are making things consistent in the right way.

The opposite interpretation is just as and perhaps more reasonable
than the one you are proposing.

Are you saying that you disagree with my general approach (and that what
I should do is slightly different from what I had planned)? Or is this
just a word of caution?

Maybe both. Not sure. :-)

My understanding is that the issue at hand is that these two subs​:

sub foo(){ 42 }
sub foo(){ return 42 }

behave differently in the face of the code like​:

${\foo}++

$ perl -le'sub foo(){return 42} ${\foo}++; print "ok";'
ok
$ perl -le'sub foo(){ 42} ${\foo}++; print "ok";'
Modification of a read-only value attempted at -e line 1.

And the question is should do something about it and if so what. Which
gives us a) do nothing, b) make them both like the version with return
(not die/not readonly), or c) make them both like the version with no
return (die/readonly).

You seem to be taking the position that a) is off the table, and that
we should take b). I am not convinced that a) should be off the table,
nor convinced that b) is better than c)​: since c) would not change the
behavior of constants it seems to me it should be preferred if we must
choose between b) and c).

You also have suggested that we should separate subs from constants. I
think this is a fine idea in principle, but there is a lot of code out
there that manufactures constants from subs without using constant.pm,
so we have a backwards compatibility issue there that needs to be
addressed.

I am not suggesting that sub(){42} stop being inlinable. The cases in
which this change would make a difference I believe are minimal (how may
people having production code triggering ‘Modification of a
read-only...’?), so this change would be mostly cosmetic. In truth,
this is just one knotty part of a larger picture. I’m mostly
interesting in fixing things like this​:

$ ./perl -Ilib -le 'sub foo(){42} for(foo) { print \$_; print \$_; }'
SCALAR(0x7f99a1806100)
SCALAR(0x7f99a18062c8)

This is under ithreads. Why am I getting two different memory addresses?

And ${\$_} is currently *modifiable* under those circumstances, too. So
there’s another backward-compatibility concern.

Also, I have found the difference between ‘42’ and ‘return 42’ to cause
surprising action at a distance. I think I was bitten by it once, so
maybe I’m biased.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jun 19, 2013

From @chipdude

On Tue, Jun 18, 2013 at 04​:24​:23PM -0400, Eric Brine wrote​:

On Tue, Jun 18, 2013 at 2​:15 PM, Chip Salzenberg <rev.chip@​gmail.com> wrote​:

Third issue​: I think there's a code smell if Perl fails to put SvREADONLY
on the SV specified as "constant". Remember, "variables don't, constants
aren't" is a joke, not a design principle.

You can do C<< void f(int i) { ++i; ... } f(5); >> in C, so why all the
fuss with f(5); in Perl?

Because they're different languages. Perl passes by reference; C does not.
Perl returns by value like C except when it doesn't -- constant subs and
lvalue subs.

I realise that the internals are very different and how, but that
shouldn't have any effect on design considerations.

That statement is pretty much the opposite of right.

@p5pRT
Copy link
Author

p5pRT commented Jun 19, 2013

From @ikegami

On Wed, Jun 19, 2013 at 2​:40 PM, Rev. Chip <rev.chip@​gmail.com> wrote​:

On Tue, Jun 18, 2013 at 04​:24​:23PM -0400, Eric Brine wrote​:

I realise that the internals are very different and how, but that

shouldn't have any effect on design considerations.

That statement is pretty much the opposite of right.

huh? code first design later???

@p5pRT
Copy link
Author

p5pRT commented Jun 19, 2013

From @ikegami

On Wed, Jun 19, 2013 at 2​:40 PM, Rev. Chip <rev.chip@​gmail.com> wrote​:

On Tue, Jun 18, 2013 at 04​:24​:23PM -0400, Eric Brine wrote​:

On Tue, Jun 18, 2013 at 2​:15 PM, Chip Salzenberg <rev.chip@​gmail.com>
wrote​:

Third issue​: I think there's a code smell if Perl fails to put
SvREADONLY
on the SV specified as "constant". Remember, "variables don't,
constants
aren't" is a joke, not a design principle.

You can do C<< void f(int i) { ++i; ... } f(5); >> in C, so why all the
fuss with f(5); in Perl?

Because they're different languages. Perl passes by reference; C does not.
Perl returns by value like C except when it doesn't -- constant subs and
lvalue subs.

I realise that the internals are very different and how, but that
shouldn't have any effect on design considerations.

That statement is pretty much the opposite of right.

(Please forget my earlier knee-jerk post.)

The issue I was addressing is whether constant literals generate mutable
values or not. It seems to me that people are saying that's something
that's simply not done elsewhere, yet I showed that's clearly not the case.
Sure, the constant in C is probably immutable, but it makes more sense to
look at the effects of that. The reason ++i doesn't croak in C is different
than the reason ++$_[0] would/wouldn't croak in Perl, but wouldn't it be
better to focus on deciding whether we want ++$_[0] to croak?

@p5pRT
Copy link
Author

p5pRT commented Jun 19, 2013

From @chipdude

On 6/19/2013 2​:26 PM, Eric Brine wrote​:

The issue I was addressing is whether constant literals generate
mutable values or not. It seems to me that people are saying that's
something that's simply not done elsewhere

Ah, I understand now. It's just that you're wrong. The above is *not*
what people are saying, not at all, not for values of "elsewhere" that
include other languages.

Hashes can be locked (as I always do with objects), Moose attributes can
be 'ro' (as I always do when I can), SVs can be readonly (as with
literal constants or when set so with CPAN help), and these all serve
otherwise unserveable purposes. It's a way of expressing intent and
promise not to change, and the VM can take advantage of that intent in
places for efficiency and error reporting. NO EXTRA COPIES. THIS IS GOOD.

Literals in Perl are readonly, and that's good; a "use constant"
constant should also be readonly, because that's equally good; and I
can't figure for the life of me why anyone would ever want otherwise.
It's just bizarre. "Constants aren't" is a joke, not a design principle.

@p5pRT
Copy link
Author

p5pRT commented Jun 20, 2013

From @ap

* Reverend Chip <rev.chip@​gmail.com> [2013-06-19 23​:55]​:

Literals in Perl are readonly, and that's good; a "use constant"
constant should also be readonly, because that's equally good; and
I can't figure for the life of me why anyone would ever want
otherwise.

No one did want otherwise.

It's just bizarre. "Constants aren't" is a joke, not a design
principle.

Repeating a nonsequitur won’t turn it into an applicable argument,
regardless of how long you keep at it. Or make it funny.

@p5pRT
Copy link
Author

p5pRT commented Jun 21, 2013

From @cpansprout

On Tue Jun 18 11​:16​:21 2013, rev.chip@​gmail.com wrote​:

There are approximately three issues here being conflated.

The reason I am conflating these, er, eight or so issues is that fixing
one affects another.

Fixing the copying under ithreads (BTW, it is also a bug under
non-threaded builds, but in fewer circumstances) will cause code that
used to modify certain values to start croaking.

Since it has the potential to break code, we need to consider the larger
picture before we can fix *anything*. This is but one example. Hence
the conflation.

Here is an
attempt to disentangle them.

First Issue​: I agree ithreads isn't DTRT if it's giving you a multiple
SV
in that for-loop snippet. It's just silly. No matter what else we
do,
fixing that is a mitzvah.

Already fixed in the sprout/padconst branch, but doubtless you will
disagree with some aspects of the implementation. But I consider it my
job to try to persuade you otherwise. :-) (Also, I know that because of
the innate entanglement of these issues my branch has the potential to
break working code.)

Second Issue​: In every Perl sub other than simple nonary subs, a final
expresion has an implicit C<return>, and the copying behavior is the
same
whether the user writes or omits C<return>. So it's a strictly
stylistic
issue, always... *except* in nonary subs consisting of a single
expressoin.
Wart much? I think it was only the convenience or excessive caution
of
the original implementor that led to this special case, not any better
reason. OTOH if it's a bug it has seniority. Can we make C<return
42>
mean C<42> in a nonary sub without breaking code? If so, we should;
if
not, then we should just sigh and move on.

I consider the seniority argument to be false (respecting of course,
your right to disagree). It is false (in my view) because it has never
actually worked as document, or, if it ever did, stopped working as
advertised a long time ago.

I (and others, too) think it was a bad design decision to conflate the
concepts of subroutines and constants. Also, I think the main purpose
was efficiency, and ‘constant’ was just a convenient term for sub(){42}
inlined, so that term was used without concern for future ramifications
of such use.

Now, witness how inconsistent it really is​:

<perlsub.pod>
Functions with a prototype of C<()> are potential candidates for
inlining. If the result after optimization and constant folding
is either a constant or a lexically-scoped scalar which has no other
references, then it will be used in place of function calls made
without C<&>.
</perlsub.pod>

How is one to know when constant folding is going to occur? It varies
from one version of perl to the next, and also depending on pragmata in
scope​:

sub rule () { '-' x 70 . "\n" }
for (rule) {
  $_ =~ s/\n/-----\n/g if needs_to_be_longer();
  print report_with_rule($_);
}

That code worked fine in perl 5.16. In perl 5.18 it croaks, because x
is now folded. (Had I realised the implications at the time, I might
not have added x folding. That changes made solely for efficiency can
break code because of the way nullary subroutines work is unfortunate.)

Also​:

sub fooo() { sprintf "%s", "hello" }
for (fooo) { $_++ } # croaks

But​:

use locale;
sub fooo() { sprintf "%s", "hello" }
for (fooo) { $_++ } # fine

So efficiency tweaks in the compiler can change the way these things are
interpreted.

Also, we have another anomaly​:

$ ./perl -Ilib -e 'sub p() { __PACKAGE__ } for (p) { $_++ }'
$ ./perl -Ilib -e 'sub p() { "main" } for (p) { $_++ }'
Modification of a read-only value attempted at -e line 1.

Another point​: Here is a quote from perlop.pod​:

<perlop.pod>
=head2 Constant Folding
X<constant folding> X<folding>

Like C, Perl does a certain amount of expression evaluation at
compile time whenever it determines that all arguments to an
operator are static and have no side effects.
</perlop.pod>

Note the last three words. If we combine what perlsub says with what
perlop says, then it is a bug for sub foo(){1+2} to behave differently
from sub foo(){1+${\2}} because constant folding is supposed to have no
side effects. And for sub foo(){1+2} to behave differently from sub
foo(){3} would contradict perlsub.

I suggest we apply this patch​:

Inline Patch
diff --git a/pod/perlsub.pod b/pod/perlsub.pod
index 027d7be..7fff252 100644
--- a/pod/perlsub.pod
+++ b/pod/perlsub.pod
@@ -1365,7 +1365,7 @@ starts scribbling on your C<@_> parameter list.  Ouch!
 This is all very powerful, of course, and should be used only in moderation
 to make the world a better place.
 
-=head2 Constant Functions
+=head2 Inlinable Functions
 X<constant>
 
 Functions with a prototype of C<()> are potential candidates for
@@ -1409,7 +1409,7 @@ the constant folding doesn't reduce them to a
single constant:

If you redefine a subroutine that was eligible for inlining, you'll get
a warning by default. (You can use this warning to tell whether or not a
-particular subroutine is considered constant.) The warning is
+particular subroutine is considered inlinable.) The warning is
considered severe enough not to be affected by the B<-w>
switch (or its absence) because previously compiled
invocations of the function will still be using the old value of the

Third issue​: I think there's a code smell if Perl fails to put
SvREADONLY
on the SV specified as "constant".

While I don’t necessarily agree, I am willing to compromise. So let’s
put SvREADONLY on things created by ‘use constant’.

Remember, "variables don't,
constants
aren't" is a joke, not a design principle. Granted that non-constant
constants might not break anything that otherwise works, but the very
idea
that someone *wants* to turn off that bit (or, equivalently, make an
unnecessary copy) is a sign that someone has greatly misunderstood the
nature of the word "constant".

Then let’s stop using that word for nullary subs whose bodies consist
solely of a single expression that always evaluates to the same value.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jun 21, 2013

From @cpansprout

On Wed Jun 19 14​:50​:33 2013, rev.chip@​gmail.com wrote​:

Hashes can be locked (as I always do with objects), Moose attributes can
be 'ro' (as I always do when I can), SVs can be readonly (as with
literal constants or when set so with CPAN help), and these all serve
otherwise unserveable purposes. It's a way of expressing intent and
promise not to change, and the VM can take advantage of that intent in
places for efficiency and error reporting. NO EXTRA COPIES. THIS IS
GOOD.

If I’m using perl, I shouldn’t have to worry about that, since perl will
take of that for me (and it can, and it does, and I plan to make it do
so even more).

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jun 27, 2013

From @rjbs

On Tue Jun 18 11​:16​:21 2013, rev.chip@​gmail.com wrote​:

There are approximately three issues here being conflated.

Chip, I appreciated your several "wait, let's clarify stuff" posts in this
thread. Thank you!

* Father Chrysostomos via RT <perlbug-followup@​perl.org> [2013-06-20T20​:57​:33]

I (and others, too) think it was a bad design decision to conflate the
concepts of subroutines and constants. Also, I think the main purpose
was efficiency, and ‘constant’ was just a convenient term for sub(){42}
inlined, so that term was used without concern for future ramifications
of such use.

[ perlsub citation ]
[ example of behavioral vicissitudes ]
[ perlop on constant folding ]

I suggest we apply this patch​:

[ diff stuff ]
-=head2 Constant Functions
+=head2 Inlinable Functions
[ more diff ]

While I don’t necessarily agree, I am willing to compromise. So let’s
put SvREADONLY on things created by ‘use constant’.

Sprout has summed up my longstanding feelings about "constant" subs. I think
they're a mess. Automatic "constification" is not reliable, and I don't see a
way to make it usefully reliable. Making C< sub x(){42} > behave like C< sub
x(){return 42} > seems like a lessening of the language's inner tensions.

Explicit constification, with 'use constant', is, well, explicit. Hooray!

(By the way, I was amused by this bit of verbal dodge in constant.pm​: "It is
not possible to have a subroutine or a keyword with the same name as a constant
in the same package." Heh.)

I am of course open to being convinced that I am totally wrong, but I feel
pretty good about this... except, how much damage will be caused?

My hunch is "not much," considering the way that constant subs have been able
to change over time, but then again, I'm guessing that sub x(){42} hasn't
changed all that much. We need some practical damage assessment. If it looks
like it's going to wreak havok, I'm not sure what the way forward along this
path would be.

--
rjbs

@p5pRT
Copy link
Author

p5pRT commented Jun 29, 2013

From @cpansprout

On Fri Jun 14 08​:39​:07 2013, sprout wrote​:

For a long time now, it has bothered me that list constants are not
inlined. I see no reason why they couldn’t be. The default storage
could be $​::{foo} = [], and then we may need a separate constlist op.

If we allow list constants to be attached to CVs (as AVs) and inlined,
what should cv_const_sv return? This is important, as it is part of the
API. Should we piggy-back on top of the existing interface, or should
we have two separate functions, cv_const_sv and cv_const_av, in case
code is not expecting the former to return an AV? (Something tells me
the latter is the right way to do this.)

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jul 3, 2013

From @cpansprout

On Wed Jun 26 17​:25​:09 2013, perl.p5p@​rjbs.manxome.org wrote​:

Sprout has summed up my longstanding feelings about "constant" subs.
I think
they're a mess. Automatic "constification" is not reliable, and I
don't see a
way to make it usefully reliable. Making C< sub x(){42} > behave like
C< sub
x(){return 42} > seems like a lessening of the language's inner
tensions.

Explicit constification, with 'use constant', is, well, explicit.
Hooray!

(By the way, I was amused by this bit of verbal dodge in constant.pm​:
"It is
not possible to have a subroutine or a keyword with the same name as a
constant
in the same package." Heh.)

I am of course open to being convinced that I am totally wrong, but I
feel
pretty good about this... except, how much damage will be caused?

My hunch is "not much," considering the way that constant subs have
been able
to change over time, but then again, I'm guessing that sub x(){42}
hasn't
changed all that much. We need some practical damage assessment. If
it looks
like it's going to wreak havok, I'm not sure what the way forward
along this
path would be.

I have a branch ready, called sprout/padconst. It hasn’t broken
anything in core except B tests, but those are extra sensitive.

Last I heard, smueller was not able to do full CPAN smokes. I suspect
this will only break tests that check to make sure certain things can’t
work. Data​::Dump​::Streamer’s tests will probably break. I think it
will break PadWalker, too.

I expect the breakage to be small enough that we could merge this to
blead and see what happens over the next few months. It should break
far fewer modules than last year’s hash and pad changes did.

May I merge it? It will allow about 10 tickets to be closed.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jul 22, 2013

From @rjbs

I was hoping for some more commentary on this, but seeing as none is forthcoming, I think you
should proceed with your plan as described. Thanks for this work!

@p5pRT
Copy link
Author

p5pRT commented Jul 26, 2013

From @cpansprout

On Fri Feb 03 08​:03​:52 2012, zefram@​fysh.org wrote​:

On a threading Perl, a list-mode refgen applied to a constant will
copy the constant rather than reference it. An srefgen, however, will
reference it, and either will work correctly on a non-threading Perl​:

That has been fixed in commit 82b84d0.

On Sat Oct 06 22​:10​:52 2012, sprout wrote​:

Interestingly, copy-on-write can affect whether a ‘constant’ is constant​:

use overload;
BEGIN { overload'constant"integer" =>=> sub { __PACKAGE__ } }
eval { ${\5} = "whatever" };
print $@​ || "no error\n";
BEGIN { overload'constant"integer" =>=> sub { "main" } }
eval { ${\5} = "whatever" };
print $@​ || "no error\n";
__END__

Ouput​:

no error
Modification of a read-only value attempted at - line 6.

This was fixed in 1913067.

On Tue Jun 11 14​:53​:37 2013, sprout wrote​:

I feel strongly that sub(){42} and sub(){return 42} should behave the
same way.

They do, as of d244020.

On Fri Jun 14 06​:09​:56 2013, nicholas wrote​:

IIRC Dave has said that the logic in pad.c needed to
cause the closure cloning routine to see that

sub foo\(\) \{ $a; \}

isn't to be treated as a closure is an utter hack.

137da2b removed that hack.

On Fri Jun 14 08​:39​:07 2013, sprout wrote​:

For a long time now, it has bothered me that list constants are not
inlined. I see no reason why they couldn’t be. The default storage
could be $​::{foo} = [], and then we may need a separate constlist op.

We don’t need a constlist op. I implemented this in commits 6f1b3ab,
15635cb and f815dc1.

The way it is currently implemented, there is nothing constant about
these, according to your definition of constant, whether under threads
or no.

15635cb fixed that, too.

On Sun Jun 16 06​:48​:26 2013, sprout wrote​:

$ ./perl -Ilib -le 'sub foo(){42} for(foo) { print \$_; print \$_; }'
SCALAR(0x7f99a1806100)
SCALAR(0x7f99a18062c8)

This is under ithreads. Why am I getting two different memory addresses?

This particular cased is fixed by 8e079c2. The surrounding commits
fix similar bugs. I’ll attach the list to #78194.

I suggest we apply this patch​:

diff --git a/pod/perlsub.pod b/pod/perlsub.pod
index 027d7be..7fff252 100644
--- a/pod/perlsub.pod
+++ b/pod/perlsub.pod
@​@​ -1365,7 +1365,7 @​@​ starts scribbling on your C<@​_> parameter list.
Ouch!
This is all very powerful, of course, and should be used only in
moderation
to make the world a better place.

-=head2 Constant Functions
+=head2 Inlinable Functions
X<constant>
etc.

This is the one unresolved issue left in this ticket. If I apply the
patch, I break the link in constant.pm’s documentation. If I change
constant.pm at the same time, I break its link when the next CPAN
release is installed in 5.18 or earlier.

POD is too limiting. Can we revive #95784 perhaps?

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jul 26, 2013

From @cpansprout

On Mon Jun 10 18​:14​:00 2013, sprout wrote​:

• For consistency (constant folding is just an optimisation, and should
not change behaviour), this will be extended to foldable operations, too
(1+2, "a"."b"), which were modifiable only under threads.

This part was done in commit 2484f8d.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jul 26, 2013

From zefram@fysh.org

Father Chrysostomos via RT wrote​:

I'm afraid the second one sounds stupid because I am biased against it
and know not how to express it convincingly.

The position to which you refer comes from a conceptual distinction
between variables and values. A value is (conceptually) inherently
immutable. A variable is a storage location that contains a value, and
is mutable in that it can contain different values at different times.
Two variables that presently contain the same value are functionally
distinguishable because one can write a new value to one of them and
observe that they now contain different values.

The position, then, is an instance of Occam's razor​: one should not
gratuitously generate variables. A non-lvalue expression, such as $a+1,
conceptually yields a value, not a variable. As the Perl language
allows the refgen operator to be applied to this expression, inevitably
we can get this value into an lvalue situation and try assigning to it.
Applying Occam's razor, this process should not have generated a variable,
and so assignment must fail. If assignment is not to fail, then we have
created a variable somewhere, and whichever operator did that ought to
be documented as having that effect. In the case of \($a+1), apparently
either the addition or the refgen operator is creating variables, either
of which is somewhat surprising.

There is a certain amount of difficulty in applying this idea to
Perl, in that Perl has historically been very weak on the distinction
between variables and values. Our SV structure serves both purposes​:
we can't have a pure value without the variability-supporting wrapper.
The closest we get to a pure value is an SV with the read-only flag set;
this is a good enough implementation for analytical purposes, but it's
really an abstraction inversion. The result of values always coming
in the structure of a variable, and the read-only flag requiring extra
effort to turn on, is that all sorts of things in Perl implicitly create
variables, and many data structures have inherent mutability that's
difficult to avoid in the rather common case where it's unwanted.

I think this state, of almost everything being mutable by default, is a
natural consequence of developing a language from modest beginnings.
A very dynamic approach to an interpreter yields quick wins, and
particularly lets you introspect quite easily. Unfortunately the
dynamic approach makes it a pain to analyse programs, so it massively
impedes compilation, optimisation, correctness proving, and other such
things that one wants to do with large programs. Each variable that
the programmer didn't actually want to be variable is an obstruction to
proving that data flows in the way the programmer relied on.

I recommend Henry Baker's paper "Equal Rights for Functional Objects"
<http​://www.pipeline.com/~hbaker1/ObjectIdentity.ps.gz> for an examination
of issues arising from the question of variability.

The end result of applying Occam's razor to the existence of variables
is a language where everything is read-only by default​: you only get
variables where explicitly requested. It's obviously not feasible
to turn Perl into this sort of language. But we'd probably have a
better language if we avoided creating variables as much as possible.
My comment above about documentation was serious​: if we're going to
implicitly create variables, the programmer ought to be able to rely on
the semantics of these variables, and so the programmer needs to know
which operations create them. When the program acquires references to
multiple variables, it's vitally important to know which of the variables
are distinct, and which are multiple references to the same variable.
If the variable-creation semantics turn out to be confusing, well, it's a
lot easier to document that an operation doesn't produce variables at all.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Jul 31, 2013

From @cpansprout

On Fri Jul 26 09​:17​:22 2013, zefram@​fysh.org wrote​:

Father Chrysostomos via RT wrote​:

I'm afraid the second one sounds stupid because I am biased against it
and know not how to express it convincingly.

The position to which you refer comes from a conceptual distinction
between variables and values. A value is (conceptually) inherently
immutable. A variable is a storage location that contains a value, and
is mutable in that it can contain different values at different times.
Two variables that presently contain the same value are functionally
distinguishable because one can write a new value to one of them and
observe that they now contain different values.

The position, then, is an instance of Occam's razor​: one should not
gratuitously generate variables. A non-lvalue expression, such as $a+1,
conceptually yields a value, not a variable. As the Perl language
allows the refgen operator to be applied to this expression, inevitably
we can get this value into an lvalue situation and try assigning to it.
Applying Occam's razor, this process should not have generated a variable,
and so assignment must fail. If assignment is not to fail, then we have
created a variable somewhere, and whichever operator did that ought to
be documented as having that effect. In the case of \($a+1), apparently
either the addition or the refgen operator is creating variables, either
of which is somewhat surprising.

Considering that \ and foreach can autovivify, I do not find that
surprising at all. In fact, you haven given me a clear way to express
something that I understood intuitively but had difficulty explaining.
I thank you for that.

If $a+$b returns a value, not a variable, then we could say that \ and
for(...) and func(...) impose ‘variable context’.

This makes it similar to the way @​a and %h work on the lhs of
assignment. The thing the expression evaluates to is an array or hash,
but bare arrays or hashes cannot be passed around in perl, so there is
no way to inspect that list as-is. Everything in perl is looked at
through scalar variables. If you write \(@​a, %h), you get scalar
variables reference the two aggregates. If you write func(@​a,%h), you
see the scalar elements (and keys). If you write scalar(@​a) you get a
brand new scalar variable with information about the array.

Likewise, \ imposes a context that requires that a scalar variable be
returned. The same for for(...) and func(...).

(Note that the term ‘variable’ does not necessarily exclude read-only
scalars. $^S is certainly a variable, but it is read-only. And in the
end distinguishing between &PL_sv_undef and $^S and considering them to
be of different types is not all that useful, even though one is
constant and the other is not--they are both read-only scalars.)

Under that model, there is no reason why something called a constant
cannot return a mutable scalar. In that case it is the context that is
reifying the constant value into a mutable scalar.

Whether we should follow that model is hard to say. The other model,
where ‘constant’ means ‘read-only scalar’ breaks CPAN modules. See
tickets #119043 and #119045.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jul 31, 2013

From @cpansprout

On Fri Jul 26 00​:28​:04 2013, sprout wrote​:

On Sun Jun 16 06​:48​:26 2013, sprout wrote​:

diff --git a/pod/perlsub.pod b/pod/perlsub.pod
index 027d7be..7fff252 100644
--- a/pod/perlsub.pod
+++ b/pod/perlsub.pod
@​@​ -1365,7 +1365,7 @​@​ starts scribbling on your C<@​_> parameter list.
Ouch!
This is all very powerful, of course, and should be used only in
moderation
to make the world a better place.

-=head2 Constant Functions
+=head2 Inlinable Functions
X<constant>
etc.

This is the one unresolved issue left in this ticket. If I apply the
patch, I break the link in constant.pm’s documentation. If I change
constant.pm at the same time, I break its link when the next CPAN
release is installed in 5.18 or earlier.

I just applied the change to the paragraph, without the header change,
in commit e4fde5c.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jul 31, 2013

From zefram@fysh.org

Father Chrysostomos via RT wrote​:

If $a+$b returns a value, not a variable, then we could say that \ and
for(...) and func(...) impose "variable context".

The usual term for this is "lvalue context". A variable (that can vary)
is an lvalue. A read-only scalar is also an lvalue.

It is instructive to compare against C. The C equivalent of Perl's \
operator is &. C's & operator requires that the operand be an lvalue
(but not necessarily mutable), and it is a compile-time error to apply
it to a non-lvalue. Perl's pass-by-reference semantics would conflict
with any attempt to prevent enreferencement of non-lvalues.

Are you going to document \'s variable-generating semantics?

-zefram

@p5pRT
Copy link
Author

p5pRT commented Jul 31, 2013

From @cpansprout

On Wed Jul 31 02​:42​:52 2013, zefram@​fysh.org wrote​:

Father Chrysostomos via RT wrote​:

If $a+$b returns a value, not a variable, then we could say that \ and
for(...) and func(...) impose "variable context".

The usual term for this is "lvalue context". A variable (that can vary)
is an lvalue. A read-only scalar is also an lvalue.

It is instructive to compare against C. The C equivalent of Perl's \
operator is &. C's & operator requires that the operand be an lvalue
(but not necessarily mutable), and it is a compile-time error to apply
it to a non-lvalue. Perl's pass-by-reference semantics would conflict
with any attempt to prevent enreferencement of non-lvalues.

Are you going to document \'s variable-generating semantics?

I’m not sure how to go about that, nor do I think it is necessary. I
always assumed that $a+$b would return a new mutable scalar, and that
\($a+$b) just references a scalar that would otherwise have been
short-lived. Nothing in the observable behaviour contradicts that view.

This bare value vs variable distinction is not something that is
mentioned anywhere in the Perl documentation. Is it something we want
to document rigidly, or will it just add to the mental burden? Most of
the time it makes no difference.

In we want to put it anywhere, it should go under the documentation for
constants, wherever that might be, since that is where it actually
matters. Apparently we don’t define the term ‘constant’ clearly anywhere.

One thing that makes this difficult is that constants are currently
inconsistent. Making constants created by overload​::constant and
constant.pm consistently return read-only scalars broke CPAN modules.

At this stage, I am willing to leave things inconsistent, as the changes
I have made so far have allowed me to fix the bugs I wanted to fix.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jul 31, 2013

From zefram@fysh.org

Father Chrysostomos via RT wrote​:

I'm not sure how to go about that, nor do I think it is necessary. I
always assumed that $a+$b would return a new mutable scalar, and that
\($a+$b) just references a scalar that would otherwise have been
short-lived. Nothing in the observable behaviour contradicts that view.

If the behaviour is consistently thus, then we could document a general
principle that certain classes of operator generate new variables.
However, it's actually not a consistent behaviour of the addition
operator, because if the operands are sufficiently constant then
it gets constant folded and generates a read-only lvalue instead.
Related inconsistency is where this bug report started.

I think it is vitally important that we document which situations create
new variables, because it makes a real difference to the semantics
of reasonable programs. If generating variables is a feature of the
language, then it is reasonable to use those variables by storing new
values in them. If we have two references to variables, the program's
behaviour is likely to depend a great deal on whether they're distinct
variables or two references to the same variable. The documentation
should provide enough information for someone using these variables to
determine which ones will be distinct. Also, since addition doesn't
*always* generate a variable, the documentation must distinguish the
variable-generating cases from the read-only-lvalue cases.

This bare value vs variable distinction is not something that is
mentioned anywhere in the Perl documentation.

It's not necessary to cast the documentation in those terms. It's a
useful abstraction in language design, but since Perl doesn't reify
the distinction it's not necessarily useful in documenting Perl.
In Perl the main concept is the scalar (or the broader concept of
scalar/array/hash/glob/etc.), which can always be referenced in an
lvalue way, so we never deal with values in isolation. The important
distinction is between variable and read-only scalars, and among variables
the important feature is which operation created them (so that we can
determine which ones are separate storage locations).

At this stage, I am willing to leave things inconsistent, as the changes
I have made so far have allowed me to fix the bugs I wanted to fix.

OK, but we do need to document the behaviour. Especially so if it's a
feature that we want to keep, but if not then we should at least say
something like "the scalar returned by this operation may be either
read-only or variable, and this may change in the future".

-zefram

@p5pRT
Copy link
Author

p5pRT commented Aug 1, 2013

From @demerphq

On 26 July 2013 18​:16, Zefram <zefram@​fysh.org> wrote​:

Father Chrysostomos via RT wrote​:

I'm afraid the second one sounds stupid because I am biased against it
and know not how to express it convincingly.

The position to which you refer comes from a conceptual distinction
between variables and values. A value is (conceptually) inherently
immutable. A variable is a storage location that contains a value, and
is mutable in that it can contain different values at different times.
Two variables that presently contain the same value are functionally
distinguishable because one can write a new value to one of them and
observe that they now contain different values.

The position, then, is an instance of Occam's razor​: one should not
gratuitously generate variables. A non-lvalue expression, such as $a+1,
conceptually yields a value, not a variable. As the Perl language
allows the refgen operator to be applied to this expression, inevitably
we can get this value into an lvalue situation and try assigning to it.
Applying Occam's razor, this process should not have generated a variable,
and so assignment must fail. If assignment is not to fail, then we have
created a variable somewhere, and whichever operator did that ought to
be documented as having that effect. In the case of \($a+1), apparently
either the addition or the refgen operator is creating variables, either
of which is somewhat surprising.

There is a certain amount of difficulty in applying this idea to
Perl, in that Perl has historically been very weak on the distinction
between variables and values. Our SV structure serves both purposes​:
we can't have a pure value without the variability-supporting wrapper.
The closest we get to a pure value is an SV with the read-only flag set;
this is a good enough implementation for analytical purposes, but it's
really an abstraction inversion. The result of values always coming
in the structure of a variable, and the read-only flag requiring extra
effort to turn on, is that all sorts of things in Perl implicitly create
variables, and many data structures have inherent mutability that's
difficult to avoid in the rather common case where it's unwanted.

I think this state, of almost everything being mutable by default, is a
natural consequence of developing a language from modest beginnings.
A very dynamic approach to an interpreter yields quick wins, and
particularly lets you introspect quite easily. Unfortunately the
dynamic approach makes it a pain to analyse programs, so it massively
impedes compilation, optimisation, correctness proving, and other such
things that one wants to do with large programs. Each variable that
the programmer didn't actually want to be variable is an obstruction to
proving that data flows in the way the programmer relied on.

I recommend Henry Baker's paper "Equal Rights for Functional Objects"
<http​://www.pipeline.com/~hbaker1/ObjectIdentity.ps.gz> for an examination
of issues arising from the question of variability.

The end result of applying Occam's razor to the existence of variables
is a language where everything is read-only by default​: you only get
variables where explicitly requested. It's obviously not feasible
to turn Perl into this sort of language. But we'd probably have a
better language if we avoided creating variables as much as possible.
My comment above about documentation was serious​: if we're going to
implicitly create variables, the programmer ought to be able to rely on
the semantics of these variables, and so the programmer needs to know
which operations create them. When the program acquires references to
multiple variables, it's vitally important to know which of the variables
are distinct, and which are multiple references to the same variable.
If the variable-creation semantics turn out to be confusing, well, it's a
lot easier to document that an operation doesn't produce variables at all.

I just wanted to thank you for this post. IMO it could be the start of
a very useful new perldoc pod file.

cheers,
Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

p5pRT commented Aug 12, 2013

From @cpansprout

On Wed Jul 31 09​:17​:27 2013, zefram@​fysh.org wrote​:

Father Chrysostomos via RT wrote​:

I'm not sure how to go about that, nor do I think it is necessary. I
always assumed that $a+$b would return a new mutable scalar, and that
\($a+$b) just references a scalar that would otherwise have been
short-lived. Nothing in the observable behaviour contradicts that view.

If the behaviour is consistently thus, then we could document a general
principle that certain classes of operator generate new variables.
However, it's actually not a consistent behaviour of the addition
operator, because if the operands are sufficiently constant then
it gets constant folded and generates a read-only lvalue instead.
Related inconsistency is where this bug report started.

Actually, I fixed that in commit 2484f8d.

I think it is vitally important that we document which situations create
new variables, because it makes a real difference to the semantics
of reasonable programs. If generating variables is a feature of the
language, then it is reasonable to use those variables by storing new
values in them. If we have two references to variables, the program's
behaviour is likely to depend a great deal on whether they're distinct
variables or two references to the same variable. The documentation
should provide enough information for someone using these variables to
determine which ones will be distinct. Also, since addition doesn't
*always* generate a variable, the documentation must distinguish the
variable-generating cases from the read-only-lvalue cases.

This bare value vs variable distinction is not something that is
mentioned anywhere in the Perl documentation.

It's not necessary to cast the documentation in those terms. It's a
useful abstraction in language design, but since Perl doesn't reify
the distinction it's not necessarily useful in documenting Perl.
In Perl the main concept is the scalar (or the broader concept of
scalar/array/hash/glob/etc.), which can always be referenced in an
lvalue way, so we never deal with values in isolation. The important
distinction is between variable and read-only scalars, and among variables
the important feature is which operation created them (so that we can
determine which ones are separate storage locations).

At this stage, I am willing to leave things inconsistent, as the changes
I have made so far have allowed me to fix the bugs I wanted to fix.

The only inconsistency left here is in constant.pm, and I documented it
in 842f391.

OK, but we do need to document the behaviour. Especially so if it's a
feature that we want to keep, but if not then we should at least say
something like "the scalar returned by this operation may be either
read-only or variable, and this may change in the future".

How does the attached patch look to you?

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Aug 12, 2013

From @cpansprout

Inline Patch
diff --git a/pod/perlfunc.pod b/pod/perlfunc.pod
index 4de31ac..390c2df 100644
--- a/pod/perlfunc.pod
+++ b/pod/perlfunc.pod
@@ -94,6 +94,13 @@ L<perlapi/PL_keyword_plugin> for the mechanism.  If you are using such
 a module, see the module's documentation for details of the syntax that
 it defines.
 
+The return values of functions are generally new variables, meaning that
+you can take references to them and modify them through those references.
+Evaluating the same function call twice (in a loop or subroutine) produces
+two different variables.  Functions returning true or false generally
+return the same two read-only scalars each time, though this is not always
+consistent and may change in the future.
+
 =head2 Perl Functions by Category
 X<function>
 
diff --git a/pod/perlop.pod b/pod/perlop.pod
index 4c26fe7..0792710 100644
--- a/pod/perlop.pod
+++ b/pod/perlop.pod
@@ -21,6 +21,13 @@ repetition or list repetition, depending on the type of the left
 operand, and C<&>, C<|> and C<^> can be either string or numeric bit
 operations.
 
+The return values of operators are generally new variables, meaning that
+you can take references to them and modify them through those references.
+Evaluating the same operator twice (in a loop or subroutine) produces two
+different variables.  Operators returning true or false generally return
+the same two read-only scalars each time, though this is not always
+consistent and may change in the future.
+
 =head2 Operator Precedence and Associativity
 X<operator, precedence> X<precedence> X<associativity>
 

@p5pRT
Copy link
Author

p5pRT commented Aug 14, 2013

From zefram@fysh.org

Father Chrysostomos via RT wrote​:

How does the attached patch look to you?

Seems woolly. As there's a localised disclaimer of future changeability,
it seems that the bulk of the new text is guaranteeing something that
will never change. But the actual statement isn't a clear guarantee​:
it says "generally" this happens but allows for exceptions. So I think
it'll mislead.

If the intention is to guarantee some behaviour, state precisely
what behaviour is being guaranteed. If the intention is to describe
non-guaranteed behaviour, add explicit disclaimer covering the whole
description.

Also, I'd be wary of widely guaranteeing creation of new variables at
this stage. That's an architectural question that needs debate before
we start closing off future options.

-zefram

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants