Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bleadperl v5.19.2-138-g137da2b breaks JKEGL/Marpa-R2-2.064000.tar.gz #13130

Closed
p5pRT opened this issue Jul 27, 2013 · 19 comments
Closed

Bleadperl v5.19.2-138-g137da2b breaks JKEGL/Marpa-R2-2.064000.tar.gz #13130

p5pRT opened this issue Jul 27, 2013 · 19 comments

Comments

@p5pRT
Copy link

p5pRT commented Jul 27, 2013

Migrated from rt.perl.org#119047 (status was 'resolved')

Searchable as RT119047$

@p5pRT
Copy link
Author

p5pRT commented Jul 27, 2013

From @andk

git bisect


  137da2b is the first bad commit
  commit 137da2b
  Author​: Father Chrysostomos <sprout@​cpan.org>
  Date​: Sun Jun 30 20​:26​:34 2013 -0700
 
  [perl #79908] Stop sub inlining from breaking closures

diagnostics


  Not yet on cpantesters and too large to reproduce, please try yourself

perl -V


  Summary of my perl5 (revision 5 version 19 subversion 3) configuration​:
  Commit id​: 137da2b
  Platform​:
  osname=linux, osvers=3.9-1-amd64, archname=x86_64-linux
  uname='linux k83 3.9-1-amd64 #1 smp debian 3.9.8-1 x86_64 gnulinux '
  config_args='-Dprefix=/home/sand/src/perl/repoperls/installed-perls/perl/v5.19.2-138-g137da2b/165a -Dmyhostname=k83 -Dinstallusrbinperl=n -Uversiononly -Dusedevel -des -Ui_db -Uuseithreads -Uuselongdouble -DDEBUGGING=-g'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=undef, usemultiplicity=undef
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-O2 -g',
  cppflags='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
  ccversion='', gccversion='4.8.1', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
  libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
  libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
  perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
  libc=, so=so, useshrplib=false, libperl=libperl.a
  gnulibc_version='2.17'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector'

  Characteristics of this binary (from libperl)​:
  Compile-time options​: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
  PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
  PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
  PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT
  USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_PERLIO
  USE_PERL_ATOF
  Built under linux
  Compiled at Jul 27 2013 22​:01​:44
  @​INC​:
  /home/sand/src/perl/repoperls/installed-perls/perl/v5.19.2-138-g137da2b/165a/lib/site_perl/5.19.3/x86_64-linux
  /home/sand/src/perl/repoperls/installed-perls/perl/v5.19.2-138-g137da2b/165a/lib/site_perl/5.19.3
  /home/sand/src/perl/repoperls/installed-perls/perl/v5.19.2-138-g137da2b/165a/lib/5.19.3/x86_64-linux
  /home/sand/src/perl/repoperls/installed-perls/perl/v5.19.2-138-g137da2b/165a/lib/5.19.3
  .

--
andreas

@p5pRT
Copy link
Author

p5pRT commented Jul 28, 2013

From @cpansprout

On Sat Jul 27 14​:28​:11 2013, andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

git bisect
----------
137da2b is the first bad commit
commit 137da2b
Author​: Father Chrysostomos <sprout@​cpan.org>
Date​: Sun Jun 30 20​:26​:34 2013 -0700

  \[perl \#79908\] Stop sub inlining from breaking closures

It is relying on a bug in perl.

It uses a single lexical variable whose value keeps changing, and
creates closures that are not really closures but snapshots of the
current value of the variable.

The cited commit fixes that bug, such that closures work properly.
(What it was relying on was an awful hack that, at the time it was
added, was *supposed* to be transparent, but got implemented in a way
that was not.)

The attached patch makes Marpa​::R2 use constant.pm to create constants.

Unfortunately, it is more complicated than it ought to be, because
constant.pm does not accept fully-qualified constant names. Is there
any reason it shouldn’t?

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jul 28, 2013

From @cpansprout

Inline Patch
diff -rup Marpa-R2-2.064000-EIBDuD-orig/lib/Marpa/R2/Internal.pm Marpa-R2-2.064000-EIBDuD/lib/Marpa/R2/Internal.pm
--- Marpa-R2-2.064000-EIBDuD-orig/lib/Marpa/R2/Internal.pm	2013-07-11 09:25:42.000000000 -0700
+++ Marpa-R2-2.064000-EIBDuD/lib/Marpa/R2/Internal.pm	2013-07-28 12:55:22.000000000 -0700
@@ -19,6 +19,7 @@ use 5.010;
 use strict;
 use warnings;
 use Carp;
+use constant;
 
 use vars qw($VERSION $STRING_VERSION);
 $VERSION        = '2.064000';
@@ -84,8 +85,10 @@ sub Marpa::R2::offset {
 
         Marpa::R2::exception("Unacceptable field name: $field")
             if $field =~ /[^A-Z0-9_]/xms;
-        my $field_name = $prefix . $field;
-        *{$field_name} = sub () {$offset};
+        local *Marpa::R2::Internal::_temp:: = $prefix;
+        package Marpa::R2::Internal::_temp;
+        no warnings;
+        constant->import($field => $offset);
     } ## end for my $field (@fields)
     return 1;
 } ## end sub Marpa::R2::offset

@p5pRT
Copy link
Author

p5pRT commented Jul 28, 2013

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jul 29, 2013

From bug-Marpa-R2@rt.cpan.org

<URL​: https://rt.cpan.org/Ticket/Display.html?id=87384 >

Thanks, Father Chrysostomos, for the fix, which seems to work on *MOST* platforms.

I incorporated it into Marpa-R2-2.065_003, which reports 4 failures on cpantesters. One seems unrelated to the fix, but the messages in the other three are of the kind that I'd expect if the constants are not being defined. As an example of one​:

Bareword "Marpa​::R2​::Internal​::Grammar​::TRACE_RULES" not allowed while "strict subs" in use at /usr/home/cpan/pit/64bit/conf/perl-5.14.1/.cpanplus/5.14.1/build/Marpa-R2-2.065_003/blib/lib/Marpa/R2/Grammar.pm line 148.

This seems to indicate that the fix did not actually succeed in adding TRACE_RULES to the namespace Marpa​::R2​::Internal​::Grammar.

The reports are from 2 BSD variants and Linux, but all are for 5.14.x​:

http​://www.cpantesters.org/cpan/report/5198f65e-f85c-11e2-8358-7df630b64f85
http​://www.cpantesters.org/cpan/report/40725a58-f859-11e2-8f75-553d14be7dd6
http​://www.cpantesters.org/cpan/report/ca977e96-f84c-11e2-8f75-553d14be7dd6

@p5pRT
Copy link
Author

p5pRT commented Jul 29, 2013

From @demerphq

On 28 July 2013 21​:57, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Sat Jul 27 14​:28​:11 2013, andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

git bisect
----------
137da2b is the first bad commit
commit 137da2b
Author​: Father Chrysostomos <sprout@​cpan.org>
Date​: Sun Jun 30 20​:26​:34 2013 -0700

  \[perl \#79908\] Stop sub inlining from breaking closures

It is relying on a bug in perl.

It uses a single lexical variable whose value keeps changing, and
creates closures that are not really closures but snapshots of the
current value of the variable.

The cited commit fixes that bug, such that closures work properly.
(What it was relying on was an awful hack that, at the time it was
added, was *supposed* to be transparent, but got implemented in a way
that was not.)

The attached patch makes Marpa​::R2 use constant.pm to create constants.

Unfortunately, it is more complicated than it ought to be, because
constant.pm does not accept fully-qualified constant names. Is there
any reason it shouldn’t?

--

Father Chrysostomos

---
via perlbug​: queue​: perl5 status​: new
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=119047

diff -rup Marpa-R2-2.064000-EIBDuD-orig/lib/Marpa/R2/Internal.pm Marpa-R2-2.064000-EIBDuD/lib/Marpa/R2/Internal.pm
--- Marpa-R2-2.064000-EIBDuD-orig/lib/Marpa/R2/Internal.pm 2013-07-11 09​:25​:42.000000000 -0700
+++ Marpa-R2-2.064000-EIBDuD/lib/Marpa/R2/Internal.pm 2013-07-28 12​:55​:22.000000000 -0700
@​@​ -19,6 +19,7 @​@​ use 5.010;
use strict;
use warnings;
use Carp;
+use constant;

use vars qw($VERSION $STRING_VERSION);
$VERSION = '2.064000';
@​@​ -84,8 +85,10 @​@​ sub Marpa​::R2​::offset {

     Marpa&#8203;::R2&#8203;::exception\("Unacceptable field name&#8203;: $field"\)
         if $field =~ /\[^A\-Z0\-9\_\]/xms;

- my $field_name = $prefix . $field;
- *{$field_name} = sub () {$offset};
+ local *Marpa​::R2​::Internal​::_temp​:: = $prefix;
+ package Marpa​::R2​::Internal​::_temp;
+ no warnings;
+ constant->import($field => $offset);
} ## end for my $field (@​fields)
return 1;
} ## end sub Marpa​::R2​::offset

Does this mean one cannot use

sub foo () { 1 }

to create a constant anymore?

If so, IMO this breaks a lot of code.

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

p5pRT commented Jul 29, 2013

From @Leont

On Mon, Jul 29, 2013 at 7​:26 PM, demerphq <demerphq@​gmail.com> wrote​:

Does this mean one cannot use

sub foo () { 1 }

to create a constant anymore?

If so, IMO this breaks a lot of code.

If I understand this correctly, the problem is that this doesn't do
exactly what it used to do​:

my $foo = 1;
for my $bar (@​whatever) {
  $foo++
  *{$bar} = sub () { $foo }
}

Because $foo itself keeps changing value.

Leon

@p5pRT
Copy link
Author

p5pRT commented Jul 29, 2013

From bug-Marpa-R2@rt.cpan.org

<URL​: https://rt.cpan.org/Ticket/Display.html?id=87384 >

FYI anyone else working on this. I decided to work around the issue.

These constants can be set at distribution build time, so I've decided to auto-generate a .pm file. This is easier all around from the point of view of getting Marpa working on all distributions.

1 similar comment
@p5pRT
Copy link
Author

p5pRT commented Jul 29, 2013

From bug-Marpa-R2@rt.cpan.org

<URL​: https://rt.cpan.org/Ticket/Display.html?id=87384 >

FYI anyone else working on this. I decided to work around the issue.

These constants can be set at distribution build time, so I've decided to auto-generate a .pm file. This is easier all around from the point of view of getting Marpa working on all distributions.

@p5pRT
Copy link
Author

p5pRT commented Jul 30, 2013

From @cpansprout

On Mon Jul 29 10​:27​:30 2013, demerphq wrote​:

Does this mean one cannot use

sub foo () { 1 }

to create a constant anymore?

No.

$ git describe
v5.19.2-208-g5c24ac0
$ ./perl -Ilib -MO=Deparse -e 'sub foo () { 1 } print foo'
sub foo () { 1 }
print 1;
-e syntax OK

It only applies to sub foo() { $variable_declared_outside }.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Jul 30, 2013

From @demerphq

On 29 July 2013 19​:47, Leon Timmermans <fawaka@​gmail.com> wrote​:

On Mon, Jul 29, 2013 at 7​:26 PM, demerphq <demerphq@​gmail.com> wrote​:

Does this mean one cannot use

sub foo () { 1 }

to create a constant anymore?

If so, IMO this breaks a lot of code.

If I understand this correctly, the problem is that this doesn't do
exactly what it used to do​:

my $foo = 1;
for my $bar (@​whatever) {
$foo++
*{$bar} = sub () { $foo }
}

Because $foo itself keeps changing value.

Ah yes, it should be​:

*$bar= eval "sub () { $foo }";

So that the $foo is frozen.

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

p5pRT commented Jul 30, 2013

From @demerphq

On 30 July 2013 05​:25, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Mon Jul 29 10​:27​:30 2013, demerphq wrote​:

Does this mean one cannot use

sub foo () { 1 }

to create a constant anymore?

No.

$ git describe
v5.19.2-208-g5c24ac0
$ ./perl -Ilib -MO=Deparse -e 'sub foo () { 1 } print foo'
sub foo () { 1 }
print 1;
-e syntax OK

It only applies to sub foo() { $variable_declared_outside }.

Ah yes, thanks. Sorry for getting confused.

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

p5pRT commented Jul 31, 2013

From bug-Marpa-R2@rt.cpan.org

<URL​: https://rt.cpan.org/Ticket/Display.html?id=87384 >

On Mon Jul 29 13​:17​:32 2013, JKEGL wrote​:

Thanks, Father Chrysostomos, for the fix, which seems to work on
*MOST* platforms.

I incorporated it into Marpa-R2-2.065_003, which reports 4 failures on
cpantesters. One seems unrelated to the fix, but the messages in the
other three are of the kind that I'd expect if the constants are not
being defined. As an example of one​:

Bareword "Marpa​::R2​::Internal​::Grammar​::TRACE_RULES" not allowed while
"strict subs" in use at /usr/home/cpan/pit/64bit/conf/perl-
5.14.1/.cpanplus/5.14.1/build/Marpa-R2-
2.065_003/blib/lib/Marpa/R2/Grammar.pm line 148.

This seems to indicate that the fix did not actually succeed in adding
TRACE_RULES to the namespace Marpa​::R2​::Internal​::Grammar.

I just had to get to the bottom of this....

It seems you omitted the ‘use constant’ part of my patch. On most perl versions it seems to be loaded by something else. Not so under 5.14.

Sorry about the extra tickets. I forwarded the patch to this bug tracker from within perl’s bug tracker, and several responses to it were sent via ‘Reply All’.

@p5pRT
Copy link
Author

p5pRT commented Jul 31, 2013

@cpansprout - Status changed from 'open' to 'resolved'

@p5pRT p5pRT closed this as completed Jul 31, 2013
@p5pRT
Copy link
Author

p5pRT commented Jul 31, 2013

From bug-Marpa-R2@rt.cpan.org

<URL​: https://rt.cpan.org/Ticket/Display.html?id=87384 >

Thanks for your patience with this issue, and with me as a slow learner. In retrospect the need for "use constant" is pretty obvious, and it explains the strange distribution of the symptoms -- where something else had required the "constant" pragma, it worked, and where nothing else had, it failed. Side effect city.

As mentioned, I decided to back out the run-time calculation of constants in Marpa​::R2. The values of the constants I needed were integers, known at thetime that the distribution was created. So my original solution was over-engineered and my new "backed out" solution is in fact better code -- simpler, and more efficient. It's implemented in my latest developer's releases.

I am happy to learn my problem with your patch was my error and not a Perl issue. It's a bit of Perl infra-structure I may need someday, so it's a relief to know it's there and it works.

On Wed Jul 31 00​:53​:31 2013, SPROUT wrote​:

On Mon Jul 29 13​:17​:32 2013, JKEGL wrote​:

Thanks, Father Chrysostomos, for the fix, which seems to work on
*MOST* platforms.

I incorporated it into Marpa-R2-2.065_003, which reports 4 failures
on
cpantesters. One seems unrelated to the fix, but the messages in the
other three are of the kind that I'd expect if the constants are not
being defined. As an example of one​:

Bareword "Marpa​::R2​::Internal​::Grammar​::TRACE_RULES" not allowed
while
"strict subs" in use at /usr/home/cpan/pit/64bit/conf/perl-
5.14.1/.cpanplus/5.14.1/build/Marpa-R2-
2.065_003/blib/lib/Marpa/R2/Grammar.pm line 148.

This seems to indicate that the fix did not actually succeed in
adding
TRACE_RULES to the namespace Marpa​::R2​::Internal​::Grammar.

I just had to get to the bottom of this....

It seems you omitted the ‘use constant’ part of my patch. On most
perl versions it seems to be loaded by something else. Not so under
5.14.

Sorry about the extra tickets. I forwarded the patch to this bug
tracker from within perl’s bug tracker, and several responses to it
were sent via ‘Reply All’.

1 similar comment
@p5pRT
Copy link
Author

p5pRT commented Jul 31, 2013

From bug-Marpa-R2@rt.cpan.org

<URL​: https://rt.cpan.org/Ticket/Display.html?id=87384 >

Thanks for your patience with this issue, and with me as a slow learner. In retrospect the need for "use constant" is pretty obvious, and it explains the strange distribution of the symptoms -- where something else had required the "constant" pragma, it worked, and where nothing else had, it failed. Side effect city.

As mentioned, I decided to back out the run-time calculation of constants in Marpa​::R2. The values of the constants I needed were integers, known at thetime that the distribution was created. So my original solution was over-engineered and my new "backed out" solution is in fact better code -- simpler, and more efficient. It's implemented in my latest developer's releases.

I am happy to learn my problem with your patch was my error and not a Perl issue. It's a bit of Perl infra-structure I may need someday, so it's a relief to know it's there and it works.

On Wed Jul 31 00​:53​:31 2013, SPROUT wrote​:

On Mon Jul 29 13​:17​:32 2013, JKEGL wrote​:

Thanks, Father Chrysostomos, for the fix, which seems to work on
*MOST* platforms.

I incorporated it into Marpa-R2-2.065_003, which reports 4 failures
on
cpantesters. One seems unrelated to the fix, but the messages in the
other three are of the kind that I'd expect if the constants are not
being defined. As an example of one​:

Bareword "Marpa​::R2​::Internal​::Grammar​::TRACE_RULES" not allowed
while
"strict subs" in use at /usr/home/cpan/pit/64bit/conf/perl-
5.14.1/.cpanplus/5.14.1/build/Marpa-R2-
2.065_003/blib/lib/Marpa/R2/Grammar.pm line 148.

This seems to indicate that the fix did not actually succeed in
adding
TRACE_RULES to the namespace Marpa​::R2​::Internal​::Grammar.

I just had to get to the bottom of this....

It seems you omitted the ‘use constant’ part of my patch. On most
perl versions it seems to be loaded by something else. Not so under
5.14.

Sorry about the extra tickets. I forwarded the patch to this bug
tracker from within perl’s bug tracker, and several responses to it
were sent via ‘Reply All’.

@p5pRT
Copy link
Author

p5pRT commented Aug 3, 2013

From bug-Marpa-R2@rt.cpan.org

<URL​: https://rt.cpan.org/Ticket/Display.html?id=87384 >

Fixed in Marpa-R2 2.066000. Thanks!

@p5pRT
Copy link
Author

p5pRT commented May 5, 2014

From @eserte

Dana Sub 27. Srp 2013, 14​:28​:11, andreas.koenig.7os6VVqR@​franz.ak.mind.de reče​:

git bisect
----------
137da2b is the first bad commit
commit 137da2b
Author​: Father Chrysostomos <sprout@​cpan.org>
Date​: Sun Jun 30 20​:26​:34 2013 -0700

[perl #79908] Stop sub inlining from breaking closures

Also affected​:
* YATT-v0.0.8 https://rt.cpan.org/Ticket/Display.html?id=95311
* Debuggit-2.05 https://rt.perl.org/Public/Bug/Display.html?id=119047
* enum-1.07 https://rt-archive.perl.org/perl5/Ticket/Display.html?id=119047

@p5pRT
Copy link
Author

p5pRT commented May 5, 2014

From [Unknown Contact. See original ticket]

Dana Sub 27. Srp 2013, 14​:28​:11, andreas.koenig.7os6VVqR@​franz.ak.mind.de reče​:

git bisect
----------
137da2b is the first bad commit
commit 137da2b
Author​: Father Chrysostomos <sprout@​cpan.org>
Date​: Sun Jun 30 20​:26​:34 2013 -0700

[perl #79908] Stop sub inlining from breaking closures

Also affected​:
* YATT-v0.0.8 https://rt.cpan.org/Ticket/Display.html?id=95311
* Debuggit-2.05 https://rt.perl.org/Public/Bug/Display.html?id=119047
* enum-1.07 https://rt-archive.perl.org/perl5/Ticket/Display.html?id=119047

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant