Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

overloading 'x' is broken #13811

Closed
p5pRT opened this issue May 8, 2014 · 11 comments
Closed

overloading 'x' is broken #13811

p5pRT opened this issue May 8, 2014 · 11 comments

Comments

@p5pRT
Copy link

p5pRT commented May 8, 2014

Migrated from rt.perl.org#121827 (status was 'resolved')

Searchable as RT121827$

@p5pRT
Copy link
Author

p5pRT commented May 8, 2014

From @salva

Created by @salva

Create the following module​:

------------overx.pm--------------------

package overx;

require overload;
overload->import('x'  => \&_x,
                 '*'  => \&_x,
                 '""' => \&_to_string,
                 '0+' => \&_length);

sub new {
    my ($class, @​a) = @​_;
    my $self = \@​a;
    bless $self, $class;
}

sub _x {
    my $a = shift;
    my $b = shift;
    my $c = [@​$a, @​$b];
    bless $c;
    print STDERR "$a x $b -> $c\n";
    $c;
}

sub _length {
    my $a = shift;
    scalar @​$a;
}

sub _to_string {
    my $a = shift;
    "{".join(', ', @​$a)."}";
}

1;

------------end-of-overx.pm--------------------

and run...

  $ perl -I. -Moverx -MData​::Dumper -e '$a = overx->new(1,'a'); print Dumper [$a, scalar(($a x $a) x $a)]'

I get...

    {1, a} x {1, a} -> {1, a, 1, a}
    {1, a, 1, a} x {1, a} -> {1, a, 1, a, 1, a}
    $VAR1 = bless( [
                     1,
                     'a'
                   ], 'overx' );
    $VAR2 = [
              bless( [
                       1,
                       'a',
                       1,
                       'a',
                       1,
                       'a'
                     ], 'overx' )
            ];

when expecting...

    {1, a} x {1, a} -> {1, a, 1, a}
    {1, a, 1, a} x {1, a} -> {1, a, 1, a, 1, a}
    $VAR1 = [
              bless( [
                       1,
                       'a'
                     ], 'overx' ),
              bless( [
                       1,
                       'a',
                       1,
                       'a',
                       1,
                       'a'
                     ], 'overx' )
            ];

Other operators work as expected.

For instance, the following generates the expected result.

  perl -I. -Moverx -MData​::Dumper -e '$a = overx->new(1,'a'); print Dumper [$a, scalar(($a * $a) * $a)]'

While the following also fails​:

  perl -I. -Moverx -MData​::Dumper -e '$a = overx->new(1,'a'); print Dumper [$a, scalar(($a * $a) x $a)]'

I can reproduce the bug also in 5.8.8.

Perl Info

Flags:
    category=library
    severity=low
    module=overload

Site configuration information for perl 5.18.2:

Configured by Debian Project at Thu Mar 27 18:28:21 UTC 2014.

Summary of my perl5 (revision 5 version 18 subversion 2) configuration:
   
  Platform:
    osname=linux, osvers=3.2.0-58-generic, archname=x86_64-linux-gnu-thread-multi
    uname='linux brownie 3.2.0-58-generic #88-ubuntu smp tue dec 3 17:37:58 utc 2013 x86_64 x86_64 x86_64 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Dldflags= -Wl,-Bsymbolic-functions -Wl,-z,relro -Dlddlflags=-shared -Wl,-Bsymbolic-functions -Wl,-z,relro -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.18 -Darchlib=/usr/lib/perl/5.18 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.18.2 -Dsitearch=/usr/local/lib/perl/5.18.2 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -Uversiononly -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.18.2 -des'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fstack-protector -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fstack-protector -fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='4.8.2', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=, so=so, useshrplib=true, libperl=libperl.so.5.18.2
    gnulibc_version='2.19'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib -fstack-protector'

Locally applied patches:
    DEBPKG:debian/cpan_definstalldirs - Provide a sensible INSTALLDIRS default for modules installed from CPAN.
    DEBPKG:debian/db_file_ver - http://bugs.debian.org/340047 Remove overly restrictive DB_File version check.
    DEBPKG:debian/doc_info - Replace generic man(1) instructions with Debian-specific information.
    DEBPKG:debian/enc2xs_inc - http://bugs.debian.org/290336 Tweak enc2xs to follow symlinks and ignore missing @INC directories.
    DEBPKG:debian/errno_ver - http://bugs.debian.org/343351 Remove Errno version check due to upgrade problems with long-running processes.
    DEBPKG:debian/libperl_embed_doc - http://bugs.debian.org/186778 Note that libperl-dev package is required for embedded linking
    DEBPKG:fixes/respect_umask - Respect umask during installation
    DEBPKG:debian/writable_site_dirs - Set umask approproately for site install directories
    DEBPKG:debian/extutils_set_libperl_path - EU:MM: Set location of libperl.a to /usr/lib
    DEBPKG:debian/no_packlist_perllocal - Don't install .packlist or perllocal.pod for perl or vendor
    DEBPKG:debian/prefix_changes - Fiddle with *PREFIX and variables written to the makefile
    DEBPKG:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to the binary targets.
    DEBPKG:debian/instmodsh_doc - Debian policy doesn't install .packlist files for core or vendor.
    DEBPKG:debian/ld_run_path - Remove standard libs from LD_RUN_PATH as per Debian policy.
    DEBPKG:debian/libnet_config_path - Set location of libnet.cfg to /etc/perl/Net as /usr may not be writable.
    DEBPKG:debian/mod_paths - Tweak @INC ordering for Debian
    DEBPKG:debian/module_build_man_extensions - http://bugs.debian.org/479460 Adjust Module::Build manual page extensions for the Debian Perl policy
    DEBPKG:debian/prune_libs - http://bugs.debian.org/128355 Prune the list of libraries wanted to what we actually need.
    DEBPKG:fixes/net_smtp_docs - [rt.cpan.org #36038] http://bugs.debian.org/100195 Document the Net::SMTP 'Port' option
    DEBPKG:debian/perlivp - http://bugs.debian.org/510895 Make perlivp skip include directories in /usr/local
    DEBPKG:debian/cpanplus_definstalldirs - http://bugs.debian.org/533707 Configure CPANPLUS to use the site directories by default.
    DEBPKG:debian/cpanplus_config_path - Save local versions of CPANPLUS::Config::System into /etc/perl.
    DEBPKG:debian/deprecate-with-apt - http://bugs.debian.org/702096 Point users to Debian packages of deprecated core modules
    DEBPKG:debian/squelch-locale-warnings - http://bugs.debian.org/508764 Squelch locale warnings in Debian package maintainer scripts
    DEBPKG:debian/skip-upstream-git-tests - Skip tests specific to the upstream Git repository
    DEBPKG:debian/patchlevel - http://bugs.debian.org/567489 List packaged patches for 5.18.2-2ubuntu1 in patchlevel.h
    DEBPKG:debian/skip-kfreebsd-crash - http://bugs.debian.org/628493 [perl #96272] Skip a crashing test case in t/op/threads.t on GNU/kFreeBSD
    DEBPKG:fixes/document_makemaker_ccflags - http://bugs.debian.org/628522 [rt.cpan.org #68613] Document that CCFLAGS should include $Config{ccflags}
    DEBPKG:debian/find_html2text - http://bugs.debian.org/640479 Configure CPAN::Distribution with correct name of html2text
    DEBPKG:debian/hurd_test_skip_stack - http://bugs.debian.org/650175 Disable failing GNU/Hurd tests dist/threads/t/stack.t
    DEBPKG:fixes/manpage_name_Test-Harness - http://bugs.debian.org/650451 [rt.cpan.org #73399] cpan/Test-Harness: add NAME headings in modules with POD
    DEBPKG:debian/makemaker-pasthru - http://bugs.debian.org/660195 [rt.cpan.org #28632] Make EU::MM pass LD through to recursive Makefile.PL invocations
    DEBPKG:debian/perl5db-x-terminal-emulator.patch - http://bugs.debian.org/668490 Invoke x-terminal-emulator rather than xterm in perl5db.pl
    DEBPKG:debian/cpan-missing-site-dirs - http://bugs.debian.org/688842 Fix CPAN::FirstTime defaults with nonexisting site dirs if a parent is writable
    DEBPKG:fixes/memoize_storable_nstore - [rt.cpan.org #77790] http://bugs.debian.org/587650 Memoize::Storable: respect 'nstore' option not respected
    DEBPKG:fixes/net_ftp_failed_command - [rt.cpan.org #37700] http://bugs.debian.org/491062 Net::FTP: cope gracefully with a failed command
    DEBPKG:fixes/perlbug-patchlist - [3541c11] http://bugs.debian.org/710842 [perl #118433] Make perlbug look up the list of local patches at run time
    DEBPKG:fixes/module_metadata_security_doc - [68cdd4b] CVE-2013-1437 documentation fix
    DEBPKG:fixes/module_metadata_taint_fix - [bff978f] http://bugs.debian.org/722210 [rt.cpan.org #88576] untaint version, if needed, in Module::Metadata
    DEBPKG:fixes/IPC-SysV-spelling - http://bugs.debian.org/730558 [rt.cpan.org #86736] Fix spelling of IPC_CREAT in IPC-SysV documentation
    DEBPKG:fixes/fix-undef-source -


@INC for perl 5.18.2:
    /etc/perl
    /usr/local/lib/perl/5.18.2
    /usr/local/share/perl/5.18.2
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.18
    /usr/share/perl/5.18
    /usr/local/lib/site_perl
    /usr/local/lib/perl/5.18.1
    /usr/local/share/perl/5.18.1
    .


Environment for perl 5.18.2:
    HOME=/home/salva
    LANG=en_US.UTF-8
    LANGUAGE=en_US:en
    LC_ADDRESS=en_US.UTF-8
    LC_IDENTIFICATION=en_US.UTF-8
    LC_MEASUREMENT=en_US.UTF-8
    LC_MONETARY=en_US.UTF-8
    LC_NAME=en_US.UTF-8
    LC_NUMERIC=en_US.UTF-8
    LC_PAPER=en_US.UTF-8
    LC_TELEPHONE=en_US.UTF-8
    LC_TIME=en_US.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/local/scala/bin:/usr/local/scala/current/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented May 10, 2014

From ambrus@math.bme.hu

On Thu May 08 07​:54​:20 2014, salva wrote​:

$ perl -I. -Moverx -MData​::Dumper -e '$a = overx->new(1,'a'); print Dumper [$a, scalar(($a x $a) x $a)]'

Could this be because the x operator in that case is parsed as list repetition, and only string repetition can be overloaded?

@p5pRT
Copy link
Author

p5pRT commented May 10, 2014

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented May 11, 2014

From @salva

----- Original Message -----

From​: Zsban Ambrus via RT <perlbug-followup@​perl.org>
To​: sfandino@​yahoo.com
Cc​:
Sent​: Saturday, May 10, 2014 3​:28 PM
Subject​: [perl #121827] overloading 'x' is broken

On Thu May 08 07​:54​:20 2014, salva wrote​:

$ perl -I. -Moverx -MData​::Dumper -e '$a =
overx->new(1,'a'); print Dumper [$a, scalar(($a x $a) x $a)]'

Could this be because the x operator in that case is parsed as list repetition,
and only string repetition can be overloaded?

No, the expression is wrapped by a "scalar" call.

In any case, the real issue is that the return value from the first
expression "escapes" from the array constructor. In other words, that is
like [$a, $b] returning ($a, [$b]). There is something here going really
wrong!

@p5pRT
Copy link
Author

p5pRT commented May 12, 2014

From @salva

The issue is caused by a 'pushmark' OP being prepended to the left-hand tree of the 'x' binary operator by the parser when the left expression is inside parenthesis.

When 'x' is not overloaded the code in pp_repeat undoes the pushmark and places the result on the parent frame, but when the operation is overloaded, the tryAMAGICbin_MG macro call just returns without removing the extra frame.

@p5pRT
Copy link
Author

p5pRT commented May 12, 2014

From [Unknown Contact. See original ticket]

The issue is caused by a 'pushmark' OP being prepended to the left-hand tree of the 'x' binary operator by the parser when the left expression is inside parenthesis.

When 'x' is not overloaded the code in pp_repeat undoes the pushmark and places the result on the parent frame, but when the operation is overloaded, the tryAMAGICbin_MG macro call just returns without removing the extra frame.

@p5pRT
Copy link
Author

p5pRT commented May 12, 2014

From @iabyn

On Sun, May 11, 2014 at 11​:30​:44AM +0200, Salvador Fandiño wrote​:

----- Original Message -----

From​: Zsban Ambrus via RT <perlbug-followup@​perl.org>
To​: sfandino@​yahoo.com
Cc​:
Sent​: Saturday, May 10, 2014 3​:28 PM
Subject​: [perl #121827] overloading 'x' is broken

On Thu May 08 07​:54​:20 2014, salva wrote​:

$ perl -I. -Moverx -MData​::Dumper -e '$a =
overx->new(1,'a'); print Dumper [$a, scalar(($a x $a) x $a)]'

Could this be because the x operator in that case is parsed as list repetition,
and only string repetition can be overloaded?

No, the expression is wrapped by a "scalar" call.

In any case, the real issue is that the return value from the first
expression "escapes" from the array constructor. In other words,
that is like [$a, $b] returning ($a, [$b]). There is something here
going really wrong!

In the 'x' case, an extra pushmark is happening after the '[ $a, ',
so that when anonlist is executed, it only collects the elements
to the right of '[ $a, ' into the anonymous list.

Whether the pushmark is extraneous, or whether its the fault of something
further along not popping the mark, I don't know.

Anyway, I haven't really got time to look any further into this yet, as it
isn't a 5.20 blocker.

--
"Procrastination grows to fill the available time"
  -- Mitchell's corollary to Parkinson's Law

@p5pRT
Copy link
Author

p5pRT commented Nov 6, 2014

From @cpansprout

On Mon May 12 04​:17​:13 2014, salva wrote​:

The issue is caused by a 'pushmark' OP being prepended to the left-
hand tree of the 'x' binary operator by the parser when the left
expression is inside parenthesis.

When 'x' is not overloaded the code in pp_repeat undoes the pushmark
and places the result on the parent frame, but when the operation is
overloaded, the tryAMAGICbin_MG macro call just returns without
removing the extra frame.

There were two stack bugs in pp_repeat, so I fixed them both in 3a100da. Then I added an optimisation in 82e4f30 that sidesteps the issue most of the time.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Nov 6, 2014

@cpansprout - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

p5pRT commented Jun 2, 2015

From @khwilliamson

Thanks for submitting this ticket

The issue should be resolved with the release today of Perl v5.22. If you find that the problem persists, feel free to reopen this ticket

--
Karl Williamson for the Perl 5 porters team

@p5pRT p5pRT closed this as completed Jun 2, 2015
@p5pRT
Copy link
Author

p5pRT commented Jun 2, 2015

@khwilliamson - Status changed from 'pending release' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant