Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Outer my hash (%...) in perl -ln misbehaves inside a globally-declared function in the script. #14699

Open
p5pRT opened this issue May 12, 2015 · 13 comments

Comments

@p5pRT
Copy link

p5pRT commented May 12, 2015

Migrated from rt.perl.org#125162 (status was 'open')

Searchable as RT125162$

@p5pRT
Copy link
Author

p5pRT commented May 12, 2015

From @shlomif

With this code running under perl -ln​:

< CODE >
#!/usr/bin/perl -ln

use strict;
use warnings;

s/\A([0-9]+)​:\s*//;
my $n = $1;
my %factors;
foreach my $f (split/\s+/, $_)
{
  die if $f !~ /\S/;
  print"F=$f\n";
  $factors{$f}++;
}
my $root = sqrt($n);

sub my_func
{
  if (!@​_)
  {
  return (1);
  }
  else
  {
  my $f = shift;
  print "F/=$f ; %factors = @​{[%factors]}\n";
  my @​r = my_func(@​_);

  my $x = 1;
  my @​ret;
  for my $e (0 .. $factors{$f})
  {
  push @​ret, map { $x * $_ } @​r;
  }
  continue
  {
  $x *= $f;
  }
  return @​ret;
  }
};

print join " ", $n, (sort { $a <=> $b } grep { $_ > 1 and $_ <= $root }
my_func(keys(%factors)));

< / CODE >

And with this input​:

< INPUT >
2​: 2
4​: 2 2
6​: 2 3
10​: 2 5
12​: 2 2 3
16​: 2 2 2 2
18​: 2 3 3
22​: 2 11
28​: 2 2 7
30​: 2 3 5
36​: 2 2 3 3
40​: 2 2 2 5
42​: 2 3 7
46​: 2 23
52​: 2 2 13
58​: 2 29
60​: 2 2 3 5
66​: 2 3 11
70​: 2 5 7
72​: 2 2 2 3 3
78​: 2 3 13
82​: 2 41
88​: 2 2 2 11
96​: 2 2 2 2 2 3
100​: 2 2 5 5
< / INPUT>

Then I'm getting this output and warnings​:

< OUTPUT >
Use of uninitialized value within %factors in foreach loop entry at euler-357-v1

F=2

F/=2 ; %factors = 2 1

2
F=2

F=2

F/=2 ; %factors = 2 1

4 2
F=2

F=3

F/=2 ; %factors = 2 1

F/=3 ; %factors = 2 1

6 2
F=2

F=5

F/=2 ; %factors = 2 1

F/=5 ; %factors = 2 1
< / OUTPUT >

As can be seen %factors inside my_func remains the same throughout the run and
there isn't a warning. I've attached the code an the input file here and they
are also available here​:

https://github.com/shlomif/possible-perl-ln-f-hash-bug

I can reproduce this issue with bleadperl as well as with perl-5.20.2 and
perl-5.10.1 (and possibly all perls in between).

I hereby disclaim any implicit or explicit ownership on the code and input file,
and put them under the Public Domain/CC-Zero/MIT/X11 licence/Same terms as
Perl/Any other licence.

Best regards,

  Shlomi Fish

--


Shlomi Fish http​://www.shlomifish.org/
NSA Factoids - http​://www.shlomifish.org/humour/bits/facts/NSA/

I wear prescription glasses so I may be half-blind, but at least I'm trying
hard not to be a complete dick.
  — https://twitter.com/shlomif/status/480765130242662400

Please reply to list if it's a mailing list post - http​://shlom.in/reply .

@p5pRT
Copy link
Author

p5pRT commented May 12, 2015

@p5pRT
Copy link
Author

p5pRT commented May 12, 2015

From @shlomif

2​: 2
4​: 2 2
6​: 2 3
10​: 2 5
12​: 2 2 3
16​: 2 2 2 2
18​: 2 3 3
22​: 2 11
28​: 2 2 7
30​: 2 3 5
36​: 2 2 3 3
40​: 2 2 2 5
42​: 2 3 7
46​: 2 23
52​: 2 2 13
58​: 2 29
60​: 2 2 3 5
66​: 2 3 11
70​: 2 5 7
72​: 2 2 2 3 3
78​: 2 3 13
82​: 2 41
88​: 2 2 2 11
96​: 2 2 2 2 2 3
100​: 2 2 5 5

@p5pRT
Copy link
Author

p5pRT commented May 12, 2015

From zefram@fysh.org

shlomif@​shlomifish.org wrote​:

As can be seen %factors inside my_func remains the same throughout the run and
there isn't a warning.

The -n wraps the program in an outer loop. The "my %factors" is
inside this loop, so the main program gets a separate %factors for
each iteration, i.e., for each line of input. However, the subroutine
my_func is only defined once, even though its definition is textually
within the loop, so it always sees the same %factors. In the current
implementation the %factors that my_func sees is actually the same as
the %factors for the first iteration of the loop.

You should make the loop explicit. This program is too complex to
comfortably use -n. Alternatively you could use "our %factors" to
make it work with -n, but that's heading in the wrong direction for a
growing program.

-zefram

@p5pRT
Copy link
Author

p5pRT commented May 12, 2015

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented May 12, 2015

From @shlomif

On Tue May 12 08​:19​:56 2015, zefram@​fysh.org wrote​:

shlomif@​shlomifish.org wrote​:

As can be seen %factors inside my_func remains the same throughout
the run and
there isn't a warning.

The -n wraps the program in an outer loop. The "my %factors" is
inside this loop, so the main program gets a separate %factors for
each iteration, i.e., for each line of input. However, the subroutine
my_func is only defined once, even though its definition is textually
within the loop, so it always sees the same %factors. In the current
implementation the %factors that my_func sees is actually the same as
the %factors for the first iteration of the loop.

You should make the loop explicit. This program is too complex to
comfortably use -n. Alternatively you could use "our %factors" to
make it work with -n, but that's heading in the wrong direction for a
growing program.

-zefram

I realised all that now and I'm sorry I didn't mention all that in my original post, but the #p5p people on irc.perl.org believed that it should at least warn in this case. Quoting from #p5p :

May 11 11​:46​:08 <rindolf> Hi all. https://github.com/shlomif/possible-perl-ln-f-hash-bug - I can reproduce a problem with the %f lexical/global hash getting nullified here with -ln on bleadperl and perl-5.20.2 built from source on Mageia Linux x86-64 5. Is this a bug?
May 11 11​:46​:09 <dipsy> [ shlomif/possible-perl-ln-f-hash-bug · GitHub ]
May 11 12​:07​:15 <Bram> rindolf​: as far as I can tell it also happens on perl v5.8.9 and possibly all versions in between
May 11 12​:07​:28 <TonyC> 5.14.2 here and it happens
May 11 12​:07​:39 <rindolf> Bram​: yes.
May 11 12​:08​:47 <TonyC> it's strange enough it should be reported
May 11 12​:10​:44 <Bram> it's not *that* strange tho
May 11 12​:11​:33 <Bram> You got to remember that -n adds a while (<>) { .... } so the subroutine effectively creates a closure I believe
May 11 12​:11​:49 <TonyC> ahh, yes
May 11 12​:12​:14 <Bram> http​://pastebin.com/raw.php?i=aHFzm5uT - tested with v5.8.9 tho
May 11 12​:12​:38 <Bram> Would've expected a warning that %z would not stay shared tho
May 11 12​:15​:23 <Bram> but maybe that warning only happens in constructs like ' sub foo { my %z; sub bar { print %z } }' and not if there the variable/sub is declared in another block which can get called multiple times
May 11 12​:17​:03 <Bram> http​://pastebin.com/raw.php?i=LVWTEeuD (again with v5.8.9)
May 11 12​:19​:15 <Bram> So the bug might be that the warning isn't show.. other then that it is somewhat expected behaviour to me
May 11 12​:20​:27 <TonyC> yep, it's reasonable behaviour
May 11 12​:20​:56 <Bram> To fix the program you could end the while (<>) { ... } loop added by -n/-p but you can't add anything in front of it so won't be able to add 'my %f;' (i.e. perl -nle 'print "in while"; } print "outside while {' so will need a global variable.. in which case it doesn't really matter where the subroutine is defined)
May 11 12​:25​:29 <Bram> Or use an anonymous sub to create the closure I guess.. but then you'll need to weaken it since you would need to reference it in the sub itself.. (or leak memory)
May 11 12​:30​:48 <Bram> TLDR version​: create a bug report for the missing warning :)

============

Regards,

-- Shlomi Fish

@p5pRT
Copy link
Author

p5pRT commented May 12, 2015

From @iabyn

On Tue, May 12, 2015 at 08​:29​:39AM -0700, Shlomi Fish via RT wrote​:

I realised all that now and I'm sorry I didn't mention all that in my
original post, but the #p5p people on irc.perl.org believed that it
should at least warn in this case. Quoting from #p5p :

If I read this all correctly, the complaint is that although this code
causes a "will not stay shared" warning​:

  sub f {
  my $x;
  sub g { $x }
  }

this doesn't​:

  while (1) {
  my $x;
  sub g { $x }
  }

I think it would be hard in this case to determine at compile time that
the 'my $x' declaration can be executed multiple times. Note that the
following *shouldn't* warn​:

  {
  my $x;
  sub g { $x }
  }

--
Any [programming] language that doesn't occasionally surprise the
novice will pay for it by continually surprising the expert.
  -- Larry Wall

@p5pRT
Copy link
Author

p5pRT commented May 12, 2015

From zefram@fysh.org

Dave Mitchell wrote​:

following *shouldn't* warn​:

{
my $x;
sub g { $x }
}

What about​:

  {
  my $x;
  sub g { $x }
  redo unless $z++;
  }

?

-zefram

@p5pRT
Copy link
Author

p5pRT commented May 12, 2015

From p5p@spam.wizbit.be

Just for reference including the code in the pastebins​:

#!/usr/bin/perl -l

use strict;
use warnings;
use Data​::Dumper;

for my $i ("a", "b", "c") {
  my %z;
  $z{$i} = $i;

  print "Outside sub​: " . join(" & ", keys %z);
  s1();

  sub s1 {
  print "In sub​: " . join(" & ", keys %z);
  }
}

__END__

$ perl x.pl
Outside sub​: a
In sub​: a
Outside sub​: b
In sub​: a
Outside sub​: c
In sub​: a

and the other​:

#!/usr/bin/perl -l

use strict;
use warnings;
use Data​::Dumper;

sub s2 {
  for my $i ("a", "b", "c") {
  my %z;
  $z{$i} = $i;

  print "Outside sub​: " . join(" & ", keys %z);
  s1();

  sub s1 {
  print "In sub​: " . join(" & ", keys %z);
  }
  }
}

s2();

__END__
Variable "%z" will not stay shared at x.pl line 17.
Outside sub​: a
In sub​: a
Outside sub​: b
In sub​: a
Outside sub​: c
In sub​: a

Would've expected the bug to get reported with the code of the first pastebin since that clearly shows what is happening and not with the original code...

Anyway, my remark was that the warning 'Variable "%z" will not stay shared' would've been nice to see in both cases since that clearly highlights that there was a problem with the code and at least gives a pointer to the issue.

@p5pRT
Copy link
Author

p5pRT commented May 12, 2015

From @iabyn

On Tue, May 12, 2015 at 05​:04​:01PM +0100, Zefram wrote​:

Dave Mitchell wrote​:

following *shouldn't* warn​:

{
my $x;
sub g { $x }
}

What about​:

 \{
     my $x;
     sub g \{ $x \}
 redo unless $z\+\+;
 \}

That would again be in the category of "too hard to detect at compile
time".

Personally even if we could detect it, I would veer towards not warning
in the while and redo cases.

--
"There's something wrong with our bloody ships today, Chatfield."
  -- Admiral Beatty at the Battle of Jutland, 31st May 1916.

@p5pRT
Copy link
Author

p5pRT commented May 13, 2015

From @epa

Zefram <zefram <at> fysh.org> writes​:

What about​:

\{
    my $x;
    sub g \{ $x \}
redo unless $z\+\+;
\}

I think this may be another case that argues for distinguishing between
ordinary naked blocks, which serve only for variable scoping, and 'a loop
executed only once', which would allow next/redo/last but be distinguished by
a special keyword.

So the new syntax 'once { ... }' would be a kind of loop, where 'redo' can be
used but the above would give a 'will not stay shared' warning. Meanwhile
an ordinary naked block { ... } would not allow the 'redo' keyword inside it.

I'm not saying that the above code is necessarily a convincing reason to start
splitting out the language syntax in this way, just that it is something to
think about.

--
Ed Avis <eda@​waniasset.com>

@p5pRT
Copy link
Author

p5pRT commented May 13, 2015

From zefram@fysh.org

Ed Avis wrote​:

an ordinary naked block { ... } would not allow the 'redo' keyword inside it.

Can't be done without silently changing the language. Even if you detect
"redo" within the block to generate a compile-time error, there's

  perl -le 'sub foo { redo unless $z++ } { print ++$i; foo(); }'

So the naked block has to remain the do-once loop, and we need some other
syntax for a pure scoping block. Such as "do { };", which already exists.

-zefram

@p5pRT
Copy link
Author

p5pRT commented May 13, 2015

From @epa

Ah yes, I always forget that 'redo' and friends are dynamically scoped, so to
speak, and not lexically tied to the loop they appear in.

Personally I consider that something of a misfeature, and I would prefer to
move away from allowing subroutines to alter their caller's control flow
(at least without explicit permission), but I recognize that many disagree.

--
Ed Avis <eda@​waniasset.com>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants