Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bleadperl v5.19.3-296-gffdb8b1 breaks BAREFOOT/Test-File-1.34.tar.gz #13224

Closed
p5pRT opened this issue Sep 4, 2013 · 31 comments
Closed

Bleadperl v5.19.3-296-gffdb8b1 breaks BAREFOOT/Test-File-1.34.tar.gz #13224

p5pRT opened this issue Sep 4, 2013 · 31 comments

Comments

@p5pRT
Copy link

p5pRT commented Sep 4, 2013

Migrated from rt.perl.org#119593 (status was 'resolved')

Searchable as RT119593$

@p5pRT
Copy link
Author

p5pRT commented Sep 4, 2013

From @andk

git bisect


commit ffdb8b1
Author​: Father Chrysostomos <sprout@​cpan.org>
Date​: Sun Sep 1 00​:30​:59 2013 -0700

  Fix two line numbers bugs involving quote-like ops

diagnostics


Also affected​: FERRENCY/Text-MicroMason-2.13.tar.gz

Sample fails not yet arrived at cpantesters. This is from Text​::MicroMason​:

  # Failed test 'first line of $@​ describes the error location'
  # at t/08-errors.t line 37.
  # 'MicroMason compilation failed​: syntax error at text template (compiled at t/08-errors.t line 35) line 9, at EOF'
  # doesn't match '(?^​:MicroMason compilation failed​: syntax error at text template \(compiled at t/08-errors.t line \d+\) line 8)'
  # MicroMason compilation failed​: syntax error at text template (compiled at t/08-errors.t line 35) line 9, at EOF
  #
  # 0 # line 1 "text template (compiled at t/08-errors.t line 35)"
  # 1 sub {
  # 2 local $SIG{__DIE__} = sub { die "MicroMason execution failed​: ", @​_ };
  # 3 my @​OUT; my $_out = sub {push @​OUT, @​_};
  # 4 my %ARGS = @​_ if ($#_ % 2);
  # 5 push @​OUT,( qq(Hello\ world\!\
  # 6 This\ ) );
  # 7 push @​OUT,( "".do{
  # 8 thing(
  # 9 } );
  # 10 push @​OUT,( qq(\ is\ a\ test\.\
  # 11 End\.\
  # 12 ) );
  # 13 join("", @​OUT)
  # 14 }
  # ** Please use Text​::MicroMason->new(-LineNumbers) for better diagnostics!
  # at t/08-errors.t line 35.
  # eval {...} called at t/08-errors.t line 35
  # Looks like you failed 1 test of 28.
  t/08-errors.t ...............
  Dubious, test returned 1 (wstat 256, 0x100)
  Failed 1/28 subtests

perl -V


Summary of my perl5 (revision 5 version 19 subversion 4) configuration​:
  Commit id​: ffdb8b1
  Platform​:
  osname=linux, osvers=3.10-2-amd64, archname=x86_64-linux
  uname='linux k83 3.10-2-amd64 #1 smp debian 3.10.7-1 (2013-08-17) x86_64 gnulinux '
  config_args='-Dprefix=/home/sand/src/perl/repoperls/installed-perls/perl/v5.19.3-296-gffdb8b1/165a -Dmyhostname=k83 -Dinstallusrbinperl=n -Uversiononly -Dusedevel -des -Ui_db -Uuseithreads -Uuselongdouble -DDEBUGGING=-g'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=undef, usemultiplicity=undef
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-O2 -g',
  cppflags='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
  ccversion='', gccversion='4.8.1', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
  libpth=/usr/local/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /usr/lib
  libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
  perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
  libc=, so=so, useshrplib=false, libperl=libperl.a
  gnulibc_version='2.17'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector'

Characteristics of this binary (from libperl)​:
  Compile-time options​: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
  PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
  PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
  PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT
  USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_PERLIO
  USE_PERL_ATOF
  Built under linux
  Compiled at Sep 3 2013 22​:52​:55
  @​INC​:
  /home/sand/src/perl/repoperls/installed-perls/perl/v5.19.3-296-gffdb8b1/165a/lib/site_perl/5.19.4/x86_64-linux
  /home/sand/src/perl/repoperls/installed-perls/perl/v5.19.3-296-gffdb8b1/165a/lib/site_perl/5.19.4
  /home/sand/src/perl/repoperls/installed-perls/perl/v5.19.3-296-gffdb8b1/165a/lib/5.19.4/x86_64-linux
  /home/sand/src/perl/repoperls/installed-perls/perl/v5.19.3-296-gffdb8b1/165a/lib/5.19.4
  .

--
andreas

@p5pRT
Copy link
Author

p5pRT commented Sep 7, 2013

From @cpansprout

On Tue Sep 03 19​:58​:33 2013, andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

git bisect
----------
commit ffdb8b1
Author​: Father Chrysostomos <sprout@​cpan.org>
Date​: Sun Sep 1 00​:30​:59 2013 -0700

Fix two line numbers bugs involving quote\-like ops

The Test​::File failure can be reduced to​:

foo(
  "@​{[]}"
. warn() # now gives line 1, not 2
. "")

The line number returned by caller() (and used by warn) is a single
number applying to the whole statement. A multiline statement only gets
one line number assigned to it in the op tree.

"@​{[]}" used to reset the line number for the surrounding statement to
the line number of the "@​{[]}" itself. Now it has stopped doing that,
although @​{[]} without quotes still does it.

The reason this happens is that PL_copline is now localised and
invalidated for the compilation of the contents of the quotes, so that
statements inside the quotes don’t pick up the line number from outside.

Test​::File’s test suite is sensitive to that, as it uses
Test​::Builder​::Tester​::line_num (which uses caller), rather than
__LINE__. __LINE__ is not subject to the one-number-per-statement
limitation.

As long as we are limited to one line number per statement, which line
should we be aiming for? To me, the first line makes sense, so it looks
as though I have partially fixed a bug here.

If necessary, though, I could allow changes to PL_copline inside the
quotes to leak out, but still protect code inside from the previous
PL_copline value from outside.

(For those who have no idea what I am talking about, PL_copline is set
at the beginning of certain possibly multiline constructs, and then
newSTATEOP [which creates the statement] uses that value for the line
number if it is set. The idea is to report beginning, rather than
ending, line numbers. Currently PL_copline is not used enough, IMO.)

diagnostics
-----------
Also affected​: FERRENCY/Text-MicroMason-2.13.tar.gz

Sample fails not yet arrived at cpantesters. This is from
Text​::MicroMason​:

# Failed test 'first line of $@​ describes the error location'
# at t/08-errors.t line 37.
# 'MicroMason compilation failed​: syntax error at
text template (compiled at t/08-errors.t line 35) line 9, at EOF'
# doesn't match '(?^​:MicroMason compilation failed​: syntax error
at text template \(compiled at t/08-errors.t line \d+\) line 8)'

So now perl reports 9 instead of 8.

# MicroMason compilation failed​: syntax error at text template
(compiled at t/08-errors.t line 35) line 9, at EOF
#
# 0 # line 1 "text template (compiled at t/08-errors.t line 35)"
# 1 sub {
# 2 local $SIG{__DIE__} = sub { die "MicroMason execution
failed​: ", @​_ };
# 3 my @​OUT; my $_out = sub {push @​OUT, @​_};
# 4 my %ARGS = @​_ if ($#_ % 2);
# 5 push @​OUT,( qq(Hello\ world\!\
# 6 This\ ) );
# 7 push @​OUT,( "".do{
# 8 thing(
# 9 } );

And the syntax error is on line 9. So that looks like a bug fix, too.

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Sep 7, 2013

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Sep 8, 2013

From @cpansprout

On Sat Sep 07 01​:20​:26 2013, sprout wrote​:

On Tue Sep 03 19​:58​:33 2013, andreas.koenig.7os6VVqR@​franz.ak.mind.de
wrote​:

git bisect
----------
commit ffdb8b1
Author​: Father Chrysostomos <sprout@​cpan.org>
Date​: Sun Sep 1 00​:30​:59 2013 -0700

Fix two line numbers bugs involving quote\-like ops

The Test​::File failure can be reduced to​:

foo(
"@​{[]}"
. warn() # now gives line 1, not 2
. "")

The line number returned by caller() (and used by warn) is a single
number applying to the whole statement. A multiline statement only gets
one line number assigned to it in the op tree.

"@​{[]}" used to reset the line number for the surrounding statement to
the line number of the "@​{[]}" itself. Now it has stopped doing that,
although @​{[]} without quotes still does it.

The reason this happens is that PL_copline is now localised and
invalidated for the compilation of the contents of the quotes, so that
statements inside the quotes don’t pick up the line number from outside.

Test​::File’s test suite is sensitive to that, as it uses
Test​::Builder​::Tester​::line_num (which uses caller), rather than
__LINE__. __LINE__ is not subject to the one-number-per-statement
limitation.

As long as we are limited to one line number per statement, which line
should we be aiming for? To me, the first line makes sense, so it looks
as though I have partially fixed a bug here.

If necessary, though, I could allow changes to PL_copline inside the
quotes to leak out, but still protect code inside from the previous
PL_copline value from outside.

(For those who have no idea what I am talking about, PL_copline is set
at the beginning of certain possibly multiline constructs, and then
newSTATEOP [which creates the statement] uses that value for the line
number if it is set. The idea is to report beginning, rather than
ending, line numbers. Currently PL_copline is not used enough, IMO.)

Maybe the best solution here is to allow entersub ops to store a line
number.

Maybe all ops should store a line number.... (I’m not the first to
suggest that. It was brought up in another ticket a long time ago.)

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Dec 22, 2013

From @cpansprout

On Sat Sep 07 18​:47​:34 2013, sprout wrote​:

On Sat Sep 07 01​:20​:26 2013, sprout wrote​:

On Tue Sep 03 19​:58​:33 2013, andreas.koenig.7os6VVqR@​franz.ak.mind.de
wrote​:

git bisect
----------
commit ffdb8b1
Author​: Father Chrysostomos <sprout@​cpan.org>
Date​: Sun Sep 1 00​:30​:59 2013 -0700

Fix two line numbers bugs involving quote\-like ops

The Test​::File failure can be reduced to​:

foo(
"@​{[]}"
. warn() # now gives line 1, not 2
. "")

The line number returned by caller() (and used by warn) is a single
number applying to the whole statement. A multiline statement only gets
one line number assigned to it in the op tree.

"@​{[]}" used to reset the line number for the surrounding statement to
the line number of the "@​{[]}" itself. Now it has stopped doing that,
although @​{[]} without quotes still does it.

The reason this happens is that PL_copline is now localised and
invalidated for the compilation of the contents of the quotes, so that
statements inside the quotes don’t pick up the line number from outside.

Test​::File’s test suite is sensitive to that, as it uses
Test​::Builder​::Tester​::line_num (which uses caller), rather than
__LINE__. __LINE__ is not subject to the one-number-per-statement
limitation.

As long as we are limited to one line number per statement, which line
should we be aiming for? To me, the first line makes sense, so it looks
as though I have partially fixed a bug here.

If necessary, though, I could allow changes to PL_copline inside the
quotes to leak out, but still protect code inside from the previous
PL_copline value from outside.

(For those who have no idea what I am talking about, PL_copline is set
at the beginning of certain possibly multiline constructs, and then
newSTATEOP [which creates the statement] uses that value for the line
number if it is set. The idea is to report beginning, rather than
ending, line numbers. Currently PL_copline is not used enough, IMO.)

Maybe the best solution here is to allow entersub ops to store a line
number.

Maybe all ops should store a line number.... (I’m not the first to
suggest that. It was brought up in another ticket a long time ago.)

Could someone else familiar with ops review my suggestions?

Should I
• make all ops store line numbers, fixing numerous bugs and
  increasing memory usage,
• introduce a new SUBOP op type that includes a line number or
• make PL_copline localisation only protect the inner value, not
  the outer?

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Dec 22, 2013

From @bulk88

On Sat Dec 21 18​:07​:08 2013, sprout wrote​:

Could someone else familiar with ops review my suggestions?

Should I
• make all ops store line numbers, fixing numerous bugs and
increasing memory usage,
• introduce a new SUBOP op type that includes a line number or
• make PL_copline localisation only protect the inner value, not
the outer?

I suggest won't fix. IMO line numbers are on a statement, not expression/substatement basis. None of the suggestions sound like the increase in memory or CPU will justify a bug about undocumented and subject to change behavior.

--
bulk88 ~ bulk88 at hotmail.com

@p5pRT
Copy link
Author

p5pRT commented Dec 22, 2013

From zefram@fysh.org

Father Chrysostomos via RT wrote​:

Should I
* make all ops store line numbers, fixing numerous bugs and
increasing memory usage,
* introduce a new SUBOP op type that includes a line number or
* make PL_copline localisation only protect the inner value, not
the outer?

None of the above. The current behaviour is not a bug; there is not
sufficient justification for changing it, especially in a way that would
substantially increase memory usage.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Dec 22, 2013

From @bulk88

On Sat Dec 21 20​:25​:13 2013, bulk88 wrote​:

On Sat Dec 21 18​:07​:08 2013, sprout wrote​:

Could someone else familiar with ops review my suggestions?

Should I
• make all ops store line numbers, fixing numerous bugs and
increasing memory usage,
• introduce a new SUBOP op type that includes a line number or
• make PL_copline localisation only protect the inner value, not
the outer?

I suggest won't fix. IMO line numbers are on a statement, not
expression/substatement basis. None of the suggestions sound like the
increase in memory or CPU will justify a bug about undocumented and
subject to change behavior.

I did think of something, what if a single bit in the op struct indicated if this op is on a new line, or previous line. The line number of the current op * is determined by walking from the COP along the the linked list to the current op * and counting the number of newline bits. The algorithm for computing the line number is very overhead intensive/slow, but takes very very little memory, just 1 bit per op struct.

--
bulk88 ~ bulk88 at hotmail.com

@p5pRT
Copy link
Author

p5pRT commented Dec 23, 2013

From zefram@fysh.org

bulk88 via RT wrote​:

I did think of something, what if a single bit in the op struct indicated
if this op is on a new line, or previous line.

Unworkable. With conditionals, multiple paths can be taken through the
ops of a statement, leading to disagreements about the line an op is on.
(Similar to existing issues about an op being reached from more than one
COP, but worse in that it'll produce totally bogus line numbers rather
than just the correct line number for the wrong code.) You also run
into trouble if there's a blank line between consecutive ops, and where
the op execution sequence goes back a line, and where a #line directive
ascribes a totally different line number to the next op, and where a
statement is split between files.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Dec 23, 2013

From perl5-porters@perl.org

Zefram wrote​:

where a
statement is split between files.

When does *that* occur?

@p5pRT
Copy link
Author

p5pRT commented Dec 23, 2013

From zefram@fysh.org

Father Chrysostomos wrote​:

Zefram wrote​:

where a
statement is split between files.

When does *that* occur?

#line can trivially make a statement notionally split between files.
A real split between input files can't occur short of parser plugins.
I erred slightly in writing what you've quoted​: I was really thinking
of cpp #include semantics, which we don't have.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Dec 26, 2013

From @jkeenan

Buddy Burden has released dev version 1.35_01 of Test-File, which appears to address the problem. Once he promotes that to a full version, we'll be able to close this ticket.

Thank you very much.
Jim Keenan

@p5pRT
Copy link
Author

p5pRT commented Jan 9, 2014

From @rurban

On Mon, Dec 23, 2013 at 6​:59 AM, Zefram <zefram@​fysh.org> wrote​:

bulk88 via RT wrote​:

I did think of something, what if a single bit in the op struct indicated
if this op is on a new line, or previous line.

Unworkable. With conditionals, multiple paths can be taken through the
ops of a statement, leading to disagreements about the line an op is on.
(Similar to existing issues about an op being reached from more than one
COP, but worse in that it'll produce totally bogus line numbers rather
than just the correct line number for the wrong code.) You also run
into trouble if there's a blank line between consecutive ops, and where
the op execution sequence goes back a line, and where a #line directive
ascribes a totally different line number to the next op, and where a
statement is split between files.

So you have no idea about my oplines branch from 2008, which moves the
copline to every op
and saves 8% space and runtime by omitting unnecessary nextstate cops.
http​://www.nntp.perl.org/group/perl.perl5.porters/2008/09/msg140023.html

It it not ready, as I did just to test the optimization idea.
What is missing are the COP search in case of warnings/die and the
fixes for the debugger.

--
Reini Urban
http​://cpanel.net/ http​://www.perl-compiler.org/

@p5pRT
Copy link
Author

p5pRT commented Jan 21, 2014

From @rjbs

On Wed Dec 25 18​:25​:51 2013, jkeenan wrote​:

Buddy Burden has released dev version 1.35_01 of Test-File, which
appears to address the problem. Once he promotes that to a full
version, we'll be able to close this ticket.

Jim​: Test-File 1.36 is out​: https://metacpan.org/release/Test-File

Are we done?

--
rjbs

@p5pRT
Copy link
Author

p5pRT commented Jan 21, 2014

From @jkeenan

On Mon Jan 20 19​:47​:17 2014, rjbs wrote​:

On Wed Dec 25 18​:25​:51 2013, jkeenan wrote​:

Buddy Burden has released dev version 1.35_01 of Test-File, which
appears to address the problem. Once he promotes that to a full
version, we'll be able to close this ticket.

Jim​: Test-File 1.36 is out​: https://metacpan.org/release/Test-File

Are we done?

I will take the ticket for the purpose of investigating and confirming.

@p5pRT
Copy link
Author

p5pRT commented Jan 21, 2014

From @jkeenan

On Tue Jan 21 04​:55​:37 2014, jkeenan wrote​:

On Mon Jan 20 19​:47​:17 2014, rjbs wrote​:

On Wed Dec 25 18​:25​:51 2013, jkeenan wrote​:

Buddy Burden has released dev version 1.35_01 of Test-File, which
appears to address the problem. Once he promotes that to a full
version, we'll be able to close this ticket.

Jim​: Test-File 1.36 is out​: https://metacpan.org/release/Test-File

Are we done?

I will take the ticket for the purpose of investigating and confirming.

Test-File v1.36 does pass all its tests at​:
This is perl 5, version 19, subversion 9 (v5.19.9 (v5.19.8-10-g568d025)) built for darwin-2level

However, reviewing the ticket, I see that it was originally filed against CPAN distro Text-MicroMason. Some of this distro's tests are still failing​:

#####
t/08-errors.t ............... 1/28
# Failed test 'first line of $@​ describes the error location'
# at t/08-errors.t line 37.
# 'MicroMason compilation failed​: syntax error at text template (compiled at t/08-errors.t line 35) line 9, at EOF'
# doesn't match '(?^​:MicroMason compilation failed​: syntax error at text template \(compiled at t/08-errors.t line \d+\) line 8)'
# MicroMason compilation failed​: syntax error at text template (compiled at t/08-errors.t line 35) line 9, at EOF
#
# 0 # line 1 "text template (compiled at t/08-errors.t line 35)"
# 1 sub {
# 2 local $SIG{__DIE__} = sub { die "MicroMason execution failed​: ", @​_ };
# 3 my @​OUT; my $_out = sub {push @​OUT, @​_};
# 4 my %ARGS = @​_ if ($#_ % 2);
# 5 push @​OUT,( qq(Hello\ world\!\
# 6 This\ ) );
# 7 push @​OUT,( "".do{
# 8 thing(
# 9 } );
# 10 push @​OUT,( qq(\ is\ a\ test\.\
# 11 End\.\
# 12 ) );
# 13 join("", @​OUT)
# 14 }
# ** Please use Text​::MicroMason->new(-LineNumbers) for better diagnostics!
# at t/08-errors.t line 35.
# eval {...} called at t/08-errors.t line 35
# Looks like you failed 1 test of 28.
t/08-errors.t ............... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/28 subtests
t/09-regression.t ........... 1/25 Use of uninitialized value in concatenation (.) or string at text template (compiled at t/09-regression.t line 24) line 6.
t/09-regression.t ........... ok
#####

At the moment, I'm not able to diagnose this further. Suggestions?

Thank you very much.
Jim Keenan

@p5pRT
Copy link
Author

p5pRT commented Feb 6, 2014

From @tonycoz

On Tue Jan 21 10​:22​:35 2014, jkeenan wrote​:

At the moment, I'm not able to diagnose this further. Suggestions?

Bisected down to​:

commit ffdb8b1
Author​: Father Chrysostomos <sprout@​cpan.org>
Date​: Sun Sep 1 00​:30​:59 2013 -0700

  Fix two line numbers bugs involving quote-like ops

  I was going to try and fix #line directives in quote-like operators,
  but I found myself fixing bug #3643 at the same time.

  Before this commit, the #line directive would last until the end of
  the quote-ilke operator, but not beyond​:

  qq{${
  print __LINE__,"\n"; # 43
  }};
  print __LINE__,"\n"; # 5

with​:

perl ../bisect.pl --start v5.12.0 -e '$code = `cat /home/tonyc/foo.compiled.pl`; eval $code; $@​ =~ /line 8/ or die'

where foo.compiled.pl contains the code generated from the template.

This appears to be the result of a bug fix, if I expand the Hello world string in the by a few more lines the error message on 5.14.2 stays on line 8, but increases correctly in blead.

I've reported this to the Text-MicroMason queue as https://rt.cpan.org/Ticket/Display.html?id=92771

Tony

@p5pRT
Copy link
Author

p5pRT commented Feb 6, 2014

From @jkeenan

On Wed Feb 05 16​:34​:26 2014, tonyc wrote​:

On Tue Jan 21 10​:22​:35 2014, jkeenan wrote​:

At the moment, I'm not able to diagnose this further. Suggestions?

Bisected down to​:

commit ffdb8b1
Author​: Father Chrysostomos <sprout@​cpan.org>
Date​: Sun Sep 1 00​:30​:59 2013 -0700

Fix two line numbers bugs involving quote-like ops

I was going to try and fix #line directives in quote-like operators,
but I found myself fixing bug #3643 at the same time.

Before this commit, the #line directive would last until the end of
the quote-ilke operator, but not beyond​:

qq{${
print __LINE__,"\n"; # 43
}};
print __LINE__,"\n"; # 5

with​:

perl ../bisect.pl --start v5.12.0 -e '$code = `cat
/home/tonyc/foo.compiled.pl`; eval $code; $@​ =~ /line 8/ or die'

where foo.compiled.pl contains the code generated from the template.

This appears to be the result of a bug fix, if I expand the Hello
world string in the by a few more lines the error message on 5.14.2
stays on line 8, but increases correctly in blead.

I've reported this to the Text-MicroMason queue as
https://rt.cpan.org/Ticket/Display.html?id=92771

Tony

Tony, if I understand you correctly, Text-MicroMason was inadvertently relying on buggy behavior in Perl 5 -- behavior which we have now corrected. If so, given that you have notified Text-MicroMason of this problem, we can close this ticket and remove a blocker for 5.20.

I will close this ticket within 7 days unless someone convinces me otherwise.

Thank you very much.
Jim Keenan

@p5pRT
Copy link
Author

p5pRT commented Feb 6, 2014

From @tonycoz

On Wed Feb 05 19​:31​:15 2014, jkeenan wrote​:

Tony, if I understand you correctly, Text-MicroMason was inadvertently
relying on buggy behavior in Perl 5 -- behavior which we have now
corrected. If so, given that you have notified Text-MicroMason of
this problem, we can close this ticket and remove a blocker for 5.20.

Exactly that.

I will close this ticket within 7 days unless someone convinces me
otherwise.

I've closed it.

Tony

@p5pRT
Copy link
Author

p5pRT commented Feb 6, 2014

@tonycoz - Status changed from 'open' to 'resolved'

@p5pRT
Copy link
Author

p5pRT commented May 2, 2014

From @andk

"Father Chrysostomos via RT" <perlbug-followup@​perl.org> writes​:

On Tue Sep 03 19​:58​:33 2013, andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

git bisect
----------
commit ffdb8b1
Author​: Father Chrysostomos <sprout@​cpan.org>
Date​: Sun Sep 1 00​:30​:59 2013 -0700

Fix two line numbers bugs involving quote\-like ops

Also affected​: IKEGAMI/Syntax-Feature-QwComments-v1.12.0.tar.gz

--
andreas

@p5pRT
Copy link
Author

p5pRT commented May 2, 2014

From @ikegami

On Sun Dec 22 04​:08​:19 2013, zefram@​fysh.org wrote​:

Father Chrysostomos via RT wrote​:

Should I
* make all ops store line numbers, fixing numerous bugs and
increasing memory usage,
* introduce a new SUBOP op type that includes a line number or
* make PL_copline localisation only protect the inner value, not
the outer?

None of the above. The current behaviour is not a bug

What do you mean by "current behaviour"? Currently, blead is issuing error messages with no line numbers, and I'd consider that a bug.

@p5pRT
Copy link
Author

p5pRT commented May 2, 2014

From @ikegami

On Fri May 02 06​:28​:45 2014, ikegami@​adaelis.com wrote​:

On Sun Dec 22 04​:08​:19 2013, zefram@​fysh.org wrote​:

Father Chrysostomos via RT wrote​:

Should I
* make all ops store line numbers, fixing numerous bugs and
increasing memory usage,
* introduce a new SUBOP op type that includes a line number or
* make PL_copline localisation only protect the inner value, not
the outer?

None of the above. The current behaviour is not a bug

What do you mean by "current behaviour"? Currently, blead is issuing
error messages with no line numbers, and I'd consider that a bug.

Or maybe that wasn't mentioned? The commit also removes " at -e line 1" from the following​:

$ ./perl -e'qw'
Can't find string terminator ";" anywhere before EOF.

@p5pRT
Copy link
Author

p5pRT commented May 2, 2014

@ikegami - Status changed from 'resolved' to 'open'

@p5pRT
Copy link
Author

p5pRT commented May 6, 2014

From @tonycoz

On Fri May 02 06​:41​:33 2014, ikegami@​adaelis.com wrote​:

On Fri May 02 06​:28​:45 2014, ikegami@​adaelis.com wrote​:

On Sun Dec 22 04​:08​:19 2013, zefram@​fysh.org wrote​:

Father Chrysostomos via RT wrote​:

Should I
* make all ops store line numbers, fixing numerous bugs and
increasing memory usage,
* introduce a new SUBOP op type that includes a line number or
* make PL_copline localisation only protect the inner value, not
the outer?

None of the above. The current behaviour is not a bug

What do you mean by "current behaviour"? Currently, blead is issuing
error messages with no line numbers, and I'd consider that a bug.

Or maybe that wasn't mentioned? The commit also removes " at -e line
1" from the following​:

$ ./perl -e'qw'
Can't find string terminator ";" anywhere before EOF.

Please try the attached patch.

Tony

@p5pRT
Copy link
Author

p5pRT commented May 6, 2014

From @tonycoz

0001-perl-119593-ensure-qw-q-and-report-line-on-a-missing.patch
From f1e03a55e00886d916008377fb3adc2913e35288 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Tue, 6 May 2014 16:27:14 +1000
Subject: [perl #119593] ensure qw//, q'' and '' report line on a missing
 terminator

---
 t/lib/croak/toke |   15 +++++++++++++++
 toke.c           |    8 ++++----
 2 files changed, 19 insertions(+), 4 deletions(-)

diff --git a/t/lib/croak/toke b/t/lib/croak/toke
index d6026d2..0572094 100644
--- a/t/lib/croak/toke
+++ b/t/lib/croak/toke
@@ -134,3 +134,18 @@ Execution of - aborted due to compilation errors.
 <<"foo
 EXPECT
 Unterminated delimiter for here document at - line 1.
+########
+# NAME Unterminated qw//
+qw/
+EXPECT
+Can't find string terminator "/" anywhere before EOF at - line 1.
+########
+# NAME Unterminated q//
+qw/
+EXPECT
+Can't find string terminator "/" anywhere before EOF at - line 1.
+########
+# NAME Unterminated ''
+'
+EXPECT
+Can't find string terminator "'" anywhere before EOF at - line 1.
diff --git a/toke.c b/toke.c
index 3d992f6..ea88183 100644
--- a/toke.c
+++ b/toke.c
@@ -6848,6 +6848,8 @@ Perl_yylex(pTHX)
 
     case '\'':
 	s = scan_str(s,!!PL_madskills,FALSE,FALSE,FALSE,NULL);
+	if (!s)
+	    missingterm(NULL);
 	COPLINE_SET_FROM_MULTI_END;
 	DEBUG_T( { printbuf("### Saw string before %s\n", s); } );
 	if (PL_expect == XOPERATOR) {
@@ -6857,8 +6859,6 @@ Perl_yylex(pTHX)
 	    else
 		no_op("String",s);
 	}
-	if (!s)
-	    missingterm(NULL);
 	pl_yylval.ival = OP_CONST;
 	TERM(sublex_start());
 
@@ -8385,9 +8385,9 @@ Perl_yylex(pTHX)
 
 	case KEY_q:
 	    s = scan_str(s,!!PL_madskills,FALSE,FALSE,FALSE,NULL);
-	    COPLINE_SET_FROM_MULTI_END;
 	    if (!s)
 		missingterm(NULL);
+	    COPLINE_SET_FROM_MULTI_END;
 	    pl_yylval.ival = OP_CONST;
 	    TERM(sublex_start());
 
@@ -8397,9 +8397,9 @@ Perl_yylex(pTHX)
 	case KEY_qw: {
 	    OP *words = NULL;
 	    s = scan_str(s,!!PL_madskills,FALSE,FALSE,FALSE,NULL);
-	    COPLINE_SET_FROM_MULTI_END;
 	    if (!s)
 		missingterm(NULL);
+	    COPLINE_SET_FROM_MULTI_END;
 	    PL_expect = XOPERATOR;
 	    if (SvCUR(PL_lex_stuff)) {
 		int warned_comma = !ckWARN(WARN_QW);
-- 
1.7.10.4

@p5pRT
Copy link
Author

p5pRT commented May 6, 2014

From @tonycoz

On Mon May 05 23​:28​:08 2014, tonyc wrote​:

On Fri May 02 06​:41​:33 2014, ikegami@​adaelis.com wrote​:

On Fri May 02 06​:28​:45 2014, ikegami@​adaelis.com wrote​:

On Sun Dec 22 04​:08​:19 2013, zefram@​fysh.org wrote​:

Father Chrysostomos via RT wrote​:

Should I
* make all ops store line numbers, fixing numerous bugs and
increasing memory usage,
* introduce a new SUBOP op type that includes a line number or
* make PL_copline localisation only protect the inner value, not
the outer?

None of the above. The current behaviour is not a bug

What do you mean by "current behaviour"? Currently, blead is issuing
error messages with no line numbers, and I'd consider that a bug.

Or maybe that wasn't mentioned? The commit also removes " at -e line
1" from the following​:

$ ./perl -e'qw'
Can't find string terminator ";" anywhere before EOF.

Please try the attached patch.

New patch with explanation added.

Tony

@p5pRT
Copy link
Author

p5pRT commented May 6, 2014

From @tonycoz

0001-perl-119593-ensure-qw-q-and-report-line-on-a-missing.patch
From bf65bd5366e6c89587d7f8f75c8ec3cdc37ac7ab Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Wed, 7 May 2014 09:33:03 +1000
Subject: [perl #119593] ensure qw//, q'' and '' report line on a missing
 terminator

scan_str() only sets PL_multi_end on success, but the qw, q amd '' cases
were calling the COPLINE_SET_FROM_MULTI_END; macro before reporting
failure, overwriting the line used for reporting errors.

For the simplest case, as in the ticket and the tests, PL_multi_end is
zero, so the error is reported without a line number.
---
 t/lib/croak/toke |   15 +++++++++++++++
 toke.c           |    8 ++++----
 2 files changed, 19 insertions(+), 4 deletions(-)

diff --git a/t/lib/croak/toke b/t/lib/croak/toke
index d6026d2..0572094 100644
--- a/t/lib/croak/toke
+++ b/t/lib/croak/toke
@@ -134,3 +134,18 @@ Execution of - aborted due to compilation errors.
 <<"foo
 EXPECT
 Unterminated delimiter for here document at - line 1.
+########
+# NAME Unterminated qw//
+qw/
+EXPECT
+Can't find string terminator "/" anywhere before EOF at - line 1.
+########
+# NAME Unterminated q//
+qw/
+EXPECT
+Can't find string terminator "/" anywhere before EOF at - line 1.
+########
+# NAME Unterminated ''
+'
+EXPECT
+Can't find string terminator "'" anywhere before EOF at - line 1.
diff --git a/toke.c b/toke.c
index 3d992f6..ea88183 100644
--- a/toke.c
+++ b/toke.c
@@ -6848,6 +6848,8 @@ Perl_yylex(pTHX)
 
     case '\'':
 	s = scan_str(s,!!PL_madskills,FALSE,FALSE,FALSE,NULL);
+	if (!s)
+	    missingterm(NULL);
 	COPLINE_SET_FROM_MULTI_END;
 	DEBUG_T( { printbuf("### Saw string before %s\n", s); } );
 	if (PL_expect == XOPERATOR) {
@@ -6857,8 +6859,6 @@ Perl_yylex(pTHX)
 	    else
 		no_op("String",s);
 	}
-	if (!s)
-	    missingterm(NULL);
 	pl_yylval.ival = OP_CONST;
 	TERM(sublex_start());
 
@@ -8385,9 +8385,9 @@ Perl_yylex(pTHX)
 
 	case KEY_q:
 	    s = scan_str(s,!!PL_madskills,FALSE,FALSE,FALSE,NULL);
-	    COPLINE_SET_FROM_MULTI_END;
 	    if (!s)
 		missingterm(NULL);
+	    COPLINE_SET_FROM_MULTI_END;
 	    pl_yylval.ival = OP_CONST;
 	    TERM(sublex_start());
 
@@ -8397,9 +8397,9 @@ Perl_yylex(pTHX)
 	case KEY_qw: {
 	    OP *words = NULL;
 	    s = scan_str(s,!!PL_madskills,FALSE,FALSE,FALSE,NULL);
-	    COPLINE_SET_FROM_MULTI_END;
 	    if (!s)
 		missingterm(NULL);
+	    COPLINE_SET_FROM_MULTI_END;
 	    PL_expect = XOPERATOR;
 	    if (SvCUR(PL_lex_stuff)) {
 		int warned_comma = !ckWARN(WARN_QW);
-- 
1.7.10.4

@p5pRT
Copy link
Author

p5pRT commented May 12, 2014

From @khwilliamson

I spent an hour or so familiarizing myself with the code and the patch, and it looks good to me. So with rjbs' approval, I've pushed it as commit 9612f77
--
Karl Williamson

@p5pRT
Copy link
Author

p5pRT commented May 12, 2014

@khwilliamson - Status changed from 'open' to 'resolved'

@p5pRT p5pRT closed this as completed May 12, 2014
@p5pRT
Copy link
Author

p5pRT commented May 14, 2014

From @ikegami

I had problems with my linux box that kept me from testing it until today.
It restores the expected error message syntax, verified manually and by
re-running the Syntax​::Feature​::QwComments test suite.

On Mon, May 12, 2014 at 3​:07 PM, Karl Williamson via RT <
perlbug-followup@​perl.org> wrote​:

I spent an hour or so familiarizing myself with the code and the patch,
and it looks good to me. So with rjbs' approval, I've pushed it as commit
9612f77
--
Karl Williamson

---
via perlbug​: queue​: perl5 status​: open
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=119593

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant