From Ulrich.Windl@rz.uni-regensburg.de

Hi!

I think this is a bug in perl 5.18.2 (as shipped in SLES12 SP4)​:

Debugging a sort routine I wrote myself I get an error that I don't understand, most likely because the line number reported (maybe even the whole error message) is wrong​:

First the effect (generic_sort is my sorting routine)​:

  DB<1> x generic_sort('', 'a')
'','a'
Argument "" isn't numeric in sort at sapgw-log-condenser.pl line 132.
at sapgw-log-condenser.pl line 132.
  main​::generic_sort('', 'a') called at (eval 7)[/usr/lib/perl5/5.18.2/perl5db.pl​:732] line 2
  eval 'no strict; ($@​, $!, $^E, $,, $/, $\\, $^W) = @​DB​::saved;package main; $^D = $^D | $DB​::db_stop;
  generic_sort(\'\', \'a\');
' called at /usr/lib/perl5/5.18.2/perl5db.pl line 732
  DB​::eval called at /usr/lib/perl5/5.18.2/perl5db.pl line 3096
  DB​::DB called at sapgw-log-condenser.pl line 15
0 ''
1 'a'
  DB<2> l 132
132​: print "'$a','$b'\n";

Then, even more confusing, I wrote a simpler test caser of my routine, named "_sort" where I deliberately forced an error​:

  DB<3> x _sort('', 'a')
'','a'
Argument "a" isn't numeric in numeric comparison (<=>) at sapgw-log-condenser.pl line 178.
at sapgw-log-condenser.pl line 178.
  main​::_sort('', 'a') called at (eval 8)[/usr/lib/perl5/5.18.2/perl5db.pl​:732] line 2
  eval 'no strict; ($@​, $!, $^E, $,, $/, $\\, $^W) = @​DB​::saved;package main; $^D = $^D | $DB​::db_stop;
  _sort(\'\', \'a\');
' called at /usr/lib/perl5/5.18.2/perl5db.pl line 732
  DB​::eval called at /usr/lib/perl5/5.18.2/perl5db.pl line 3096
  DB​::DB called at sapgw-log-condenser.pl line 15
Argument "" isn't numeric in numeric comparison (<=>) at sapgw-log-condenser.pl line 178.
at sapgw-log-condenser.pl line 178.
  main​::_sort('', 'a') called at (eval 8)[/usr/lib/perl5/5.18.2/perl5db.pl​:732] line 2
  eval 'no strict; ($@​, $!, $^E, $,, $/, $\\, $^W) = @​DB​::saved;package main; $^D = $^D | $DB​::db_stop;
  _sort(\'\', \'a\');
' called at /usr/lib/perl5/5.18.2/perl5db.pl line 732
  DB​::eval called at /usr/lib/perl5/5.18.2/perl5db.pl line 3096
  DB​::DB called at sapgw-log-condenser.pl line 15
0 ''
1 'a'
  DB<4> l 178
178​: return $a <=> $b;
  DB<5>

So what you see is that in the first case the line reported cannot trigger the error; even the lines around it can't.
In contrast the second example reports the correct line.

Here are the relevant lines from the source (prefixed by the line numbers)​:

  15 require 5.018_000;
  16 use strict;
  17 use English;

  129 sub generic_sort(@​)
  130 {
  131 return sort {
  132 print "'$a','$b'\n";
  133 if ($a eq $b) { # trivial case
  134 return 0;
  135 } else {
  136 my ($c, $d) = ($a, $b);
  137
  138 while ($c && $d) {
... the only occurrence of a numeric comparison is this, protected by some regular expression matches​:
  151 if ((my $result = $cm <=> $dm) != 0) {
  152 return $result;
...
  166 }
  167 }
  168 } @​_;
  169 }

  171 sub _sort(@​)
  172 {
  173 return sort {
  174 print "'$a','$b'\n";
  175 if ($a eq $b) { # trivial case
  176 return 0;
  177 } else {
  178 return $a <=> $b; # force an error
  179 }
  180 } @​_;
  181 }

The print in line 132 was added AFTER I had observed the error. Before adding line 132, the error line reported also was 132.
Also I had added a print statement that should output before <=> is ever reached, but the print never happens.

More obscure when I set a breakpoint at line 132, and then step through​:
main​::(sapgw-log-condenser.pl​:15)​: require 5.018_000;
  DB<1> b 133
  DB<2> x generic_sort('', 'a')
initial​: '','a'
main​::generic_sort(sapgw-log-condenser.pl​:133)​:
133​: if ($a eq $b) { # trivial case
  DB<<3>> n
main​::generic_sort(sapgw-log-condenser.pl​:136)​:
136​: my ($c, $d) = ($a, $b);
  DB<<3>> n
main​::generic_sort(sapgw-log-condenser.pl​:138)​:
138​: while ($c && $d) {
  DB<<3>> n
Argument "" isn't numeric in sort at sapgw-log-condenser.pl line 132.
...

P.S.​: I did not post my sorting routine, as the code is probably not correct yet, but still the bug described does not make it easier for me to debug and fix.

perl -V
Summary of my perl5 (revision 5 version 18 subversion 2) configuration​:

  Platform​:
  osname=linux, osvers=3.12.61-52.141-default, archname=x86_64-linux-thread-multi
  uname='linux sheep08 3.12.61-52.141-default #1 smp tue aug 14 07​:09​:09 utc 2018 (b9c454e) x86_64 x86_64 x86_64 gnulinux '
  config_args='-ds -e -Dprefix=/usr -Dvendorprefix=/usr -Dinstallusrbinperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm -Dd_dbm_open -Duseshrplib=true -Doptimize=-fmessage-length=0 -grecord-gcc-switches -fstack-protector -O2 -Wall -D_FORTIFY_SOURCE=2 -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -g -Wall -pipe -Accflags=-DPERL_USE_SAFE_PUTENV -Dotherlibdirs=/usr/lib/perl5/site_perl -Dinc_version_list=5.18.0/x86_64-linux-thread-multi 5.18.0 5.18.1/x86_64-linux-thread-multi 5.18.1'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-fmessage-length=0 -grecord-gcc-switches -fstack-protector -O2 -Wall -D_FORTIFY_SOURCE=2 -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -g -Wall -pipe',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector'
  ccversion='', gccversion='4.8.5', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -L/usr/local/lib64 -fstack-protector'
  libpth=/lib64 /usr/lib64 /usr/local/lib64
  libs=-lm -ldl -lcrypt -lpthread
  perllibs=-lm -ldl -lcrypt -lpthread
  libc=/lib64/libc-2.19.so, so=so, useshrplib=true, libperl=libperl.so
  gnulibc_version='2.19'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/usr/lib/perl5/5.18.2/x86_64-linux-thread-multi/CORE'
  cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib64 -fstack-protector'

Characteristics of this binary (from libperl)​:
  Compile-time options​: HAS_TIMES MULTIPLICITY PERLIO_LAYERS
  PERL_DONT_CREATE_GVSV
  PERL_HASH_FUNC_ONE_AT_A_TIME_HARD
  PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
  PERL_PRESERVE_IVUV PERL_SAWAMPERSAND
  PERL_USE_SAFE_PUTENV USE_64_BIT_ALL USE_64_BIT_INT
  USE_ITHREADS USE_LARGE_FILES USE_LOCALE
  USE_LOCALE_COLLATE USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC USE_PERLIO USE_PERL_ATOF
  USE_REENTRANT_API
  Built under linux
  Compiled at Aug 17 2018 14​:16​:32
  @​INC​:
  /usr/lib/perl5/site_perl/5.18.2/x86_64-linux-thread-multi
  /usr/lib/perl5/site_perl/5.18.2
  /usr/lib/perl5/vendor_perl/5.18.2/x86_64-linux-thread-multi
  /usr/lib/perl5/vendor_perl/5.18.2
  /usr/lib/perl5/5.18.2/x86_64-linux-thread-multi
  /usr/lib/perl5/5.18.2
  /usr/lib/perl5/site_perl
  .

From @tonycoz

On Thu, 25 Jul 2019 01​:20​:48 -0700, Ulrich.Windl@​rz.uni-regensburg.de wrote​:

Hi!

I think this is a bug in perl 5.18.2 (as shipped in SLES12 SP4)​:

Debugging a sort routine I wrote myself I get an error that I don't
understand, most likely because the line number reported (maybe even
the whole error message) is wrong​:

Could your sort comparison block be returning "" ?

$ perl5.18.3 -wle 'my @​x = sort { "" } qw(1 2 3)'
Argument "" isn't numeric in sort at -e line 1.

Tony

The RT System itself - Status changed from 'new' to 'open'

From Ulrich.Windl@rz.uni-regensburg.de

"Tony Cook via RT" <perlbug-followup@​perl.org> schrieb am 06.08.2019 um 08​:16
in Nachricht <rt-4.0.24-9652-1565072215-957.134312-94-0@​perl.org>​:
On Thu, 25 Jul 2019 01​:20​:48 -0700, Ulrich.Windl@​rz.uni-regensburg.de wrote​:
Hi!

I think this is a bug in perl 5.18.2 (as shipped in SLES12 SP4)​:

Debugging a sort routine I wrote myself I get an error that I don't
understand, most likely because the line number reported (maybe even
the whole error message) is wrong​:

Could your sort comparison block be returning "" ?

It could be that some if..else branch was incomplete and did not return an explicit value.
Actually what I did in the meantime was to convert the block to use a subroutine. Then the line numbers were correct, and I was able to complete the routine.

The code now looks like this​:
# generic comparer
sub generic_cmp($)
{
  my $num_re = shift;

  return sub ($$) {
  my ($a, $b) = @​_;
  my $result = $a cmp $b;

  while ($result != 0) {
...
  }
  return $result;
  };
}

sub generic_sort(@​)
{
  my $cmp = generic_cmp(RE_UNSIGNED_INT);

  return sort { $cmp->($a, $b) } @​_;
}

Note​: It may be less efficient, but at least it works and is debuggable ;-)

$ perl5.18.3 -wle 'my @​x = sort { "" } qw(1 2 3)'
Argument "" isn't numeric in sort at -e line 1.

Tony

From @tonycoz

On Tue, 06 Aug 2019 00​:32​:16 -0700, Ulrich.Windl@​rz.uni-regensburg.de wrote​:

"Tony Cook via RT" <perlbug-followup@​perl.org> schrieb am
06.08.2019 um 08​:16
in Nachricht <rt-4.0.24-9652-1565072215-957.134312-94-0@​perl.org>​:
On Thu, 25 Jul 2019 01​:20​:48 -0700, Ulrich.Windl@​rz.uni-regensburg.de
wrote​:
Hi!

I think this is a bug in perl 5.18.2 (as shipped in SLES12 SP4)​:

Debugging a sort routine I wrote myself I get an error that I don't
understand, most likely because the line number reported (maybe even
the whole error message) is wrong​:

Could your sort comparison block be returning "" ?

It could be that some if..else branch was incomplete and did not
return an explicit value.
Actually what I did in the meantime was to convert the block to use a
subroutine. Then the line numbers were correct, and I was able to
complete the routine.

Do you still consider this an issue?

Note that perl isn't complaining about a numeric comparison in your generic_sort() case, but does in the _sort() case, so I do think it was the return value of the sort block causing the problem.

In which case the line number is only off by one, possibly because the parser is looking ahead.

Tony

From Ulrich.Windl@rz.uni-regensburg.de

"Tony Cook via RT" <perlbug-followup@​perl.org> schrieb am 08.08.2019 um 07​:19
in Nachricht <rt-4.0.24-561-1565241581-1461.134312-94-0@​perl.org>​:
On Tue, 06 Aug 2019 00​:32​:16 -0700, Ulrich.Windl@​rz.uni-regensburg.de wrote​:

"Tony Cook via RT" <perlbug-followup@​perl.org> schrieb am
06.08.2019 um 08​:16
in Nachricht <rt-4.0.24-9652-1565072215-957.134312-94-0@​perl.org>​:
On Thu, 25 Jul 2019 01​:20​:48 -0700, Ulrich.Windl@​rz.uni-regensburg.de
wrote​:
Hi!

I think this is a bug in perl 5.18.2 (as shipped in SLES12 SP4)​:

Debugging a sort routine I wrote myself I get an error that I don't
understand, most likely because the line number reported (maybe even
the whole error message) is wrong​:

Could your sort comparison block be returning "" ?

It could be that some if..else branch was incomplete and did not
return an explicit value.
Actually what I did in the meantime was to convert the block to use a
subroutine. Then the line numbers were correct, and I was able to
complete the routine.

Do you still consider this an issue?

Note that perl isn't complaining about a numeric comparison in your
generic_sort() case, but does in the _sort() case, so I do think it was the
return value of the sort block causing the problem.

In which case the line number is only off by one, possibly because the
parser is looking ahead.

Tony

Well, I think the original error message with the (wrong) line number is confusing a lot, just as the attempt to single-step the code in the debugger was. Having to use a named subroutine just to debug the case seems non-optimal.
I suspect the true cause for the issue is the perl compiler creating incorrect line number labels for the code.