Skip Menu |
Report information
Id: 131663
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: andreas.koenig.7os6VVqR [at] franz.ak.mind.de
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: unknown
Perl Version: (no value)
Fixed In: (no value)



Subject: Bleadperl v5.27.0-132-g13f4dd346e breaks JDDPAUSE/MarpaX-Languages-M4-0.017.tar.gz
To: perlbug [...] perl.org
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Date: Tue, 27 Jun 2017 08:00:56 +0200
CC: slaven [...] rezic.de
Download (untitled) / with headers
text/plain 4.5k
Slaven digged it out, the second on this day, delightfully hard to track. bisect ------ commit 13f4dd346e6f3b61534a20f246de3a80b3feb743 Author: Abigail <abigail@abigail.be> Date: Tue Jun 6 18:51:37 2017 +0200 Forbid out of range Unicode code points. diagnostics ----------- Note: no evidence available for threaded perls; you need not try because of https://rt.cpan.org/Ticket/Display.html?id=109841 : t/m4.t ................. 73/? : : # Failed test 'regexp - GNU emacs' : # at t/m4.t line 96. : # got: ' : # : # : # : # ' : # expected: '5 : # -1 : # *** Unix *** nix *** : # : # ' : : # Failed test 'patsubst' : # at t/m4.t line 96. : # got: 'OBS: GNUs not Unix : # OBS: GNUs OBS: not OBS: Unix : # : # : # GN not : # GNUs NOT Unix : # ' : # expected: 'OBS: GNUs not Unix : # OBS: GNUs OBS: not OBS: Unix : # (GNUs)() (not)() (Unix)() : # (GNUs) (not) (Unix) : # GN not : # GNUs NOT Unix : # ' : : # Failed test 'patsubst - composites upcase/downcase/capitalize' : # at t/m4.t line 96. : # got: ' : # GNUS NOT UNIX : # gnus not unix : # : # ' : # expected: ' : # GNUS NOT UNIX : # gnus not unix : # Gnus Not Unix : # ' : t/m4.t ................. 120/? : # Looks like you failed 3 tests of 141. : t/m4.t ................. Dubious, test returned 3 (wstat 768, 0x300) : Failed 3/141 subtests perl -V ------- Summary of my perl5 (revision 5 version 27 subversion 1) configuration: Commit id: 13f4dd346e6f3b61534a20f246de3a80b3feb743 Platform: osname=linux osvers=4.9.0-2-amd64 archname=x86_64-linux uname='linux k93msid 4.9.0-2-amd64 #1 smp debian 4.9.18-1 (2017-03-30) x86_64 gnulinux ' config_args='-Dprefix=/home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.27.0-132-g13f4dd346e/89ad -Dmyhostname=k93msid -Dinstallusrbinperl=n -Uversiononly -Dusedevel -des -Ui_db -Dlibswanted=cl pthread socket inet nsl gdbm dbm malloc dl ld sun m crypt sec util c cposix posix ucb BSD gdbm_compat -Uuseithreads -Uuselongdouble -DDEBUGGING=-g' hint=recommended useposix=true d_sigaction=define useithreads=undef usemultiplicity=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='cc' ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' optimize='-O2 -g' cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='' gccversion='6.3.0 20170406' gccosandvers='' intsize=4 longsize=8 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long' ivsize=8 nvtype='double' nvsize=8 Off_t='off_t' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='cc' ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/6/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.24.so so=so useshrplib=false libperl=libperl.a gnulibc_version='2.24' Dynamic Linking: dlsrc=dl_dlopen.xs dlext=so d_dlsymun=undef ccdlflags='-Wl,-E' cccdlflags='-fPIC' lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector-strong' Characteristics of this binary (from libperl): Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_MALLOC_WRAP PERL_OP_PARENT PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF Built under linux Compiled at Jun 27 2017 05:08:23 @INC: /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.27.0-132-g13f4dd346e/89ad/lib/site_perl/5.27.1/x86_64-linux /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.27.0-132-g13f4dd346e/89ad/lib/site_perl/5.27.1 /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.27.0-132-g13f4dd346e/89ad/lib/5.27.1/x86_64-linux /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.27.0-132-g13f4dd346e/89ad/lib/5.27.1 -- andreas
RT-Send-CC: perl5-porters [...] perl.org
I'll take a look at this. -- Karl Williamson
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 4.1k
On Tue, 27 Jun 2017 12:27:03 -0700, khw wrote: Show quoted text
> I'll take a look at this.
I have traced this down to these lines in the distro: use re::engine::GNU 0.019; $regexp = qr/$regexpString/sm; no re::engine::GNU; And, the GNU engine as interpreted by re::engine::GNU is calling back to Perl's utf8_to_uvchr() with code points that are no longer legal because they are above IV_MAX, in particular 0xFF...FF80. (I lost track of how many F's, and its not important enough to figure it out.) If I use Perl's engine by commenting out the 'use re...', this error doesn't happen, though other failures occur, due presumably to the differences in semantics in GNU's. A backtrace yields: (gdb) bt #0 Perl_utf8n_to_uvchr_error (s=0x7fffffffc307 "", curlen=13, retlen=0x0, flags=0, errors=0x7fffffffc244) at utf8.c:1630 #1 0x000055555587f9e8 in Perl__force_out_malformed_utf8_message (p=0x7fffffffc2fa "\377\200\217\277\277\277\277\277\277\277\277\276\200", e=0x7fffffffc308 "", flags=0, die_here=true) at utf8.c:90 #2 0x000055555588476b in S_is_utf8_common_with_len (p=0x7fffffffc2fa "\377\200\217\277\277\277\277\277\277\277\277\276\200", e=0x7fffffffc308 "", swash=0x555555be0860 <PL_utf8_swash_ptrs>, swashname=0x5555559994a0 "XPosixWord", invlist=0x555555be2f50) at utf8.c:2816 #3 0x00005555558852dc in Perl__is_utf8_FOO_with_len (classnum=0 '\000', p=0x7fffffffc2fa "\377\200\217\277\277\277\277\277\277\277\277\276\200", e=0x7fffffffc308 "") at utf8.c:2975 #4 0x000055555588333d in Perl__is_uni_FOO (classnum=0 '\000', c=18446744073709551488) at utf8.c:2460 #5 0x00007fffeec0d10a in rpl_Perl_isalnum (c=18446744073709551488) at regex_internal.h:1101 #6 0x00007fffeec19fd5 in build_charclass (trans=0x0, sbcset=0x55555add9250, mbcset=0x55555aeb4fe0, char_class_alloc=0x7fffffffc3f8, class_name=0x7fffeec2aa84 "alnum", syntax=0) at regcomp.c:3576 #7 0x00007fffeec1a903 in build_charclass_op (dfa=0x55555aebe080, trans=0x0, class_name=0x7fffeec2aa84 "alnum", extra=0x7fffeec2b0c4 "_", non_match=false, err=0x7fffffffc624) at regcomp.c:3645 #8 0x00007fffeec183a8 in parse_expression (regexp=0x7fffffffc630, preg=0x55555ae5a278, token=0x7fffffffc5c0, syntax=64, nest=0, err=0x7fffffffc624) at regcomp.c:2428 #9 0x00007fffeec17be0 in parse_branch (regexp=0x7fffffffc630, preg=0x55555ae5a278, token=0x7fffffffc5c0, syntax=64, nest=0, err=0x7fffffffc624) at regcomp.c:2237 #10 0x00007fffeec179fc in parse_reg_exp (regexp=0x7fffffffc630, preg=0x55555ae5a278, token=0x7fffffffc5c0, syntax=64, nest=0, err=0x7fffffffc624) at regcomp.c:2189 #11 0x00007fffeec178da in parse (regexp=0x7fffffffc630, preg=0x55555ae5a278, syntax=64, err=0x7fffffffc624) at regcomp.c:2158 #12 0x00007fffeec14563 in re_compile_internal (preg=0x55555ae5a278, pattern=0x55555aea1e80 "\\<[a-z]\\w+", length=10, syntax=64, is_utf8=true) at regcomp.c:813 #13 0x00007fffeec2645f in GNU_comp (pattern=0x555558cf3000, flags=259) at GNU.xs:439 #14 0x0000555555674dd8 in Perl_re_op_compile (patternp=0x55555a24db30, pat_count=1, expr=0x0, eng=0x7fffeee2f340 <engine_GNU>, old_re=0x55555ac02138, is_bare_re=0x7fffffffd02a, orig_rx_flags=259, pm_flags=1073742083) at regcomp.c:7016 #15 0x00005555557de49e in Perl_pp_regcomp () at pp_ctl.c:108 #16 0x00005555556cdbdf in Perl_runops_debug () at dump.c:2465 #17 0x00005555555c8e45 in S_run_body (oldscope=1) at perl.c:2543 #18 0x00005555555c83fd in perl_run (my_perl=0x555555be1010) at perl.c:2471 #19 0x0000555555580a9e in main (argc=3, argv=0x7fffffffd418, env=0x7fffffffd438) at perlmain.c:123 Stack position #5 0x00007fffeec0d10a in rpl_Perl_isalnum (c=18446744073709551488) indicates that the re::engine function rpl_Perl_isalnum() is being called with a code point (c=) that is now illegal in perl. It calls the perl function Perl__is_uni_FOO, which no longer accepts this input value. So this is the problem. Perl calls the substitute engine in frame #13, and it calls functions within itself until the call at #4. (It has its own regcomp.c, so that isn't ours in those frames.) At this point, I'm uncertain what to do. I'm pretty sure this isn't Perl's bug. We deprecated use of these code points for 2 cycles, and now they're illegal. -- Karl Williamson
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 394b
On Wed, 26 Jul 2017 16:31:02 GMT, khw wrote: Show quoted text
> > At this point, I'm uncertain what to do. I'm pretty sure this isn't > Perl's bug. We deprecated use of these code points for 2 cycles, and > now they're illegal.
If that's the case, then can you open a bug ticket at https://rt.cpan.org/Dist/Display.html?Name=MarpaX-Languages-M4? Thank you very much. -- James E Keenan (jkeenan@cpan.org)
Subject: Re: [perl #131663] Bleadperl v5.27.0-132-g13f4dd346e breaks JDDPAUSE/MarpaX-Languages-M4-0.017.tar.gz
From: Karl Williamson <public [...] khwilliamson.com>
Date: Sat, 29 Jul 2017 11:57:06 -0600
CC: perl5-porters [...] perl.org
To: perlbug-followup [...] perl.org
Download (untitled) / with headers
text/plain 488b
On 07/26/2017 04:53 PM, James E Keenan via RT wrote: Show quoted text
> On Wed, 26 Jul 2017 16:31:02 GMT, khw wrote: >
>> >> At this point, I'm uncertain what to do. I'm pretty sure this isn't >> Perl's bug. We deprecated use of these code points for 2 cycles, and >> now they're illegal.
> > If that's the case, then can you open a bug ticket at https://rt.cpan.org/Dist/Display.html?Name=MarpaX-Languages-M4? > > Thank you very much. >
Now done https://rt.cpan.org/Ticket/Display.html?id=122641
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 140b
The owner has already issued a fix for this, in re::engine::GNU 0.022. I have tested it, and the test no longer fails. -- Karl Williamson


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org