Skip Menu |
Report information
Id: 133292
Status: open
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: eda [at] waniasset.com
Cc:
AdminCc:

Operating System: Linux
PatchStatus: (no value)
Severity: Wishlist
Type: core
Perl Version: 5.22.2
Fixed In: (no value)



To: <perlbug [...] perl.org>
Date: Thu, 21 Jun 2018 10:40:04 +0100
Subject: Feature: builtin to apply double-quote interpolation to a scalar
From: <eda [...] waniasset.com>
Download (untitled) / with headers
text/plain 8.1k
This is a bug report for perl from eda@waniasset.com, generated with the help of perlbug 1.40 running under perl 5.22.2. ----------------------------------------------------------------- [Please describe your issue here] Sometimes you might want to make a template string with variables, and interpolate them later. my $template = 'hello $name'; foreach my $name (qw(Fred Jim)) { my $s = eval '"' . $template . '"'; die $@ if $@; say $s; } That trick with eval is awkward and unhygienic (what if $template contains a " character?). You could instead do some s///e magic to find variable names and interpolate them, but that too is error-prone and hard to generalize to all the things that doublequote interpolation supports. Presumably the perl interpreter has an internal routine that performs double-quote interpolation on a string. That should be exposed as a builtin function, so you can say my $s = interpolate($template); and get broadly the same semantics as the eval trick above, but without having to rely on a particular delimiter character that doesn't appear in the string (or the string not ending in backslash), and throwing errors rather than leaving them in $@. I know there are many excellent templating systems out there. This is for much more basic uses, particularly those where you started off with a doublequoted string and then wanted to separate the definition of that bit of code from where it is used. [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=wishlist --- Site configuration information for perl 5.22.2: Configured by Red Hat, Inc. at Fri Nov 4 14:35:02 UTC 2016. Summary of my perl5 (revision 5 version 22 subversion 2) configuration: Platform: osname=linux, osvers=4.7.9-200.fc24.x86_64, archname=x86_64-linux-thread-multi uname='linux buildvm-12.phx2.fedoraproject.org 4.7.9-200.fc24.x86_64 #1 smp thu oct 20 14:26:16 utc 2016 x86_64 x86_64 x86_64 gnulinux ' config_args='-des -Doptimize=none -Dccflags=-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -Dldflags=-Wl,-z,relro -Dccdlflags=-Wl,--enable-new-dtags -Wl,-z,relro -Dlddlflags=-shared -Wl,-z,relro -Dshrpdir=/usr/lib64 -DDEBUGGING=-g -Dversion=5.22.2 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5 -Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl -Darchlib=/usr/lib64/perl5 -Dvendorarch=/usr/lib64/perl5/vendor_perl -Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Duseshrplib -Dusethreads -Duseithreads -Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/usr/bin -Dusesitecustomize' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fwrapv -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize=' -g', cppflags='-D_REENTRANT -D_GNU_SOURCE -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fwrapv -fno-strict-aliasing -I/usr/local/include' ccversion='', gccversion='5.3.1 20160406 (Red Hat 5.3.1-6)', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='gcc', ldflags ='-Wl,-z,relro -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib64 /lib64 /usr/lib64 /usr/local/lib /usr/lib /lib/../lib64 /usr/lib/../lib64 /lib libs=-lpthread -lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat perllibs=-lpthread -lresolv -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.22.so, so=so, useshrplib=true, libperl=libperl.so gnulibc_version='2.22' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,--enable-new-dtags -Wl,-z,relro ' cccdlflags='-fPIC', lddlflags='-shared -Wl,-z,relro -L/usr/local/lib -fstack-protector-strong' Locally applied patches: Fedora Patch1: Removes date check, Fedora/RHEL specific Fedora Patch3: support for libdir64 Fedora Patch4: use libresolv instead of libbind Fedora Patch5: USE_MM_LD_RUN_PATH Fedora Patch6: Skip hostname tests, due to builders not being network capable Fedora Patch7: Dont run one io test due to random builder failures Fedora Patch15: Define SONAME for libperl.so Fedora Patch16: Install libperl.so to -Dshrpdir value Fedora Patch22: Document Math::BigInt::CalcEmu requires Math::BigInt (CPAN RT#85015) Fedora Patch26: Make *DBM_File desctructors thread-safe (RT#61912) Fedora Patch27: Make PadlistNAMES() lvalue again (CPAN RT#101063) Fedora Patch28: Make magic vtable writable as a work-around for Coro (CPAN RT#101063) Fedora Patch29: Fix duplicating PerlIO::encoding when spawning threads (RT#31923) Fedora Patch30: Do not let XSLoader load relative paths (CVE-2016-6185) Fedora Patch31: Avoid loading optional modules from default . (CVE-2016-1238) Fedora Patch32: Fix a crash in lexical scope warnings (RT#128597) Fedora Patch33: Do not mangle errno from failed socket calls (RT#128316) Fedora Patch34: Fix crash in "evalbytes S" (RT#129196) Fedora Patch35: Fix crash in "evalbytes S" (RT#129196) Fedora Patch36: Fix crash in "evalbytes S" (RT#129196) Fedora Patch37: Fix crash in splice (RT#129164, RT#129166, RT#129167) Fedora Patch38: Fix string overrun in Perl_gv_fetchmethod_pvn_flags (RT#129267) Fedora Patch39: Fix string overrun in Perl_gv_fetchmethod_pvn_flags (RT#129267) Fedora Patch40: Fix string overrun in Perl_gv_fetchmethod_pvn_flags (RT#129267) Fedora Patch41: Fix string overrun in Perl_gv_fetchmethod_pvn_flags (RT#129267) Fedora Patch42: Fix string overrun in Perl_gv_fetchmethod_pvn_flags (RT#129267) Fedora Patch43: Fix crash when matching UTF-8 string with non-UTF-8 substrings (RT#129350) Fedora Patch44: Fix parsing perl options in shell bang line (RT#129336) Fedora Patch45: Fix firstchar bitmap under UTF-8 with prefix optimization (RT#129950) Fedora Patch46: Avoid infinite loop in h2xs tool if enum and type have the same name (RT130001) Fedora Patch47: Fix stack handling when calling chdir without an argument (RT#129130) Fedora Patch200: Link XS modules to libperl.so with EU::CBuilder on Linux Fedora Patch201: Link XS modules to libperl.so with EU::MM on Linux --- @INC for perl 5.22.2: /home/eda/lib64/perl5/ /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 --- Environment for perl 5.22.2: HOME=/home/eda LANG=en_GB.UTF-8 LANGUAGE (unset) LC_COLLATE=C LC_CTYPE=en_GB.UTF-8 LC_MESSAGES=en_GB.UTF-8 LC_MONETARY=en_GB.UTF-8 LC_NUMERIC=en_GB.UTF-8 LC_TIME=en_GB.UTF-8 LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/home/eda/bin:/home/eda/bin:/usr/local/bin:/usr/bin:/sbin:/usr/sbin:/sbin:/usr/sbin PERL5LIB=/home/eda/lib64/perl5/ PERL_BADLANG (unset) SHELL=/bin/bash
To: perl5-porters [...] perl.org
Date: Mon, 25 Jun 2018 08:45:01 +0100
Subject: Re: [perl #133292] Feature: builtin to apply double-quote interpolation to a scalar
From: Dave Mitchell <davem [...] iabyn.com>
Download (untitled) / with headers
text/plain 1.4k
On Thu, Jun 21, 2018 at 02:40:44AM -0700, Ed Avis (via RT) wrote: Show quoted text
> Sometimes you might want to make a template string with variables, and > interpolate them later. > > my $template = 'hello $name'; > foreach my $name (qw(Fred Jim)) { > my $s = eval '"' . $template . '"'; > die $@ if $@; > say $s; > } > > That trick with eval is awkward and unhygienic (what if $template > contains a " character?). You could instead do some s///e magic to > find variable names and interpolate them, but that too is error-prone > and hard to generalize to all the things that doublequote > interpolation supports. > > Presumably the perl interpreter has an internal routine that performs > double-quote interpolation on a string. That should be exposed as a > builtin function, so you can say > > my $s = interpolate($template);
Well, not really. The lexer has code to understand a double-quoted string, and alter what stream of tokens it passes to the parser. So "result=$foo[$x + 1]\n" gets lexed into the stream of tokens STRINGIFY '(' THING('result=') ADDOP(concat) '$' PRIVATEREF(padany) '[' '$' BAREWORD('x') ADDOP('+') THING(1) ']' ADDOP(concat) THING("\n") so the parser thinks that the source code was actually 'result=' . $foo[$x + 1] . "\n". There is no simple function that can be exposed. -- A walk of a thousand miles begins with a single step... then continues for another 1,999,999 or so.
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 402b
Thanks. I forgot that string interpolation is converted to concatenation at compile time. So the way to implement an interpolate() operator would be to make a new parser at run time and parse the argument's value as a doublequoted string, then evaluate that. It would still be preferable to the trick with eval() because you don't need to worry about the delimiter character appearing in the string.
Date: Mon, 25 Jun 2018 11:05:16 +0100
From: Dave Mitchell <davem [...] iabyn.com>
CC: perl5-porters [...] perl.org
Subject: Re: [perl #133292] Feature: builtin to apply double-quote interpolation to a scalar
To: Ed Avis via RT <perlbug-followup [...] perl.org>
Download (untitled) / with headers
text/plain 705b
On Mon, Jun 25, 2018 at 01:43:44AM -0700, Ed Avis via RT wrote: Show quoted text
> Thanks. I forgot that string interpolation is converted to > concatenation at compile time. So the way to implement an interpolate() > operator would be to make a new parser at run time and parse the > argument's value as a doublequoted string, then evaluate that. It would > still be preferable to the trick with eval() because you don't need to > worry about the delimiter character appearing in the string.
It would have to be implemented using the underlying eval mechanism, which would still be very slow. -- Counsellor Troi states something other than the blindingly obvious. -- Things That Never Happen in "Star Trek" #16
Date: Mon, 25 Jun 2018 11:05:16 +0100
To: Ed Avis via RT <perlbug-followup [...] perl.org>
From: Dave Mitchell <davem [...] iabyn.com>
Subject: Re: [perl #133292] Feature: builtin to apply double-quote interpolation to a scalar
CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 705b
On Mon, Jun 25, 2018 at 01:43:44AM -0700, Ed Avis via RT wrote: Show quoted text
> Thanks. I forgot that string interpolation is converted to > concatenation at compile time. So the way to implement an interpolate() > operator would be to make a new parser at run time and parse the > argument's value as a doublequoted string, then evaluate that. It would > still be preferable to the trick with eval() because you don't need to > worry about the delimiter character appearing in the string.
It would have to be implemented using the underlying eval mechanism, which would still be very slow. -- Counsellor Troi states something other than the blindingly obvious. -- Things That Never Happen in "Star Trek" #16
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 626b
On Mon, 25 Jun 2018 01:43:44 -0700, ed wrote: Show quoted text
> Thanks. I forgot that string interpolation is converted to > concatenation at compile time. So the way to implement an > interpolate() operator would be to make a new parser at run time and > parse the argument's value as a doublequoted string, then evaluate > that. It would still be preferable to the trick with eval() because > you don't need to worry about the delimiter character appearing in the > string.
You could just forbid some unusual character from the source. The obvious choice is NUL: $source =~ /\0/ and die; my $result = eval "qq\0$source\0"; Tony


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org