Feature: builtin to apply double-quote interpolation to a scalar #16592

p5pRT · 2018-06-21T09:40:43Z

Migrated from rt.perl.org#133292 (status was 'open')

Searchable as RT133292$

p5pRT · 2018-06-21T09:40:44Z

From @epa

Created by @epa

Sometimes you might want to make a template string with variables, and
interpolate them later.

my $template = 'hello $name';
foreach my $name (qw(Fred Jim)) {
my $s = eval '"' . $template . '"';
die $@ if $@;
say $s;
}

That trick with eval is awkward and unhygienic (what if $template
contains a " character?). You could instead do some s///e magic to
find variable names and interpolate them, but that too is error-prone
and hard to generalize to all the things that doublequote
interpolation supports.

Presumably the perl interpreter has an internal routine that performs
double-quote interpolation on a string. That should be exposed as a
builtin function, so you can say

my $s = interpolate($template);

and get broadly the same semantics as the eval trick above, but
without having to rely on a particular delimiter character that
doesn't appear in the string (or the string not ending in backslash),
and throwing errors rather than leaving them in $@.

I know there are many excellent templating systems out there. This is
for much more basic uses, particularly those where you started off
with a doublequoted string and then wanted to separate the definition
of that bit of code from where it is used.

Perl Info


Flags:
    category=core
    severity=wishlist

Site configuration information for perl 5.22.2:

Configured by Red Hat, Inc. at Fri Nov  4 14:35:02 UTC 2016.

Summary of my perl5 (revision 5 version 22 subversion 2) configuration:
   
  Platform:
    osname=linux, osvers=4.7.9-200.fc24.x86_64, archname=x86_64-linux-thread-multi
    uname='linux buildvm-12.phx2.fedoraproject.org 4.7.9-200.fc24.x86_64 #1 smp thu oct 20 14:26:16 utc 2016 x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Doptimize=none -Dccflags=-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches  -m64 -mtune=generic -Dldflags=-Wl,-z,relro  -Dccdlflags=-Wl,--enable-new-dtags -Wl,-z,relro  -Dlddlflags=-shared -Wl,-z,relro  -Dshrpdir=/usr/lib64 -DDEBUGGING=-g -Dversion=5.22.2 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl5 -Dsitearch=/usr/local/lib64/perl5 -Dprivlib=/usr/share/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl -Darchlib=/usr/lib64/perl5 -Dvendorarch=/usr/lib64/perl5/vendor_perl -Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64 /lib64 /usr/lib64 -Duseshrplib -Dusethreads -Duseithreads -Dusedtrace=/usr/bin/dtrace -Duselargefiles -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/usr/bin -Dusesitecustomize'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fwrapv -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='  -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fwrapv -fno-strict-aliasing -I/usr/local/include'
    ccversion='', gccversion='5.3.1 20160406 (Red Hat 5.3.1-6)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags ='-Wl,-z,relro  -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib64 /lib64 /usr/lib64 /usr/local/lib /usr/lib /lib/../lib64 /usr/lib/../lib64 /lib
    libs=-lpthread -lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -lresolv -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.22.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.22'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,--enable-new-dtags -Wl,-z,relro '
    cccdlflags='-fPIC', lddlflags='-shared -Wl,-z,relro  -L/usr/local/lib -fstack-protector-strong'

Locally applied patches:
    Fedora Patch1: Removes date check, Fedora/RHEL specific
    Fedora Patch3: support for libdir64
    Fedora Patch4: use libresolv instead of libbind
    Fedora Patch5: USE_MM_LD_RUN_PATH
    Fedora Patch6: Skip hostname tests, due to builders not being network capable
    Fedora Patch7: Dont run one io test due to random builder failures
    Fedora Patch15: Define SONAME for libperl.so
    Fedora Patch16: Install libperl.so to -Dshrpdir value
    Fedora Patch22: Document Math::BigInt::CalcEmu requires Math::BigInt (CPAN RT#85015)
    Fedora Patch26: Make *DBM_File desctructors thread-safe (RT#61912)
    Fedora Patch27: Make PadlistNAMES() lvalue again (CPAN RT#101063)
    Fedora Patch28: Make magic vtable writable as a work-around for Coro (CPAN RT#101063)
    Fedora Patch29: Fix duplicating PerlIO::encoding when spawning threads (RT#31923)
    Fedora Patch30: Do not let XSLoader load relative paths (CVE-2016-6185)
    Fedora Patch31: Avoid loading optional modules from default . (CVE-2016-1238)
    Fedora Patch32: Fix a crash in lexical scope warnings (RT#128597)
    Fedora Patch33: Do not mangle errno from failed socket calls (RT#128316)
    Fedora Patch34: Fix crash in "evalbytes S" (RT#129196)
    Fedora Patch35: Fix crash in "evalbytes S" (RT#129196)
    Fedora Patch36: Fix crash in "evalbytes S" (RT#129196)
    Fedora Patch37: Fix crash in splice (RT#129164, RT#129166, RT#129167)
    Fedora Patch38: Fix string overrun in Perl_gv_fetchmethod_pvn_flags (RT#129267)
    Fedora Patch39: Fix string overrun in Perl_gv_fetchmethod_pvn_flags (RT#129267)
    Fedora Patch40: Fix string overrun in Perl_gv_fetchmethod_pvn_flags (RT#129267)
    Fedora Patch41: Fix string overrun in Perl_gv_fetchmethod_pvn_flags (RT#129267)
    Fedora Patch42: Fix string overrun in Perl_gv_fetchmethod_pvn_flags (RT#129267)
    Fedora Patch43: Fix crash when matching UTF-8 string with non-UTF-8 substrings (RT#129350)
    Fedora Patch44: Fix parsing perl options in shell bang line (RT#129336)
    Fedora Patch45: Fix firstchar bitmap under UTF-8 with prefix optimization (RT#129950)
    Fedora Patch46: Avoid infinite loop in h2xs tool if enum and type have the same name (RT130001)
    Fedora Patch47: Fix stack handling when calling chdir without an argument (RT#129130)
    Fedora Patch200: Link XS modules to libperl.so with EU::CBuilder on Linux
    Fedora Patch201: Link XS modules to libperl.so with EU::MM on Linux


@INC for perl 5.22.2:
    /home/eda/lib64/perl5/
    /usr/local/lib64/perl5
    /usr/local/share/perl5
    /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl
    /usr/lib64/perl5
    /usr/share/perl5


Environment for perl 5.22.2:
    HOME=/home/eda
    LANG=en_GB.UTF-8
    LANGUAGE (unset)
    LC_COLLATE=C
    LC_CTYPE=en_GB.UTF-8
    LC_MESSAGES=en_GB.UTF-8
    LC_MONETARY=en_GB.UTF-8
    LC_NUMERIC=en_GB.UTF-8
    LC_TIME=en_GB.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/eda/bin:/home/eda/bin:/usr/local/bin:/usr/bin:/sbin:/usr/sbin:/sbin:/usr/sbin
    PERL5LIB=/home/eda/lib64/perl5/
    PERL_BADLANG (unset)
    SHELL=/bin/bash

p5pRT · 2018-06-25T07:45:45Z

From @iabyn

On Thu, Jun 21, 2018 at 02:40:44AM -0700, Ed Avis (via RT) wrote:

Sometimes you might want to make a template string with variables, and
interpolate them later.

my $template = 'hello $name';
foreach my $name (qw(Fred Jim)) {
my $s = eval '"' . $template . '"';
die $@ if $@;
say $s;
}

That trick with eval is awkward and unhygienic (what if $template
contains a " character?). You could instead do some s///e magic to
find variable names and interpolate them, but that too is error-prone
and hard to generalize to all the things that doublequote
interpolation supports.

Presumably the perl interpreter has an internal routine that performs
double-quote interpolation on a string. That should be exposed as a
builtin function, so you can say
my $s = interpolate$$template$;

Well, not really. The lexer has code to understand a double-quoted string,
and alter what stream of tokens it passes to the parser. So

"result=$foo[$x + 1]\n"

gets lexed into the stream of tokens

STRINGIFY
'('
THING('result=')
ADDOP(concat)
'$'
PRIVATEREF(padany)
'['
'$'
BAREWORD('x')
ADDOP('+')
THING(1)
']'
ADDOP(concat)
THING("\n")

so the parser thinks that the source code was actually

'result=' . $foo[$x + 1] . "\n".

There is no simple function that can be exposed.

--
A walk of a thousand miles begins with a single step...
then continues for another 1,999,999 or so.

p5pRT · 2018-06-25T07:45:45Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2018-06-25T08:43:44Z

From @epa

Thanks. I forgot that string interpolation is converted to concatenation at compile time. So the way to implement an interpolate() operator would be to make a new parser at run time and parse the argument's value as a doublequoted string, then evaluate that. It would still be preferable to the trick with eval() because you don't need to worry about the delimiter character appearing in the string.

p5pRT · 2018-06-25T10:06:16Z

From @iabyn

On Mon, Jun 25, 2018 at 01:43:44AM -0700, Ed Avis via RT wrote:

Thanks. I forgot that string interpolation is converted to
concatenation at compile time. So the way to implement an interpolate()
operator would be to make a new parser at run time and parse the
argument's value as a doublequoted string, then evaluate that. It would
still be preferable to the trick with eval() because you don't need to
worry about the delimiter character appearing in the string.

It would have to be implemented using the underlying eval mechanism, which
would still be very slow.

--
Counsellor Troi states something other than the blindingly obvious.
-- Things That Never Happen in "Star Trek" #16

p5pRT · 2018-06-25T10:06:39Z

From @iabyn

On Mon, Jun 25, 2018 at 01:43:44AM -0700, Ed Avis via RT wrote:

Thanks. I forgot that string interpolation is converted to
concatenation at compile time. So the way to implement an interpolate()
operator would be to make a new parser at run time and parse the
argument's value as a doublequoted string, then evaluate that. It would
still be preferable to the trick with eval() because you don't need to
worry about the delimiter character appearing in the string.

It would have to be implemented using the underlying eval mechanism, which
would still be very slow.

--
Counsellor Troi states something other than the blindingly obvious.
-- Things That Never Happen in "Star Trek" #16

p5pRT · 2018-08-13T01:25:53Z

From @tonycoz

On Mon, 25 Jun 2018 01:43:44 -0700, ed wrote:

Thanks. I forgot that string interpolation is converted to
concatenation at compile time. So the way to implement an
interpolate() operator would be to make a new parser at run time and
parse the argument's value as a doublequoted string, then evaluate
that. It would still be preferable to the trick with eval() because
you don't need to worry about the delimiter character appearing in the
string.

You could just forbid some unusual character from the source. The
obvious choice is NUL:

$source =~ /\0/ and die;
my $result = eval "qq\0$source\0";

Tony

khwilliamson · 2022-04-17T19:41:50Z

Is this closable?

iabyn · 2022-04-20T09:04:06Z

I think so

p5pRT added Wishlist affects-5.22 distro-Linux type-core labels Oct 19, 2019

xenu removed the affects-5.22 label Nov 19, 2021

demerphq closed this as completed Apr 20, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature: builtin to apply double-quote interpolation to a scalar #16592

Feature: builtin to apply double-quote interpolation to a scalar #16592

p5pRT commented Jun 21, 2018

p5pRT commented Jun 21, 2018

p5pRT commented Jun 25, 2018

p5pRT commented Jun 25, 2018

p5pRT commented Jun 25, 2018

p5pRT commented Jun 25, 2018

p5pRT commented Jun 25, 2018

p5pRT commented Aug 13, 2018

khwilliamson commented Apr 17, 2022

iabyn commented Apr 20, 2022

Feature: builtin to apply double-quote interpolation to a scalar #16592

Feature: builtin to apply double-quote interpolation to a scalar #16592

Comments

p5pRT commented Jun 21, 2018

p5pRT commented Jun 21, 2018

From @epa

Created by @epa

p5pRT commented Jun 25, 2018

From @iabyn

p5pRT commented Jun 25, 2018

p5pRT commented Jun 25, 2018

From @epa

p5pRT commented Jun 25, 2018

From @iabyn

p5pRT commented Jun 25, 2018

From @iabyn

p5pRT commented Aug 13, 2018

From @tonycoz

khwilliamson commented Apr 17, 2022

iabyn commented Apr 20, 2022