From @masak

<hoelzro> I found some behavior using heredocs, and I'm not really
sure what the right thing to do is
<hoelzro> https://gist.github.com/hoelzro/3b1ff9951908c9ce5aa4

(Gist shows the following heredoc with a \n in it, which gives a
de-indentation warning at compile time.)

  my $here = qq​:to/END_TEXT/;
  foo\nbar
  END_TEXT

<hoelzro> it's concerning how leading whitespace removal is handled
when one has an embedded newline via \n in a heredoc
<timotimo> oh, interesting
<timotimo> i hadn't considered newlines that are escaped inside
heredocs like that when i wrote the new dedenting code
<hoelzro> timotimo​: I hadn't considered it either until I was
converting code that used a single non-heredoc line and came across
that =/
<hoelzro> I would file a ticket, but I don't even know if that's a
bug. I don't even know what I expect it should do!
<lucasb> Is there a way to dedent the heredoc literal text before
expanding newline escape sequences?
<hoelzro> TimToady may have to weigh in
<masak> I'm... not sure I feel that's a bug...
<hoelzro> masak​: "not sure" is the thing with me too =/
<masak> I can understand the argument that this \n newline is not part
of the heredoc's actual newlines, but...
<masak> ...I'm also not ready to tear up compiler internals just to
fix this one.
<masak> but let's see if TimToady has some nice insight about it.
<masak> it's kind of related to (but not the same as)
http://strangelyconsistent.org/blog/here-be-heredocs under the heading
"Indentation, revisited"
<colomon> why would you want to embed a \n in a heredoc?
<masak> colomon​: that's a good, but separate question :)
<masak> we're exploring "what happens when you do?"
<masak> I don't believe it should be illegal.
<hoelzro> colomon, masak​: there's no good reason, I just found it interesting
<masak> it is interesting.
<hoelzro> I was converting a long single double quoted string with
multiple embedded newlines into a heredoc while refactoring a program
<masak> I mean, from the user's point of view, a \n in a heredoc isn't
the same as a hard newline in a heredoc.
<hoelzro> right
<colomon> what happens if you embed a string with a newline in it? I
actually do that quite a bit in my (mostly p5) scripts
<masak> colomon​: read the blog post I linked.
<jnthn> I seem to remember I did it one way in the original herdocs
impl in Rakudo, and something was off and got changed (probably what
masak++ is referencing)
<masak> yes.
<TimToady> re \n in heredocs, the dedent should be happening before \n
interpolation, just as it happens before $I-contain-a-newline
interpolation
<TimToady> so re http://irclog.perlgeek.de/perl6/2015-07-03#i_10845432
I'd call it a bug

(URL referencing hoelzro's original example at the top of this report)

<TimToady> recall that dedent is only for the purposes of establishing
a false margin on the literal text, so from the user's perspective it
really needs to happen before any kind of string processing
<TimToady> I can think of several ways to fix it
<TimToady> one is to treat \n interpolation more like $newline interpolation
<TimToady> another is to substitute a sentinel character for \n inside
heredoc parsing, and only dedent after the sentinel (and replace the
sentinel with a real \n)
* masak submits TimToady-approved heredoc \n de-indentation rakudobug

From @jnthn

On Fri Jul 03 12​:29​:34 2015, masak wrote​:

<hoelzro> I found some behavior using heredocs, and I'm not really
sure what the right thing to do is
<hoelzro> https://gist.github.com/hoelzro/3b1ff9951908c9ce5aa4

(Gist shows the following heredoc with a \n in it, which gives a
de-indentation warning at compile time.)

my $here = qq​:to/END_TEXT/;
foo\nbar
END_TEXT

<hoelzro> it's concerning how leading whitespace removal is handled
when one has an embedded newline via \n in a heredoc
<timotimo> oh, interesting
<timotimo> i hadn't considered newlines that are escaped inside
heredocs like that when i wrote the new dedenting code
<hoelzro> timotimo​: I hadn't considered it either until I was
converting code that used a single non-heredoc line and came across
that =/
<hoelzro> I would file a ticket, but I don't even know if that's a
bug. I don't even know what I expect it should do!
<lucasb> Is there a way to dedent the heredoc literal text before
expanding newline escape sequences?
<hoelzro> TimToady may have to weigh in
<masak> I'm... not sure I feel that's a bug...
<hoelzro> masak​: "not sure" is the thing with me too =/
<masak> I can understand the argument that this \n newline is not part
of the heredoc's actual newlines, but...
<masak> ...I'm also not ready to tear up compiler internals just to
fix this one.
<masak> but let's see if TimToady has some nice insight about it.
<masak> it's kind of related to (but not the same as)
http://strangelyconsistent.org/blog/here-be-heredocs under the heading
"Indentation, revisited"
<colomon> why would you want to embed a \n in a heredoc?
<masak> colomon​: that's a good, but separate question :)
<masak> we're exploring "what happens when you do?"
<masak> I don't believe it should be illegal.
<hoelzro> colomon, masak​: there's no good reason, I just found it
interesting
<masak> it is interesting.
<hoelzro> I was converting a long single double quoted string with
multiple embedded newlines into a heredoc while refactoring a program
<masak> I mean, from the user's point of view, a \n in a heredoc isn't
the same as a hard newline in a heredoc.
<hoelzro> right
<colomon> what happens if you embed a string with a newline in it? I
actually do that quite a bit in my (mostly p5) scripts
<masak> colomon​: read the blog post I linked.
<jnthn> I seem to remember I did it one way in the original herdocs
impl in Rakudo, and something was off and got changed (probably what
masak++ is referencing)
<masak> yes.
<TimToady> re \n in heredocs, the dedent should be happening before \n
interpolation, just as it happens before $I-contain-a-newline
interpolation
<TimToady> so re http://irclog.perlgeek.de/perl6/2015-07-03#i_10845432
I'd call it a bug

(URL referencing hoelzro's original example at the top of this report)

<TimToady> recall that dedent is only for the purposes of establishing
a false margin on the literal text, so from the user's perspective it
really needs to happen before any kind of string processing
<TimToady> I can think of several ways to fix it
<TimToady> one is to treat \n interpolation more like $newline
interpolation
<TimToady> another is to substitute a sentinel character for \n inside
heredoc parsing, and only dedent after the sentinel (and replace the
sentinel with a real \n)
* masak submits TimToady-approved heredoc \n de-indentation rakudobug

Turns out there was a neater way than a sentinel​: in the case of being inside a heredoc, code-gen \n and \r\n as an AST node, not just a string. This triggers the same codepaths that deal with interpolation.

Tests in S02-literals/quoting.t.

/jnthn

The RT System itself - Status changed from 'new' to 'open'

@jnthn - Status changed from 'open' to 'resolved'