Skip Menu |
Report information

To: perlbug [...] perl.org
Date: Wed, 12 Dec 2018 17:41:01 -0500
Subject: What do we mean by "paragraph mode"?
From: James E Keenan <jkeenan [...] pobox.com>
Download (untitled) / with headers
text/plain 2.9k
Summary: This ticket replaces RT 133703 and proposes improved documentation and testing for the so-called "paragraph mode", i.e., processing of records in a file while $/ set to the empty string. I. Background: In https://rt.perl.org/Ticket/Display.html?id=133703, I submitted a patch to sv.c intended to address a "Comparison result is always the same" warning reported by LTGM.com analysis of the Perl 5 core distribution. Dave Mitchell pointed out where the patch was wrong and, in the course of his discussion, mentioned that the section of sv.c at issue governed so-called "paragraph mode", that is, what happens when you read a file when the input record separator has been set to an empty string: ##### $/ = ''; ##### While poking around in the code and the test suite, I became convinced: (a) that I didn't understand "paragraph mode" very well; (b) that "paragraph mode" wasn't well documented; (c) that "paragraph mode" wasn't thoroughly tested in the core distribution's test suite; and (d) that as a consequence of (b) and (c), it might contain bugs. I spent several days working on this. I no longer think "paragraph mode" has bugs, but I'm more convinced that it is under-documented and under-tested. In this RT I propose better documentation of paragraph mode and additional tests. II. Paragraph Mode as Currently Found in the Core Distribution From this point forward I'll treat "paragraph mode" and "setting $/ to an empty string" as equivalent. In the core distribution $/ (or $INPUT_RECORD_SEPARATOR) is defined in pod/perlvar.pod and is mainly discussed in perlfaq5 and perlfaq6. Please see this attachment: ##### paragraph_mode_in_core_distribution.pod ##### ... for a thorough discussion of paragraph mode in the core distribution, including instances in code found in cpan/, ext/, dist/ and lib/. You can also find this discussion on the web at: ##### http://thenceforward.net/perl/misc/paragraph_mode_in_core_distribution.html ##### III. Proposed Additional Documentation I believe that the best place to put additional discussion of paragraph mode will be in perlfaq5. Please review the patch attached ##### 0001-More-detailed-explanation-of-paragraph-mode.patch ##### Note that perlfaq5 is maintained upstream on CPAN, so once P5P is on-board with the patch, I will submit it as a github issue at https://github.com/perl-doc-cats/perlfaq/issues. IV. Proposed Additional Testing Please see the program attached: ##### paragraph_mode.t ##### To facilitate discussion, I've written this test program in a modern style using Test::More, File::Temp, CPAN module Data::Dump and subroutines. When we bring this into the core distribution, I'll adapt it for inclusion under, say, t/op/, where I'll have to use only the testing functions provided by t/test.pl. What is important for discussion now is: Do these tests thoroughly cover what we mean by "paragraph mode"? Thank you very much. Jim Keenan

Message body not shown because it is not plain text.

Message body is not shown because sender requested not to inline it.

Download paragraph_mode.t
text/plain 13.1k

Message body is not shown because sender requested not to inline it.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 2.5k
On Wed, 12 Dec 2018 14:41:11 -0800, jkeenan@pobox.com wrote: Show quoted text
> Summary: This ticket replaces RT 133703 and proposes improved > documentation and testing for the so-called "paragraph mode", i.e., > processing of records in a file while $/ set to the empty string. > > I. Background: > > In https://rt.perl.org/Ticket/Display.html?id=133703, I submitted a > patch to sv.c intended to address a "Comparison result is always the > same" warning reported by LTGM.com analysis of the Perl 5 core > distribution. Dave Mitchell pointed out where the patch was wrong and, > in the course of his discussion, mentioned that the section of sv.c at > issue governed so-called "paragraph mode", that is, what happens when > you read a file when the input record separator has been set to an empty > string: > > ##### > $/ = ''; > ##### > > While poking around in the code and the test suite, I became convinced: > > (a) that I didn't understand "paragraph mode" very well; > > (b) that "paragraph mode" wasn't well documented; > > (c) that "paragraph mode" wasn't thoroughly tested in the core > distribution's test suite; and > > (d) that as a consequence of (b) and (c), it might contain bugs. > > I spent several days working on this. I no longer think "paragraph > mode" has bugs, but I'm more convinced that it is under-documented and > under-tested. In this RT I propose better documentation of paragraph > mode and additional tests.
If the behaviour of <> with $/ = "" is unclear, any improvements to the documentation belong in either the documentation for <> (aka readline in pod/perlfunc.pod) or for $/ ($/ in pod/perlvar.pod), not in perlfaq. Given the other reference documentation of readline/$/ combined behaviour is in perlvar, it probably belongs there. Show quoted text
> To facilitate discussion, I've written this test program in a modern > style using Test::More, File::Temp, CPAN module Data::Dump and > subroutines. When we bring this into the core distribution, I'll adapt > it for inclusion under, say, t/op/, where I'll have to use only the > testing functions provided by t/test.pl. > > What is important for discussion now is: Do these tests thoroughly > cover what we mean by "paragraph mode"?
One suggestion I'd make for the tests is to include a brief description of the test case in the is()/is_deeply() calls, since normal default test failure out includes the name of the test - it doesn't include the note() output. In your case you might pass a test name prefix to perform_tests() and include that as part of the name supplied to is()/is_deeply(). Tony
From: James E Keenan <jkeenan [...] pobox.com>
Subject: Re: [perl #133722] What do we mean by "paragraph mode"?
Date: Thu, 13 Dec 2018 18:44:30 -0500
To: perlbug-followup [...] perl.org
Download (untitled) / with headers
text/plain 2.7k
On 12/12/18 11:11 PM, Tony Cook via RT wrote: Show quoted text
> On Wed, 12 Dec 2018 14:41:11 -0800, jkeenan@pobox.com wrote:
>> Summary: This ticket replaces RT 133703 and proposes improved >> documentation and testing for the so-called "paragraph mode", i.e., >> processing of records in a file while $/ set to the empty string. >> >> I. Background: >> >> In https://rt.perl.org/Ticket/Display.html?id=133703, I submitted a >> patch to sv.c intended to address a "Comparison result is always the >> same" warning reported by LTGM.com analysis of the Perl 5 core >> distribution. Dave Mitchell pointed out where the patch was wrong and, >> in the course of his discussion, mentioned that the section of sv.c at >> issue governed so-called "paragraph mode", that is, what happens when >> you read a file when the input record separator has been set to an empty >> string: >> >> ##### >> $/ = ''; >> ##### >> >> While poking around in the code and the test suite, I became convinced: >> >> (a) that I didn't understand "paragraph mode" very well; >> >> (b) that "paragraph mode" wasn't well documented; >> >> (c) that "paragraph mode" wasn't thoroughly tested in the core >> distribution's test suite; and >> >> (d) that as a consequence of (b) and (c), it might contain bugs. >> >> I spent several days working on this. I no longer think "paragraph >> mode" has bugs, but I'm more convinced that it is under-documented and >> under-tested. In this RT I propose better documentation of paragraph >> mode and additional tests.
> > If the behaviour of <> with $/ = "" is unclear, any improvements to the documentation belong in either the documentation for <> (aka readline in pod/perlfunc.pod) or for $/ ($/ in pod/perlvar.pod), not in perlfaq. Given the other reference documentation of readline/$/ combined behaviour is in perlvar, it probably belongs there. >
>> To facilitate discussion, I've written this test program in a modern >> style using Test::More, File::Temp, CPAN module Data::Dump and >> subroutines. When we bring this into the core distribution, I'll adapt >> it for inclusion under, say, t/op/, where I'll have to use only the >> testing functions provided by t/test.pl. >> >> What is important for discussion now is: Do these tests thoroughly >> cover what we mean by "paragraph mode"?
> > One suggestion I'd make for the tests is to include a brief description of the test case in the is()/is_deeply() calls, since normal default test failure out includes the name of the test - it doesn't include the note() output. > > In your case you might pass a test name prefix to perform_tests() and include that as part of the name supplied to is()/is_deeply(). > > Tony >
Please review the two new patches attached. 0001-More-specific-documentation-of-paragraph-mode.patch 0002-Thoroughly-test-paragraph-mode.patch Thank you very much. Jim Keenan

Message body is not shown because sender requested not to inline it.

Message body is not shown because sender requested not to inline it.

RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 236b
On Thu, 13 Dec 2018 15:44:48 -0800, jkeenan@pobox.com wrote: Show quoted text
> Please review the two new patches attached. > > 0001-More-specific-documentation-of-paragraph-mode.patch > 0002-Thoroughly-test-paragraph-mode.patch >
That's fine. Tony
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 486b
On Wed, 19 Dec 2018 03:09:34 GMT, tonyc wrote: Show quoted text
> On Thu, 13 Dec 2018 15:44:48 -0800, jkeenan@pobox.com wrote:
> > Please review the two new patches attached. > > > > 0001-More-specific-documentation-of-paragraph-mode.patch > > 0002-Thoroughly-test-paragraph-mode.patch > >
> > That's fine. > > Tony
Merged to blead in commits 440af013a4cf7804b7a9560b14d5e569e8ad9c09 and bf8c3686f6dbea493e3c066a78983bba9ad605fb. Will monitor for a few days. -- James E Keenan (jkeenan@cpan.org)
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 604b
On Wed, 19 Dec 2018 14:46:07 GMT, jkeenan wrote: Show quoted text
> On Wed, 19 Dec 2018 03:09:34 GMT, tonyc wrote:
> > On Thu, 13 Dec 2018 15:44:48 -0800, jkeenan@pobox.com wrote:
> > > Please review the two new patches attached. > > > > > > 0001-More-specific-documentation-of-paragraph-mode.patch > > > 0002-Thoroughly-test-paragraph-mode.patch > > >
> > > > That's fine. > > > > Tony
> > Merged to blead in commits 440af013a4cf7804b7a9560b14d5e569e8ad9c09 > and bf8c3686f6dbea493e3c066a78983bba9ad605fb. > > Will monitor for a few days.
No failures observed; resolving ticket. -- James E Keenan (jkeenan@cpan.org)
Download (untitled) / with headers
text/plain 313b
Thank you for filing this report. You have helped make Perl better. With the release today of Perl 5.30.0, this and 160 other issues have been resolved. Perl 5.30.0 may be downloaded via: https://metacpan.org/release/XSAWYERX/perl-5.30.0 If you find that the problem persists, feel free to reopen this ticket.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org