Skip Menu |
Report information
Id: 23813
Status: resolved
Priority: 0/
Queue: perl5

Owner: jkeenan <jkeenan [at] cpan.org>
Requestors: ncleaton <nick [at] cleaton.net>
Cc:
AdminCc:

Operating System: All
PatchStatus: Applied
Severity: low
Type: docs
Perl Version:
  • 5.8.0
  • 5.25.2
Fixed In: 5.27.1



To: perlbug [...] perl.org
Subject: misleading example in perldoc -f flock
CC: nick [...] cleaton.net
From: Nick Cleaton <nick [...] cleaton.net>
Date: Mon, 15 Sep 2003 19:41:17 +0100
Download (untitled) / with headers
text/plain 4.5k
This is a bug report for perl from nick@cleaton.net, generated with the help of perlbug 1.34 running under perl v5.8.0. ----------------------------------------------------------------- [Please enter your report here] The output of "perldoc -f flock" includes the following example code: Here's a mailbox appender for BSD systems. use Fcntl ':flock'; # import LOCK_* constants sub lock { flock(MBOX,LOCK_EX); # and, in case someone appended # while we were waiting... seek(MBOX, 0, 2); } sub unlock { flock(MBOX,LOCK_UN); } open(MBOX, ">>/usr/spool/mail/$ENV{'USER'}") or die "Can't open mailbox: $!"; lock(); print MBOX $msg,"\n\n"; unlock(); ... which appears to have been written by someone under the misapprehension that opening a file for append is the same as opening for write and then seeking to the end. This is not the case. When a file is opened in append mode, all writes are appended to the then end of file, irrespective of the current file position. From fopen(3) on FreeBSD: ``a'' Open for writing. The file is created if it does not exist. The stream is positioned at the end of the file. Subsequent writes to the file will always end up at the then current end of file, irrespective of any intervening fseek(3) or similar. and the corresponding extract from open(2): Opening a file with O_APPEND set causes each write on the file to be appended to the end. I've confirmed this behavior on FreeBSD by adding a sleep to the example and running several copies of it in parallel. The messages are all appended to the end with the seek() commented out, or even replaced with a seek() to the middle or the start of the file. Reading this example a couple of years ago left me with a false impression of how O_APPEND works, and I've only just now stumbled across the truth. [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=docs severity=low --- Site configuration information for perl v5.8.0: Configured by nick at Mon Jul 22 22:30:59 BST 2002. Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration: Platform: osname=freebsd, osvers=4.6-release-p1, archname=i386-freebsd uname='freebsd lt1.cleaton.net 4.6-release-p1 freebsd 4.6-release-p1 #0: mon jul 1 11:11:48 bst 2002 root@:asyncobjasyncsrcsyslaptop i386 ' config_args='-des -Dprefix=/u/usr/local/perl-5.8.0' hint=recommended, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -I/usr/local/include', optimize='-O', cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -I/usr/local/include' ccversion='', gccversion='2.95.3 20010315 (release) [FreeBSD]', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags ='-Wl,-E -L/usr/local/lib' libpth=/usr/lib /usr/local/lib libs=-lgdbm -lm -lc -lcrypt -lutil perllibs=-lm -lc -lcrypt -lutil libc=, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' ' cccdlflags='-DPIC -fpic', lddlflags='-shared -L/usr/local/lib' Locally applied patches: --- @INC for perl v5.8.0: /u/usr/local/perl-5.8.0/lib/5.8.0/i386-freebsd /u/usr/local/perl-5.8.0/lib/5.8.0 /u/usr/local/perl-5.8.0/lib/site_perl/5.8.0/i386-freebsd /u/usr/local/perl-5.8.0/lib/site_perl/5.8.0 /u/usr/local/perl-5.8.0/lib/site_perl . --- Environment for perl v5.8.0: HOME=/home/nick LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin:/home/nick/bin:/home/local/bin:/home/local/bin:/home/local/bin PERL_BADLANG (unset) SHELL=/usr/local/bin/bash
To: perl5-porters [...] perl.org
Subject: Re: [perl #23813] misleading example in perldoc -f flock
Date: Mon, 15 Sep 2003 16:17:19 -0400
From: Mark Jason Dominus <mjd [...] plover.com>
RT-Send-Cc:
Download (untitled) / with headers
text/plain 672b
Show quoted text
> ... which appears to have been written by someone under the > misapprehension that opening a file for append is the same as > opening for write and then seeking to the end.
If memory serves me correctly, on old unix systems, it *was* the same; the special semantics of append mode weren't introduced until later. In fact, if memory serves, the mode argument of open() was originally 0, 1 or 2; to indicate reading, writing, or reading+writing; the special O_APPEND (and related) flags and the semantics they represent didn't exist until later. Whether any systems with the old semantics still exist, or, of they exist, whether they are still relevant, I don't know.
To: perlbug [...] perl.com
Subject: Re: [perl #23813] misleading example in perldoc -f flock
Date: Mon, 15 Sep 2003 16:32:05 -0400
From: Mark Jason Dominus <mjd [...] plover.com>
RT-Send-Cc:
Download (untitled) / with headers
text/plain 1.2k
Rochkind (_Advanced Unix Programming_) says that O_APPEND was introduced in System III (released 1982). Presumably it didn't get into BSD until sometime after that. The example in question does specify that it's "a mailbox appender for BSD systems". I don't know if the author (probably Tom C.) was thinking of the O_APPEND issue or if the BSDness refers only to the fact that the mailbox is in /usr/spool/mail/$USER and that flock-style locking is used to prevent concurrent access. I'll ask Tom what he was thinking. I said: Show quoted text
>
> > ... which appears to have been written by someone under the > > misapprehension that opening a file for append is the same as > > opening for write and then seeking to the end.
> > If memory serves me correctly, on old unix systems, it *was* the same; > the special semantics of append mode weren't introduced until later. > In fact, if memory serves, the mode argument of open() was originally > 0, 1 or 2; to indicate reading, writing, or reading+writing; the > special O_APPEND (and related) flags and the semantics they represent > didn't exist until later. > > Whether any systems with the old semantics still exist, or, of they > exist, whether they are still relevant, I don't know. >
To: perl5-porters [...] perl.org
CC: bugs-bitbucket [...] netlabs.develooper.com
Subject: Re: [perl #23813] misleading example in perldoc -f flock
From: Gisle Aas <gisle [...] ActiveState.com>
Date: 22 Sep 2003 04:58:26 -0700
RT-Send-Cc:
Download (untitled) / with headers
text/plain 1.4k
"nick@cleaton.net (via RT)" <perlbug-followup@perl.org> writes: Show quoted text
> The output of "perldoc -f flock" includes the following example > code: > > > Here's a mailbox appender for BSD systems. > > use Fcntl ':flock'; # import LOCK_* constants > > sub lock { > flock(MBOX,LOCK_EX); > # and, in case someone appended > # while we were waiting... > seek(MBOX, 0, 2); > } > > sub unlock { > flock(MBOX,LOCK_UN); > } > > open(MBOX, ">>/usr/spool/mail/$ENV{'USER'}") > or die "Can't open mailbox: $!"; > > lock(); > print MBOX $msg,"\n\n"; > unlock(); > > > ... which appears to have been written by someone under the > misapprehension that opening a file for append is the same as > opening for write and then seeking to the end. > > This is not the case. When a file is opened in append mode, all > writes are appended to the then end of file, irrespective of the > current file position.
The lock is still needed here since a single print might result in multiple low level writes. Potentially the seek() could be removed from the lock function. Is this what you suggest? It also seems like this example need to make sure to flush the MBOX before it unlocks the handle. It would also be wise to check the outcome of the system calls. Regards, Gisle
Date: Mon, 22 Sep 2003 13:48:48 +0100
From: Nick Cleaton <nick [...] cleaton.net>
To: Gisle Aas <perlbug-followup [...] perl.org>
Subject: Re: [perl #23813] misleading example in perldoc -f flock
RT-Send-Cc:
Download (untitled) / with headers
text/plain 1.8k
On Mon, Sep 22, 2003 at 11:59:33AM -0000, Gisle Aas wrote: Show quoted text
> "nick@cleaton.net (via RT)" <perlbug-followup@perl.org> writes: >
> > The output of "perldoc -f flock" includes the following example > > code: > > > > > > Here's a mailbox appender for BSD systems. > > > > use Fcntl ':flock'; # import LOCK_* constants > > > > sub lock { > > flock(MBOX,LOCK_EX); > > # and, in case someone appended > > # while we were waiting... > > seek(MBOX, 0, 2); > > } > > > > sub unlock { > > flock(MBOX,LOCK_UN); > > } > > > > open(MBOX, ">>/usr/spool/mail/$ENV{'USER'}") > > or die "Can't open mailbox: $!"; > > > > lock(); > > print MBOX $msg,"\n\n"; > > unlock(); > > > > > > ... which appears to have been written by someone under the > > misapprehension that opening a file for append is the same as > > opening for write and then seeking to the end. > > > > This is not the case. When a file is opened in append mode, all > > writes are appended to the then end of file, irrespective of the > > current file position.
> > The lock is still needed here since a single print might result in > multiple low level writes. Potentially the seek() could be removed > from the lock function. Is this what you suggest?
Either that or adjust the comment so that it notes that the seek isn't needed on modern unices: flock(MBOX,LOCK_EX); # and, in case we're running on a very old UNIX # variant without the modern O_APPEND semantics... seek(MBOX, 0, 2); Show quoted text
> It also seems like this example need to make sure to flush the MBOX > before it unlocks the handle. It would also be wise to check the > outcome of the system calls.
Yes. -- Nick
To: perl5-porters [...] perl.org
Subject: Re: [perl #23813] misleading example in perldoc -f flock
Date: Mon, 22 Sep 2003 09:15:36 -0400
From: Mark Jason Dominus <mjd [...] plover.com>
RT-Send-Cc:
Download (untitled) / with headers
text/plain 611b
Gisle Aas <gisle@ActiveState.com>: Show quoted text
> The lock is still needed here since a single print might result in > multiple low level writes. Potentially the seek() could be removed > from the lock function. Is this what you suggest?
I believe that Nick was suggesting that the seek could be removed. Show quoted text
> It also seems like this example need to make sure to flush the MBOX > before it unlocks the handle.
I thought so too, but the manual says that that is now longer necessary: To avoid the possibility of miscoordination, Perl now flushes FILEHANDLE before locking or unlocking it.
To: Mark Jason Dominus <mjd [...] plover.com>
CC: perl5-porters [...] perl.org
Subject: Re: [perl #23813] misleading example in perldoc -f flock
From: Gisle Aas <gisle [...] ActiveState.com>
Date: 22 Sep 2003 07:07:12 -0700
RT-Send-Cc:
Download (untitled) / with headers
text/plain 447b
Mark Jason Dominus <mjd@plover.com> writes: Show quoted text
> > It also seems like this example need to make sure to flush the MBOX > > before it unlocks the handle.
> > I thought so too, but the manual says that that is now longer necessary: > > To avoid the possibility of miscoordination, Perl now flushes > FILEHANDLE before locking or unlocking it.
Cool. In what version can we depend on this behaviour? Regards, Gisle
Date: Mon, 22 Sep 2003 10:45:00 -0400
From: Rick Delaney <rick [...] bort.ca>
To: Gisle Aas <gisle [...] ActiveState.com>
CC: perl5-porters [...] perl.org
Subject: Re: [perl #23813] misleading example in perldoc -f flock
RT-Send-Cc:
Download (untitled) / with headers
text/plain 531b
On Mon, Sep 22, 2003 at 07:07:12AM -0700, Gisle Aas wrote: Show quoted text
> Mark Jason Dominus <mjd@plover.com> writes: >
> > To avoid the possibility of miscoordination, Perl now flushes > > FILEHANDLE before locking or unlocking it.
> > Cool. In what version can we depend on this behaviour?
IIRC, 5.004. perl5004delta.pod confirms my recollection: =item flock is now supported on more platforms, prefers fcntl to lockf when emulating, and always flushes before (un)locking. -- Rick Delaney rick@bort.ca
To: perl5-porters [...] perl.org
Subject: Re: [perl #23813] misleading example in perldoc -f flock
Date: Mon, 22 Sep 2003 11:10:48 -0400
From: Mark Jason Dominus <mjd [...] plover.com>
RT-Send-Cc:
Download (untitled) / with headers
text/plain 513b
Gisle Aas <gisle@ActiveState.com>: Show quoted text
> Mark Jason Dominus <mjd@plover.com> writes: >
> > > It also seems like this example need to make sure to flush the MBOX > > > before it unlocks the handle.
> > > > I thought so too, but the manual says that that is now longer necessary: > > > > To avoid the possibility of miscoordination, Perl now flushes > > FILEHANDLE before locking or unlocking it.
> > Cool. In what version can we depend on this behaviour?
Starting in 5.005_03.
To: perl5-porters [...] perl.org
Subject: Re: [perl #23813] misleading example in perldoc -f flock
Date: Mon, 22 Sep 2003 11:12:44 -0400
From: Mark Jason Dominus <mjd [...] plover.com>
RT-Send-Cc:
Download (untitled) / with headers
text/plain 377b
Mark Jason Dominus <mjd@plover.com>: Show quoted text
> Gisle Aas <gisle@ActiveState.com>:
> > > > Cool. In what version can we depend on this behaviour?
> > Starting in 5.005_03. > > >
Rick Delaney said 5.004. He is probably right. I based my conclusion on the appearance of the guarantee in the perlfunc manual. But the feature could of course have been in Perl earlier than that.
To: perl5-porters [...] perl.org
Subject: Re: [perl #23813] misleading example in perldoc -f flock
From: merlyn [...] stonehenge.com (Randal L. Schwartz)
Date: 22 Sep 2003 10:01:10 -0700
RT-Send-Cc:
Download (untitled) / with headers
text/plain 1.2k
Show quoted text
>>>>> "Nick@Cleaton" == Nick@Cleaton Net <perlbug-followup@perl.org> writes:
Nick@Cleaton> ... which appears to have been written by someone under the Nick@Cleaton> misapprehension that opening a file for append is the same as Nick@Cleaton> opening for write and then seeking to the end. Nick@Cleaton> This is not the case. When a file is opened in append mode, all Nick@Cleaton> writes are appended to the then end of file, irrespective of the Nick@Cleaton> current file position. From fopen(3) on FreeBSD: It was *indeed* precisely the case on V7 Unix. The example probably even dates back to when Perl ran on V7 Unix and early BSD (*real* BSD, not foo-BSD) versions. Even after you opened for append, another process could extend the file, and you'd be writing in the middle. Wasn't safe to seek until after the flock(). However, I'd be hard-pressed to find a modern Unix on which append-mode didn't include the forced-seek mode. -- Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095 <merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/> Perl/Unix/security consulting, Technical writing, Comedy, etc. etc. See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 399b
On Mon Sep 22 10:01:30 2003, merlyn@stonehenge.com wrote: Show quoted text
> >>>>> "Nick@Cleaton" == Nick@Cleaton Net <perlbug-followup@perl.org>
> writes: >
The example found in Perl 5.16.0 under 'perldoc -f flock' (excerpt attached) has been revised since the OP filed this ticket nine years ago. Could Nick@Cleaton and the other commenters review this and see where we stand? Thank you very much. Jim Keenan
Here's a mailbox appender for BSD systems. use Fcntl qw(:flock SEEK_END); # import LOCK_* and SEEK_END constants sub lock { my ($fh) = @_; flock($fh, LOCK_EX) or die "Cannot lock mailbox - $!\n"; # and, in case someone appended while we were waiting... seek($fh, 0, SEEK_END) or die "Cannot seek - $!\n"; } sub unlock { my ($fh) = @_; flock($fh, LOCK_UN) or die "Cannot unlock mailbox - $!\n"; } open(my $mbox, ">>", "/usr/spool/mail/$ENV{'USER'}") or die "Can't open mailbox: $!"; lock($mbox); print $mbox $msg,"\n\n"; unlock($mbox);
CC: perl5-porters [...] perl.org
Subject: Re: [perl #23813] misleading example in perldoc -f flock
Date: Sat, 26 May 2012 20:44:11 -0600
To: perlbug-followup [...] perl.org
From: Tom Christiansen <tchrist [...] perl.com>
Download (untitled) / with headers
text/plain 320b
So I think what that's saying is that the seek is immaterial because of the append mode on the open, which I believe is correct. That means there doesn't need to be lock/unlock functions to do niceties, but one can just call flock directly. Well, there's still the buffering concern on some systems, I suppose. --tom
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 525b
On Sat May 26 19:44:41 2012, tom christiansen wrote: Show quoted text
> So I think what that's saying is that the seek is immaterial > because of the append mode on the open, which I believe is > correct. That means there doesn't need to be lock/unlock > functions to do niceties, but one can just call flock directly. > Well, there's still the buffering concern on some systems, I suppose. > > --tom >
Tom, does that mean that this nearly ten-year-old RT is closable? (It appears closable to me, FWIW.) Thank you very much. Jim Keenan
Download (untitled) / with headers
text/plain 222b
I agree with Nick's complaint and with the other observations in this thread. The seek may not be necessary, but this comment at least explains why, whereas removing the seek doesn't teach anyone anything. Patch attached.
Subject: 0001-RT-23813-perlfunc.pod-explain-seek-in-O_APPEND.patch
From c2bde2eb1618d5608461cb065d688f7dc76a5d0e Mon Sep 17 00:00:00 2001 From: Dan Collins <dcollinsn@gmail.com> Date: Tue, 5 Jul 2016 19:02:12 -0400 Subject: [PATCH] [RT #23813] perlfunc.pod: explain seek in O_APPEND In all modern systems, seek has no effect in O_APPEND. As explained by Nick Cleaton at that RT, the comment was written in a time when: ...opening a file for append [was] the same as opening for write and then seeking to the end. This is not the case. When a file is opened in append mode, all writes are appended to the then end of file, irrespective of the current file position. Corrected comment was contributed by him. --- pod/perlfunc.pod | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pod/perlfunc.pod b/pod/perlfunc.pod index 5c778f1..36eb96b 100644 --- a/pod/perlfunc.pod +++ b/pod/perlfunc.pod @@ -2653,8 +2653,8 @@ Here's a mailbox appender for BSD systems. sub lock { my ($fh) = @_; flock($fh, LOCK_EX) or die "Cannot lock mailbox - $!\n"; - - # and, in case someone appended while we were waiting... + # and, in case we're running on a very old UNIX + # variant without the modern O_APPEND semantics... seek($fh, 0, SEEK_END) or die "Cannot seek - $!\n"; } -- 2.8.1
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 482b
On Tue, 05 Jul 2016 23:13:20 GMT, dcollinsn@gmail.com wrote: Show quoted text
> I agree with Nick's complaint and with the other observations in this > thread. The seek may not be necessary, but this comment at least > explains why, whereas removing the seek doesn't teach anyone anything. > > Patch attached.
List: Is the documentation change to perlfunc proposed by Dan Collins acceptable? If so, is this 13-year-old ticket closable? Thank you very much. -- James E Keenan (jkeenan@cpan.org)
Subject: Re: [perl #23813] [PATCH] misleading example in perldoc -f flock
From: Dave Mitchell <davem [...] iabyn.com>
To: James E Keenan via RT <perlbug-followup [...] perl.org>
Date: Wed, 29 Mar 2017 12:14:05 +0100
CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 692b
On Sun, Feb 26, 2017 at 02:58:55PM -0800, James E Keenan via RT wrote: Show quoted text
> On Tue, 05 Jul 2016 23:13:20 GMT, dcollinsn@gmail.com wrote:
> > I agree with Nick's complaint and with the other observations in this > > thread. The seek may not be necessary, but this comment at least > > explains why, whereas removing the seek doesn't teach anyone anything. > > > > Patch attached.
> > List: Is the documentation change to perlfunc proposed by Dan Collins > acceptable?
Yes, but lets apply it after 5.26.... Show quoted text
> If so, is this 13-year-old ticket closable?
...and then close the ticket. -- Indomitable in retreat, invincible in advance, insufferable in victory -- Churchill on Montgomery
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 847b
On Wed, 29 Mar 2017 11:14:36 GMT, davem wrote: Show quoted text
> On Sun, Feb 26, 2017 at 02:58:55PM -0800, James E Keenan via RT wrote:
> > On Tue, 05 Jul 2016 23:13:20 GMT, dcollinsn@gmail.com wrote:
> > > I agree with Nick's complaint and with the other observations in this > > > thread. The seek may not be necessary, but this comment at least > > > explains why, whereas removing the seek doesn't teach anyone anything. > > > > > > Patch attached.
> > > > List: Is the documentation change to perlfunc proposed by Dan Collins > > acceptable?
> > Yes, but lets apply it after 5.26.... >
> > If so, is this 13-year-old ticket closable?
> > ...and then close the ticket. >
Pushed to blead in commit 5c3085ee8edac26825ef44056ea5aba97493d7ab. Marking ticket Resolved. (It only took 13 years!) Thank you very much. -- James E Keenan (jkeenan@cpan.org)


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org