Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PATCH] misleading example in perldoc -f flock #6767

Closed
p5pRT opened this issue Sep 15, 2003 · 21 comments
Closed

[PATCH] misleading example in perldoc -f flock #6767

p5pRT opened this issue Sep 15, 2003 · 21 comments

Comments

@p5pRT
Copy link

p5pRT commented Sep 15, 2003

Migrated from rt.perl.org#23813 (status was 'resolved')

Searchable as RT23813$

@p5pRT
Copy link
Author

p5pRT commented Sep 15, 2003

From nick@cleaton.net

Created by nick@cleaton.net

The output of "perldoc -f flock" includes the following example
code​:

  Here's a mailbox appender for BSD systems.

  use Fcntl '​:flock'; # import LOCK_* constants

  sub lock {
  flock(MBOX,LOCK_EX);
  # and, in case someone appended
  # while we were waiting...
  seek(MBOX, 0, 2);
  }

  sub unlock {
  flock(MBOX,LOCK_UN);
  }

  open(MBOX, ">>/usr/spool/mail/$ENV{'USER'}")
  or die "Can't open mailbox​: $!";

  lock();
  print MBOX $msg,"\n\n";
  unlock();

... which appears to have been written by someone under the
misapprehension that opening a file for append is the same as
opening for write and then seeking to the end.

This is not the case. When a file is opened in append mode, all
writes are appended to the then end of file, irrespective of the
current file position. From fopen(3) on FreeBSD​:

``a'' Open for writing. The file is created if it does not
  exist. The stream is positioned at the end of the file.
  Subsequent writes to the file will always end up at the
  then current end of file, irrespective of any intervening
  fseek(3) or similar.

and the corresponding extract from open(2)​:

  Opening a file with O_APPEND set causes each write on the
  file to be appended to the end.

I've confirmed this behavior on FreeBSD by adding a sleep to the
example and running several copies of it in parallel. The
messages are all appended to the end with the seek() commented
out, or even replaced with a seek() to the middle or the start of
the file.

Reading this example a couple of years ago left me with a false
impression of how O_APPEND works, and I've only just now stumbled
across the truth.

Perl Info

Flags:
    category=docs
    severity=low

Site configuration information for perl v5.8.0:

Configured by nick at Mon Jul 22 22:30:59 BST 2002.

Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration:
  Platform:
    osname=freebsd, osvers=4.6-release-p1, archname=i386-freebsd
    uname='freebsd lt1.cleaton.net 4.6-release-p1 freebsd 4.6-release-p1 #0: mon jul 1 11:11:48 bst 2002 root@:asyncobjasyncsrcsyslaptop i386 '
    config_args='-des -Dprefix=/u/usr/local/perl-5.8.0'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -I/usr/local/include',
    optimize='-O',
    cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -I/usr/local/include'
    ccversion='', gccversion='2.95.3 20010315 (release) [FreeBSD]', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags ='-Wl,-E  -L/usr/local/lib'
    libpth=/usr/lib /usr/local/lib
    libs=-lgdbm -lm -lc -lcrypt -lutil
    perllibs=-lm -lc -lcrypt -lutil
    libc=, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' '
    cccdlflags='-DPIC -fpic', lddlflags='-shared  -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.8.0:
    /u/usr/local/perl-5.8.0/lib/5.8.0/i386-freebsd
    /u/usr/local/perl-5.8.0/lib/5.8.0
    /u/usr/local/perl-5.8.0/lib/site_perl/5.8.0/i386-freebsd
    /u/usr/local/perl-5.8.0/lib/site_perl/5.8.0
    /u/usr/local/perl-5.8.0/lib/site_perl
    .


Environment for perl v5.8.0:
    HOME=/home/nick
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin:/home/nick/bin:/home/local/bin:/home/local/bin:/home/local/bin
    PERL_BADLANG (unset)
    SHELL=/usr/local/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Sep 15, 2003

From @mjdominus

... which appears to have been written by someone under the
misapprehension that opening a file for append is the same as
opening for write and then seeking to the end.

If memory serves me correctly, on old unix systems, it *was* the same;
the special semantics of append mode weren't introduced until later.
In fact, if memory serves, the mode argument of open() was originally
0, 1 or 2; to indicate reading, writing, or reading+writing; the
special O_APPEND (and related) flags and the semantics they represent
didn't exist until later.

Whether any systems with the old semantics still exist, or, of they
exist, whether they are still relevant, I don't know.

@p5pRT
Copy link
Author

p5pRT commented Sep 15, 2003

From @mjdominus

Rochkind (_Advanced Unix Programming_) says that O_APPEND was
introduced in System III (released 1982). Presumably it didn't get
into BSD until sometime after that.

The example in question does specify that it's "a mailbox appender for
BSD systems". I don't know if the author (probably Tom C.) was
thinking of the O_APPEND issue or if the BSDness refers only to the
fact that the mailbox is in /usr/spool/mail/$USER and that flock-style
locking is used to prevent concurrent access. I'll ask Tom what he
was thinking.

I said​:

... which appears to have been written by someone under the
misapprehension that opening a file for append is the same as
opening for write and then seeking to the end.

If memory serves me correctly, on old unix systems, it *was* the same;
the special semantics of append mode weren't introduced until later.
In fact, if memory serves, the mode argument of open() was originally
0, 1 or 2; to indicate reading, writing, or reading+writing; the
special O_APPEND (and related) flags and the semantics they represent
didn't exist until later.

Whether any systems with the old semantics still exist, or, of they
exist, whether they are still relevant, I don't know.

@p5pRT
Copy link
Author

p5pRT commented Sep 22, 2003

From @gisle

"nick@​cleaton.net (via RT)" <perlbug-followup@​perl.org> writes​:

The output of "perldoc -f flock" includes the following example
code​:

   Here's a mailbox appender for BSD systems\.

       use Fcntl '&#8203;:flock'; \# import LOCK\_\* constants

       sub lock \{
           flock\(MBOX\,LOCK\_EX\);
           \# and\, in case someone appended
           \# while we were waiting\.\.\.
           seek\(MBOX\, 0\, 2\);
       \}

       sub unlock \{
           flock\(MBOX\,LOCK\_UN\);
       \}

       open\(MBOX\, ">>/usr/spool/mail/$ENV\{'USER'\}"\)
               or die "Can't open mailbox&#8203;: $\!";

       lock\(\);
       print MBOX $msg\,"\\n\\n";
       unlock\(\);

... which appears to have been written by someone under the
misapprehension that opening a file for append is the same as
opening for write and then seeking to the end.

This is not the case. When a file is opened in append mode, all
writes are appended to the then end of file, irrespective of the
current file position.

The lock is still needed here since a single print might result in
multiple low level writes. Potentially the seek() could be removed
from the lock function. Is this what you suggest?

It also seems like this example need to make sure to flush the MBOX
before it unlocks the handle. It would also be wise to check the
outcome of the system calls.

Regards,
Gisle

@p5pRT
Copy link
Author

p5pRT commented Sep 22, 2003

From nick@cleaton.net

On Mon, Sep 22, 2003 at 11​:59​:33AM -0000, Gisle Aas wrote​:

"nick@​cleaton.net (via RT)" <perlbug-followup@​perl.org> writes​:

The output of "perldoc -f flock" includes the following example
code​:

   Here's a mailbox appender for BSD systems\.

       use Fcntl '&#8203;:flock'; \# import LOCK\_\* constants

       sub lock \{
           flock\(MBOX\,LOCK\_EX\);
           \# and\, in case someone appended
           \# while we were waiting\.\.\.
           seek\(MBOX\, 0\, 2\);
       \}

       sub unlock \{
           flock\(MBOX\,LOCK\_UN\);
       \}

       open\(MBOX\, ">>/usr/spool/mail/$ENV\{'USER'\}"\)
               or die "Can't open mailbox&#8203;: $\!";

       lock\(\);
       print MBOX $msg\,"\\n\\n";
       unlock\(\);

... which appears to have been written by someone under the
misapprehension that opening a file for append is the same as
opening for write and then seeking to the end.

This is not the case. When a file is opened in append mode, all
writes are appended to the then end of file, irrespective of the
current file position.

The lock is still needed here since a single print might result in
multiple low level writes. Potentially the seek() could be removed
from the lock function. Is this what you suggest?

Either that or adjust the comment so that it notes that the seek isn't
needed on modern unices​:

  flock(MBOX,LOCK_EX);
  # and, in case we're running on a very old UNIX
  # variant without the modern O_APPEND semantics...
  seek(MBOX, 0, 2);

It also seems like this example need to make sure to flush the MBOX
before it unlocks the handle. It would also be wise to check the
outcome of the system calls.

Yes.

--
Nick

@p5pRT
Copy link
Author

p5pRT commented Sep 22, 2003

From @mjdominus

Gisle Aas <gisle@​ActiveState.com>​:

The lock is still needed here since a single print might result in
multiple low level writes. Potentially the seek() could be removed
from the lock function. Is this what you suggest?

I believe that Nick was suggesting that the seek could be removed.

It also seems like this example need to make sure to flush the MBOX
before it unlocks the handle.

I thought so too, but the manual says that that is now longer necessary​:

  To avoid the possibility of miscoordination, Perl now flushes
  FILEHANDLE before locking or unlocking it.

@p5pRT
Copy link
Author

p5pRT commented Sep 22, 2003

From @gisle

Mark Jason Dominus <mjd@​plover.com> writes​:

It also seems like this example need to make sure to flush the MBOX
before it unlocks the handle.

I thought so too, but the manual says that that is now longer necessary​:

           To avoid the possibility of miscoordination\, Perl now flushes
           FILEHANDLE before locking or unlocking it\.

Cool. In what version can we depend on this behaviour?

Regards,
Gisle

@p5pRT
Copy link
Author

p5pRT commented Sep 22, 2003

From rick@bort.ca

On Mon, Sep 22, 2003 at 07​:07​:12AM -0700, Gisle Aas wrote​:

Mark Jason Dominus <mjd@​plover.com> writes​:

           To avoid the possibility of miscoordination\, Perl now flushes
           FILEHANDLE before locking or unlocking it\.

Cool. In what version can we depend on this behaviour?

IIRC, 5.004. perl5004delta.pod confirms my recollection​:

=item flock

is now supported on more platforms, prefers fcntl to lockf when
emulating, and always flushes before (un)locking.

--
Rick Delaney
rick@​bort.ca

@p5pRT
Copy link
Author

p5pRT commented Sep 22, 2003

From @mjdominus

Gisle Aas <gisle@​ActiveState.com>​:

Mark Jason Dominus <mjd@​plover.com> writes​:

It also seems like this example need to make sure to flush the MBOX
before it unlocks the handle.

I thought so too, but the manual says that that is now longer necessary​:

           To avoid the possibility of miscoordination\, Perl now flushes
           FILEHANDLE before locking or unlocking it\.

Cool. In what version can we depend on this behaviour?

Starting in 5.005_03.

@p5pRT
Copy link
Author

p5pRT commented Sep 22, 2003

From @mjdominus

Mark Jason Dominus <mjd@​plover.com>​:

Gisle Aas <gisle@​ActiveState.com>​:

Cool. In what version can we depend on this behaviour?

Starting in 5.005_03.

Rick Delaney said 5.004. He is probably right. I based my conclusion
on the appearance of the guarantee in the perlfunc manual. But the
feature could of course have been in Perl earlier than that.

@p5pRT
Copy link
Author

p5pRT commented Sep 22, 2003

From @RandalSchwartz

"Nick@​Cleaton" == Nick@​Cleaton Net <perlbug-followup@​perl.org> writes​:

Nick@​Cleaton> ... which appears to have been written by someone under the
Nick@​Cleaton> misapprehension that opening a file for append is the same as
Nick@​Cleaton> opening for write and then seeking to the end.

Nick@​Cleaton> This is not the case. When a file is opened in append mode, all
Nick@​Cleaton> writes are appended to the then end of file, irrespective of the
Nick@​Cleaton> current file position. From fopen(3) on FreeBSD​:

It was *indeed* precisely the case on V7 Unix. The example probably
even dates back to when Perl ran on V7 Unix and early BSD (*real* BSD,
not foo-BSD) versions. Even after you opened for append, another process
could extend the file, and you'd be writing in the middle. Wasn't safe
to seek until after the flock().

However, I'd be hard-pressed to find a modern Unix on which append-mode
didn't include the forced-seek mode.

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@​stonehenge.com> <URL​:http​://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

@p5pRT
Copy link
Author

p5pRT commented May 27, 2012

From @jkeenan

On Mon Sep 22 10​:01​:30 2003, merlyn@​stonehenge.com wrote​:

"Nick@​Cleaton" == Nick@​Cleaton Net <perlbug-followup@​perl.org>
writes​:

The example found in Perl 5.16.0 under 'perldoc -f flock' (excerpt
attached) has been revised since the OP filed this ticket nine years ago.

Could Nick@​Cleaton and the other commenters review this and see where we
stand?

Thank you very much.
Jim Keenan

@p5pRT
Copy link
Author

p5pRT commented May 27, 2012

From @jkeenan

Here's a mailbox appender for BSD systems.

  use Fcntl qw(​:flock SEEK_END); # import LOCK_* and SEEK_END constants

  sub lock {
  my ($fh) = @​_;
  flock($fh, LOCK_EX) or die "Cannot lock mailbox - $!\n";

  # and, in case someone appended while we were waiting...
  seek($fh, 0, SEEK_END) or die "Cannot seek - $!\n";
  }

  sub unlock {
  my ($fh) = @​_;
  flock($fh, LOCK_UN) or die "Cannot unlock mailbox - $!\n";
  }

  open(my $mbox, ">>", "/usr/spool/mail/$ENV{'USER'}")
  or die "Can't open mailbox​: $!";

  lock($mbox);
  print $mbox $msg,"\n\n";
  unlock($mbox);

@p5pRT
Copy link
Author

p5pRT commented May 27, 2012

From tchrist@perl.com

So I think what that's saying is that the seek is immaterial
because of the append mode on the open, which I believe is
correct. That means there doesn't need to be lock/unlock
functions to do niceties, but one can just call flock directly.
Well, there's still the buffering concern on some systems, I suppose.

--tom

@p5pRT
Copy link
Author

p5pRT commented Jun 30, 2013

From @jkeenan

On Sat May 26 19​:44​:41 2012, tom christiansen wrote​:

So I think what that's saying is that the seek is immaterial
because of the append mode on the open, which I believe is
correct. That means there doesn't need to be lock/unlock
functions to do niceties, but one can just call flock directly.
Well, there's still the buffering concern on some systems, I suppose.

--tom

Tom, does that mean that this nearly ten-year-old RT is closable? (It
appears closable to me, FWIW.)

Thank you very much.
Jim Keenan

@p5pRT
Copy link
Author

p5pRT commented Jul 5, 2016

From @dcollinsn

I agree with Nick's complaint and with the other observations in this thread. The seek may not be necessary, but this comment at least explains why, whereas removing the seek doesn't teach anyone anything.

Patch attached.

@p5pRT
Copy link
Author

p5pRT commented Jul 5, 2016

From @dcollinsn

0001-RT-23813-perlfunc.pod-explain-seek-in-O_APPEND.patch
From c2bde2eb1618d5608461cb065d688f7dc76a5d0e Mon Sep 17 00:00:00 2001
From: Dan Collins <dcollinsn@gmail.com>
Date: Tue, 5 Jul 2016 19:02:12 -0400
Subject: [PATCH] [RT #23813] perlfunc.pod: explain seek in O_APPEND

In all modern systems, seek has no effect in O_APPEND. As explained
by Nick Cleaton at that RT, the comment was written in a time when:

    ...opening a file for append [was] the same as opening for
    write and then seeking to the end.

    This is not the case.  When a file is opened in append mode,
    all writes are appended to the then end of file, irrespective
    of the current file position.

Corrected comment was contributed by him.
---
 pod/perlfunc.pod | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/pod/perlfunc.pod b/pod/perlfunc.pod
index 5c778f1..36eb96b 100644
--- a/pod/perlfunc.pod
+++ b/pod/perlfunc.pod
@@ -2653,8 +2653,8 @@ Here's a mailbox appender for BSD systems.
     sub lock {
         my ($fh) = @_;
         flock($fh, LOCK_EX) or die "Cannot lock mailbox - $!\n";
-
-        # and, in case someone appended while we were waiting...
+        # and, in case we're running on a very old UNIX
+        # variant without the modern O_APPEND semantics...
         seek($fh, 0, SEEK_END) or die "Cannot seek - $!\n";
     }
 
-- 
2.8.1

@p5pRT
Copy link
Author

p5pRT commented Feb 26, 2017

From @jkeenan

On Tue, 05 Jul 2016 23​:13​:20 GMT, dcollinsn@​gmail.com wrote​:

I agree with Nick's complaint and with the other observations in this
thread. The seek may not be necessary, but this comment at least
explains why, whereas removing the seek doesn't teach anyone anything.

Patch attached.

List​: Is the documentation change to perlfunc proposed by Dan Collins acceptable?

If so, is this 13-year-old ticket closable?

Thank you very much.
--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Mar 29, 2017

From @iabyn

On Sun, Feb 26, 2017 at 02​:58​:55PM -0800, James E Keenan via RT wrote​:

On Tue, 05 Jul 2016 23​:13​:20 GMT, dcollinsn@​gmail.com wrote​:

I agree with Nick's complaint and with the other observations in this
thread. The seek may not be necessary, but this comment at least
explains why, whereas removing the seek doesn't teach anyone anything.

Patch attached.

List​: Is the documentation change to perlfunc proposed by Dan Collins
acceptable?

Yes, but lets apply it after 5.26....

If so, is this 13-year-old ticket closable?

...and then close the ticket.

--
Indomitable in retreat, invincible in advance, insufferable in victory
  -- Churchill on Montgomery

@p5pRT
Copy link
Author

p5pRT commented Jun 1, 2017

From @jkeenan

On Wed, 29 Mar 2017 11​:14​:36 GMT, davem wrote​:

On Sun, Feb 26, 2017 at 02​:58​:55PM -0800, James E Keenan via RT wrote​:

On Tue, 05 Jul 2016 23​:13​:20 GMT, dcollinsn@​gmail.com wrote​:

I agree with Nick's complaint and with the other observations in this
thread. The seek may not be necessary, but this comment at least
explains why, whereas removing the seek doesn't teach anyone anything.

Patch attached.

List​: Is the documentation change to perlfunc proposed by Dan Collins
acceptable?

Yes, but lets apply it after 5.26....

If so, is this 13-year-old ticket closable?

...and then close the ticket.

Pushed to blead in commit 5c3085e. Marking ticket Resolved. (It only took 13 years!)

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Jun 1, 2017

@jkeenan - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant