Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect math when reading a padded number from shared memory #11634

Closed
p5pRT opened this issue Sep 5, 2011 · 19 comments
Closed

Incorrect math when reading a padded number from shared memory #11634

p5pRT opened this issue Sep 5, 2011 · 19 comments

Comments

@p5pRT
Copy link

p5pRT commented Sep 5, 2011

Migrated from rt.perl.org#98480 (status was 'resolved')

Searchable as RT98480$

@p5pRT
Copy link
Author

p5pRT commented Sep 5, 2011

From markgr@online.no

Created by markgr@online.no

Synopsis

When reading a number from shared memory that is padded with /0`s and then doing
math on it, perl will always do the calculation using the original value of the
buffer, even if the value in the buffer changes however, it will "print" the
modified value.

Detail

If the fourth field in shmwrite is larger than the value being written, the
shared memory is padded with /0 as mentioned in the texts. If a process reads
that value with shmread, including the padding, into a variable, perl will be
able to print that value and do math correctly using it. However, if a new
value is written to the shared memory segment (with the padding), a subsequent
shmread into the same variable as used previously will result in any math being
done using the original value though printing the value will give the correct
result.

Work arounds

I have posted this on perl monks http​://www.perlmonks.org/?node_id=924278 and
it seems stripping the padding, copying the buffer variable before hand or
using undef on the buffer variable before each shmread return perls to sanity :)

Example code

The following code does nothing sensible (not even error checking) but it does
demonstrate the problem on at least perl 5.10.0 on OSX and 5.8.4 on Solaris.

Thanks

Mark

#!/usr/bin/perl

use IPC​::SysV qw(IPC_STAT IPC_PRIVATE IPC_CREAT IPC_EXCL S_IRUSR S_IWUSR IPC_RMID);

$key=shmget(IPC_PRIVATE,100,IPC_CREAT | IPC_EXCL | S_IRUSR | S_IWUSR);

my $ret=fork();

if($ret>0){
  my $int=0;
  while(1){
  ++$int;
  shmwrite($key,$int,0,99);
  sleep 60;
  }
} else {
  my $t=0;
  while(1){
  shmread($key,$message,0,99);
  ++$t;
  $result=$t-$message;
  print "$t - $message is $result\n";
  sleep 10;
  }
}

Perl Info

Flags:
    category=core
    severity=low

Site configuration information for perl 5.10.0:

Configured by _mdnsresponder at Thu Jun 25 21:51:41 PDT 2009.

Summary of my perl5 (revision 5 version 10 subversion 0) configuration:
  Platform:
    osname=darwin, osvers=10.0, archname=darwin-thread-multi-2level
    uname='darwin b79.apple.com 10.0 darwin kernel version 10.0.0d8: tue may 5 19:29:59 pdt 2009; root:xnu-1437.2~2release_i386 i386 '
    config_args='-ds -e -Dprefix=/usr -Dccflags=-g  -pipe  -Dldflags= -Dman3ext=3pm -Duseithreads -Duseshrplib -Dinc_version_list=none -Dcc=gcc-4.2'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc-4.2', ccflags ='-arch x86_64 -arch i386 -arch ppc -g -pipe -fno-common -DPERL_DARWIN -fno-strict-aliasing -I/usr/local/include',
    optimize='-Os',
    cppflags='-g -pipe -fno-common -DPERL_DARWIN -fno-strict-aliasing -I/usr/local/include'
    ccversion='', gccversion='4.2.1 (Apple Inc. build 5646)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc-4.2 -mmacosx-version-min=10.6', ldflags ='-arch x86_64 -arch i386 -arch ppc -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib
    libs=-ldbm -ldl -lm -lutil -lc
    perllibs=-ldl -lm -lutil -lc
    libc=/usr/lib/libc.dylib, so=dylib, useshrplib=true, libperl=libperl.dylib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-arch x86_64 -arch i386 -arch ppc -bundle -undefined dynamic_lookup -L/usr/local/lib'

Locally applied patches:
    /Library/Perl/Updates/<version> comes before system perl directories
    installprivlib and installarchlib points to the Updates directory


@INC for perl 5.10.0:
    /Library/Perl/Updates/5.10.0
    /System/Library/Perl/5.10.0/darwin-thread-multi-2level
    /System/Library/Perl/5.10.0
    /Library/Perl/5.10.0/darwin-thread-multi-2level
    /Library/Perl/5.10.0
    /Network/Library/Perl/5.10.0/darwin-thread-multi-2level
    /Network/Library/Perl/5.10.0
    /Network/Library/Perl
    /System/Library/Perl/Extras/5.10.0/darwin-thread-multi-2level
    /System/Library/Perl/Extras/5.10.0
    .


Environment for perl 5.10.0:
    DYLD_LIBRARY_PATH (unset)
    HOME=/Users/markgrant
    LANG (unset)
    LANGUAGE (unset)
    LC_CTYPE=UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Sep 6, 2011

From zefram@fysh.org

Mark Grant wrote​:

When reading a number from shared memory that is padded with /0`s and then doing
math on it, perl will always do the calculation using the original value of the
buffer, even if the value in the buffer changes however, it will "print" the
modified value.

Yes, it will. Perl maintains a scalar's numeric value separately from
its string value. Normally, a process that modifies the string value
will clear the numeric value to force reconversion. However, if the
string value is modified by virtue of being shared memory (shm, mmap,
threads), the modification can't be processed the way a language-visible
modification would be. Viewing shared memory as a Perl scalar is
inherently a hack, and you can't expect it to work for all purposes.

I recommend that you always explicitly stringify the scalar when reading
it for non-string purposes. E.g., write '"$shared" + 1' instead of
'$shared + 1'.

-zefram

@p5pRT
Copy link
Author

p5pRT commented Sep 6, 2011

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Sep 6, 2011

From @Leont

On Mon, Sep 5, 2011 at 10​:32 PM, Mark Grant <perlbug-followup@​perl.org> wrote​:

# New Ticket Created by  Mark Grant
# Please include the string​:  [perl #98480]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=98480 >

It seems shmread doesn't turn off SvIOK on the variable when writing
to it, even though it should.

The patch attached is a quick fix, but I'm wondering if there aren't
more corner cases in this code. It looks terribly fragile to me.

Leon

@p5pRT
Copy link
Author

p5pRT commented Sep 6, 2011

From @Leont

shmread.patch
diff --git a/doio.c b/doio.c
index cecc574..1759fea 100644
--- a/doio.c
+++ b/doio.c
@@ -2304,7 +2304,7 @@ Perl_do_shmio(pTHX_ I32 optype, SV **mark, SV **sp)
 	/* suppress warning when reading into undef var (tchrist 3/Mar/00) */
 	if (! SvOK(mstr))
 	    sv_setpvs(mstr, "");
-	SvPV_force_nolen(mstr);
+	SvPOK_only(mstr);
 	mbuf = SvGROW(mstr, (STRLEN)msize+1);
 
 	Copy(shm + mpos, mbuf, msize, char);

@p5pRT
Copy link
Author

p5pRT commented Sep 6, 2011

From @Leont

On Tue, Sep 6, 2011 at 10​:40 AM, Zefram <zefram@​fysh.org> wrote​:

Viewing shared memory as a Perl scalar is
inherently a hack, and you can't expect it to work for all purposes.

No, shmread copies the data, it does not map it into a scalar. My
module SysV​::SharedMem does exactly that, and it goes into quite a bit
of internal pain this kind of problem doesn't happen ;-)

Leon

@p5pRT
Copy link
Author

p5pRT commented Sep 6, 2011

From @cpansprout

On Tue Sep 06 04​:44​:45 2011, LeonT wrote​:

On Mon, Sep 5, 2011 at 10​:32 PM, Mark Grant <perlbug-
followup@​perl.org> wrote​:

# New Ticket Created by �Mark Grant
# Please include the string​: �[perl #98480]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=98480 >

It seems shmread doesn't turn off SvIOK on the variable when writing
to it, even though it should.

The patch attached is a quick fix, but I'm wondering if there aren't
more corner cases in this code. It looks terribly fragile to me.

Leon

Is it possible to write tests for this sort of thing?

@p5pRT
Copy link
Author

p5pRT commented Sep 6, 2011

From @Leont

On Tue, Sep 6, 2011 at 5​:31 PM, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Tue Sep 06 04​:44​:45 2011, LeonT wrote​:

It seems shmread doesn't turn off SvIOK on the variable when writing
to it, even though it should.

The patch attached is a quick fix, but I'm wondering if there aren't
more corner cases in this code. It looks terribly fragile to me.

Leon

Is it possible to write tests for this sort of thing?

Sure. I've updated my patch and attached a test for this issue, but I
don't really know where to put it. The SysV primitives are currently
not tested at all, and IPC​::SysV just doesn't seem to be the right
place.

Leon

@p5pRT
Copy link
Author

p5pRT commented Sep 6, 2011

From @Leont

shmread.patch
diff --git a/doio.c b/doio.c
index cecc574..6f6d6b8 100644
--- a/doio.c
+++ b/doio.c
@@ -2304,7 +2304,8 @@ Perl_do_shmio(pTHX_ I32 optype, SV **mark, SV **sp)
 	/* suppress warning when reading into undef var (tchrist 3/Mar/00) */
 	if (! SvOK(mstr))
 	    sv_setpvs(mstr, "");
-	SvPV_force_nolen(mstr);
+	sv_upgrade(mstr, SVt_PV);
+	SvPOK_only(mstr);
 	mbuf = SvGROW(mstr, (STRLEN)msize+1);
 
 	Copy(shm + mpos, mbuf, msize, char);

@p5pRT
Copy link
Author

p5pRT commented Sep 6, 2011

From @Leont

shm2.pl

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2011

From @Leont

On Tue, Sep 6, 2011 at 5​:31 PM, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Tue Sep 06 04​:44​:45 2011, LeonT wrote​:

On Mon, Sep 5, 2011 at 10​:32 PM, Mark Grant <perlbug-
followup@​perl.org> wrote​:

# New Ticket Created by �Mark Grant
# Please include the string​: �[perl #98480]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=98480 >

It seems shmread doesn't turn off SvIOK on the variable when writing
to it, even though it should.

The patch attached is a quick fix, but I'm wondering if there aren't
more corner cases in this code. It looks terribly fragile to me.

Leon

Is it possible to write tests for this sort of thing?

Fix with tests attached

Leon

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2011

From @Leont

0001-Added-SysV-shared-memory-tests.patch
From 2395d6ca00592a30ca1a71d0539471816293883f Mon Sep 17 00:00:00 2001
From: Leon Timmermans <fawaka@gmail.com>
Date: Tue, 13 Sep 2011 18:34:28 +0200
Subject: [PATCH 1/2] Added SysV shared memory tests

Tests are based on IPC-SysV's tests, though I had to remove a lot for it
not to rely on IPC::SharedMem.
---
 MANIFEST   |    1 +
 t/io/shm.t |   75 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 76 insertions(+), 0 deletions(-)
 create mode 100644 t/io/shm.t

diff --git a/MANIFEST b/MANIFEST
index 16121f8..a52d7a7 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -4810,6 +4810,7 @@ t/io/print.t			See if print commands work
 t/io/pvbm.t			See if PVBMs break IO commands
 t/io/read.t			See if read works
 t/io/say.t			See if say works
+t/io/shm.t			See if SysV shared memory works
 t/io/tell.t			See if file seeking works
 t/io/through.t			See if pipe passes data intact
 t/io/utf8.t			See if file seeking works
diff --git a/t/io/shm.t b/t/io/shm.t
new file mode 100644
index 0000000..a450679
--- /dev/null
+++ b/t/io/shm.t
@@ -0,0 +1,75 @@
+################################################################################
+#
+#  $Revision: 6 $
+#  $Author: mhx $
+#  $Date: 2010/03/07 16:01:42 +0100 $
+#
+################################################################################
+#
+#  Version 2.x, Copyright (C) 2007-2010, Marcus Holland-Moritz <mhx@cpan.org>.
+#  Version 1.x, Copyright (C) 1999, Graham Barr <gbarr@pobox.com>.
+#
+#  This program is free software; you can redistribute it and/or
+#  modify it under the same terms as Perl itself.
+#
+################################################################################
+
+BEGIN {
+  if ($ENV{'PERL_CORE'}) {
+    chdir 't' if -d 't';
+    @INC = '../lib' if -d '../lib' && -d '../ext';
+  }
+
+  require Test::More; import Test::More;
+  require Config; import Config;
+
+  if ($ENV{'PERL_CORE'} && $Config{'extensions'} !~ m[\bIPC/SysV\b]) {
+    plan(skip_all => 'IPC::SysV was not built');
+  }
+}
+
+if ($Config{'d_shm'} ne 'define') {
+  plan(skip_all => '$Config{d_shm} undefined');
+}
+
+use sigtrap qw/die normal-signals error-signals/;
+use IPC::SysV qw/ IPC_PRIVATE S_IRWXU IPC_RMID /;
+
+my $key;
+END { shmctl $key, IPC_RMID, 0 if defined $key }
+
+{
+	local $SIG{SYS} = sub { plan(skip_all => "SIGSYS caught") } if exists $SIG{SYS};
+	$key = shmget IPC_PRIVATE, 8, S_IRWXU;
+}
+
+if (not defined $key) {
+  my $info = "IPC::SharedMem->new failed: $!";
+  if ($! == &IPC::SysV::ENOSPC || $! == &IPC::SysV::ENOSYS ||
+      $! == &IPC::SysV::ENOMEM || $! == &IPC::SysV::EACCES) {
+    plan(skip_all => $info);
+  }
+  else {
+    die $info;
+  }
+}
+else {
+	plan(tests => 11);
+	pass('acquired shared mem');
+}
+
+ok(shmwrite($key, pack("N", 4711), 0, 4), 'write(offs=0)');
+ok(shmwrite($key, pack("N", 210577), 4, 4), 'write(offs=4)');
+
+my $var;
+ok(shmread($key, $var, 0, 4), 'read(offs=0) returned ok');
+is($var, pack("N", 4711), 'read(offs=0) correct');
+ok(shmread($key, $var, 4, 4), 'read(offs=4) returned ok');
+is($var, pack("N", 210577), 'read(offs=4) correct');
+
+ok(shmwrite($key, "Shared", 1, 6), 'write(offs=1)');
+
+ok(shmread($key, $var, 1, 6), 'read(offs=1) returned ok');
+is($var, 'Shared', 'read(offs=1) correct');
+ok(shmwrite($key,"Memory", 0, 6), 'write(offs=0)');
+
-- 
1.7.4.1

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2011

From @Leont

0002-Fix-98480-math-when-reading-shared-memory.patch
From 38c932d1f18ff2003ccac01607311506b81e1714 Mon Sep 17 00:00:00 2001
From: Leon Timmermans <fawaka@gmail.com>
Date: Tue, 13 Sep 2011 20:13:22 +0200
Subject: [PATCH 2/2] Fix #98480 math when reading shared memory

shmread didn't unset SvIOK properly, causing a read into a SVIV to have
an incorrect numeric value. This patch fixes that and adds tests.
---
 doio.c     |    3 ++-
 t/io/shm.t |    8 +++++++-
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/doio.c b/doio.c
index cb77cf6..1edf8d2 100644
--- a/doio.c
+++ b/doio.c
@@ -2278,7 +2278,8 @@ Perl_do_shmio(pTHX_ I32 optype, SV **mark, SV **sp)
 	/* suppress warning when reading into undef var (tchrist 3/Mar/00) */
 	if (! SvOK(mstr))
 	    sv_setpvs(mstr, "");
-	SvPV_force_nolen(mstr);
+	sv_upgrade(mstr, SVt_PV);
+	SvPOK_only(mstr);
 	mbuf = SvGROW(mstr, (STRLEN)msize+1);
 
 	Copy(shm + mpos, mbuf, msize, char);
diff --git a/t/io/shm.t b/t/io/shm.t
index a450679..46bb2e1 100644
--- a/t/io/shm.t
+++ b/t/io/shm.t
@@ -54,7 +54,7 @@ if (not defined $key) {
   }
 }
 else {
-	plan(tests => 11);
+	plan(tests => 13);
 	pass('acquired shared mem');
 }
 
@@ -73,3 +73,9 @@ ok(shmread($key, $var, 1, 6), 'read(offs=1) returned ok');
 is($var, 'Shared', 'read(offs=1) correct');
 ok(shmwrite($key,"Memory", 0, 6), 'write(offs=0)');
 
+my $number = 1;
+my $int = 2;
+shmwrite $key, $int, 0, 1;
+shmread $key, $number, 0, 1;
+is("$number", $int, qq{"\$id" eq "$int"});
+cmp_ok($number + 0, '==', $int, "\$id + 0 == $int");
-- 
1.7.4.1

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2011

From @Leont

On Tue, Sep 6, 2011 at 5​:31 PM, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Tue Sep 06 04​:44​:45 2011, LeonT wrote​:

On Mon, Sep 5, 2011 at 10​:32 PM, Mark Grant <perlbug-
followup@​perl.org> wrote​:

# New Ticket Created by �Mark Grant
# Please include the string​: �[perl #98480]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=98480 >

It seems shmread doesn't turn off SvIOK on the variable when writing
to it, even though it should.

The patch attached is a quick fix, but I'm wondering if there aren't
more corner cases in this code. It looks terribly fragile to me.

Leon

Is it possible to write tests for this sort of thing?

Fix with tests attached

Leon

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2011

From @cpansprout

On Wed Sep 14 08​:12​:07 2011, LeonT wrote​:

On Tue, Sep 6, 2011 at 5​:31 PM, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Tue Sep 06 04​:44​:45 2011, LeonT wrote​:

On Mon, Sep 5, 2011 at 10​:32 PM, Mark Grant <perlbug-
followup@​perl.org> wrote​:

# New Ticket Created by �Mark Grant
# Please include the string​: �[perl #98480]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=98480 >

It seems shmread doesn't turn off SvIOK on the variable when writing
to it, even though it should.

The patch attached is a quick fix, but I'm wondering if there aren't
more corner cases in this code. It looks terribly fragile to me.

Leon

Is it possible to write tests for this sort of thing?

Fix with tests attached

Leon

It seems you’ve submitted one of those ‘stealth’ patches. For some
strange reason, I always find those hard to apply.

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2011

From @cpansprout

On Wed Sep 14 08​:11​:13 2011, LeonT wrote​:

On Tue, Sep 6, 2011 at 5​:31 PM, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Tue Sep 06 04​:44​:45 2011, LeonT wrote​:

On Mon, Sep 5, 2011 at 10​:32 PM, Mark Grant <perlbug-
followup@​perl.org> wrote​:

# New Ticket Created by �Mark Grant
# Please include the string​: �[perl #98480]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=98480 >

It seems shmread doesn't turn off SvIOK on the variable when writing
to it, even though it should.

The patch attached is a quick fix, but I'm wondering if there aren't
more corner cases in this code. It looks terribly fragile to me.

Leon

Is it possible to write tests for this sort of thing?

Fix with tests attached

Leon

*That’s* odd! Your message showed up twice in RT, once without the
patches (hence my comment about stealth patches).

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2011

From @Leont

On Wed, Sep 14, 2011 at 6​:56 PM, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

*That’s* odd! Your message showed up twice in RT, once without the
patches (hence my comment about stealth patches).

It seems gmail screwed up, it appears in my mailbox twice too, once
with attachments and once without. Never had this happen before…

Leon

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2011

From @cpansprout

On Wed Sep 14 08​:11​:13 2011, LeonT wrote​:

On Tue, Sep 6, 2011 at 5​:31 PM, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

On Tue Sep 06 04​:44​:45 2011, LeonT wrote​:

On Mon, Sep 5, 2011 at 10​:32 PM, Mark Grant <perlbug-
followup@​perl.org> wrote​:

# New Ticket Created by �Mark Grant
# Please include the string​: �[perl #98480]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=98480 >

It seems shmread doesn't turn off SvIOK on the variable when writing
to it, even though it should.

The patch attached is a quick fix, but I'm wondering if there aren't
more corner cases in this code. It looks terribly fragile to me.

Leon

Is it possible to write tests for this sort of thing?

Fix with tests attached

Leon

Thank you. Applied as 2d5385e and af8ff72.

@p5pRT
Copy link
Author

p5pRT commented Sep 14, 2011

@cpansprout - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant