Skip Menu |
Report information
Id: 27555
Status: open
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: jamie <jamie [at] shareable.org>
muir [at] idiom.com
Cc: davidnicol <davidnico [at] cpan.org>
AdminCc:

Operating System: Linux
PatchStatus: (no value)
Severity: medium
Type:
Perl Version: 5.8.3
Fixed In: (no value)



CC: root [...] localhost
Subject: TIED $x = \$y{z}; delete $y{z} -- behaves badly
Date: Tue, 9 Mar 2004 13:57:07 -0800 (PST)
To: perlbug [...] perl.org
From: muir [...] charm.idiom.com (David Muir Sharnoff)
Download (untitled) / with headers
text/plain 4.4k
This is a bug report for perl from muir@idiom.com, generated with the help of perlbug 1.34 running under perl v5.8.3. ----------------------------------------------------------------- [Please enter your report here] Several related bugs... 1. After deleting a reference to a tied hash value, an assignment through the reference will re-create the tied hash key and reconnect the reference to the hash value. If there is more than one such reference, then when one reference reconnects, the others disconnect. 2. With untied hashes, creating a reference to hash value that doesn't exist will create the hash key. With tied hashes this does not happen. The tie interface probably needs some new methods. 3. In most of perl, two references to the same object are themselves the same. refaddr($ref1) == refaddr($ref2) iff $ref1 and $ref2 refer to the same thing. With tied hases, this is not the case. I haven't checked the behavior of tied ARRAYs with respect to these things but I wouldn't be suprised if they had the same problems. I'll upload a regression test. [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=medium --- Site configuration information for perl v5.8.3: Configured by Debian Project at Sun Feb 15 17:22:09 EST 2004. Summary of my perl5 (revision 5.0 version 8 subversion 3) configuration: Platform: osname=linux, osvers=2.4.22-xfs+ti1211, archname=i386-linux-thread-multi uname='linux kosh 2.4.22-xfs+ti1211 #1 sat oct 25 10:11:37 est 2003 i686 gnulinux ' config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i386-linux -Dprefix=/usr -Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.8.3 -Dsitearch=/usr/local/lib/perl/5.8.3 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.3 -Dd_dosuid -des' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef useithreads=define usemultiplicity=define useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O3', cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -I/usr/local/include' ccversion='', gccversion='3.3.3 20040125 (prerelease) (Debian)', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt perllibs=-ldl -lm -lpthread -lc -lcrypt libc=/lib/libc-2.3.2.so, so=so, useshrplib=true, libperl=libperl.so.5.8.3 gnulibc_version='2.3.2' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic' cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib' Locally applied patches: --- @INC for perl v5.8.3: /etc/perl /usr/local/lib/perl/5.8.3 /usr/local/share/perl/5.8.3 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.8 /usr/share/perl/5.8 /usr/local/lib/site_perl /usr/local/lib/perl/5.8.2 /usr/local/share/perl/5.8.2 . --- Environment for perl v5.8.3: HOME=/home/muir LANG=C LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=.:/home/muir/bin/charm:/home/muir/bin:/home/muir/bin/share:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/shbin:/usr/local/sbin:/usr/local/bin:/usr/local/ptybin:/usr/X11R6/bin:/usr/bin/X11:/usr/local/tex/bin:/usr/ucb:/usr/bin:/bin:/etc:/usr/etc:/usr/games:/lib:/usr/lib:/usr/local/java/bin:/usr/lib/uucp:/usr/openwin/bin:/usr/openwin/bin/xview:/usr/openwin/demo:/usr/adm:/home/muir/tmp PERL_BADLANG (unset) SHELL=/bin/tcsh
Subject: bug-refs.t
Download bug-refs.t
text/plain 1.9k
#!/usr/bin/perl use warnings; use strict; use Scalar::Util qw(refaddr reftype blessed); use Test::More tests => 82; # # References to tied hash values are all unique. They each have # their own address. References remain even when the hash key # is deleted. # # Sometimes the references can become disconnected from the underlying # hash. They'll reconnect on assignement. # # When a reference reconnects after assignement, any other references # disconnect. (not shown) # print "# block at ".__LINE__."\n"; { my %x; tie %x, 'Hash1', {}; $x{y} = 7; my $a = \$x{y}; delete $x{y}; $x{y} = 9; my $b = \$x{y}; my $c = \$x{y}; ok($$a == 7, "The \$a reference should be disconnected"); ok(refaddr($b) eq refaddr($c), "References to the same thing should be the same"); delete $x{y}; $$c = 17; ok($$b != 17, "Post-delete, references should be disconnected"); ok($x{y} != 17, "Post-delete, references should be disconnected"); my $d = \$x{y}; $$a = 12; ok($x{y} != 12, "Post-disconnect, reconnect shouldn't happen"); my $q = \$x{q}; ok(exists($x{q}), "creating a reference creates a key"); } package Hash1; sub TIEHASH { my $pkg = shift; return bless [ @_ ], $pkg; } sub FETCH { my $self = shift; my $key = shift; my ($underlying) = @$self; return $underlying->{$key}; } sub STORE { my $self = shift; my $key = shift; my $value = shift; my ($underlying) = @$self; return ($underlying->{$key} = $value); } sub DELETE { my ($self, $key) = @_; my ($underlying) = @$self; return delete($underlying->{$key}); } sub CLEAR { my $self = shift; my ($underlying) = @$self; %$underlying = (); } sub EXISTS { my $self = shift; my $key = shift; my ($underlying) = @$self; return exists $underlying->{$key}; } sub FIRSTKEY { my $self = shift; my ($underlying) = @$self; keys %$underlying; return each %$underlying; } sub NEXTKEY { my $self = shift; my ($underlying) = @$self; return each %$underlying; }
Subject: Re: [perl #27555] TIED $x = \$y{z}; delete $y{z} -- behaves badly
Date: Wed, 24 Mar 2004 23:30:32 +0000
To: perl5-porters [...] perl.org
From: Dave Mitchell <davem [...] fdisolutions.com>
Download (untitled) / with headers
text/plain 1.8k
On Tue, Mar 09, 2004 at 09:58:08PM -0000, David Muir Sharnoff wrote: Show quoted text
> Several related bugs... > > 1. After deleting a reference to a tied hash value, an assignment > through the reference will re-create the tied hash key and reconnect > the reference to the hash value. > > If there is more than one such reference, then when one reference > reconnects, the others disconnect. > > 2. With untied hashes, creating a reference to hash value that doesn't > exist will create the hash key. With tied hashes this does not happen. > The tie interface probably needs some new methods. > > 3. In most of perl, two references to the same object are themselves > the same. refaddr($ref1) == refaddr($ref2) iff $ref1 and $ref2 refer > to the same thing. With tied hases, this is not the case. > > I haven't checked the behavior of tied ARRAYs with respect to these > things but I wouldn't be suprised if they had the same problems.
What you are seeing here is various user-level manfifestions of the fact that the expression $tied_hash{$key} actually creates a "proxy" lvalue object, which when accessed calls FETCH, and when assigned to, calls STORE. When you create a reference to a tied element, you are really creating a reference to this proxy object. This allows the following to work: $ref = \$tied_hash{foo}; $$ref = 1; # does $obj->STORE('foo', 1); $$ref = 2; # does $obj->STORE('foo', 2); $$ref = 3; # does $obj->STORE('foo', 3); $x = $$ref; # does $obj->FETCH('foo'); $y = $$ref; # does $obj->FETCH('foo'); There's no way to achieve these seamantics without the rough edges showing slightly. If you think about it, its amazing that taking a reference works in the first place, since there isn't a real value who's reference can actually be taken. Dave. -- "The GPL violates the U.S. Constitution, together with copyright, antitrust and export control laws" -- SCO smoking crack again.
CC: davidnico [...] cpan.org
Subject: Re: [perl #27555] TIED $x = \$y{z}; delete $y{z} -- behaves badly
Date: Wed, 24 Mar 2004 17:19:06 -0800 (PST)
To: perlbug-followup [...] perl.org
From: David Muir Sharnoff <muir [...] idiom.com>
Download (untitled) / with headers
text/plain 1.7k
I know this stuff isn't easy. I'm impressed it works as well as it does. In investigating this, I started reading perl's source code for the first time and I was really impressed at how small it is. The density is amizingly high. Totally inscrutible without spending a few weeks learning the psudo-language it's written in though. I do think this bug can be fixed. The simplest problem: binding to a value doesn't cause it to exist can be fixed by doing: $obj->STORE('foo', undef) unless exists $obj->EXISTS('foo'); For the harder issue that $obj->DELETE('foo') should disconnect all existing references, addional bookkeeping is required. What I did to work around this in my persistence module is that I register all such references with the tied hash object (which knows about this problem) and then whenever $obj->DELETE is called, the tied has object notifies all the dangling references to disconnect from the hash. The disconnect routine connects dangling references to the same key together. I can't do this completely transparently because I can't prevent users from doing $ref1 = $ref2. What I require at that point is that they call workaround27555($ref1); Perl could do this automatically by automatically registering such hashes somewhere and then intercepting tied DELETE and tied CLEAR to deal with it. I'm not sure how to fix the refaddr() problem. I work around it but my workaround requires traversing all available structures. Not pretty and a lot of work for an odd case. I think that work would only be worth it in the context of the larger goal of making it possible for tied objects to truely emulate untied objects. This is a worthy goal and it requires more than just fixing my bugs to achieve it. -Dave
CC: perlbug-followup [...] perl.org
Subject: Re: [perl #27555] TIED $x = \$y{z}; delete $y{z} -- behaves badly
Date: Thu, 25 Mar 2004 12:09:41 +0000
To: David Muir Sharnoff <muir [...] idiom.com>
From: hv [...] crypt.org
David Muir Sharnoff <muir@idiom.com> wrote: :For the harder issue that $obj->DELETE('foo') should disconnect all :existing references, addional bookkeeping is required. What I did :to work around this in my persistence module is that I register all :such references with the tied hash object (which knows about this :problem) and then whenever $obj->DELETE is called, the tied has :object notifies all the dangling references to disconnect from the :hash. The disconnect routine connects dangling references to the :same key together. I can't do this completely transparently because :I can't prevent users from doing $ref1 = $ref2. What I require :at that point is that they call workaround27555($ref1); : :Perl could do this automatically by automatically registering such :hashes somewhere and then intercepting tied DELETE and tied CLEAR :to deal with it. Without thinking about it too deeply, it feels like this is something you should be able to solve by ensuring that the refs taken rely on reading through a weak reference, and that DELETE frees the last concrete reference. See Scalar::Util::weaken(). Hugo
CC: jamie [...] shareable.org
Subject: Reference to tied hash element caches value too aggressively
Date: Mon, 5 Apr 2004 00:12:45 +0100
To: perlbug [...] perl.org
From: Jamie Lokier <jamie [...] shareable.org>
Download (untitled) / with headers
text/plain 6.4k
This is a bug report for perl from jamie@shareable.org, generated with the help of perlbug 1.34 running under perl v5.8.0. ----------------------------------------------------------------- [Please enter your report here] It is possible to take a reference to an element of a tied hash, like this: my $ref = \$tied_hash{key}; Taking the reference like that creates a special kind of pseudo-reference which remembers the key that was used. It doesn't call FETCH or STORE or any other method. Reading from the reference (C<my $value = $$ref>) calls the tied hash's FETCH method the _first_ time it's read, and then caches the result. Writing to the reference (C<$$ref = ...>) calls the tied hash's STORE method and invalidates the cached result, so that FETCH will be called next time the reference is read. This is very clever and sensible and useful. Confusion comes about when the hash is written to directly, or via another pseudo-reference using the same key. This code prints "new_value": $tied_hash{key} = "old_value"; my $ref = \$tied_hash{key}; $tied_hash{key} = "new_value"; print $$ref; Whereas this code prints "old_value" _if_ C<$tied_hash> is tied: $tied_hash{key} = "old_value"; my $ref = \$tied_hash{key}; "$$ref"; # Force non-void context. $tied_hash{key} = "new_value"; print $$ref; This also prints "old_value" if C<$tied_hash> is tied: $tied_hash{key} = "old_value"; my $ref1 = \$tied_hash{key}; my $ref2 = \$tied_hash{key}; "$$ref1"; # Force non-void context. $$ref2 = "new_value"; print $$ref1; My complaint is that this cacheing of fetched values in a pseudo-reference is slightly too aggressive: it makes references to tied hash elements behave semantically too differently from ordinary hashes, and it isn't necessary or difficult to fix. Ideally, any write to that key of the tied hash should invalidate the cached pseudo-reference value. (Really ideally, all pseudo-references using the same key would be equal , too). It may well be to much too state to keep track of, remembering the pseudo-references of each key individually. So instead, I suggest any write to a tied hash should invalidate _all_ cached pseudo-references which are associated with that hash. That wouldn't be much state to keep track of. Example program: perl -l <<'END' sub TIEHASH { return bless {}, $_[0] } sub FETCH { print "Fetch"; return $_[0]->{$_[1]} } sub STORE { print "Store"; $_[0]->{$_[1]} = $_[2] } tie %HASH, __PACKAGE__; my $r = \$HASH{key}; $$r = "value1"; print $$r; print $HASH{key}; print $$r; $$r = "value2"; print $$r; print $$r; $HASH{key} = "value3"; print $$r; print $HASH{key}; print $$r; END Output of the above program: Store Fetch value1 Fetch value1 value1 Store Fetch value2 value2 Store value2 Fetch value3 value2 See how the "value2" lines after the last "Store" should logically say "value3". I think this quirk of pseudo-references is too subtle, and is likely to cause subtle bugs in programs, especially code that is given a tied hash and doesn't know it, treating it like a normal hash. Thanks, -- Jamie [Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=medium --- Site configuration information for perl v5.8.0: Configured by bhcompile' cf_email='bhcompile at Wed Aug 13 11:45:59 EDT 2003. Summary of my rderl (revision 5.0 version 8 subversion 0) configuration: Platform: osname=linux, osvers=2.4.21-1.1931.2.382.entsmp, archname=i386-linux-thread-multi uname='linux str' config_args='-des -Doptimize=-O2 -g -pipe -march=i386 -mcpu=i686 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dinstallprefix=/usr -Dprefix=/usr -Darchname=i386-linux -Dvendorprefix=/usr -Dsiteprefix=/usr -Dotherlibdirs=/usr/lib/perl5/5.8.0 -Duseshrplib -Dusethreads -Duseithreads -Duselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr' hint=recommended, useposix=true, d_sigaction=define usethreads=define use5005threads=undef' useithreads=define usemultiplicity= useperlio= d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=un uselongdouble= usemymalloc=, bincompat5005=undef Compiler: cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm', optimize='', cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -I/usr/local/include -I/usr/include/gdbm' ccversion='', gccversion='3.2.2 20030222 (Red Hat Linux 3.2.2-5)', gccosandvers='' gccversion='3.2.2 200302' intsize=r, longsize=r, ptrsize=5, doublesize=8, byteorder=1234 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long' k', ivsize=4' ivtype='l, nvtype='double' o_nonbl', nvsize=, Off_t='', lseeksize=8 alignbytes=4, prototype=define Linker and Libraries: ld='gcc' l', ldflags =' -L/u' libpth=/usr/local/lib /lib /usr/lib libs=-lnsl -lgdbm -ldb -ldl -lm -lpthread -lc -lcrypt -lutil perllibs= libc=/lib/libc-2.3.2.so, so=so, useshrplib=true, libperl=libper gnulibc_version='2.3.2' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so', d_dlsymun=undef, ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5/5.8.0/i386-linux-thread-multi/CORE' cccdlflags='-fPIC' ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5', lddlflags='s Unicode/Normalize XS/A' Locally applied patches: MAINT18379 --- @INC for perl v5.8.0: /usr/lib/perl5/5.8.0/i386-linux-thread-multi /usr/lib/perl5/5.8.0 /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.0 /usr/lib/perl5/vendor_perl /usr/lib/perl5/5.8.0/i386-linux-thread-multi /usr/lib/perl5/5.8.0 . --- Environment for perl v5.8.0: HOME=/home/jamie LANG=en_GB.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH (unset) LOGDIR (unset) PATH=/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/jamie/bin PERL_BADLANG (unset) SHELL=/bin/bash dlflags='-share (unset)
Download (untitled) / with headers
text/plain 470b
Just noticed something else. An assignment though a reference to a tied hash key will disassociate any other similar references. This happens even if there has never been a delete. print "# block at ".__LINE__."\n"; { my %x; tie %x, 'Hash1', {}; $x{y} = 7; my $a = \$x{y}; my $b = \$x{y}; $x{y} = 9; ok($$a == 9); ok($$b == 9); $$a = 10; ok($$b == 9); # bug }
This should be merged with bug # 27555.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org