Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Misleading error message from Storable #7436

Open
p5pRT opened this issue Jul 24, 2004 · 11 comments
Open

Misleading error message from Storable #7436

p5pRT opened this issue Jul 24, 2004 · 11 comments

Comments

@p5pRT
Copy link

p5pRT commented Jul 24, 2004

Migrated from rt.perl.org#30807 (status was 'open')

Searchable as RT30807$

@p5pRT
Copy link
Author

p5pRT commented Jul 24, 2004

From rick@bort.ca

This is a bug report for perl from rick@​bort.ca,
generated with the help of perlbug 1.34 running under perl v5.8.3.


% perl -MStorable=thaw -e 'thaw("a")'
Magic number checking on storable string failed at ../../lib/Storable.pm (autosplit into ../../lib/auto/Storable/thaw.al) line 358, at -e line 1

So far so good.

% perl -MStorable=thaw -e 'thaw("aa")'
Storable binary image v48.97 more recent than I am (v2.6) at ../../lib/Storable.pm (autosplit into ../../lib/auto/Storable/thaw.al) line 358, at -e line 1

It would be nice if the second had the same error as the first.



Flags​:
  category=library
  severity=low


Site configuration information for perl v5.8.3​:

Configured by Debian Project at Sun Feb 15 17​:22​:09 EST 2004.

Summary of my perl5 (revision 5.0 version 8 subversion 3) configuration​:
  Platform​:
  osname=linux, osvers=2.4.22-xfs+ti1211, archname=i386-linux-thread-multi
  uname='linux kosh 2.4.22-xfs+ti1211 #1 sat oct 25 10​:11​:37 est 2003 i686 gnulinux '
  config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i386-linux -Dprefix=/usr -Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.8.3 -Dsitearch=/usr/local/lib/perl/5.8.3 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.3 -Dd_dosuid -des'
  hint=recommended, useposix=true, d_sigaction=define
  usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
  useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
  use64bitint=undef use64bitall=undef uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize='-O3',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -I/usr/local/include'
  ccversion='', gccversion='3.3.3 20040125 (prerelease) (Debian)', gccosandvers=''
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
  ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=4, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -L/usr/local/lib'
  libpth=/usr/local/lib /lib /usr/lib
  libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
  perllibs=-ldl -lm -lpthread -lc -lcrypt
  libc=/lib/libc-2.3.2.so, so=so, useshrplib=true, libperl=libperl.so.5.8.3
  gnulibc_version='2.3.2'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
  cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'

Locally applied patches​:
 


@​INC for perl v5.8.3​:
  /etc/perl
  /usr/local/lib/perl/5.8.3
  /usr/local/share/perl/5.8.3
  /usr/lib/perl5
  /usr/share/perl5
  /usr/lib/perl/5.8
  /usr/share/perl/5.8
  /usr/local/lib/site_perl
  /usr/local/lib/perl/5.8.2
  /usr/local/share/perl/5.8.2
  .


Environment for perl v5.8.3​:
  HOME=/home/rick
  LANG=en_US
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/usr/local/bin​:/usr/bin​:/bin​:/usr/bin/X11​:/usr/games​:/home/rick/bin
  PERL_BADLANG (unset)
  SHELL=/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Dec 3, 2011

From @jkeenan

On Sat Jul 24 08​:20​:35 2004, rickdelaney wrote​:

This is a bug report for perl from rick@​bort.ca,
generated with the help of perlbug 1.34 running under perl v5.8.3.
-----------------------------------------------------------------
% perl -MStorable=thaw -e 'thaw("a")'
Magic number checking on storable string failed at
../../lib/Storable.pm (autosplit into
../../lib/auto/Storable/thaw.al) line 358, at -e line 1

So far so good.

% perl -MStorable=thaw -e 'thaw("aa")'
Storable binary image v48.97 more recent than I am (v2.6) at
../../lib/Storable.pm (autosplit into
../../lib/auto/Storable/thaw.al) line 358, at -e line 1

It would be nice if the second had the same error as the first.

I can confirm that, notwithstanding minor changes in the error messages,
this bug is still present in the Storable that ships with Perl 5.14.2​:
$ perl -MStorable=thaw -e 'thaw("a")'
Magic number checking on storable string failed at
/usr/local/lib/perl5/5.14.2/darwin-2level/Storable.pm line 416, at -e line 1

$ perl -MStorable=thaw -e 'thaw("aa")'
Storable binary image v48.97 more recent than I am (v2.8) at
/usr/local/lib/perl5/5.14.2/darwin-2level/Storable.pm line 416, at -e line 1

Thank you very much.
Jim Keenan

@p5pRT
Copy link
Author

p5pRT commented Dec 3, 2011

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jul 3, 2012

From @doy

I'm inclined to call this not a bug. How is perl supposed to
differentiate between a string of bytes like the one in this example and
a string of bytes that represents the result of Storable​::freeze? What
is the exact behavior you want here?

@p5pRT
Copy link
Author

p5pRT commented Jul 3, 2012

From rick@bort.ca

On Jul 02 2012, Jesse Luehrs via RT wrote​:

I'm inclined to call this not a bug. How is perl supposed to
differentiate between a string of bytes like the one in this example and
a string of bytes that represents the result of Storable​::freeze? What
is the exact behavior you want here?

I think that my original problem was that I was doing thaw($x) where $x
was not a frozen object. Ideally, I would like Storable to tell me what
my bug is. I find the "magic number check failed" error enough to point
me in the right direction. But telling me that the version of the
frozen object is wrong is just confusing since it is not a frozen object
at all.

I don't know the structure of the Storable binary image but I would
think that if it has a magic number check that it should check that
first and not get to version checking until that passed. If that is
what it is doing then I think the magic number was not very well chosen
if it will pass ascii data. I understand if it's not possible to change
the storage format without messing other things up. But then I think
the bug needs to be marked as WONTFIX.

--
Rick Delaney
rick@​bort.ca

@p5pRT
Copy link
Author

p5pRT commented Jul 7, 2016

From @dcollinsn

Storable.xs​:

  /*
  * The "magic number" is only for files, not when freezing in memory.
  */

a is decimal 97 is ascii. The first two bytes are interpreted as major.minor version, but the first byte is first rightshifted - apparently the least significant bit is used to encode something else.

So, "aa" actually survives partway through magic_check(), when Storable discovers that 48.97 is much too high a version. "a" is the unusual case, because there isn't a second byte to read​:

  if (version_major > 1)
  GETMARK(version_minor);

And there is no character to be had, so it returns from magic_check without setting any other diagnostics.

In any event - there is no magic number that we can possibly check for thaw() from memory. The first two bytes are the version number. There are a number of different ways we can get weird error messages. So we should really just give a better error message in the cases that are most likely to be accidentally stumbled on.

I think that's the "version more recent" error as well as the "byte order not compatible" error (which you can get by passing in "\004\012\000xxxx"). Any other error message requires a crafted string like "\004\012\01012345678xxxx", which seems unlikely to happen by accident.

Will upload the patch momentarily.

@p5pRT
Copy link
Author

p5pRT commented Jul 7, 2016

From @dcollinsn

As promised - I propose this patch which changes the text of two error messages to propose that the data might also be "corrupt or not a Storable binary image". Tests are updated with the new error text.

Porting/cmp_version is "failing", but I'm not sure what to do with that information, or if that's for the pumpking to $VERSION++.

@p5pRT
Copy link
Author

p5pRT commented Jul 7, 2016

From @dcollinsn

0001-RT-30807-Clearer-error-for-thaw-text.patch
From e01cf874b04d1704a9b5bb40a7e3f476d83f5a52 Mon Sep 17 00:00:00 2001
From: Dan Collins <dcollinsn@gmail.com>
Date: Wed, 6 Jul 2016 23:26:32 -0400
Subject: [PATCH] [RT #30807] Clearer error for thaw('text')

---
 dist/Storable/Storable.xs |  6 +++---
 dist/Storable/t/croak.t   |  2 +-
 dist/Storable/t/malice.t  | 10 +++++-----
 3 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/dist/Storable/Storable.xs b/dist/Storable/Storable.xs
index 83cd001..3be1e07 100644
--- a/dist/Storable/Storable.xs
+++ b/dist/Storable/Storable.xs
@@ -6006,7 +6006,7 @@ static SV *magic_check(pTHX_ stcxt_t *cxt)
                 croak_now = 0;  /* Don't croak yet.  */
         }
         if (croak_now) {
-            CROAK(("Storable binary image v%d.%d more recent than I am (v%d.%d)",
+            CROAK(("Stored data corrupt, not a Storable binary image, or binary image version v%d.%d more recent than I am (v%d.%d)",
                    version_major, version_minor,
                    STORABLE_BIN_MAJOR, STORABLE_BIN_MINOR));
         }
@@ -6040,12 +6040,12 @@ static SV *magic_check(pTHX_ stcxt_t *cxt)
     if (SvTRUE(perl_get_sv("Storable::interwork_56_64bit", GV_ADD))) {
         if ((c != (sizeof (byteorderstr_56) - 1))
             || memNE(buf, byteorderstr_56, c))
-            CROAK(("Byte order is not compatible"));
+            CROAK(("Stored data corrupt, not a Storable binary image, or byte order is not compatible"));
     } else
 #endif
     {
         if ((c != (sizeof (byteorderstr) - 1)) || memNE(buf, byteorderstr, c))
-            CROAK(("Byte order is not compatible"));
+            CROAK(("Stored data corrupt, not a Storable binary image, or byte order is not compatible"));
     }
 
     current = buf + c;
diff --git a/dist/Storable/t/croak.t b/dist/Storable/t/croak.t
index ecd2bf8..366ff52 100644
--- a/dist/Storable/t/croak.t
+++ b/dist/Storable/t/croak.t
@@ -28,7 +28,7 @@ print "1..2\n";
 
 for my $test (1,2) {
   eval {thaw "\xFF\xFF"};
-  if ($@ =~ /Storable binary image v127.255 more recent than I am \(v2\.\d+\)/)
+  if ($@ =~ /Stored data corrupt, not a Storable binary image, or binary image version v127.255 more recent than I am \(v2\.\d+\)/)
     {
       print "ok $test\n";
     } else {
diff --git a/dist/Storable/t/malice.t b/dist/Storable/t/malice.t
index 867a0d7..ace6543 100644
--- a/dist/Storable/t/malice.t
+++ b/dist/Storable/t/malice.t
@@ -165,7 +165,7 @@ sub test_things {
 
     local $Storable::accept_future_minor = 0;
     test_corrupt ($copy, $sub,
-                  "/^Storable binary image v$header->{major}\.$minor6 more recent than I am \\(v$header->{major}\.$minor\\)/",
+                  "/^Stored data corrupt, not a Storable binary image, or binary image version v$header->{major}\.$minor6 more recent than I am \\(v$header->{major}\.$minor\\)/",
                   "higher minor");
   }
 
@@ -173,14 +173,14 @@ sub test_things {
   my $major1 = $header->{major} + 1;
   substr ($copy, $file_magic, 1) = chr 2*$major1;
   test_corrupt ($copy, $sub,
-                "/^Storable binary image v$major1\.$header->{minor} more recent than I am \\(v$header->{major}\.$minor\\)/",
+                "/^Stored data corrupt, not a Storable binary image, or binary image version v$major1\.$header->{minor} more recent than I am \\(v$header->{major}\.$minor\\)/",
                 "higher major");
 
   # Continue messing with the previous copy
   my $minor1 = $header->{minor} - 1;
   substr ($copy, $file_magic + 1, 1) = chr $minor1;
   test_corrupt ($copy, $sub,
-                "/^Storable binary image v$major1\.$minor1 more recent than I am \\(v$header->{major}\.$minor\\)/",
+                "/^Stored data corrupt, not a Storable binary image, or binary image version v$major1\.$minor1 more recent than I am \\(v$header->{major}\.$minor\\)/",
               "higher major, lower minor");
 
   my $where;
@@ -191,7 +191,7 @@ sub test_things {
     substr ($copy, $file_magic + 3, length $header->{byteorder})
       = reverse $header->{byteorder};
 
-    test_corrupt ($copy, $sub, "/^Byte order is not compatible/",
+    test_corrupt ($copy, $sub, "/^Stored data corrupt, not a Storable binary image, or byte order is not compatible/",
                   "byte order");
     $where = $file_magic + 3 + length $header->{byteorder};
     foreach (['intsize', "Integer"],
@@ -235,7 +235,7 @@ sub test_things {
     # local $Storable::DEBUGME = 1;
     local $Storable::accept_future_minor = 0;
     test_corrupt ($copy, $sub,
-                  "/^Storable binary image v$header->{major}\.$minor6 more recent than I am \\(v$header->{major}\.$minor\\)/",
+                  "/^Stored data corrupt, not a Storable binary image, or binary image version v$header->{major}\.$minor6 more recent than I am \\(v$header->{major}\.$minor\\)/",
                   "higher minor");
   }
 }
-- 
2.8.1

@p5pRT
Copy link
Author

p5pRT commented Jul 7, 2016

From @jkeenan

On Wed Jul 06 20​:51​:35 2016, dcollinsn@​gmail.com wrote​:

As promised - I propose this patch which changes the text of two error
messages to propose that the data might also be "corrupt or not a
Storable binary image". Tests are updated with the new error text.

Porting/cmp_version is "failing", but I'm not sure what to do with
that information, or if that's for the pumpking to $VERSION++.

If you modify a *.xs file in a distribution, you also need to increment $VERSION in the corresponding *.pm file. See second patch attached.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Jul 7, 2016

From @jkeenan

30807-0002-Increment-Storable-VERSION-to-2.57.patch
From 5f7fabc05a5e4119cc07f1b4eb592ac8f7a99d5d Mon Sep 17 00:00:00 2001
From: James E Keenan <jkeenan@cpan.org>
Date: Thu, 7 Jul 2016 08:19:58 -0400
Subject: [PATCH] Increment Storable $VERSION to 2.57.

For RT #30807; addition to patch submitted by dcollins.
---
 dist/Storable/Storable.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/dist/Storable/Storable.pm b/dist/Storable/Storable.pm
index c8f6db1..e928401 100644
--- a/dist/Storable/Storable.pm
+++ b/dist/Storable/Storable.pm
@@ -22,7 +22,7 @@ package Storable; @ISA = qw(Exporter);
 
 use vars qw($canonical $forgive_me $VERSION);
 
-$VERSION = '2.56';
+$VERSION = '2.57';
 
 BEGIN {
     if (eval { local $SIG{__DIE__}; require Log::Agent; 1 }) {
-- 
1.9.1

@jkeenan
Copy link
Contributor

jkeenan commented Mar 2, 2020

From @dcollinsn

Storable.xs​:

/*

  • The "magic number" is only for files, not when freezing in memory.
    */

a is decimal 97 is ascii. The first two bytes are interpreted as major.minor version, but the first byte is first rightshifted - apparently the least significant bit is used to encode something else.

So, "aa" actually survives partway through magic_check(), when Storable discovers that 48.97 is much too high a version. "a" is the unusual case, because there isn't a second byte to read​:

if (version_major > 1)
GETMARK(version_minor);

And there is no character to be had, so it returns from magic_check without setting any other diagnostics.

In any event - there is no magic number that we can possibly check for thaw() from memory. The first two bytes are the version number. There are a number of different ways we can get weird error messages. So we should really just give a better error message in the cases that are most likely to be accidentally stumbled on.

I think that's the "version more recent" error as well as the "byte order not compatible" error (which you can get by passing in "\004\012\000xxxx"). Any other error message requires a crafted string like "\004\012\01012345678xxxx", which seems unlikely to happen by accident.

Will upload the patch momentarily.

@DCOLLINS back in 2016 you proposed changes to Storable's error messages (see 1st patch in this gh-issue). Do you wish to pursue these?

Thank you very much.
Jim Keenan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants