From @steve-m-hay

This is a bug report for perl from steve.hay@​uk.radan.com,
generated with the help of perlbug 1.34 running under perl v5.8.1.

Hashes tied to SDBM_File don't seem to be able to cope with certain kinds of
data. I've experienced breakage with "real" data involving many small,
similar
keys and large (but not too large!) values; the test program below
reproduces
the problem using data like that​:

WARNING! This program will create a *very* large file, so make sure you
don't run out of space in your filesystem!

=====
use strict;
use warnings;

use Fcntl;
use SDBM_File;

tie my %h, 'SDBM_File', 'testsdbm', O_RDWR | O_CREAT | O_TRUNC, 0666
  or die "Couldn't tie SDBM file 'testsdbm'​: $!\n";

for my $i (0 .. 255) {
  for my $j (0 .. 255) {
  printf "%03d - %03d\n", $i, $j;
  for my $k (0 .. 5) {
  my $key = chr($i) . chr($j) . chr($k);
  my $val = ' ' x 1000;
  $h{$key} = $val;
  }
  }
}

untie %h;

The combined length of the key + value here is always 1003, which is
within the 1008 limit imposed by sdbm, so there should be no problem
there.

However, the above program always crashes with this error​:

  sdbm store returned -1, errno 22, key "..."

at the point where i==1 and j==0.

I've hacked the ext/SDBM_File/sdbm/sdbm.c file in the Perl source to
include some fprintf(stderr, ...) statements in pertinent places, and I
have found that it is the last lseek() within the main while { ... }
loop in makroom() that fails. The offset that is being passed to that
lseek() is -130153472 (when formatted with %ld, which should be correct
since the offset is a "long").

At the point where it failed, the .pag file was 1.87GB (to be exact, it
was 2,017,331,200 bytes), so I wondered if this might be a "large file"
problem with a 2GB limit since my Perl doesn't have large file support?

However, I retried the same program using ActivePerl Build 806 (which
has large file support enabled) and got the same error at the same
point.

The same program completes without error if I just change SDBM_File to
DB_File. The (single) file which it creates is only about 0.5GB when the
program is finished!

Flags​:
  category=library
  severity=medium

Site configuration information for perl v5.8.1​:

Configured by steveh at Thu Sep 25 17​:01​:07 2003.

Summary of my perl5 (revision 5 version 8 subversion 1) configuration​:
  Platform​:
  osname=MSWin32, osvers=4.0, archname=MSWin32-x86-perlio
  uname=''
  config_args='undef'
  hint=recommended, useposix=true, d_sigaction=undef
  usethreads=undef use5005threads=undef useithreads=undef
usemultiplicity=undef
  useperlio=define d_sfio=undef uselargefiles=undef usesocks=undef
  use64bitint=undef use64bitall=undef uselongdouble=undef
  usemymalloc=y, bincompat5005=undef
  Compiler​:
  cc='cl', ccflags ='-nologo -Gf -W3 -MD -DNDEBUG -O1 -DWIN32
-D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -DPERL_DEBUGGING_MSTATS
-DUSE_PERLIO -DPERL_MSVCRT_READFIX',
  optimize='-MD -DNDEBUG -O1',
  cppflags='-DWIN32'
  ccversion='', gccversion='', gccosandvers=''
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
  d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10
  ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=4
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='link', ldflags ='-nologo -nodefaultlib -release
-libpath​:"C​:\perl5\lib\CORE" -machine​:x86'
  libpth=C​:\PROGRA1\MICROS2\VC98\lib
  libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib version.lib
odbc32.lib odbccp32.lib msvcrt.lib
  perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib
winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib
oleaut32.lib netapi32.lib uuid.lib wsock32.lib mpr.lib winmm.lib
version.lib odbc32.lib odbccp32.lib msvcrt.lib
  libc=msvcrt.lib, so=dll, useshrplib=yes, libperl=perl58.lib
  gnulibc_version='undef'
  Dynamic Linking​:
  dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
  cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -release
-libpath​:"C​:\perl5\lib\CORE" -machine​:x86'

Locally applied patches​:
 

@​INC for perl v5.8.1​:
  C​:/perl5/lib
  C​:/perl5/site/lib
  .

Environment for perl v5.8.1​:
  HOME (unset)
  LANG (unset)
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
 
PATH=C​:\WINDOWS\system32;C​:\WINDOWS;C​:\WINDOWS\System32\Wbem;C​:\perl5\bin
  PERL5LIB (unset)
  PERL_BADLANG (unset)
  SHELL (unset)

From @jkeenan

On Tue Oct 14 08​:54​:57 2003, shay wrote​:

However, the above program always crashes with this error​:

sdbm store returned \-1\, errno 22\, key "\.\.\."

at the point where i==1 and j==0.

With the attached program 24206.pl, I was able to set how large a matrix
I wanted to try. I had no problem at 127, but when I tried the OP's
suggested 255, my program got farther than his but ultimately died with
a similar error.

069 - 128
sdbm store returned -1, errno 22, key "E?" at 24206.pl line 25.

This was on Darwin (Mac OS X 10.4.11).

Does anyone have any insight into this?

Thank you very much.
Jim Keenan

From @jkeenan

24206.pl

The RT System itself - Status changed from 'new' to 'open'

From @tonycoz

On Thu, Dec 01, 2011 at 06​:06​:28PM -0800, James E Keenan via RT wrote​:

On Tue Oct 14 08​:54​:57 2003, shay wrote​:

However, the above program always crashes with this error​:

sdbm store returned \-1\, errno 22\, key "\.\.\."

at the point where i==1 and j==0.

With the attached program 24206.pl, I was able to set how large a matrix
I wanted to try. I had no problem at 127, but when I tried the OP's
suggested 255, my program got farther than his but ultimately died with
a similar error.

069 - 128
sdbm store returned -1, errno 22, key "E?" at 24206.pl line 25.

This was on Darwin (Mac OS X 10.4.11).

Does anyone have any insight into this?

I suspect you're running into a 32-bit long issue.

On x64 Linux with a 64-bit long your test ran successfully, producing​:

-rw-r--r-- 1 tony tony 1048576 Dec 2 20​:54 testsdbm.dir
-rw-r--r-- 1 tony tony 8589907968 Dec 2 20​:54 testsdbm.pag

which a 64-bit offset can seek within successfully, not not 32-bit.

Tony

Yes, definitely a 32-bit issue on Windows. I saw this bug today with a script we typically run in production on Linux and wanted to see if it can be run on Windows. On Linux, our script creates a ~64GB .pag file. On Windows, it fails with "sdbm store returned -1, errno 22" when the .pag file reaches almost 2**31 bytes, 2147471360 to be exact.

Is this connected to issue #12227?

It might be partly connected to #12227.

Another problem is "long" is 32-bits for both 32 and 64-bit Win32: https://docs.microsoft.com/en-us/cpp/cpp/integer-limits?view=msvc-170

and sdbm.c casts its page numbers to long when calculating the seek offsets, which limits the offsets to long.