Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

miniperl creates 8gb output under shell redirection #12615

Closed
p5pRT opened this issue Nov 25, 2012 · 14 comments
Closed

miniperl creates 8gb output under shell redirection #12615

p5pRT opened this issue Nov 25, 2012 · 14 comments
Assignees
Labels
Closable? We might be able to close this ticket, but we need to check with the reporter distro-mswin32 miniperl miniperl, minitest and similar 'make' targets type-core

Comments

@p5pRT
Copy link

p5pRT commented Nov 25, 2012

Migrated from rt.perl.org#115904 (status was 'open')

Searchable as RT115904$

@p5pRT
Copy link
Author

p5pRT commented Nov 25, 2012

From @wchristian

Created by @wchristian

In any checkout with the following git commit id, or after
925798f, the following commands
will cause a file containing 8GB of \o to be written​:

  cd win32
  dmake .\config.h ..\git_version.h
  cd ..
  miniperl -e "open STDERR_COPY, '>&STDERR'; print 1;" 1>log 2>&1
  perl -e "print -s 'log'"

This is on Windows 7, using the MinGW included in
ActivePerl 5.12.4. I also tried with the MinGW included in the
latest Strawberry Perl, but that resulted only in compile errors.

Here's a full log of the broken behavior of miniperl​:

  https://gist.github.com/4142366

Here's a full log of miniperl acting correctly​:

  https://gist.github.com/4142377

The commit in question includes this line in the commit message​:

  - Set lseeksize/lseektype to 4/long in new gc64* files
  (see f7e8b52)

The mentioned commit has this message, which seems to be
related to the problem​:

  Change 25208 switched off USE_LARGE_FILES in win32/config_H.*
  but left LSEEKSIZE/Off_t_size and Off_t as 8 and __int64 (or
  long long) respectively. Similarly change 25215 switched off
  uselargefiles in win32/config.* but left lseeksize and
  lseektype as 8 and __int64 (or long long) respectively. Change
  25216 fixed the Borland settings in win32/config.bc on the
  basis that Borland should always be using 4 and long, but
  really all the other files should be using 4 and long for
  their default values as well to match the default values of
  USE_LARGE_FILES and uselargefiles. Having done that, we must
  then reverse the logic for fiddling with these values in
  win32/config_sh.PL​: they are now changed to 8 and __int64 (or
  long long) if uselargefiles *is* defined (except for Borland,
  which always wants 4 and long).

Perl Info

Flags:
     category=core
     severity=medium

Site configuration information for perl 5.12.4:

Configured by gecko at Mon Jun 20 18:32:45 2011.

Summary of my perl5 (revision 5 version 12 subversion 4) configuration:

   Platform:
     osname=MSWin32, osvers=5.2, archname=MSWin32-x86-multi-thread
     uname=''
     config_args='undef'
     hint=recommended, useposix=true, d_sigaction=undef
     useithreads=define, usemultiplicity=define
     useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
     use64bitint=undef, use64bitall=undef, uselongdouble=undef
     usemymalloc=n, bincompat5005=undef
   Compiler:
     cc='C:/Perl/site/bin/gcc.exe', ccflags ='-DNDEBUG -DWIN32 -D_CONSOLE  
-DNO_STRICT -DHAVE_DES_FCRYPT -DUSE_SITECUSTOMIZE -DPERL_IMPLICIT_CONTEXT  
-DPERL_IMPLICIT_SYS -DUSE_PERLIO -D_USE_32BIT_TIME_T -DPERL_MSVCRT_READFIX  
-DHASATTRIBUTE -fno-strict-aliasing -mms-bitfields',
     optimize='-O2',
     cppflags='-DWIN32'
     ccversion='', gccversion='3.4.5 (mingw-vista special r3)',  
gccosandvers=''
     intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
     d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=8
     ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64',  
lseeksize=8
     alignbytes=8, prototype=define
   Linker and Libraries:
     ld='C:\Perl\site\bin\g++.exe', ldflags ='-L"C:\Perl\lib\CORE"'
     libpth=\lib
     libs=-lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32  
-lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm  
-lversion -lodbc32 -lodbccp32 -lcomctl32 -lmsvcrt
     perllibs=-lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32  
-lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm  
-lversion -lodbc32 -lodbccp32 -lcomctl32 -lmsvcrt
     libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl512.lib
     gnulibc_version=''
   Dynamic Linking:
     dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
     cccdlflags=' ', lddlflags='-mdll -L"C:\Perl\lib\CORE"'

Locally applied patches:
     ACTIVEPERL_LOCAL_PATCHES_ENTRY
     c6fbf28 [perl #71806] perldb does not setup %dbline with the shebang  
option -d
     1fd8fa4 Add Wolfram Humann to AUTHORS
     f120055 make string-append on win32 100 times faster
     a2a8d15 Define _USE_32BIT_TIME_T for VC6 and VC7
     007cfe1 Don't pretend to support really old VC++ compilers
     6d8f7c9 Get rid of obsolete PerlCRT.dll support
     d956618 Make Term::ReadLine::findConsole fall back to STDIN if  
/dev/tty can't be opened
     321e50c Escape patch strings before embedding them in patchlevel.h


@INC for perl 5.12.4:
     C:/Perl/site/lib/MSWin32-x86-multi-thread
     C:/Perl/site/lib
     C:/Perl/lib
     .


Environment for perl 5.12.4:
     CYGWIN=nodosfilewarning
     HOME (unset)
     LANG=en_US.utf8
     LANGUAGE (unset)
     LD_LIBRARY_PATH (unset)
     LOGDIR (unset)
     PATH=C:\Program Files (x86)\ActiveState Komodo IDE 7\;C:\Program Files  
(x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files (x86)\PC  
Connectivity Solution\;C:\Perl64-16\site\bin;C:\Perl64-16\bin;C:\Program  
Files (x86)\ImageMagick-6.7.7-Q16;%CommonProgramFiles%\Microsoft  
Shared\Windows  
Live;C:\Perl\site\bin;C:\Perl\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program  
Files\Intel\WiFi\bin\;C:\Program Files\Common  
Files\Intel\WirelessCommon\;C:\Program Files\ThinkPad\Bluetooth  
Software\;C:\Program Files\ThinkPad\Bluetooth  
Software\syswow64;C:\SWTOOLS\ReadyApps;C:\Program Files  
(x86)\Lenovo\Access Connections\;c:\Program Files  
(x86)\PuTTY\;c:\cygwin\bin\;C:\Program Files  
(x86)\QuickTime\QTSystem\;C:\Program Files  
(x86)\Intel\Services\IPT\;C:\Program Files  
(x86)\GtkSharp\2.12\bin;d:\j\MeCab\bin\;c:\Python27\;c:\Python27\Scripts\;c:\Program  
Files (x86)\CMake  
2.8\bin\;C:\strawberry\c\bin;C:\strawberry\perl\site\bin;C:\strawberry\perl\bin;C:\Program  
Files (x86)\Calibre2\;C:\Program Files (x86)\GNU\GnuPG\pub;C:\Program  
Files\TortoiseSVN\bin;C:\Program Files (x86)\Git\cmd;C:\Program  
Files\TortoiseGit\bin;C:\Ruby193\bin;C:\Program Files\Common  
Files\Microsoft Shared\Windows Live;C:\Program  
Files\Intel\WiFi\bin\;C:\Program Files\Common  
Files\Intel\WirelessCommon\;C:\Program Files (x86)\OpenVPN\bin;C:\Program  
Files\Mercurial
     PERL_BADLANG (unset)
     PERL_JSON_BACKEND=JSON::XS
     PERL_STRICTURES_EXTRA=1
     PERL_YAML_BACKEND=YAML::XS
     SHELL (unset)

@p5pRT
Copy link
Author

p5pRT commented Nov 25, 2012

From @wchristian

Small correction, of course i mean \0, not \o. It is a bit early in the
morning here.

@p5pRT
Copy link
Author

p5pRT commented Nov 25, 2012

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Nov 25, 2012

From @wchristian

An extra piece of info i found out with kmx, apparently the resulting
miniperl executable thinks​:

This is perl 5, version 15, subversion 2 (007BB008) built for MSWin32-x86

This seems wrong. Might it be caused by the makefile not correctly
setting up a 64 bit compile?

@p5pRT
Copy link
Author

p5pRT commented Nov 27, 2012

From @bulk88

On Sat Nov 24 20​:43​:13 2012, walde.christian@​googlemail.com wrote​:

This is a bug report for perl from walde.christian@​googlemail.com,
generated with the help of perlbug 1.39 running under perl 5.12.4.

-----------------------------------------------------------------
[Please describe your issue here]

In any checkout with the following git commit id, or after
925798f, the following commands
will cause a file containing 8GB of \o to be written​:

cd win32
dmake .\config.h ..\git_version.h
cd ..
miniperl -e "open STDERR_COPY, '>&STDERR'; print 1;" 1>log 2>&1
perl -e "print -s 'log'"

I couldn't reproduce it with my VC miniperl (the commit is probably
5c26a17 but I will soon update)
_______________________________________________________
C​:\p517\src\lib>..\miniperl -V
Summary of my perl5 (revision 5 version 17 subversion 6) configuration​:

  Platform​:
  osname=MSWin32, osvers=5.2, archname=MSWin32-x86-multi-thread
  uname=''
  config_args='undef'
  hint=recommended, useposix=true, d_sigaction=undef
  useithreads=define, usemultiplicity=define
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=undef, use64bitall=undef, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cl', ccflags ='-nologo -GF -W3 -MD -Zi -DNDEBUG -Od -GL -GS-
-arch​:SSE2
-DWIN32 -D_CONSOLE -DNO_STRICT -D_CRT_SECURE_NO_DEPRECATE
-D_CRT_NONSTDC_NO_DEPR
ECATE -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT
-DPERL_IMPLICIT_SYS -DUSE
_PERLIO',
  optimize='-MD -Zi -DNDEBUG -Od -GL -GS- -arch​:SSE2',
  cppflags='-DWIN32'
  ccversion='15.00.30729.01', gccversion='', gccosandvers=''
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
  d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=8
  ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64',
lseeksi
ze=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='link', ldflags ='-nologo -nodefaultlib -debug -opt​:ref,icf -ltcg
-libpa
th​:"c​:\p517\lib\CORE" -machine​:x86 "/manifestdependency​:type='Win32'
name='Micr
osoft.Windows.Common-Controls' version='6.0.0.0'
processorArchitecture='*' publi
cKeyToken='6595b64144ccf1df' language='*'"'
  libpth=\lib
  libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.l
ib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib
uuid.lib ws2_32
.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib
msvcrt.
lib
  perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg
32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib
uuid.lib ws
2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib
comctl32.lib msv
crt.lib
  libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl517.lib
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
  cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug
-opt​:ref,icf -l
tcg -libpath​:"c​:\p517\lib\CORE" -machine​:x86
"/manifestdependency​:type='Win32'
name='Microsoft.Windows.Common-Controls' version='6.0.0.0'
processorArchitectur
e='*' publicKeyToken='6595b64144ccf1df' language='*'"'

Characteristics of this binary (from libperl)​:
  Compile-time options​: HAS_TIMES HAVE_INTERP_INTERN MULTIPLICITY
  PERLIO_LAYERS PERL_DONT_CREATE_GVSV
  PERL_EXTERNAL_GLOB PERL_IMPLICIT_CONTEXT
  PERL_IS_MINIPERL PERL_MALLOC_WRAP PERL_PRESERVE_IVUV
  USE_ITHREADS USE_LARGE_FILES USE_LOCALE
  USE_LOCALE_COLLATE USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC USE_PERLIO USE_PERL_ATOF
  USE_SITECUSTOMIZE
  Built under MSWin32
  Compiled at Nov 7 2012 23​:42​:27
  @​INC​:
  .
________________________________________________________
I got 1 byte long file on Sever 2003 x64, with 32 bit perl. I know there
was a #win32 conversation on this bug recently but I didn't log it and
dont remember what it said.
--
bulk88 ~ bulk88 at hotmail.com

@p5pRT
Copy link
Author

p5pRT commented Nov 27, 2012

From @wchristian

On Tue Nov 27 02​:36​:39 2012, bulk88 wrote​:

I couldn't reproduce it with my VC miniperl

Thanks for checking. So it seems this issue is only with MinGW. (I'm
specifically using the MinGW included with ActivePerl 5.12.4.)

@p5pRT
Copy link
Author

p5pRT commented Nov 27, 2012

From @bulk88

On Tue Nov 27 03​:37​:05 2012, Mithaldu wrote​:

On Tue Nov 27 02​:36​:39 2012, bulk88 wrote​:

I couldn't reproduce it with my VC miniperl

Thanks for checking. So it seems this issue is only with MinGW. (I'm
specifically using the MinGW included with ActivePerl 5.12.4.)

Also tried with 9566f14 (blead today),
VC 2003, 32 bit on x64 2003, couldnt reproduce.
--
bulk88 ~ bulk88 at hotmail.com

@p5pRT
Copy link
Author

p5pRT commented Nov 27, 2012

From @wchristian

Alright, after a lengthy session of looking at things with bulk88,
another aspect of the problem is now clear​:

I compiled perl on Win7-64, using the MinGW included in 32 bit Perl
distros. Due to the OS, the makefile decides to use 64 bit header files,
but due to the compiler the resulting binary is actually a 32 bit binary,
using 64 bit configuration values.

The -v/-V output of miniperl demonstrates this​: https​://
gist.github.com/7673e573f642a7896d70 and analysis of the PE headers
supports it.

At first blush one could think that this means the correct solution is to
use a 64 bit compiler instead. I have started a smoke of blead with a 64
bit MinGW (gcc version 4.5.4 (rubenvb-4.5.4)) and its miniperl does not
exhibit the problematic 8 gb behavior. However now it just doesn't output
anything anymore. The command mentioned in the first message of this bug
results in the file "log" being 0 bytes long.

@p5pRT
Copy link
Author

p5pRT commented Nov 29, 2012

From @bulk88

On Tue Nov 27 06​:44​:11 2012, Mithaldu wrote​:

Alright, after a lengthy session of looking at things with bulk88,
another aspect of the problem is now clear​:

I compiled perl on Win7-64, using the MinGW included in 32 bit Perl
distros. Due to the OS, the makefile decides to use 64 bit header files,
but due to the compiler the resulting binary is actually a 32 bit binary,
using 64 bit configuration values.

The -v/-V output of miniperl demonstrates this​: https​://
gist.github.com/7673e573f642a7896d70 and analysis of the PE headers
supports it.

At first blush one could think that this means the correct solution is to
use a 64 bit compiler instead. I have started a smoke of blead with a 64
bit MinGW (gcc version 4.5.4 (rubenvb-4.5.4)) and its miniperl does not
exhibit the problematic 8 gb behavior. However now it just doesn't output
anything anymore. The command mentioned in the first message of this bug
results in the file "log" being 0 bytes long.

I looked at Mithaldu's miniperl' asm code, in pp_sysread, the SSize_t
vars were 64 bit ints. His miniperl is 32 bit asm code. It is as if you
forgot to uncomment WIN64 at the top of the makefile, but no
architecture is being forced from the makefile to GCC I think, so GCC
compiles as "default" (32 bits in this case), even though perl's headers
and config.h and makefile are set for x64 perl.
--
bulk88 ~ bulk88 at hotmail.com

@p5pRT
Copy link
Author

p5pRT commented Dec 14, 2013

From @jkeenan

On Tue Nov 27 06​:44​:11 2012, Mithaldu wrote​:

Alright, after a lengthy session of looking at things with bulk88,
another aspect of the problem is now clear​:

I compiled perl on Win7-64, using the MinGW included in 32 bit Perl
distros. Due to the OS, the makefile decides to use 64 bit header files,
but due to the compiler the resulting binary is actually a 32 bit binary,
using 64 bit configuration values.

The -v/-V output of miniperl demonstrates this​: https​://
gist.github.com/7673e573f642a7896d70 and analysis of the PE headers
supports it.

At first blush one could think that this means the correct solution is to
use a 64 bit compiler instead. I have started a smoke of blead with a 64
bit MinGW (gcc version 4.5.4 (rubenvb-4.5.4)) and its miniperl does not
exhibit the problematic 8 gb behavior. However now it just doesn't output
anything anymore. The command mentioned in the first message of this bug
results in the file "log" being 0 bytes long.

Mithaldu, bulk88​: Would it be possible to get an update on the status of this ticket?

Thank you very much.
Jim Keenan

@p5pRT
Copy link
Author

p5pRT commented Dec 15, 2013

From @bulk88

On Sat Dec 14 13​:25​:05 2013, jkeenan wrote​:

Mithaldu, bulk88​: Would it be possible to get an update on the status
of this ticket?

Thank you very much.
Jim Keenan

I didn't compile the perl causing the 8 GB file, I only debugged the binaries Mithaldu sent me.

--
bulk88 ~ bulk88 at hotmail.com

@toddr
Copy link
Member

toddr commented Feb 13, 2020

@wchristian barring further information, I don't see anything actionable here. Are we ok to close?

@toddr toddr added the Closable? We might be able to close this ticket, but we need to check with the reporter label Feb 13, 2020
@xenu xenu removed the affects-5.12 label Nov 19, 2021
@jkeenan jkeenan added the miniperl miniperl, minitest and similar 'make' targets label Jun 9, 2022
@jkeenan jkeenan self-assigned this Jul 11, 2022
@jkeenan
Copy link
Contributor

jkeenan commented Jul 11, 2022

@wchristian barring further information, I don't see anything actionable here. Are we ok to close?

Self-assigning for the purpose of closing in 7 days unless the discussion is moved forward.

@jkeenan
Copy link
Contributor

jkeenan commented Jul 18, 2022

@wchristian barring further information, I don't see anything actionable here. Are we ok to close?

Self-assigning for the purpose of closing in 7 days unless the discussion is moved forward.

Closing as per schedule.

@jkeenan jkeenan closed this as completed Jul 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Closable? We might be able to close this ticket, but we need to check with the reporter distro-mswin32 miniperl miniperl, minitest and similar 'make' targets type-core
Projects
None yet
Development

No branches or pull requests

4 participants