Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

reading a line from a filehandle results in eating all ram #15062

Open
p5pRT opened this issue Nov 24, 2015 · 6 comments
Open

reading a line from a filehandle results in eating all ram #15062

p5pRT opened this issue Nov 24, 2015 · 6 comments
Labels
distro-Linux type-core

Comments

@p5pRT
Copy link

p5pRT commented Nov 24, 2015

Migrated from rt.perl.org#126719 (status was 'open')

Searchable as RT126719$
@p5pRT
Copy link
Author

p5pRT commented Nov 24, 2015

From michael.adamcik@pro-pos.at

Created by michael.adamcik@pro-pos.at

This is a bug report for perl from michael.adamcik@​pro-pos.at,
generated with the help of perlbug 1.40 running under perl 5.20.2.

-----------------------------------------------------------------
opening a html file with the encoding detected by io​::html, in this case
"​:encoding(iso-2022-jp)", reading a (specific) line results in eating up
all memory/swap.
i reported the bug also to the maintainer of io​::html.

i will attach a small script and the html which procuce this behaviour.

Perl Info 

Flags:
     category=core
     severity=critical

Site configuration information for perl 5.20.2:

Configured by Debian Project at Sat May 16 11:36:34 UTC 2015.

Summary of my perl5 (revision 5 version 20 subversion 2) configuration:

   Platform:
     osname=linux, osvers=3.16.0-4-amd64, 
archname=x86_64-linux-gnu-thread-multi
     uname='linux himalia 3.16.0-4-amd64 #1 smp debian 
3.16.7-ckt9-3~deb8u1 (2015-04-24) x86_64 gnulinux '
     config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN 
-D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector-strong -Wformat 
-Werror=format-security -Dldflags= -Wl,-z,relro -Dlddlflags=-shared 
-Wl,-z,relro -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu 
-Dprefix=/usr -Dprivlib=/usr/share/perl/5.20 
-Darchlib=/usr/lib/x86_64-linux-gnu/perl/5.20 -Dvendorprefix=/usr 
-Dvendorlib=/usr/share/perl5 
-Dvendorarch=/usr/lib/x86_64-linux-gnu/perl5/5.20 
-Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.20.2 
-Dsitearch=/usr/local/lib/x86_64-linux-gnu/perl/5.20.2 
-Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 
-Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 
-Duse64bitint -Dman1ext=1 -Dman3ext=3perl 
-Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio 
-Uusenm -Ui_libutil -Uversiononly -DDEBUGGING=-g -Doptimize=-O2 
-Duseshrplib -Dlibperl=libperl.so.5.20.2 -des'
     hint=recommended, useposix=true, d_sigaction=define
     useithreads=define, usemultiplicity=define
     use64bitint=define, use64bitall=define, uselongdouble=undef
     usemymalloc=n, bincompat5005=undef
   Compiler:
     cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv 
-fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE 
-D_FILE_OFFSET_BITS=64',
     optimize='-O2 -g',
     cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv 
-fno-strict-aliasing -pipe -I/usr/local/include'
     ccversion='', gccversion='4.9.2', gccosandvers=''
     intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
     d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
     ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', 
lseeksize=8
     alignbytes=8, prototype=define
   Linker and Libraries:
     ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
     libpth=/usr/local/lib 
/usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed 
/usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib 
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
     libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
     perllibs=-ldl -lm -lpthread -lc -lcrypt
     libc=libc-2.19.so, so=so, useshrplib=true, libperl=libperl.so.5.20
     gnulibc_version='2.19'
   Dynamic Linking:
     dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
     cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib 
-fstack-protector'

Locally applied patches:
     DEBPKG:debian/cpan_definstalldirs - Provide a sensible INSTALLDIRS 
default for modules installed from CPAN.
     DEBPKG:debian/db_file_ver - http://bugs.debian.org/340047 Remove 
overly restrictive DB_File version check.
     DEBPKG:debian/doc_info - Replace generic man(1) instructions with 
Debian-specific information.
     DEBPKG:debian/enc2xs_inc - http://bugs.debian.org/290336 Tweak 
enc2xs to follow symlinks and ignore missing @INC directories.
     DEBPKG:debian/errno_ver - http://bugs.debian.org/343351 Remove 
Errno version check due to upgrade problems with long-running processes.
     DEBPKG:debian/libperl_embed_doc - http://bugs.debian.org/186778 
Note that libperl-dev package is required for embedded linking
     DEBPKG:fixes/respect_umask - Respect umask during installation
     DEBPKG:debian/writable_site_dirs - Set umask approproately for site 
install directories
     DEBPKG:debian/extutils_set_libperl_path - EU:MM: set location of 
libperl.a under /usr/lib
     DEBPKG:debian/no_packlist_perllocal - Don't install .packlist or 
perllocal.pod for perl or vendor
     DEBPKG:debian/prefix_changes - Fiddle with *PREFIX and variables 
written to the makefile
     DEBPKG:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to the 
binary targets.
     DEBPKG:debian/instmodsh_doc - Debian policy doesn't install 
.packlist files for core or vendor.
     DEBPKG:debian/ld_run_path - Remove standard libs from LD_RUN_PATH 
as per Debian policy.
     DEBPKG:debian/libnet_config_path - Set location of libnet.cfg to 
/etc/perl/Net as /usr may not be writable.
     DEBPKG:debian/mod_paths - Tweak @INC ordering for Debian
     DEBPKG:debian/module_build_man_extensions - 
http://bugs.debian.org/479460 Adjust Module::Build manual page 
extensions for the Debian Perl policy
     DEBPKG:debian/prune_libs - http://bugs.debian.org/128355 Prune the 
list of libraries wanted to what we actually need.
     DEBPKG:fixes/net_smtp_docs - [rt.cpan.org #36038] 
http://bugs.debian.org/100195 Document the Net::SMTP 'Port' option
     DEBPKG:debian/perlivp - http://bugs.debian.org/510895 Make perlivp 
skip include directories in /usr/local
     DEBPKG:debian/deprecate-with-apt - http://bugs.debian.org/747628 
Point users to Debian packages of deprecated core modules
     DEBPKG:debian/squelch-locale-warnings - 
http://bugs.debian.org/508764 Squelch locale warnings in Debian package 
maintainer scripts
     DEBPKG:debian/skip-upstream-git-tests - Skip tests specific to the 
upstream Git repository
     DEBPKG:debian/patchlevel - http://bugs.debian.org/567489 List 
packaged patches for 5.20.2-6 in patchlevel.h
     DEBPKG:debian/skip-kfreebsd-crash - http://bugs.debian.org/628493 
[perl #96272] Skip a crashing test case in t/op/threads.t on GNU/kFreeBSD
     DEBPKG:fixes/document_makemaker_ccflags - 
http://bugs.debian.org/628522 [rt.cpan.org #68613] Document that CCFLAGS 
should include $Config{ccflags}
     DEBPKG:debian/find_html2text - http://bugs.debian.org/640479 
Configure CPAN::Distribution with correct name of html2text
     DEBPKG:debian/perl5db-x-terminal-emulator.patch - 
http://bugs.debian.org/668490 Invoke x-terminal-emulator rather than 
xterm in perl5db.pl
     DEBPKG:debian/cpan-missing-site-dirs - 
http://bugs.debian.org/688842 Fix CPAN::FirstTime defaults with 
nonexisting site dirs if a parent is writable
     DEBPKG:fixes/memoize_storable_nstore - [rt.cpan.org #77790] 
http://bugs.debian.org/587650 Memoize::Storable: respect 'nstore' option 
not respected
     DEBPKG:debian/regen-skip - Skip a regeneration check in unrelated 
git repositories
     DEBPKG:fixes/regcomp-mips-optim - [perl #122817] 
http://bugs.debian.org/754054 Downgrade the optimization of regcomp.c on 
mips and mipsel due to a gcc-4.9 bug
     DEBPKG:debian/makemaker-pasthru - http://bugs.debian.org/758471 
Pass LD settings through to subdirectories
     DEBPKG:fixes/perldoc-less-R - [rt.cpan.org #98636] 
http://bugs.debian.org/758689 Tell the 'less' pager to allow terminal 
escape sequences
     DEBPKG:fixes/pod_man_reproducible_date - 
http://bugs.debian.org/759405 Support POD_MAN_DATE in Pod::Man for the 
left-hand footer
     DEBPKG:fixes/io_uncompress_gunzip_inmemory - 
http://bugs.debian.org/747363 [rt.cpan.org #95494] Fix gunzip to 
in-memory file handle
     DEBPKG:fixes/socket_test_recv_fix - http://bugs.debian.org/758718 
[perl #122657] Compare recv return value to peername in socket test
     DEBPKG:fixes/hurd_socket_recv_todo - http://bugs.debian.org/758718 
[perl #122657] TODO checking the result of recv() on hurd
     DEBPKG:fixes/regexp-performance - [0fa70a0] 
http://bugs.debian.org/777556 [perl #123743] simpify and speed up 
/.*.../ handling
     DEBPKG:fixes/failed_require_diagnostics - 
http://bugs.debian.org/781120 [perl #123270] Report inaccesible file on 
failed require
     DEBPKG:fixes/array-cloning - http://bugs.debian.org/779357 [perl 
#124127] [902d169] fix cloning arrays with unused elements
     DEBPKG:fixes/perldb-threads - http://bugs.debian.org/779357 [perl 
#124127] [41ef2c6] lib/perl5db.pl: Restore noop lock prototype
     DEBPKG:debian/locale-robustness - http://bugs.debian.org/782068 
[perl #124310] Make t/run/locale.t survive missing locales masked by LC_ALL
     DEBPKG:fixes/gcc5-errno - http://bugs.debian.org/778060 [perl 
#123784] [816b056] Fix Errno.pm generation for gcc-5.0
     DEBPKG:fixes/h2ph-hex-constants - http://bugs.debian.org/778060 
[perl #123784] [3bea78d] h2ph: correct handling of hex constants for the 
preamble
     DEBPKG:fixes/h2ph-test-inc - http://bugs.debian.org/778060 [perl 
#123784] [3359391] lib/h2ph.t to test generated t/_h2ph_pre.ph instead 
of the system one
     DEBPKG:fixes/podman-utc - http://bugs.debian.org/780259 Make the 
embedded date from Pod::Man reproducible
     DEBPKG:fixes/podman-utc-docs - http://bugs.debian.org/780259 
Documentation and test suite updates for UTC fix
     DEBPKG:fixes/podman-empty-date - http://bugs.debian.org/780259 
Support an empty POD_MAN_DATE environment variable
     DEBPKG:fixes/podman-pipe - http://bugs.debian.org/777405 Better 
errors for man pages from standard input
     DEBPKG:debian/pod2man-customized - Update porting/customized.dat 
for pod2man modifications


@INC for perl 5.20.2:
     /etc/perl
     /usr/local/lib/x86_64-linux-gnu/perl/5.20.2
     /usr/local/share/perl/5.20.2
     /usr/lib/x86_64-linux-gnu/perl5/5.20
     /usr/share/perl5
     /usr/lib/x86_64-linux-gnu/perl/5.20
     /usr/share/perl/5.20
     /usr/local/lib/site_perl
     .


Environment for perl 5.20.2:
     HOME=/home/mike
     LANG=de_AT.UTF-8
     LANGUAGE (unset)
     LD_LIBRARY_PATH (unset)
     LOGDIR (unset)
     PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
     PERL_BADLANG (unset)
     SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Nov 24, 2015

From michael.adamcik@pro-pos.at

bug.pl

@p5pRT
Copy link
Author

p5pRT commented Nov 24, 2015

From @tonycoz

On Mon Nov 23 17​:40​:12 2015, michael.adamcik@​pro-pos.at wrote​:

opening a html file with the encoding detected by io​::html, in this case
"​:encoding(iso-2022-jp)", reading a (specific) line results in eating up
all memory/swap.
i reported the bug also to the maintainer of io​::html.

i will attach a small script and the html which procuce this behaviour.

This looks like a bug in Encode for that encoding, it isn't consuming any of the supplied data, so PerlIO​::encoding can't fill it's buffer.

As to the memory usage, I suspect that's from the temp svs returned by the call to the decode() method. In the usual case there would only be one or two temps before the code fell through to the FREETMPS at the end of PerlIOEncode_fill(), but in this case the code is looping so the memory doesn't get freed.

Tony

@p5pRT
Copy link
Author

p5pRT commented Nov 24, 2015

From @tonycoz

126713d.pl

@p5pRT
Copy link
Author

p5pRT commented Nov 24, 2015

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Nov 24, 2015

From @tonycoz

On Mon Nov 23 21​:51​:09 2015, tonyc wrote​:

On Mon Nov 23 17​:40​:12 2015, michael.adamcik@​pro-pos.at wrote​:

opening a html file with the encoding detected by io​::html, in this
case
"​:encoding(iso-2022-jp)", reading a (specific) line results in eating
up
all memory/swap.
i reported the bug also to the maintainer of io​::html.

i will attach a small script and the html which procuce this
behaviour.

This looks like a bug in Encode for that encoding, it isn't consuming
any of the supplied data, so PerlIO​::encoding can't fill it's buffer.

Reported upstream as https://rt.cpan.org/Ticket/Display.html?id=109555

Perhaps PerlIOEncode_fill() should be more robust - if it reaches end-of-file on the lower layer and the decoder still isn't returning any data, croak.

Tony

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
distro-Linux type-core
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@xenu @p5pRT