Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Custom cop_warnings take up 17 bytes per COP. #16249

Open
p5pRT opened this issue Nov 15, 2017 · 5 comments
Open

Custom cop_warnings take up 17 bytes per COP. #16249

p5pRT opened this issue Nov 15, 2017 · 5 comments
Assignees
@toddr
Labels
type-core
Projects
toddr

Comments

@p5pRT
Copy link

p5pRT commented Nov 15, 2017

Migrated from rt.perl.org#132451 (status was 'open')

Searchable as RT132451$
@p5pRT
Copy link
Author

p5pRT commented Nov 15, 2017

From @toddr

Created by @toddr

Every COP (line of perl code) stores a 17 byte (mostly) binary string in the COP
struct's cop_warnings field. This string is also known as ${^WARNING_BITS}.
Originally this meant that every line of code had 17 bytes allocated
to the every
line of Perl code to store this information. Way back in 2006, Nick
Clark optimized
this by having 3 of the most common warnings not actually be strings,
but instead
be special pointers (pWARN_ALL, pWARN_NONE, pWARN_STD) which literally
point to memory address 0,1,2. This saved a ton of memory but it means that
anything not in this special class will still take up more space at run time.

An example of this problem can be seen here​:

$> perl -e 'use warnings "once";
foreach (1 .. 10000)
{ eval "sub foo$_ { print qq{hello world\n};}";}
open $proc, "/proc/self/status";
print readline($proc);' | grep RSS

VmRSS​: 30896 kB

$> perl -e 'use warnings;
foreach (1 .. 10000)
{ eval "sub foo$_ { print qq{hello world\n};}";}
open $proc, "/proc/self/status";
print readline($proc);' | grep RSS

VmRSS​: 30632 kB

This example is a little contrived but shows the problem. My suggestion is
that we just store the shared strings in PL_strtab. I would prefer not to
refcount their use and just skip the free when the COP is let go but I'm
not committed to that position.

I'm vaguely concerned with XS poking at this field an manipulating it at
run time. I've not found any examples of that on CPAN but I don't feel
I've made a thorough search.

I'm working on a proposed patch for this. If anyone has concerns,
please let me know.

Perl Info 

Flags:
    category=core
    severity=low
    module=

Site configuration information for perl 5.27.6:

Configured by root at Fri Nov 10 21:43:02 CST 2017.

Summary of my perl5 (revision 5 version 27 subversion 6) configuration:
  Commit id: 8d6363863eb2ae5eb05a83c01c276a321727c828
  Platform:
    osname=linux
    osvers=3.10.0-693.2.2.el7.x86_64
    archname=x86_64-linux-thread-multi
    uname='linux toddr.dev.cpanel.net
 3.10.0-693.2.2.el7.x86_64 #1 smp
tue sep 12 22:26:13 utc 2017 x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Dusedevel -Uversiononly -Dcc=gcc -Dusethreads
-Dprefix= -DDEBUGGING'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='gcc'
    ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -DDEBUGGING
-fno-strict-aliasing -pipe -fstack-protector-strong
-I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
-D_FORTIFY_SOURCE=2'
    optimize='-O2 -g'
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -DDEBUGGING
-fno-strict-aliasing -pipe -fstack-protector-strong
-I/usr/local/include'
    ccversion=''
    gccversion='4.8.5 20150623 (Red Hat 4.8.5-16)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='gcc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib /lib/../lib64 /usr/lib/../lib64
/lib /lib64 /usr/lib64 /usr/local/lib64
    libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.17.so


    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version='2.17'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector-strong'



@INC for perl 5.27.6:
    /root/projects/perl/lib
    /usr/local/lib/perl5/site_perl/5.27.6/x86_64-linux-thread-multi
    /usr/local/lib/perl5/site_perl/5.27.6
    /usr/local/lib/perl5/5.27.6/x86_64-linux-thread-multi
    /usr/local/lib/perl5/5.27.6


Environment for perl 5.27.6:
    HOME=/root
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH=/root/projects/perl
    LOGDIR (unset)
    PATH=/bin
    PERL_BADLANG (unset)
    SHELL=/bin/zsh

@p5pRT
Copy link
Author

p5pRT commented Sep 4, 2018 •
edited by toddr

From @jkeenan

Have you been able to make progress on that patch?

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Sep 4, 2018

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Sep 4, 2018

From @toddr

On Mon, 03 Sep 2018 17​:47​:59 -0700, jkeenan wrote​:

Have you been able to make progress on that patch?

Thank you very much.

I have not but we're looking at B​::C for 5.28.0 right now, so let me get back to you in a week.

@toddr toddr self-assigned this Feb 17, 2020
@toddr toddr added this to Want to work on it. in toddr Feb 17, 2020
@atoomic
Copy link
Member

atoomic commented Mar 10, 2020

one liner to see the issue

# pWARN_STD
╰─> perl -e 'use warnings; eval "sub foo$_ { print qq{hello world\n};}" for (1..100_000); print qx{grep RSS /proc/$$/status}'
VmRSS:	  293304 kB
# warnings once
╰─> perl -e 'use warnings "once"; eval "sub foo$_ { print qq{hello world\n};}" for (1..100_000); print qx{grep RSS /proc/$$/status}'
VmRSS:	  297888 kB

atoomic added a commit to atoomic/perl5 that referenced this issue Mar 10, 2020
@atoomic
Try using sharepv for cop_warnings
b68d524 
GH Perl#16249

This is an attempt to use PL_strtab
to store the cop PVs.

Work in Progress...
@xenu xenu removed the affects-5.27 label Nov 19, 2021
@xenu xenu removed the Severity Low label Dec 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@toddr toddr

Labels
type-core
Projects
toddr
  
Want to work on it.
Milestone
No milestone
Development

No branches or pull requests
4 participants
@toddr @xenu @atoomic @p5pRT