Skip Menu |
Report information
Id: 94234
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: sartak <sartak [at] gmail.com>
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: Patch
Perl Version: (no value)
Fixed In: (no value)



Subject: [PATCH] New DTrace probe for changes to global phase
Date: Wed, 6 Jul 2011 23:58:29 -0400
To: perlbug [...] perl.org
From: Sartak <sartak [...] gmail.com>

Message body is not shown because sender requested not to inline it.

Message body is not shown because sender requested not to inline it.

Download (untitled) / with headers
text/plain 3.5k
Hi porters,

I've recently developed somewhat of a DTrace kick, and I figured what better way to shake that out of my system than by improving Perl's DTrace support? As of 5.10.1, we have probes for subroutine entry and exit, which *does* get us a long way, but we could do even better. There are some patches floating around (like at http://fosiki.com/blog/2008/02/15/blead-perl-dtrace-probes/ ) to improve Perl's DTrace support in other ways. For this first new probe (an easy and useful one: global phase changes) I decided to piggy-back on top of Florian++'s new ${^GLOBAL_PHASE} variable, since that provides more visibility than the aforelinked patch, and is easily understood in terms of tangible Perl features.

My first patch does a tiny bit of refactoring to change all those literal:

PL_phase = PERL_PHASE_END;

assignments to use a new macro instead:

PERL_SET_PHASE(PERL_PHASE_END);

The second patch, the one with the actual meat in it, adds to that new PERL_SET_PHASE macro a DTrace probe. This new probe gives DTrace users additional visibility into _when_ certain things happen. For example, this DTrace invocation:

sudo dtrace -qZn ':perl::phase-change /copyinstr(arg0) == "END"/ { self->cleaning_up = 1 } :perl::sub-entry /self->cleaning_up && copyinstr(arg0) != "END"/ { printf("%s::%s at %s line %d\n", copyinstr(arg3), copyinstr(arg0), copyinstr(arg1), arg2) }'

will list all of the function calls made during interpreter cleanup (excluding the sneaky sorta-function "END"). You might use this to help diagnose why the interpreter doesn't exit immediately after the main processing is done. You can play with it by running this one-liner in another shell:

perl -MFile::Temp -MTest::Builder -e 'sub foo {} sub bar {} END { foo } bar'

which for me gives:

main::foo at -e line 1
Test::Builder::_ending at /Users/sartak/.perl/perls/dtrace/lib/5.15.0/Test/Builder.pm line 2389
Test::Builder::__ANON__ at /Users/sartak/.perl/perls/dtrace/lib/5.15.0/Test/Builder.pm line 1592
File::Temp::cleanup at /Users/sartak/.perl/perls/dtrace/lib/5.15.0/File/Temp.pm line 877
Test::Builder::DESTROY at /Users/sartak/.perl/perls/dtrace/lib/5.15.0/Test/Builder.pm line 376
Test::Builder::parent at /Users/sartak/.perl/perls/dtrace/lib/5.15.0/Test/Builder.pm line 361

Other use cases might be to investigate how many system calls are being made at compile time, for performance tuning something like Moose (which is my actual interest here). This DTrace invocation:

dtrace -qZn ':perl::phase-change /copyinstr(arg0) == "START"/ { self->interesting = 1 } :perl::phase-change /copyinstr(arg0) == "RUN"/ { self->interesting = 0 } syscall::: /self->interesting/ { @[probefunc] = count() } END { trunc(@, 3) }'

produces the top three most-used syscalls for the above Perl one-liner:

  read                                                            152
  ioctl                                                           156
  stat64                                                          470

Please let me know if there are any reservations about these patches. Keep in mind that unless you specifically use -Dusedtrace, DTrace support carries NO performance penalty. DTrace does quite a bit of gymnastics to ensure that there is no performance penalty during runtime, but as I understand it, Perl's DTrace support is not quite at that ideal yet. I'll have to investigate that one, since from what I can see in the perl source there should be no latent penalty.

Cheers,
Shawn

P.S. Is there a leading DTrace expert already on p5p, or did I just volunteer myself for the position? :P
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 3.9k
On Wed Jul 06 20:58:57 2011, sartak wrote: Show quoted text
> Hi porters, > > I've recently developed somewhat of a DTrace kick, and I figured what
better Show quoted text
> way to shake that out of my system than by improving Perl's DTrace
support? Show quoted text
> As of 5.10.1, we have probes for subroutine entry and exit, which
*does* get Show quoted text
> us a long way, but we could do even better. There are some patches
floating Show quoted text
> around (like at > http://fosiki.com/blog/2008/02/15/blead-perl-dtrace-probes/) to > improve Perl's DTrace support in other ways. For this first new probe > (an easy and useful one: global phase changes) I decided to piggy-back on > top of Florian++'s new ${^GLOBAL_PHASE} variable, since that provides more > visibility than the aforelinked patch, and is easily understood in
terms of Show quoted text
> tangible Perl features. > > My first patch does a tiny bit of refactoring to change all those literal: > > PL_phase = PERL_PHASE_END; > > assignments to use a new macro instead: > > PERL_SET_PHASE(PERL_PHASE_END); > > The second patch, the one with the actual meat in it, adds to that new > PERL_SET_PHASE macro a DTrace probe. This new probe gives DTrace users > additional visibility into _when_ certain things happen. For example, this > DTrace invocation: > > sudo dtrace -qZn ':perl::phase-change /copyinstr(arg0) == "END"/ { > self->cleaning_up = 1 } :perl::sub-entry /self->cleaning_up && > copyinstr(arg0) != "END"/ { printf("%s::%s at %s line %d\n", > copyinstr(arg3), copyinstr(arg0), copyinstr(arg1), arg2) }' > > will list all of the function calls made during interpreter cleanup > (excluding the sneaky sorta-function "END"). You might use this to help > diagnose why the interpreter doesn't exit immediately after the main > processing is done. You can play with it by running this one-liner in > another shell: > > perl -MFile::Temp -MTest::Builder -e 'sub foo {} sub bar {} END { foo
} bar' Show quoted text
> > which for me gives: > > main::foo at -e line 1 > Test::Builder::_ending at > /Users/sartak/.perl/perls/dtrace/lib/5.15.0/Test/Builder.pm line 2389 > Test::Builder::__ANON__ at > /Users/sartak/.perl/perls/dtrace/lib/5.15.0/Test/Builder.pm line 1592 > File::Temp::cleanup at > /Users/sartak/.perl/perls/dtrace/lib/5.15.0/File/Temp.pm line 877 > Test::Builder::DESTROY at > /Users/sartak/.perl/perls/dtrace/lib/5.15.0/Test/Builder.pm line 376 > Test::Builder::parent at > /Users/sartak/.perl/perls/dtrace/lib/5.15.0/Test/Builder.pm line 361 > > Other use cases might be to investigate how many system calls are
being made Show quoted text
> at compile time, for performance tuning something like Moose (which is my > actual interest here). This DTrace invocation: > > dtrace -qZn ':perl::phase-change /copyinstr(arg0) == "START"/ { > self->interesting = 1 } :perl::phase-change /copyinstr(arg0) == "RUN"/ { > self->interesting = 0 } syscall::: /self->interesting/ { @[probefunc] = > count() } END { trunc(@, 3) }' > > produces the top three most-used syscalls for the above Perl one-liner: > > read 152 > ioctl 156 > stat64 470 > > Please let me know if there are any reservations about these patches. Keep > in mind that unless you specifically use -Dusedtrace, DTrace support
carries Show quoted text
> NO performance penalty. DTrace does quite a bit of gymnastics to
ensure that Show quoted text
> there is no performance penalty during runtime, but as I understand it, > Perl's DTrace support is not quite at that ideal yet. I'll have to > investigate that one, since from what I can see in the perl source there > should be no latent penalty. > > Cheers, > Shawn > > P.S. Is there a leading DTrace expert already on p5p, or did I just > volunteer myself for the position? :P
I think you did. :-) We now have a nuncupative policy that new feature have to be accompanied by documentation. I don’t know whether that applies to your patch, but in any case the examples you gave above would be great in the documentation. Where is this DTrace stuff documented, anyway?
Subject: Re: [perl #94234] perlbug AutoReply: [PATCH] New DTrace probe for changes to global phase
Date: Mon, 11 Jul 2011 17:40:11 -0400
To: perlbug-followup [...] perl.org
From: Sartak <sartak [...] gmail.com>

Message body is not shown because sender requested not to inline it.

I've tried my hand at writing a DTrace test file. Anything I can do to improve it?

Shawn
Subject: Re: [perl #94234] [PATCH] New DTrace probe for changes to global phase
Date: Mon, 11 Jul 2011 17:49:36 -0400
To: perlbug-followup [...] perl.org
From: Sartak <sartak [...] gmail.com>

Message body is not shown because sender requested not to inline it.

Download (untitled) / with headers
text/plain 731b
2011/7/10 Father Chrysostomos via RT <perlbug-followup@perl.org>
Show quoted text
We now have a nuncupative policy that new feature have to be accompanied
by documentation. I don’t know whether that applies to your patch, but
in any case the examples you gave above would be great in the documentation.
Show quoted text

Where is this DTrace stuff documented, anyway?

I plan to write a perldtrace document that walks through all the probes Perl provides, with examples like what I wrote. DTrace support apparently isn't documented at all outside of INSTALL and perldelta files, partly because there seems to be no good place for such documentation.

I'm also providing a slightly updated test file patch which now respects $Config{dtrace}.

Shawn
Subject: Re: [perl #94234] [PATCH] New DTrace probe for changes to global phase
Date: Mon, 11 Jul 2011 20:54:12 -0400
To: perlbug-followup [...] perl.org
From: Sartak <sartak [...] gmail.com>

Message body is not shown because sender requested not to inline it.

Download (untitled) / with headers
text/plain 879b
2011/7/11 Sartak <sartak@gmail.com>
Show quoted text
2011/7/10 Father Chrysostomos via RT <perlbug-followup@perl.org>
We now have a nuncupative policy that new feature have to be accompanied
by documentation. I don’t know whether that applies to your patch, but
in any case the examples you gave above would be great in the documentation.

Where is this DTrace stuff documented, anyway?

I plan to write a perldtrace document that walks through all the probes Perl provides, with examples like what I wrote. DTrace support apparently isn't documented at all outside of INSTALL and perldelta files, partly because there seems to be no good place for such documentation.


And so I did! See attached. :) Once this patch and my test patch make it to blead I'll refactor my original GLOBAL_PHASE dtrace probe patch to amend perldtrace.pod and dtrace.t.

Shawn
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.3k
On Mon Jul 11 14:50:14 2011, sartak wrote: Show quoted text
> 2011/7/10 Father Chrysostomos via RT <perlbug-followup@perl.org> >
> > We now have a nuncupative policy that new feature have to be accompanied > > by documentation. I don’t know whether that applies to your patch, but > > in any case the examples you gave above would be great in the > > documentation.
The main reason for that policy is to avoid introducing feature without actually documenting them. In this case, however, we are no worse off with your patches, so I have applied them as ca7b837b4, cb3f81892d3 and b9a2454e3a. Thank you. Now I eagerly await your perldtrace.pod! :-) This feels strange. A few years ago I didn’t even know how to use diff, so you had to explain it to me (http://www.nntp.perl.org/group/perl.perl5.porters/;msgid=e7e9b6ab0703101555i1cf12927wf50b8cec77e0b731@mail.gmail.com), and now I’m applying your patches. Show quoted text
> > Where is this DTrace stuff documented, anyway? > >
> > I plan to write a perldtrace document that walks through all the
probes Perl Show quoted text
> provides, with examples like what I wrote. DTrace support apparently isn't > documented at all outside of INSTALL and perldelta files, partly because > there seems to be no good place for such documentation. > > I'm also providing a slightly updated test file patch which now respects > $Config{dtrace}.
You did not import %Config, so I changed the test to use $Config::Config{...}.
RT-Send-CC: perl5-porters [...] perl.org
On Mon Jul 11 17:54:53 2011, sartak wrote: Show quoted text
> 2011/7/11 Sartak <sartak@gmail.com> >
> > 2011/7/10 Father Chrysostomos via RT <perlbug-followup@perl.org> > >
> >> We now have a nuncupative policy that new feature have to be
accompanied Show quoted text
> >> by documentation. I don’t know whether that applies to your patch, but > >> in any case the examples you gave above would be great in the > >> documentation. > >>
> >
> >> Where is this DTrace stuff documented, anyway? > >>
> > > > I plan to write a perldtrace document that walks through all the probes > > Perl provides, with examples like what I wrote. DTrace support
apparently Show quoted text
> > isn't documented at all outside of INSTALL and perldelta files, partly > > because there seems to be no good place for such documentation. > > > >
> And so I did! See attached. :) Once this patch and my test patch make
it to Show quoted text
> blead I'll refactor my original GLOBAL_PHASE dtrace probe patch to amend > perldtrace.pod and dtrace.t. > > Shawn
(I should learn to reload the RT page before responding.) OK, now I’m going to apply this patch and run the porting tests....
Download (untitled) / with headers
text/plain 1.2k
On Mon Jul 11 20:41:03 2011, sprout wrote: Show quoted text
> On Mon Jul 11 17:54:53 2011, sartak wrote:
> > 2011/7/11 Sartak <sartak@gmail.com> > >
> > > 2011/7/10 Father Chrysostomos via RT <perlbug-followup@perl.org> > > >
> > >> We now have a nuncupative policy that new feature have to be
> accompanied
> > >> by documentation. I don’t know whether that applies to your
patch, but Show quoted text
> > >> in any case the examples you gave above would be great in the > > >> documentation. > > >>
> > >
> > >> Where is this DTrace stuff documented, anyway? > > >>
> > > > > > I plan to write a perldtrace document that walks through all the
probes Show quoted text
> > > Perl provides, with examples like what I wrote. DTrace support
> apparently
> > > isn't documented at all outside of INSTALL and perldelta files, partly > > > because there seems to be no good place for such documentation. > > > > > >
> > And so I did! See attached. :) Once this patch and my test patch make
> it to
> > blead I'll refactor my original GLOBAL_PHASE dtrace probe patch to amend > > perldtrace.pod and dtrace.t. > > > > Shawn
> > (I should learn to reload the RT page before responding.) > > OK, now I’m going to apply this patch and run the porting tests....
Where do you think this be listed in perl.pod?
Download (untitled) / with headers
text/plain 1.3k
On Mon Jul 11 20:43:42 2011, sprout wrote: Show quoted text
> On Mon Jul 11 20:41:03 2011, sprout wrote:
> > On Mon Jul 11 17:54:53 2011, sartak wrote:
> > > 2011/7/11 Sartak <sartak@gmail.com> > > >
> > > > 2011/7/10 Father Chrysostomos via RT <perlbug-followup@perl.org> > > > >
> > > >> We now have a nuncupative policy that new feature have to be
> > accompanied
> > > >> by documentation. I don’t know whether that applies to your
> patch, but
> > > >> in any case the examples you gave above would be great in the > > > >> documentation. > > > >>
> > > >
> > > >> Where is this DTrace stuff documented, anyway? > > > >>
> > > > > > > > I plan to write a perldtrace document that walks through all the
> probes
> > > > Perl provides, with examples like what I wrote. DTrace support
> > apparently
> > > > isn't documented at all outside of INSTALL and perldelta files,
partly Show quoted text
> > > > because there seems to be no good place for such documentation. > > > > > > > >
> > > And so I did! See attached. :) Once this patch and my test patch make
> > it to
> > > blead I'll refactor my original GLOBAL_PHASE dtrace probe patch to
amend Show quoted text
> > > perldtrace.pod and dtrace.t. > > > > > > Shawn
> > > > (I should learn to reload the RT page before responding.) > > > > OK, now I’m going to apply this patch and run the porting tests....
> > Where do you think this be listed in perl.pod?
s/be/should be/;
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.6k
On Mon Jul 11 20:43:57 2011, sprout wrote: Show quoted text
> On Mon Jul 11 20:43:42 2011, sprout wrote:
> > On Mon Jul 11 20:41:03 2011, sprout wrote:
> > > On Mon Jul 11 17:54:53 2011, sartak wrote:
> > > > 2011/7/11 Sartak <sartak@gmail.com> > > > >
> > > > > 2011/7/10 Father Chrysostomos via RT <perlbug-followup@perl.org> > > > > >
> > > > >> We now have a nuncupative policy that new feature have to be
> > > accompanied
> > > > >> by documentation. I don’t know whether that applies to your
> > patch, but
> > > > >> in any case the examples you gave above would be great in the > > > > >> documentation. > > > > >>
> > > > >
> > > > >> Where is this DTrace stuff documented, anyway? > > > > >>
> > > > > > > > > > I plan to write a perldtrace document that walks through all the
> > probes
> > > > > Perl provides, with examples like what I wrote. DTrace support
> > > apparently
> > > > > isn't documented at all outside of INSTALL and perldelta files,
> partly
> > > > > because there seems to be no good place for such documentation. > > > > > > > > > >
> > > > And so I did! See attached. :) Once this patch and my test patch
make Show quoted text
> > > it to
> > > > blead I'll refactor my original GLOBAL_PHASE dtrace probe patch to
> amend
> > > > perldtrace.pod and dtrace.t. > > > > > > > > Shawn
> > > > > > (I should learn to reload the RT page before responding.) > > > > > > OK, now I’m going to apply this patch and run the porting tests....
> > > > Where do you think this be listed in perl.pod?
> > s/be/should be/;
I’ve just put it in the reference section. I hope that’s OK. Anyway, thank you. I’ve just added your new pod file with commit f993ad5b93ddc1f.
Subject: Re: [perl #94234] [PATCH] New DTrace probe for changes to global phase
Date: Tue, 12 Jul 2011 09:08:39 -0400
To: perlbug-followup [...] perl.org
From: Sartak <sartak [...] gmail.com>

Message body is not shown because sender requested not to inline it.

Download (untitled) / with headers
text/plain 483b
Thanks for cleaning up and applying my patches, Father Chrysostomos. :)

I'm reopening this ticket because, like I said, "Once this patch and my test patch make it to blead I'll refactor my original GLOBAL_PHASE dtrace probe patch to amend perldtrace.pod and dtrace.t". See attached! I patched perldelta as well.

Feels more right to have this final commit be in the same ticket for continuity. Any unrelated dtrace patches I cook up will get their own tickets.

Thanks again!
Shawn
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 587b
On Tue Jul 12 06:09:19 2011, sartak wrote: Show quoted text
> Thanks for cleaning up and applying my patches, Father Chrysostomos. :) > > I'm reopening this ticket because, like I said, "Once this patch and
my test Show quoted text
> patch make it to blead I'll refactor my original GLOBAL_PHASE dtrace probe > patch to amend perldtrace.pod and dtrace.t". See attached! I patched > perldelta as well. > > Feels more right to have this final commit be in the same ticket for > continuity. Any unrelated dtrace patches I cook up will get their own > tickets. > > Thanks again! > Shawn
Thank you. Applied as 2b67939388.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org