Skip Menu |
Report information
Id: 132142
Status: open
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: andreas.koenig.7os6VVqR [at] franz.ak.mind.de
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: BBC
Perl Version: (no value)
Fixed In: (no value)

Attachments


Date: Thu, 21 Sep 2017 23:20:26 +0200
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Subject: Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
To: perlbug [...] perl.org
CC: Slaven Rezic <srezic [...] cpan.org>
Download (untitled) / with headers
text/plain 3.7k
As usual, cudos to discoverer Slaven! bisect ------ commit f6107ca24b4cf22dcf7fd69d65612ad718c48fca Author: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> Date: Wed Aug 30 22:35:17 2017 +0100 Strengthen weak refs when sorting in-place diagnostics ----------- http://www.cpantesters.org/cpan/report/86732923 perl -V ------- Summary of my perl5 (revision 5 version 27 subversion 4) configuration: Commit id: ae59d0fba73f3a4b9f082a26fad09e6cb2553dd2 Platform: osname=linux osvers=4.9.0-2-amd64 archname=x86_64-linux uname='linux k93msid 4.9.0-2-amd64 #1 smp debian 4.9.18-1 (2017-03-30) x86_64 gnulinux ' config_args='-Dprefix=/home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.27.3-41-gae59d0fba7/3c10 -Dmyhostname=k93msid -Dinstallusrbinperl=n -Uversiononly -Dusedevel -des -Ui_db -Dlibswanted=cl pthread socket inet nsl gdbm dbm malloc dl ld sun m crypt sec util c cposix posix ucb BSD gdbm_compat -Uuseithreads -Uuselongdouble -DEBUGGING=none' hint=recommended useposix=true d_sigaction=define useithreads=undef usemultiplicity=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='cc' ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2' optimize='-O2' cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='' gccversion='6.3.0 20170406' gccosandvers='' intsize=4 longsize=8 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long' ivsize=8 nvtype='double' nvsize=8 Off_t='off_t' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='cc' ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/6/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.24.so so=so useshrplib=false libperl=libperl.a gnulibc_version='2.24' Dynamic Linking: dlsrc=dl_dlopen.xs dlext=so d_dlsymun=undef ccdlflags='-Wl,-E' cccdlflags='-fPIC' lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong' Characteristics of this binary (from libperl): Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_MALLOC_WRAP PERL_OP_PARENT PERL_PRESERVE_IVUV PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF Built under linux Compiled at Sep 8 2017 09:50:13 %ENV: PERL="/tmp/basesmoker-reloperl-OiBV/bin/perl" PERL5LIB="" PERL5OPT="" PERL5_CPANPLUS_IS_RUNNING="23384" PERL5_CPAN_IS_RUNNING="23384" PERL_CANARY_STABILITY_NOPROMPT="1" PERL_MM_USE_DEFAULT="1" PERL_USE_UNSAFE_INC="1" @INC: /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.27.3-41-gae59d0fba7/3c10/lib/site_perl/5.27.4/x86_64-linux /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.27.3-41-gae59d0fba7/3c10/lib/site_perl/5.27.4 /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.27.3-41-gae59d0fba7/3c10/lib/5.27.4/x86_64-linux /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.27.3-41-gae59d0fba7/3c10/lib/5.27.4 . -- andreas
To: perl5-porters [...] perl.org
Date: Fri, 22 Sep 2017 03:12:37 +0100
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
From: ilmari [...] ilmari.org (Dagfinn Ilmari Mannsåker)
Download (untitled) / with headers
text/plain 1.5k
(Andreas J. Koenig) (via RT) <perlbug-followup@perl.org> writes: Show quoted text
> bisect > ------ > commit f6107ca24b4cf22dcf7fd69d65612ad718c48fca > Author: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> > Date: Wed Aug 30 22:35:17 2017 +0100 > > Strengthen weak refs when sorting in-place > > diagnostics > ----------- > http://www.cpantesters.org/cpan/report/86732923
AnyEvent is relying on in-place sorting of its array of pending timers keeping the references weak, and when it doesn't, user code can't cancel a timer by dropping its reference to it. Weakening all the references in the array after sorting it fixes the issue, but that only works for arrays where all the references are weak. If an array has a mix of strong and weak references one would have to record which references were weak before the sort, and then go through the sorted array and re-weaken them afterwards. With the old behaviour, getting the strengthening effect was trivial: one could assign the array to itself before or after the sort, or defeat the in-place sort altogether with the =()= trick. This makes me think that despite all the work that's gone into making the in-place sort optimisation invisible, the weak ref special case needs to be kept, i.e. the above commit reverted. I still think sv_rvunweaken() is a useful API function, so the commit that added that should be kept. - ilmari -- "The surreality of the universe tends towards a maximum" -- Skud's Law "Never formulate a law or axiom that you're not prepared to live with the consequences of." -- Skud's Meta-Law
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
From: Zefram <zefram [...] fysh.org>
Date: Fri, 22 Sep 2017 03:48:18 +0100
To: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 739b
Dagfinn Ilmari Mannsaker wrote: Show quoted text
>With the old behaviour, getting the strengthening effect was trivial:
With the new behaviour, one can fairly trivially get the weakness-preserving effect by wrapping the possibly-weak references in an extra layer of reference. Show quoted text
>This makes me think that despite all the work that's gone into making >the in-place sort optimisation invisible, the weak ref special case >needs to be kept, i.e. the above commit reverted.
-1. The optimisation should be a pure optimisation. If the useful behaviour of the old faulty optimisation were somehow not easily reproduced, it should be made available by explicit means, not kept as a funny special case of an operator that normally means something else. -zefram
To: perl5-porters [...] perl.org
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Date: Mon, 25 Sep 2017 21:59:19 +0200
Download (untitled) / with headers
text/plain 382b
Also affected: MGRIMES/AnyEvent-Filesys-Notify-1.21.tar.gz Result type: intermittent failures. Up to v5.27.3-33-gae2cf9f629 I ran make test in a loop for several minutes and produced only passes, no fails. With v5.27.3-34-gf6107ca24b and later I could produce both passes and fails, it was just a matter of running a dozen or so tests to produce both kinds of result. -- andreas
Date: Tue, 26 Sep 2017 08:23:10 +1000
From: Tony Cook <tony [...] develop-help.com>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
To: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 832b
On Fri, Sep 22, 2017 at 03:48:18AM +0100, Zefram wrote: Show quoted text
> Dagfinn Ilmari Mannsaker wrote:
> >With the old behaviour, getting the strengthening effect was trivial:
> > With the new behaviour, one can fairly trivially get the > weakness-preserving effect by wrapping the possibly-weak references in > an extra layer of reference. >
> >This makes me think that despite all the work that's gone into making > >the in-place sort optimisation invisible, the weak ref special case > >needs to be kept, i.e. the above commit reverted.
> > -1. The optimisation should be a pure optimisation. If the useful > behaviour of the old faulty optimisation were somehow not easily > reproduced, it should be made available by explicit means, not kept as > a funny special case of an operator that normally means something else.
I agree. Tony
Date: Wed, 27 Sep 2017 00:16:36 +0100
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
From: ilmari [...] ilmari.org (Dagfinn Ilmari Mannsåker)
To: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 528b
Zefram <zefram@fysh.org> writes: Show quoted text
> Dagfinn Ilmari Mannsaker wrote:
>>With the old behaviour, getting the strengthening effect was trivial:
> > With the new behaviour, one can fairly trivially get the > weakness-preserving effect by wrapping the possibly-weak references in > an extra layer of reference.
Attached is a patch for AnyEvent that does this for its pending timer list. It's pretty straight-forward, but I'm not in the mood for facing MLEHMANN's wrath over perl5-porters breaking his code by submitting it to him.
From eb30dd3aab9c33204418c77f7f81d3acb6e5b3eb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dagfinn=20Ilmari=20Manns=C3=A5ker?= <ilmari@ilmari.org> Date: Tue, 26 Sep 2017 22:03:48 +0100 Subject: [PATCH] Use strong references to weak references in the @timer Perl 5.27.4 fixes the in-place sort optimisation (@a = sort @a) to strengthen references, like assignment should. However, we need to keep the references to timers weak, so callers can cancel them by dropping their reference. To achieve this, make the actual references in @timer strong references to weak references to the value returned to the user, and filter out ones that have gone undef when necessary. --- lib/AnyEvent/Loop.pm | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/lib/AnyEvent/Loop.pm b/lib/AnyEvent/Loop.pm index fb9351a..f57eab0 100644 --- a/lib/AnyEvent/Loop.pm +++ b/lib/AnyEvent/Loop.pm @@ -202,22 +202,26 @@ sub one_event { # first sort timers if required (slow) if ($MNOW >= $need_sort) { $need_sort = 1e300; - @timer = sort { $a->[0] <=> $b->[0] } @timer; + @timer = sort { $$a->[0] <=> $$b->[0] } grep { defined $$_ } @timer; + } + else { + # shift off any cancelled timers at the start of the queue + shift @timer while @timer and not defined ${$timer[0]}; } # handle all pending timers - if (@timer && $timer[0][0] <= $MNOW) { + if (@timer && ${$timer[0]}->[0] <= $MNOW) { do { - my $timer = shift @timer; - $timer->[1] && $timer->[1]($timer); - } while @timer && $timer[0][0] <= $MNOW; + my $timer = ${shift @timer}; + $timer && $timer->[1] && $timer->[1]($timer); + } while @timer && ${$timer[0]}->[0] <= $MNOW; } else { # poll for I/O events, we do not do this when there # were any pending timers to ensure that one_event returns # quickly when some timers have been handled my ($wait, @vec, $fds) - = (@timer && $timer[0][0] < $need_sort ? $timer[0][0] : $need_sort) - $MNOW; + = (@timer && ${$timer[0]}->[0] < $need_sort ? ${$timer[0]}->[0] : $need_sort) - $MNOW; $wait = $wait < MAXWAIT ? $wait + ROUNDUP : MAXWAIT; $wait = 0 if @idle; @@ -252,7 +256,7 @@ sub one_event { } elsif (AnyEvent::WIN32 && $fds && $! == AnyEvent::Util::WSAEINVAL) { # buggy microshit windoze asks us to route around it CORE::select undef, undef, undef, $wait if $wait; - } elsif (!@timer || $timer[0][0] > $MNOW && !$fds) { + } elsif (!@timer || ${$timer[0]}->[0] > $MNOW && !$fds) { $$$_ && $$$_->() for @idle = grep $$$_, @idle; } } @@ -319,8 +323,8 @@ sub timer($$$) { if ($interval) { $self = [$MNOW + $after , sub { $_[0][0] = List::Util::max $_[0][0] + $interval, $MNOW; - push @timer, $_[0]; - weaken $timer[-1]; + weaken(my $weak = $_[0]); + push @timer, \$weak; $need_sort = $_[0][0] if $_[0][0] < $need_sort; &$cb; }]; @@ -328,8 +332,8 @@ sub timer($$$) { $self = [$MNOW + $after, $cb]; } - push @timer, $self; - weaken $timer[-1]; + weaken(my $weakself = $self); + push @timer, \$weakself; $need_sort = $self->[0] if $self->[0] < $need_sort; $self -- 2.7.4
Download (untitled) / with headers
text/plain 1.1k
Show quoted text
>>This makes me think that despite all the work that's gone into making >>the in-place sort optimisation invisible, the weak ref special case >>needs to be kept, i.e. the above commit reverted.
> > -1. The optimisation should be a pure optimisation. If the useful > behaviour of the old faulty optimisation were somehow not easily > reproduced, it should be made available by explicit means, not kept as > a funny special case of an operator that normally means something else.
Normally I would agree, but the previous behaviour has been there as long as in-place sort has, i.e. since 5.10. I also noticed that the in-place reverse optimisation (since 5.12) also preservers weakness. Presumably we want to fix that too (as I've done on the ilmari/unweaken-inplace-reverse branch). Another "inconsistency" is that in-place reverse preserves the deletedness of array elements. - ilmari -- - Twitter seems more influential [than blogs] in the 'gets reported in the mainstream press' sense at least. - Matt McLeod - That'd be because the content of a tweet is easier to condense down to a mainstream media article. - Calle Dybedahl
To: perl5-porters [...] perl.org
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
From: Zefram <zefram [...] fysh.org>
Date: Wed, 27 Sep 2017 02:02:39 +0100
Download (untitled) / with headers
text/plain 785b
Dagfinn Ilmari Mannsaker wrote: Show quoted text
> It's pretty straight-forward, but I'm not in the mood for facing >MLEHMANN's wrath over perl5-porters breaking his code by submitting it >to him.
You could just stick it in the RT queue. (And face even more wrath.) Show quoted text
>Normally I would agree, but the previous behaviour has been there as >long as in-place sort has, i.e. since 5.10.
It's been a bug all that time. Show quoted text
>in-place reverse optimisation (since 5.12) also preservers weakness. >Presumably we want to fix that too
Yes. Show quoted text
> Another "inconsistency" is >that in-place reverse preserves the deletedness of array elements.
That's not detectable, is it? Preserving deletedness seems like it would just avoid allocating some unnecessary SVs. -zefram
To: perl5-porters [...] perl.org
Date: Wed, 27 Sep 2017 10:19:23 +0100
From: ilmari [...] ilmari.org (Dagfinn Ilmari Mannsåker)
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
Download (untitled) / with headers
text/plain 696b
Zefram <zefram@fysh.org> writes: Show quoted text
> Dagfinn Ilmari Mannsaker wrote: >
>> Another "inconsistency" is >>that in-place reverse preserves the deletedness of array elements.
> > That's not detectable, is it? Preserving deletedness seems like it > would just avoid allocating some unnecessary SVs.
$ perl -E '@a=(1..3); delete $a[1]; @a = reverse @a; say 0+exists $a[1]' 0 -- - Twitter seems more influential [than blogs] in the 'gets reported in the mainstream press' sense at least. - Matt McLeod - That'd be because the content of a tweet is easier to condense down to a mainstream media article. - Calle Dybedahl
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
To: perl5-porters [...] perl.org
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
Date: Fri, 17 Nov 2017 21:38:52 +0100
Download (untitled) / with headers
text/plain 232b
What is the state of this ticket? From looking at it it looks stalled. Was there any additional activity? Has anybody summarised? Reported to Marc? Provided a patch? Considered improving? Considered reverting? Thanks! -- andreas
CC: Sawyer X <xsawyerx [...] gmail.com>
To: perl5-porters [...] perl.org
From: ilmari [...] ilmari.org (Dagfinn Ilmari Mannsåker)
Date: Sat, 18 Nov 2017 00:08:46 +0000
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
Andreas Koenig <andreas.koenig.7os6VVqR@franz.ak.mind.de> writes: Show quoted text
> What is the state of this ticket? From looking at it it looks stalled. > Was there any additional activity? > > Has anybody summarised? Reported to Marc? Provided a patch? Considered > improving? Considered reverting?
I provided a patch in https://rt.perl.org/Public/Bug/Display.html?id=132142#txn-1493873, but just realised there's a simpler way to do it, which I've attached. As mentioned in that previous message, I'm not in the mood to confront Marc over this, but Sawyer graciously offered to do that for me. Sawyer, do you feel like sending these two patches to Marc and suggest he applies one of them to keep AnyEvent working on blead? Note that AnyEvent's own test suite doesn't trigger the bug, but AnyEvent::HTTP does. - ilmari -- - Twitter seems more influential [than blogs] in the 'gets reported in the mainstream press' sense at least. - Matt McLeod - That'd be because the content of a tweet is easier to condense down to a mainstream media article. - Calle Dybedahl
From 8e12a666543e50043419630d5e4ff00c7dd277e2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dagfinn=20Ilmari=20Manns=C3=A5ker?= <ilmari@ilmari.org> Date: Fri, 17 Nov 2017 23:59:30 +0000 Subject: [PATCH] Re-weaken timer references after sorting Perl 5.27.4 fixes the in-place sort optimisation (@a = sort @a) to strengthen references, like assignment should. However, we need to keep the references to timers weak, so callers can cancel them by dropping their reference. --- lib/AnyEvent/Loop.pm | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/lib/AnyEvent/Loop.pm b/lib/AnyEvent/Loop.pm index fb9351a..f7145fd 100644 --- a/lib/AnyEvent/Loop.pm +++ b/lib/AnyEvent/Loop.pm @@ -173,6 +173,9 @@ BEGIN { $round = 0.001 if $round < 0.001; # 1ms is enough for us $round -= $round * 1e-2; # 0.1 => 0.099 eval "sub ROUNDUP() { $round }"; + + my $unweakens = $] >= 5.027004 ? 1 : 0; + eval "sub INPLACE_SORT_UNWEAKENS() { $unweakens }"; } _update_clock; @@ -203,6 +206,9 @@ sub one_event { if ($MNOW >= $need_sort) { $need_sort = 1e300; @timer = sort { $a->[0] <=> $b->[0] } @timer; + if (INPLACE_SORT_UNWEAKENS) { + weaken($_) for @timer; + } } # handle all pending timers -- 2.7.4
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.1k
On Fri, 17 Nov 2017 16:09:12 -0800, ilmari wrote: Show quoted text
> Andreas Koenig <andreas.koenig.7os6VVqR@franz.ak.mind.de> writes: >
> > What is the state of this ticket? From looking at it it looks stalled. > > Was there any additional activity? > > > > Has anybody summarised? Reported to Marc? Provided a patch? Considered > > improving? Considered reverting?
> > I provided a patch in > https://rt.perl.org/Public/Bug/Display.html?id=132142#txn-1493873, but > just realised there's a simpler way to do it, which I've attached. > > As mentioned in that previous message, I'm not in the mood to confront > Marc over this, but Sawyer graciously offered to do that for me. > > Sawyer, do you feel like sending these two patches to Marc and suggest > he applies one of them to keep AnyEvent working on blead? Note that > AnyEvent's own test suite doesn't trigger the bug, but AnyEvent::HTTP > does.
I didn't receive an email for this correspondence so I only saw the ping from Andreas. I have just sent an email to Marc with the latest patch, asking to apply the patch and release a new version. P5P was CC'ed on the email as well. Thank you for producing the patches, Ilmari.
Date: Thu, 19 Apr 2018 22:48:29 +0100
From: Dave Mitchell <davem [...] iabyn.com>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
CC: perl5-porters [...] perl.org
To: Sawyer X via RT <perlbug-followup [...] perl.org>
Download (untitled) / with headers
text/plain 1.9k
On Sun, Nov 19, 2017 at 06:10:28AM -0800, Sawyer X via RT wrote: Show quoted text
> On Fri, 17 Nov 2017 16:09:12 -0800, ilmari wrote:
> > Andreas Koenig <andreas.koenig.7os6VVqR@franz.ak.mind.de> writes: > >
> > > What is the state of this ticket? From looking at it it looks stalled. > > > Was there any additional activity? > > > > > > Has anybody summarised? Reported to Marc? Provided a patch? Considered > > > improving? Considered reverting?
> > > > I provided a patch in > > https://rt.perl.org/Public/Bug/Display.html?id=132142#txn-1493873, but > > just realised there's a simpler way to do it, which I've attached. > > > > As mentioned in that previous message, I'm not in the mood to confront > > Marc over this, but Sawyer graciously offered to do that for me. > > > > Sawyer, do you feel like sending these two patches to Marc and suggest > > he applies one of them to keep AnyEvent working on blead? Note that > > AnyEvent's own test suite doesn't trigger the bug, but AnyEvent::HTTP > > does.
> > I didn't receive an email for this correspondence so I only saw the ping from Andreas. I have just sent an email to Marc with the latest patch, asking to apply the patch and release a new version. P5P was CC'ed on the email as well.
My understanding of the current state of this 5.28 blocker ticket is: 1) A bug fix to blead removed an inadvertent user-visible difference between '@b = sort @a' and '@a sort @a'; the latter is internally optimised, but such optimisations shouldn't make visible changes to sort behaviour. 2) AnyEvent relied on this now-fixed buggy behaviour; 3) A patch has been privately submitted to Marc; 4) There has not been a new release of AnyEvent yet; 5) There was a brief discussion about the desirability of an explicit in-place sort feature. I propose a) this ticket is removed from 5.28 blockers. b) is kept open in case anyone wants to propose an in-place sort syntax or mechanism. -- Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's Law.
To: "Dave Mitchell via RT" <perlbug-followup [...] perl.org>
Date: Fri, 20 Apr 2018 06:50:54 +0200
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Download (untitled) / with headers
text/plain 3.1k
Show quoted text
>>>>> On Thu, 19 Apr 2018 14:48:47 -0700, "Dave Mitchell via RT" <perlbug-followup@perl.org> said:
Show quoted text
> On Sun, Nov 19, 2017 at 06:10:28AM -0800, Sawyer X via RT wrote:
>> On Fri, 17 Nov 2017 16:09:12 -0800, ilmari wrote:
>> > Andreas Koenig <andreas.koenig.7os6VVqR@franz.ak.mind.de> writes: >> >
>> > > What is the state of this ticket? From looking at it it looks stalled. >> > > Was there any additional activity? >> > > >> > > Has anybody summarised? Reported to Marc? Provided a patch? Considered >> > > improving? Considered reverting?
>> > >> > I provided a patch in >> > https://rt.perl.org/Public/Bug/Display.html?id=132142#txn-1493873, but >> > just realised there's a simpler way to do it, which I've attached. >> > >> > As mentioned in that previous message, I'm not in the mood to confront >> > Marc over this, but Sawyer graciously offered to do that for me. >> > >> > Sawyer, do you feel like sending these two patches to Marc and suggest >> > he applies one of them to keep AnyEvent working on blead? Note that >> > AnyEvent's own test suite doesn't trigger the bug, but AnyEvent::HTTP >> > does.
>> >> I didn't receive an email for this correspondence so I only saw the >> ping from Andreas. I have just sent an email to Marc with the latest >> patch, asking to apply the patch and release a new version. P5P was >> CC'ed on the email as well.
Show quoted text
> My understanding of the current state of this 5.28 blocker ticket is:
Show quoted text
> 1) A bug fix to blead removed an inadvertent user-visible difference > between '@b = sort @a' and '@a sort @a'; the latter is internally optimised, > but such optimisations shouldn't make visible changes to sort > behaviour.
It has been pointed out that the "internally optimized behaviour" of '@a = sort @a' was documented as a way to get an in-place sort. It was established behaviour for a long time. Given that it was both proudly announced and well established, I find the term "bug fix" for its removal problematic. This is a change in bahaviour. As such it may be a bug fix for one party and a breakage for the other. Fair discussion should try to balance the thinking and take this into account. Show quoted text
> 2) AnyEvent relied on this now-fixed buggy behaviour;
The problematic terms continue. Show quoted text
> 3) A patch has been privately submitted to Marc;
But the patch has issues. Discussion about it has been avoided and prevented. Show quoted text
> 4) There has not been a new release of AnyEvent yet;
What could that probably mean? Show quoted text
> 5) There was a brief discussion about the desirability of an explicit > in-place sort feature.
Why was it so brief? Originally the in-place sort was deemed as a feature. To remove it, is now called a bug fix. I would say seeking for solutions with the decisive extra bit of effort should be possible. Show quoted text
> I propose
Show quoted text
> a) this ticket is removed from 5.28 blockers. > b) is kept open in case anyone wants to propose an in-place sort syntax or > mechanism.
I would suggest a revert with a chance to retry for 5.30, but with the mandate to try to find a way to retain in-place sort. Show quoted text
> Hofstadter's Law: It always takes longer than you expect, even when you > take into account Hofstadter's Law.
Long live Hofstadter, -- andreas
To: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>, Dave Mitchell via RT <perlbug-followup [...] perl.org>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
From: Tom Molesworth via perl5-porters <perl5-porters [...] perl.org>
Date: Fri, 20 Apr 2018 13:36:53 +0800
Download (untitled) / with headers
text/plain 2.2k


On 20 April 2018 at 12:50, Andreas Koenig <andreas.koenig.7os6VVqR@franz.ak.mind.de> wrote:
Show quoted text
It has been pointed out that the "internally optimized behaviour" of '@a
= sort @a' was documented as a way to get an in-place sort. It was
established behaviour for a long time. Given that it was both proudly
announced and well established, I find the term "bug fix" for its
removal problematic. This is a change in bahaviour. As such it may be a
bug fix for one party and a breakage for the other. Fair discussion
should try to balance the thinking and take this into account.

I believe the original release note from perl5100delta was:

>  In-place sorting
>    Sorting arrays in place ("@a = sort @a") is now optimized to avoid making
>    a temporary copy of the array.
>
>    Likewise, "reverse sort ..." is now optimized to sort in reverse, avoiding
>    the generation of a temporary intermediate list.

Since there was no corresponding documentation update to sort itself, and no mention of how this would affect magic or weakrefs, I think it would be reasonable to conclude that this is intended purely as a non-user-visible optimisation? If the behaviour is intentional, then why (in 5.26) does it only happen for sort, but not the reverse sort that was explicitly called out in the same release note?
 
Show quoted text

  > 2) AnyEvent relied on this now-fixed buggy behaviour;

The problematic terms continue.

The sort optimisation was added in 5.10. AnyEvent has test passes (and code mention of) 5.8.0. If that's correct - I haven't looked at the relevant code to confirm - then I think relying on in-place sort optimisation could rightly be described as a bug?
 
When I mentioned the in-place sort issue to ilmari originally, I was under the impression that it was a well-known bug that no one had ever got around to fixing (similar to the case with plain reverse).

It's something I've run into a few times in the last few years, is more of an annoyance than a genuine complaint, but does seem like an arbitrary behavioural difference that should be discouraged rather than documented. To me, it's very clearly a bug, and inconsistent: we have a fine example of in-place modification in chomp(), I'd expect similar syntax for an in-place sort or reverse feature.

cheers,

Tom

To: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
From: Dave Mitchell <davem [...] iabyn.com>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
CC: Dave Mitchell via RT <perlbug-followup [...] perl.org>
Date: Fri, 20 Apr 2018 08:29:08 +0100
Download (untitled) / with headers
text/plain 3.2k
On Fri, Apr 20, 2018 at 06:50:54AM +0200, Andreas Koenig wrote: Show quoted text
> It has been pointed out that the "internally optimized behaviour" of '@a > = sort @a' was documented as a way to get an in-place sort.
Again, as the author of that optimisation, 1) it was my intention that it should make no user-visible difference apart from speed; 2) if I had been aware of the weakrefs behaviour at the time, I would have implemented the optimisation in a way that strengthened weak refs; 3) there is no documentation for the sort function which describes this behaviour. The 5.10.1 perldelta mentions in-place sorting in the 'Performance Enhancements' section, and makes no particular claim about whether weak refs are strengthened, or any implication that a change in behaviour is intended. Show quoted text
> It was > established behaviour for a long time.
Any bug fix must by definition change established behaviour. Show quoted text
> Given that it was both proudly > announced and well established,
We'll have to disagree about the "announced" bit. Show quoted text
> I find the term "bug fix" for its > removal problematic. This is a change in bahaviour. As such it may be a > bug fix for one party and a breakage for the other. Fair discussion > should try to balance the thinking and take this into account.
My personal feeling is that avoiding unexpected differences in behaviour between '@a = sort @a' and '@b = sort @a' outweighs the cost of the change in behaviour. Other people may feel differently. Show quoted text
> > 3) A patch has been privately submitted to Marc;
> > But the patch has issues. Discussion about it has been avoided and > prevented.
Unusually for CPAN authors, Marc doesn't use any publicly-facing bug tracking system for his modules, so it is hard to have a discussions about such issues. Even when Marc isn't banned from this list, it is extremely difficult to have any sort of technical discussion with him, since he often resorts to personal attacks. I have, on this list, been repeatedly called a liar by him. When I submitted a patch to him for one of his modules, his initial reply was to tell me that my attitude was atrocious. This sort of behaviour is a strong disincentive for anyone to engage with him. Show quoted text
> > 4) There has not been a new release of AnyEvent yet;
> > What could that probably mean?
Again, due to the lack of a public bug tracker, that's hard to guess. Perhaps it's a poor patch; perhaps it a good patch which Marc hasn't had time to apply yet? Show quoted text
> > 5) There was a brief discussion about the desirability of an explicit > > in-place sort feature.
> > Why was it so brief?
Presumably because no-one could come up with any good suggestions. Show quoted text
> Originally the in-place sort was deemed as a > feature.
Again, I disagree. Show quoted text
> To remove it, is now called a bug fix. I would say seeking for > solutions with the decisive extra bit of effort should be possible.
Many things are possible. In practice, perl gets 1000 new RT tickets per year, plus a further 150 security tickets; there are many things which occupy our time and energy. Show quoted text
> I would suggest a revert with a chance to retry for 5.30, but with the > mandate to try to find a way to retain in-place sort.
Does anyone else here any feelings either way? -- "I do not resent criticism, even when, for the sake of emphasis, it parts for the time with reality". -- Winston Churchill, House of Commons, 22nd Jan 1941.
To: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
CC: Dave Mitchell via RT <perlbug-followup [...] perl.org>
From: Dave Mitchell <davem [...] iabyn.com>
Date: Wed, 25 Apr 2018 11:18:53 +0100
Download (untitled) / with headers
text/plain 691b
On Fri, Apr 20, 2018 at 08:29:08AM +0100, Dave Mitchell wrote: Show quoted text
> On Fri, Apr 20, 2018 at 06:50:54AM +0200, Andreas Koenig wrote:
[ snip discussion about @a = sort @a optimisation not converting weak ref back to strong refs ] Show quoted text
> > I would suggest a revert with a chance to retry for 5.30, but with the > > mandate to try to find a way to retain in-place sort.
> > Does anyone else here any feelings either way?
From the silence, I'm assuming no-one else wants this change reverted before 5.28. -- A power surge on the Bridge is rapidly and correctly diagnosed as a faulty capacitor by the highly-trained and competent engineering staff. -- Things That Never Happen in "Star Trek" #9
Date: Wed, 25 Apr 2018 13:45:46 -0600
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
CC: Dave Mitchell via RT <perlbug-followup [...] perl.org>
From: Karl Williamson <public [...] khwilliamson.com>
To: Dave Mitchell <davem [...] iabyn.com>, Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Download (untitled) / with headers
text/plain 595b
On 04/25/2018 04:18 AM, Dave Mitchell wrote: Show quoted text
> On Fri, Apr 20, 2018 at 08:29:08AM +0100, Dave Mitchell wrote:
>> On Fri, Apr 20, 2018 at 06:50:54AM +0200, Andreas Koenig wrote:
> [ snip discussion about @a = sort @a optimisation not converting weak > ref back to strong refs ]
>>> I would suggest a revert with a chance to retry for 5.30, but with the >>> mandate to try to find a way to retain in-place sort.
>> >> Does anyone else here any feelings either way?
>
>>From the silence, I'm assuming no-one else wants this change reverted
> before 5.28. >
I could support a temporary revert.
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.1k
On Wed, 25 Apr 2018 10:20:08 GMT, davem wrote: Show quoted text
> On Fri, Apr 20, 2018 at 08:29:08AM +0100, Dave Mitchell wrote:
> > On Fri, Apr 20, 2018 at 06:50:54AM +0200, Andreas Koenig wrote:
> [ snip discussion about @a = sort @a optimisation not converting weak > ref back to strong refs ]
> > > I would suggest a revert with a chance to retry for 5.30, but with the > > > mandate to try to find a way to retain in-place sort.
> > > > Does anyone else here any feelings either way?
> > From the silence, I'm assuming no-one else wants this change reverted > before 5.28. >
We're so close to release that I think reverting would open up more problems than it would solve. MLEHMANN and his defenders are going to complain regardless of what we do. We've lived with the code in its current state, so we have a fair idea of what its impact will be. Unless someone wants to revert in a branch and test most of CPAN against that branch, and unless such a process has demonstrably better results than the current state of blead, I think we should proceed on course. Let's face the music and dance. Jim Keenan -- James E Keenan (jkeenan@cpan.org)
To: "James E Keenan via RT" <perlbug-followup [...] perl.org>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
CC: perl5-porters [...] perl.org
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Date: Thu, 26 Apr 2018 06:21:50 +0200
Download (untitled) / with headers
text/plain 522b
Show quoted text
>>>>> On Wed, 25 Apr 2018 12:59:11 -0700, "James E Keenan via RT" <perlbug-followup@perl.org> said:
Show quoted text
> MLEHMANN and his defenders are going to complain regardless of what > we do.
While you try to make this an argument about my gang vs your gang, something much more important is at stake. We're not arguing about the author of that test, we're arguing about impacts of breaking code written in the language that made CPAN possible on the userbase thereof. Please keep that in mind. Thank you very much, -- andreas
To: Karl Williamson <public [...] khwilliamson.com>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
CC: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>, Dave Mitchell via RT <perlbug-followup [...] perl.org>
From: Dave Mitchell <davem [...] iabyn.com>
Date: Thu, 26 Apr 2018 12:28:46 +0100
Download (untitled) / with headers
text/plain 943b
On Wed, Apr 25, 2018 at 01:45:46PM -0600, Karl Williamson wrote: Show quoted text
> On 04/25/2018 04:18 AM, Dave Mitchell wrote:
> > On Fri, Apr 20, 2018 at 08:29:08AM +0100, Dave Mitchell wrote:
> > > On Fri, Apr 20, 2018 at 06:50:54AM +0200, Andreas Koenig wrote:
> > [ snip discussion about @a = sort @a optimisation not converting weak > > ref back to strong refs ]
> > > > I would suggest a revert with a chance to retry for 5.30, but with the > > > > mandate to try to find a way to retain in-place sort.
> > > > > > Does anyone else here any feelings either way?
> >
> > > From the silence, I'm assuming no-one else wants this change reverted
> > before 5.28. > >
> > I could support a temporary revert.
I have pushed a branch, smoke-me/davem/weak_sort, which contains (I hope) a revert of the relevant commits, in case we decide to go ahead. -- "Procrastination grows to fill the available time" -- Mitchell's corollary to Parkinson's Law
CC: perl5-porters [...] perl.org
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
From: Sawyer X <xsawyerx [...] gmail.com>
To: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>, James E Keenan via RT <perlbug-followup [...] perl.org>
Date: Thu, 26 Apr 2018 14:51:06 +0300
Download (untitled) / with headers
text/plain 942b
Alright, everyone. It's time. I'm calling it. We revert this fix for 5.28 and leave it for 5.30. I have a lot to say about this and I'm incredibly frustrated and unhappy with this situation and the conclusion we have to reach, but I think at this point it's the right call. I have privately emailed Dave about this and he now provided the branch. Thank you, Dave. On 04/26/2018 07:21 AM, Andreas Koenig wrote: Show quoted text
>>>>>> On Wed, 25 Apr 2018 12:59:11 -0700, "James E Keenan via RT" <perlbug-followup@perl.org> said:
> > MLEHMANN and his defenders are going to complain regardless of what > > we do.
> > While you try to make this an argument about my gang vs your gang, > something much more important is at stake. We're not arguing about the > author of that test, we're arguing about impacts of breaking code > written in the language that made CPAN possible on the userbase thereof. > Please keep that in mind. > > Thank you very much,
Date: Fri, 27 Apr 2018 10:47:18 +0100
To: Sawyer X <xsawyerx [...] gmail.com>
CC: perl5-porters [...] perl.org
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
From: Dave Mitchell <davem [...] iabyn.com>
Download (untitled) / with headers
text/plain 195b
On Thu, Apr 26, 2018 at 02:51:06PM +0300, Sawyer X wrote: Show quoted text
> We revert this fix for 5.28 and leave it for 5.30.
Revert now merged into blead. -- I thought I was wrong once, but I was mistaken.
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
Date: Sun, 10 Jun 2018 20:52:25 +0200
To: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.2k
After some digging I finally found the original tickets that started this story. Enjoy the timeline below that now provides the for so long missing piece. It would be cool if it could help this issue receive the appropriate amount of fairness. 2004-01-25: https://rt.perl.org/Public/Bug/Display.html?id=25263 RT issue From: Dan Jacobson Subject: sort -u Summary: A feature request for 'sort -u' 2004-01-26: https://rt.perl.org/Public/Bug/Display.html?id=25263#txn-72227 Within same RT ticket From: Tels Summary: adding feature request for sorting in-place 2004-02-20: https://rt.perl.org/Public/Bug/Display.html?id=25263#txn-79013 Within same RT ticket From: Dave Mitchell Citation: "I've now applied change #22349 to bleed, which makes @a = sort @a act inplace." 2017-09-21: https://rt.perl.org/Public/Bug/Display.html?id=132142 BBC ticket From: Andreas Koenig Subject: Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz 2017-11-19: https://www.nntp.perl.org/group/perl.perl5.porters/2017/12/msg248259.html From: Sawyer X Subject: Re: Patch for AnyEvent::HTTP Citations: "This bug will stay fixed." 2018-04-26: https://rt.perl.org/Public/Bug/Display.html?id=132142#txn-1550198 From: Sawyer X Citation: "We revert this fix for 5.28 and leave it for 5.30." -- andreas
To: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
From: Dave Mitchell <davem [...] iabyn.com>
CC: perl5-porters [...] perl.org
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
Date: Sun, 10 Jun 2018 21:59:18 +0100
On Sun, Jun 10, 2018 at 08:52:25PM +0200, Andreas Koenig wrote: Show quoted text
> After some digging I finally found the original tickets that started > this story. Enjoy the timeline below that now provides the for so long > missing piece. It would be cool if it could help this issue receive the > appropriate amount of fairness.
I am genuinely puzzled by your email. In what way do you feel that this issue is currently not receiving "the appropriate amount of fairness", and in what way do you feel that the provided links help in improving fairness? In particular, reading through the 2004 ticket in its entirety only confirms in my mind that the work I did on in-place sort back then was intended as an internal optimisation to save memory memory and CPU, and I am still of the opinion that if, back in 2004 someone had pointed out that my optimisation failed to strengthen weak refs, then I would have regarded it as a bug and fixed it. -- Technology is dominated by two types of people: those who understand what they do not manage, and those who manage what they do not understand.
To: "Dave Mitchell via RT" <perlbug-followup [...] perl.org>
Date: Sun, 24 Jun 2018 11:34:30 +0200
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Download (untitled) / with headers
text/plain 1.9k
Show quoted text
>>>>> On Sun, 10 Jun 2018 14:00:03 -0700, "Dave Mitchell via RT" <perlbug-followup@perl.org> said:
Show quoted text
> On Sun, Jun 10, 2018 at 08:52:25PM +0200, Andreas Koenig wrote:
>> After some digging I finally found the original tickets that started >> this story. Enjoy the timeline below that now provides the for so long >> missing piece. It would be cool if it could help this issue receive the >> appropriate amount of fairness.
Show quoted text
> I am genuinely puzzled by your email. In what way do you feel that this > issue is currently not receiving "the appropriate amount of fairness", > and in what way do you feel that the provided links help in improving > fairness?
Thanks for asking. I would suggest re-reading the one sentence: : It would be cool if I could say: sort @array; or @array = sort_in_place : @array; and get an inplace sort Ask yourself what in-place-sort means and answer the following questions: - does an in-place-sort change any element within the array? - is an in-place-sort conceptually an assignment? - did the as per 2004-02-20 committed solution provide an in-place-sort disguised as an assignment? - was it clever to provide an in-place-sort disguised as an assignement? My tentative answers: - no - no - yes - probably not Show quoted text
> In particular, reading through the 2004 ticket in its entirety only > confirms in my mind that the work I did on in-place sort back then was > intended as an internal optimisation to save memory memory and CPU, and I > am still of the opinion that if, back in 2004 someone had pointed out that > my optimisation failed to strengthen weak refs, then I would have regarded > it as a bug and fixed it.
There is no doubt that your intentions back then and today are consistent. But a more complete view on the whole matter should have revealed by now that there are at least two ways to read and argue the story. I sincerely believe it cannot be too hard for an average bystander of good will to argue both ways. -- andreas
Date: Mon, 25 Jun 2018 09:14:45 +0100
From: Dave Mitchell <davem [...] iabyn.com>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
CC: Dave Mitchell via RT <perlbug-followup [...] perl.org>
To: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Download (untitled) / with headers
text/plain 1.4k
On Sun, Jun 24, 2018 at 11:34:30AM +0200, Andreas Koenig wrote: Show quoted text
> There is no doubt that your intentions back then and today are > consistent. But a more complete view on the whole matter should have > revealed by now that there are at least two ways to read and argue the > story. > > I sincerely believe it cannot be too hard for an average bystander of > good will to argue both ways.
The basic question question is whether a perl programmer could have a reasonable belief that sort has a special-cased and supported feature, in that the specific syntax @array1 = sort @array1 behaves differently than, e.g. @array2 = sort @array1; @array1 = @array1 as regards things like strengthening weak references. Since there is absolutely no mention of this in the documentation for sort, there would be an extremely strong assumption of no. There is an entry in perldelta talking about an 'in-place' optimisation, with no mention of weak refs etc. Finally there is the RT ticket, originally concerning a 'sort -u' feature request, where at one point, Tels suggested, in the context of *saving memory*, that it would be nice to have a in-place sort feature, for which he proposed a distinct new syntax (which was not taken up). I really cannot see how any of this justifies not strengthening weak refs as being anything but a bug. -- "Procrastination grows to fill the available time" -- Mitchell's corollary to Parkinson's Law
To: "Dave Mitchell via RT" <perlbug-followup [...] perl.org>
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
Date: Wed, 27 Jun 2018 08:21:43 +0200
Download (untitled) / with headers
text/plain 480b
Show quoted text
>>>>> On Mon, 25 Jun 2018 01:15:33 -0700, "Dave Mitchell via RT" <perlbug-followup@perl.org> said:
Show quoted text
> On Sun, Jun 24, 2018 at 11:34:30AM +0200, Andreas Koenig wrote:
In your previous posting you asked me "in what way do you feel that the provided links help in improving fairness". In my response I asked you 4 questions. I would be interesting what your answers to the four questions are. And why did they not help you to argue it both ways? Thank you very much. -- andreas
Date: Wed, 27 Jun 2018 08:54:48 +0100
To: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
From: Dave Mitchell <davem [...] iabyn.com>
CC: Dave Mitchell via RT <perlbug-followup [...] perl.org>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
Download (untitled) / with headers
text/plain 1.7k
On Wed, Jun 27, 2018 at 08:21:43AM +0200, Andreas Koenig wrote: Show quoted text
> >>>>> On Mon, 25 Jun 2018 01:15:33 -0700, "Dave Mitchell via RT" <perlbug-followup@perl.org> said:
>
> > On Sun, Jun 24, 2018 at 11:34:30AM +0200, Andreas Koenig wrote:
> > In your previous posting you asked me "in what way do you feel that the > provided links help in improving fairness". In my response I asked you 4 > questions. I would be interesting what your answers to the four > questions are. And why did they not help you to argue it both ways?
Show quoted text
> Ask yourself what in-place-sort means and answer the following > questions: > > - does an in-place-sort change any element within the array? > - is an in-place-sort conceptually an assignment? > - did the as per 2004-02-20 committed solution provide an in-place-sort > disguised as an assignment?
> > - was it clever to provide an in-place-sort disguised as an assignement?
> > My tentative answers: > > - no > - no > - yes > - probably not
There is ambiguity in the term 'in-place' - it can either refer to something at the user level, or to describe an internal optimisation. If the phrase 'in-place' was being used to describe a user-level feature, then my answers would be: no no no no If the phrase 'in-place' was being used to describe the implementation details of an optimisation, then yes yes no N/A NB I am of the firm opinion that the 2004-02-20 commit was entirely related to the second interpretation. Here's a question for you. What in the documentation for sort, might lead a reasonable programmer to assume that @a = sort @a is intended behave specially and (amongst other things), not strengthen weak references? -- Diplomacy is telling someone to go to hell in such a way that they'll look forward to the trip
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
Date: Thu, 05 Jul 2018 21:57:22 +0200
To: "Dave Mitchell via RT" <perlbug-followup [...] perl.org>
Download (untitled) / with headers
text/plain 4.3k
Show quoted text
>>>>> On Wed, 27 Jun 2018 01:09:40 -0700, "Dave Mitchell via RT" <perlbug-followup@perl.org> said:
Show quoted text
> On Wed, Jun 27, 2018 at 08:21:43AM +0200, Andreas Koenig wrote:
>> >>>>> On Mon, 25 Jun 2018 01:15:33 -0700, "Dave Mitchell via RT" <perlbug-followup@perl.org> said:
>>
>> > On Sun, Jun 24, 2018 at 11:34:30AM +0200, Andreas Koenig wrote:
>> >> In your previous posting you asked me "in what way do you feel that the >> provided links help in improving fairness". In my response I asked you 4 >> questions. I would be interesting what your answers to the four >> questions are. And why did they not help you to argue it both ways?
Show quoted text
>> Ask yourself what in-place-sort means and answer the following >> questions: >> >> - does an in-place-sort change any element within the array? >> - is an in-place-sort conceptually an assignment? >> - did the as per 2004-02-20 committed solution provide an in-place-sort >> disguised as an assignment?
>> > - was it clever to provide an in-place-sort disguised as an assignement?
>> >> My tentative answers: >> >> - no >> - no >> - yes >> - probably not
Show quoted text
> There is ambiguity in the term 'in-place' - it can either refer to > something at the user level, or to describe an internal > optimisation.
An ambiguity there is, what a great step forward, thank you! Show quoted text
> If the phrase 'in-place' was being used to describe a user-level feature, > then my answers would be:
Show quoted text
> no > no > no
Could you extend on that? Especially, how big is the difference between an in-place-sort and the solution provided by the February 2004 commit? I mean, look how perfectly well allocated the elements of the array are before and after the sort: % perl -le 'use Devel::Peek; @a = qw(2 1);Dump @a;@a = sort @a;Dump @a;' 2>&1 | grep at SV = PVAV(0x561c2532bf08) at 0x561c253493d8 SV = PV(0x561c2532ada0) at 0x561c2532a1e0 SV = PV(0x561c2532ae80) at 0x561c2532a3f0 SV = PVAV(0x561c2532bf08) at 0x561c253493d8 SV = PV(0x561c2532ae80) at 0x561c2532a3f0 SV = PV(0x561c2532ada0) at 0x561c2532a1e0 Both head and body of every PV involved stay the same. This you can't get with an ordinary assignment. Show quoted text
> no
Show quoted text
> If the phrase 'in-place' was being used to describe the implementation > details of an optimisation, then
Show quoted text
> yes > yes > no > N/A
Show quoted text
> NB I am of the firm opinion that the 2004-02-20 commit was entirely > related to the second interpretation.
But let's recall, that there is a finally acknowledged ambiguity for the reader of the cited thread (https://rt.perl.org/Public/Bug/Display.html?id=25263#txn-72227); Show quoted text
> Here's a question for you. What in the documentation for sort, might lead > a reasonable programmer to assume that @a = sort @a is intended behave > specially and (amongst other things), not strengthen weak references?
It has already been answered above that it was at least documented in perl5100delta with this text: : =head2 In-place sorting : : Sorting arrays in place (C<@a = sort @a>) is now optimized to avoid : making a temporary copy of the array. But there is more to it. Documentation is always more than what projects provide as the manpages. What is being discussed on mailing lists and in RT tickets is also relevant for the interpretation of the development of a language. Much more so in 2004 than in 2018 one might add. And finally Perl is a leading part of the open source culture, which is one of the reasons we use and love it. So it *may* have happenen that people picked up partially from discussions and partially from reading the sources. I'm not saying this is what happened, I don't know. I believe the discussion in the ticket and the announcement on perl5100delta and some small experiments with Devel::Peek as I showed above can give a reasonable picture of the state of the feature. To sum up: a reasonable perl programmer who was following the developement on perl5-porters around the time of perl 5.8.4 might have understood that this in-place sorting was a powerful feature. Readers who were not reading perl5-porters at that time, can now read the above linked thread and try to understand how the reasonable programmer might have taken it up. The reasonable programmer might not have noticed that the feature did not find its way into the documentation. He might have used it in a module and uploaded it to CPAN. In fact this did happen indeed. -- andreas
Date: Thu, 5 Jul 2018 22:38:24 +0100
To: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
From: Dave Mitchell <davem [...] iabyn.com>
CC: Dave Mitchell via RT <perlbug-followup [...] perl.org>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
Download (untitled) / with headers
text/plain 1.7k
On Thu, Jul 05, 2018 at 09:57:22PM +0200, Andreas Koenig wrote: Show quoted text
> To sum up: a reasonable perl programmer who was following the > developement on perl5-porters around the time of perl 5.8.4 might have > understood that this in-place sorting was a powerful feature. Readers > who were not reading perl5-porters at that time, can now read the above > linked thread and try to understand how the reasonable programmer might > have taken it up. The reasonable programmer might not have noticed that > the feature did not find its way into the documentation. He might have > used it in a module and uploaded it to CPAN. In fact this did happen > indeed.
This is just beyond absurd. If we apply that criteria to perl generally, there is essentially no bug we can fix, no change we can ever make, in case someone made a comment on a p5p thread 14 years ago, which by squinting hard enough could be interpreted as supporting the buggy behaviour. So hooray, perl is complete. We can all go away and work on other things now. A *reasonable* perl programmer would have looked at the documentation for sort, noted that there was absolutely no mention of "in-place" sorting or @a = sort @a having special behaviour, and either concluded that no special behaviour was intended, or enquired with p5p as to why the documentation was lacking, and whether this was an oversight. PS - I don't think for one minute the use of the phrase "in-place" was ambiguous *in context*, in either the p5p discussion or the perldelta; it was only ambiguous in the sense that your specific questions about it provided no context as to what behaviour was being referring to. -- A walk of a thousand miles begins with a single step... then continues for another 1,999,999 or so.
From: ilmari [...] ilmari.org (Dagfinn Ilmari Mannsåker)
CC: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>, Dave Mitchell via RT <perlbug-followup [...] perl.org>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
Date: Mon, 09 Jul 2018 12:16:35 +0100
To: Dave Mitchell <davem [...] iabyn.com>
Download (untitled) / with headers
text/plain 913b
Dave Mitchell <davem@iabyn.com> writes: Show quoted text
> A *reasonable* perl programmer would have looked at the documentation for > sort, noted that there was absolutely no mention of "in-place" sorting or > @a = sort @a having special behaviour, and either concluded that no > special behaviour was intended, or enquired with p5p as to why the > documentation was lacking, and whether this was an oversight.
FWIW: My original change came abuot exactly because someone on IRC was surprised by the undocumented special behaviour of in-place sort not strengthening weakrefs like a normal array assignment or non-in-place sort would. - ilmari -- - Twitter seems more influential [than blogs] in the 'gets reported in the mainstream press' sense at least. - Matt McLeod - That'd be because the content of a tweet is easier to condense down to a mainstream media article. - Calle Dybedahl
To: Dave Mitchell <davem [...] iabyn.com>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
Date: Sat, 14 Jul 2018 10:15:11 +0200
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
CC: Dave Mitchell via RT <perlbug-followup [...] perl.org>
Download (untitled) / with headers
text/plain 1.9k
Show quoted text
>>>>> On Thu, 5 Jul 2018 22:38:24 +0100, Dave Mitchell <davem@iabyn.com> said:
Show quoted text
> A *reasonable* perl programmer would have looked at the documentation for > sort, noted that there was absolutely no mention of "in-place" sorting or > @a = sort @a having special behaviour, and either concluded that no > special behaviour was intended, or enquired with p5p as to why the > documentation was lacking, and whether this was an oversight.
I'm wary whenever somebody argues that something would have needed to be done by somebody else. Especially so, when this somebody is a user of the language that the speaker maintains. Being a perl5-porter means being subject to a higher standard to what reasonable behaviour is than for a user of the language. In other words, a porter can never blame a user for unsatisficing documentation. We have a case that what the language maintainer demands from the language user has not happened. But it is never too late to adjust documentation. I would argue that - by having the documentation in the delta manpage, - not taking it over into the perfunc/sort manpage and - not noticing the missing part of the documentation for 13 years there is a case of a customary right on the side of all users of the language to keep the behaviour as implemented. Especially so, as the behaviour is useful and actively deployed in a high profile CPAN module that has a high number of direct and indirect users. Alternatives have not been suggested yet and might exist that might lead to an even better solution for all parties. Show quoted text
> PS - I don't think for one minute the use of the phrase "in-place" was > ambiguous *in context*, in either the p5p discussion or the perldelta; it > was only ambiguous in the sense that your specific questions about it > provided no context as to what behaviour was being referring to.
I'm not sure I can parse this sentence correctly. I think I have provided quite some context. If you think otherwise, feel free to ask. -- andreas
From: Dave Mitchell <davem [...] iabyn.com>
CC: Dave Mitchell via RT <perlbug-followup [...] perl.org>
To: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Date: Sat, 14 Jul 2018 09:47:01 +0100
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
Download (untitled) / with headers
text/plain 2.2k
On Sat, Jul 14, 2018 at 10:15:11AM +0200, Andreas Koenig wrote: Show quoted text
> I would argue that > > - by having the documentation in the delta manpage,
There was no such documentation in perldelta. There was mention of a new optimisation, not of a new sort language feature. There is a strong ethos within core development that new optimisations shouldn't change user-visible behaviour where possible. If they do, that's usually considered a bug, or if the trade off is worth it, something to be documented as a gotcha. Note also that the change was back-ported to 5.8.4 - an odd thing to do for a changed language feature. Show quoted text
> - not taking it over into the perfunc/sort manpage and
There was nothing to take over. Show quoted text
> - not noticing the missing part of the documentation for 13 years
There was nothing missing. Show quoted text
> there is a case of a customary right on the side of all users of the > language to keep the behaviour as implemented.
So do you agree that the behaviour which has been present since 5.000 and was accidentally changed in 5.8.4, should be reverted to the 5.000 state, since users have an expectation that the implemented behaviour should be kept? Show quoted text
> Alternatives have not been suggested yet and might exist that might lead > to an even better solution for all parties.
I'm open to suggestions. Show quoted text
> > PS - I don't think for one minute the use of the phrase "in-place" was > > ambiguous *in context*, in either the p5p discussion or the perldelta; it > > was only ambiguous in the sense that your specific questions about it > > provided no context as to what behaviour was being referring to.
> > I'm not sure I can parse this sentence correctly. I think I have > provided quite some context. If you think otherwise, feel free to ask.
You asked me some questions about the meaning of 'in-place'. I pointed out the that phrase could potentially have two meanings, and I wasn't clear which meaning you were referring to, so I gave answers for both meanings. I further pointed out that where that phrase had been used in perldelta / p5p discussion, I thought it was clear from the context, which meaning was intended there - I wasn't conceding that perldelta was ambiguous. -- O Unicef Clearasil! Gibberish and Drivel! -- "Bored of the Rings"
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Date: Sat, 14 Jul 2018 13:03:16 +0200
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
To: "Dave Mitchell via RT" <perlbug-followup [...] perl.org>
Download (untitled) / with headers
text/plain 3.4k
Show quoted text
>>>>> On Sat, 14 Jul 2018 01:47:25 -0700, "Dave Mitchell via RT" <perlbug-followup@perl.org> said:
Show quoted text
> On Sat, Jul 14, 2018 at 10:15:11AM +0200, Andreas Koenig wrote:
>> I would argue that >> >> - by having the documentation in the delta manpage,
Show quoted text
> There was no such documentation in perldelta. There was mention of a new > optimisation, not of a new sort language feature.
I have cited and llinked to the original places which sentences I speak about and I maintain that one could read it differently. Show quoted text
> There is a strong ethos within core development that new optimisations > shouldn't change user-visible behaviour where possible. If they do, > that's usually considered a bug, or if the trade off is worth it, > something to be documented as a gotcha.
It was not noticed for 13 years. Show quoted text
> Note also that the change was back-ported to 5.8.4 - an odd thing to do > for a changed language feature.
I'll cite my timeline again from my previous posting: 2004-02-20: https://rt.perl.org/Public/Bug/Display.html?id=25263#txn-79013 Within same RT ticket From: Dave Mitchell Citation: "I've now applied change #22349 to bleed, which makes @a = sort @a act inplace." And one line from man perlhist: 5.8.4 2004-Apr-21 For my reading, this is not a backport, it was properly injected into blead between 5.8.3 and 5.8.4. Show quoted text
>> - not taking it over into the perfunc/sort manpage and
Show quoted text
> There was nothing to take over.
Show quoted text
>> - not noticing the missing part of the documentation for 13 years
Show quoted text
> There was nothing missing.
Show quoted text
>> there is a case of a customary right on the side of all users of the >> language to keep the behaviour as implemented.
Show quoted text
> So do you agree that the behaviour which has been present since 5.000 and > was accidentally changed in 5.8.4, should be reverted to the 5.000 state, > since users have an expectation that the implemented behaviour should be > kept?
The changes in RT ticket 25263 got applause for 13 years. I have no reason to doubt the achievement that came with it. Compare that with the time between Ilmari's patch and the BBC ticket: 22 days. Show quoted text
>> Alternatives have not been suggested yet and might exist that might lead >> to an even better solution for all parties.
Show quoted text
> I'm open to suggestions.
First of all I would suggest to document existing behaviour for v5.8.4 - v5.28 and write tests that make sure we have it covered this time around. In a second iteration I would hope somebody (hopefully with decent language designing capabilities) comes up with something that allows future users to choose which behaviour they want. Show quoted text
>> > PS - I don't think for one minute the use of the phrase "in-place" was >> > ambiguous *in context*, in either the p5p discussion or the perldelta; it >> > was only ambiguous in the sense that your specific questions about it >> > provided no context as to what behaviour was being referring to.
>> >> I'm not sure I can parse this sentence correctly. I think I have >> provided quite some context. If you think otherwise, feel free to ask.
Show quoted text
> You asked me some questions about the meaning of 'in-place'. I pointed > out the that phrase could potentially have two meanings, and > I wasn't clear which meaning you were referring to, so I gave answers > for both meanings. I further pointed out that where that phrase had been > used in perldelta / p5p discussion, I thought it was clear from the > context, which meaning was intended there - I wasn't conceding that > perldelta was ambiguous.
Acknowledged. -- andreas
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
Date: Sun, 22 Jul 2018 16:28:36 +0200
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
CC: Dave Mitchell via RT <perlbug-followup [...] perl.org>
To: Dave Mitchell <davem [...] iabyn.com>
Download (untitled) / with headers
text/plain 2.2k
Show quoted text
>>>>> On Sat, 14 Jul 2018 09:47:01 +0100, Dave Mitchell <davem@iabyn.com> said:
Show quoted text
>> - not taking it over into the perfunc/sort manpage and
Show quoted text
> There was nothing to take over.
Show quoted text
>> - not noticing the missing part of the documentation for 13 years
Show quoted text
> There was nothing missing.
In my previous posting I overlooked this very interesting point in your argumentation. I think it more than deserves an answer. perl 5.8.4 introduced (thanks to your effort!) a remarkable user-tunable optimization on memory usage and you really firmly assert it is not worth documenting? Let me present figures how remarkable and how user-tunable it was and still is (lower is better on all figures): | | max VmPeak | | perl | @b=sort@a | @a=sort@a | |--------+-----------+-----------| | 5.8.3 | 194524 | 194524 | | 5.8.4 | 194532 | 114644 | | 5.28.0 | 171160 | 123768 | The point is that it is not only user-visible but actually user-tunable. Your arguments about not having user-visible effects collapse under these figures. What sense does it make to implement and announce an optimization that in fact requires cooperation from the user and then not to document it? And then to claim that it is intended as being non-user-visible? -- andreas PS: full testrun: % for t in `seq 1 3`; do for p in perl-5.8.3/6fdb perl-5.8.4/6fdb v5.28.0/da1c; do /home/sand/src/perl/repoperls/installed-perls/host/k93msid/$p/bin/perl -le ' my @a; push @a, substr(1000000+rand(9000000),0,7) while @a < 1000000; @a = sort @a; print "$]: ".join"",`cat /proc/$$/status` =~ /VmPeak:\s*(\d+)/; '; /home/sand/src/perl/repoperls/installed-perls/host/k93msid/$p/bin/perl -le ' my @a; push @a, substr(1000000+rand(9000000),0,7) while @a < 1000000; @b = sort @a; print "$]: ".join"",`cat /proc/$$/status` =~ /VmPeak:\s*(\d+)/; '; done; done 5.008003: 194524 5.008003: 194524 5.008004: 114644 5.008004: 194532 5.028000: 123768 5.028000: 171160 5.008003: 194524 5.008003: 194524 5.008004: 114644 5.008004: 194532 5.028000: 123768 5.028000: 171160 5.008003: 194524 5.008003: 194524 5.008004: 114644 5.008004: 194532 5.028000: 123768 5.028000: 171160
CC: Dave Mitchell via RT <perlbug-followup [...] perl.org>
From: Dave Mitchell <davem [...] iabyn.com>
To: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
Date: Sun, 22 Jul 2018 16:36:24 +0100
Download (untitled) / with headers
text/plain 1.8k
On Sun, Jul 22, 2018 at 04:28:36PM +0200, Andreas Koenig wrote: Show quoted text
> The point is that it is not only user-visible but actually user-tunable. > Your arguments about not having user-visible effects collapse under > these figures. What sense does it make to implement and announce an > optimization that in fact requires cooperation from the user and then > not to document it? And then to claim that it is intended as being > non-user-visible?
Most optimisations I do are "user tunable" by that definition. Did you know that if you do $a + $b, and ensure that $a and $b have only ever contained integers (so for example have never held strings or been used in a string context) then the addition is a lot faster? Is that documented in perlop? No. It is documented in perldelta? Quite possibly, in the "Optimizations" section for whatever release I tweaked pp_add() et al. I think the position you have taken is extreme, and if we used that position to inform future perl bug fixes, perl would development would come to a complete halt. Really I am utterly confounded by the stance you are taking. Perl has a strong ethos of maintaining backwards compatibility where reasonable, to the extent that many many people shout at us for causing the stagnation of perl 5. But we have never committed to maintaining 100% backwards compatibility at all costs. Each decision is made on its own merits, with a big dose of pragmatism. As I have said before, I am willing to debate the pragmatic pros and cons of whether it is better to make this change in behaviour. What I am not prepared to accept, is that digging out a 14 year old RT ticket, which at no no point proposes that weak references should be preserved, has somehow bound us in perpetuity to preserve weak references. -- Indomitable in retreat, invincible in advance, insufferable in victory -- Churchill on Montgomery
To: Dave Mitchell <davem [...] iabyn.com>
CC: Dave Mitchell via RT <perlbug-followup [...] perl.org>
Subject: Re: [perl #132142] Bleadperl v5.27.3-34-gf6107ca24b breaks MLEHMANN/AnyEvent-HTTP-2.23.tar.gz
Date: Sun, 29 Jul 2018 19:29:44 +0200
From: Andreas Koenig <andreas.koenig.7os6VVqR [...] franz.ak.mind.de>
Show quoted text
>>>>> On Sun, 22 Jul 2018 16:36:24 +0100, Dave Mitchell <davem@iabyn.com> said:
Show quoted text
> On Sun, Jul 22, 2018 at 04:28:36PM +0200, Andreas Koenig wrote:
>> The point is that it is not only user-visible but actually user-tunable. >> Your arguments about not having user-visible effects collapse under >> these figures. What sense does it make to implement and announce an >> optimization that in fact requires cooperation from the user and then >> not to document it? And then to claim that it is intended as being >> non-user-visible?
Show quoted text
> Most optimisations I do are "user tunable" by that definition.
Show quoted text
> Did you know that if you do $a + $b, and ensure that $a and $b have only > ever contained integers (so for example have never held strings or been > used in a string context) then the addition is a lot faster? Is that > documented in perlop? No. It is documented in perldelta? Quite possibly, > in the "Optimizations" section for whatever release I tweaked pp_add() et > al.
We have talked about it already. Perl is open source and as such the documentation more often than not is the code itself. This is good enough when everybody agrees. But when two developers interpret the source differently, then we must find a balance on how we deal with those differing interpretations. Then we must discuss what exactly we consider the desired behaviour. It's one of the basic fundaments of how open source works and why we engage in it. Show quoted text
> I think the position you have taken is extreme, and if we used that > position to inform future perl bug fixes, perl would development would > come to a complete halt.
This is without doubt an interesting question. But watch out, there is a bit of a grammatical twist in there and I guess it is because you edited this sentence after having written it. Maybe it indicates, that you are yourself in doubt about it? In general extremism allegations against dissenters turn me off. Show quoted text
> Really I am utterly confounded by the stance you are taking. Perl has a > strong ethos of maintaining backwards compatibility where reasonable, to > the extent that many many people shout at us for causing the stagnation of > perl 5.
Yes, but breaking stuff on CPAN without careful consideration is also not what we need. Show quoted text
> But we have never committed to maintaining 100% backwards > compatibility at all costs. Each decision is made on its own merits, with > a big dose of pragmatism.
Indeed. That's what I'm asking for: the appropriate amount of pragmatism. On a case-by-case basis. A careful evaluation, how to develop the language out of a conflict without brutally removing valuable features. Show quoted text
> As I have said before, I am willing to debate the pragmatic pros and cons > of whether it is better to make this change in behaviour.
I believe I have made a lot of my points. At least I have digged out the original RT ticket that was leading to your commit. I find that in-place sorting without changing any of the elements of an array is a feature that we should not throw away. Show quoted text
> What I am not prepared to accept, is that digging out a 14 year old > RT ticket, which at no no point proposes that weak references should > be preserved, has somehow bound us in perpetuity to preserve weak > references.
Your argument is very hard to parse for me, please explain what your argument is: - "digging": is it the fact that I was digging in the history what you find unacceptable? Are you claiming some right to be forgotten about old RT tickets? Or is digging an dirty inappropriate for language debate? What else could you have against digging? - "a 14 year old": is it that the discussion is 14 years old? would it be more acceptable when the discussion would be more recent? Or older? How many years would be OK for you? And justified for which reason? Remember, that *my* argument also points out the 14 years. I'm arguing, it took 14 years until Tom Molesworth and Ilmari discovered this behaviour. For me it is a reason to treat it a feature that just has not been documented. - "at no point proposes that weak references": the ticket said "in-place" and one reading of "in-place" means: leave each element alone. Leaving each element alone means of course not changing weak references. One could argue that the ticket was underspecified. But why would that be a justification to reject any different interpretation than your own? I know you disagree with my reading but then it's simply something we have to agree that we disagree. - "perpetuity": what are you arguing against perpetuity? Has a language to be bent and redefined in all its appearances every day to be useful? Perpetuity is despite not being realistic, a blessing. For a programmer acting as-if it were for the perpetuity should be a liability, not a shame. I repeat my quest: it is time to document existing behaviour for v5.8.4 - v5.28 and write tests that make sure we have it covered this time around. In a second iteration I would hope somebody (hopefully with decent language designing capabilities) comes up with something that allows future users to choose which behaviour they want. -- andreas


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org