Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data::Dumper Feature Request: CondenseRanges option #15069

Open
p5pRT opened this issue Dec 5, 2015 · 9 comments
Open

Data::Dumper Feature Request: CondenseRanges option #15069

p5pRT opened this issue Dec 5, 2015 · 9 comments
Labels
dist-Data-Dumper issues in the dual-life blead-first Data-Dumper distribution Wishlist

Comments

@p5pRT
Copy link

p5pRT commented Dec 5, 2015

Migrated from rt.perl.org#126812 (status was 'open')

Searchable as RT126812$

@p5pRT
Copy link
Author

p5pRT commented Dec 5, 2015

From @kentfredric

There's a few features that exist in a few other Dumper tools that are simple enough that I imagine Data​::Dumper can steal them.

One of these is the trick Data​::Dump does to condense ranges.

Data​::Dump recognises arrays of length >= 3 items that consist of monotonic integral steps and simplifies them to a range expression​:

For [1,2,3,4,5,6,7,8,9, 10], where Data​::Dumper presently spews 10 lines on indentation levels other than 0, Data​::Dump elegantly emits

[ 1 .. 10 ]

Obviously this is a feature that would be opt-in, as anything currently relying on string comparison of dumped structures in the wild randomly changing would be bad.

Why a feature for Data​::Dumper and not "Just use the other module"?

The other module doesn't do sub { } deparsing, which somewhat limits the ability to get both sub deparsing and simplified lists.

Data​::Dump also only handles forward ranges, and I think we can possibly do a better job, and maybe support
  [ 10, 9, 8, 7,6,5,4,3,2, 1 ]
emitting
  [ reverse 1 .. 10 ]

In a perfect world it could simplify any OEIS entry, but I can settle for a few higher-frequency common cases.


Flags​:
  category=core
  severity=low


Site configuration information for perl 5.22.0​:

Configured by kent at Fri Jun 19 08​:03​:55 NZST 2015.

Summary of my perl5 (revision 5 version 22 subversion 0) configuration​:
 
  Platform​:
  osname=linux, osvers=4.0.0-gentoo, archname=x86_64-linux
  uname='linux katipo2 4.0.0-gentoo #23 smp preempt sat apr 25 06​:58​:21 nzst 2015 x86_64 intel(r) core(tm) i5-2410m cpu @​ 2.30ghz genuineintel gnulinux '
  config_args='-de -Dprefix=/home/kent/perl5/perlbrew/perls/5.22.0 -Dusecbacktrace -Doptimize= -fno-stack-protector -O3 -march=native -mtune=native -Dman1dir=none -Dman3dir=none -Accflags= -fno-stack-protector -DPERL_HASH_FUNC_SDBM -DUSE_C_BACKTRACE_ON_ERROR -Aldflags= -fno-stack-protector -lbfd -Aeval​:scriptdir=/home/kent/perl5/perlbrew/perls/5.22.0/bin'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=undef, usemultiplicity=undef
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-fno-stack-protector -DPERL_HASH_FUNC_SDBM -DUSE_C_BACKTRACE_ON_ERROR -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -DUSE_C_BACKTRACE -g -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
  optimize=' -fno-stack-protector -O3 -march=native -mtune=native',
  cppflags='-fno-stack-protector -DPERL_HASH_FUNC_SDBM -DUSE_C_BACKTRACE_ON_ERROR -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong'
  ccversion='', gccversion='4.9.2', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -fno-stack-protector -lbfd -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/include-fixed /usr/lib /usr/local/lib /lib/../lib64 /usr/lib/../lib64 /lib /lib64 /usr/lib64 /usr/local/lib64
  libs=-lpthread -lnsl -lnm -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
  perllibs=-lpthread -lnsl -lnm -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.20.so, so=so, useshrplib=false, libperl=libperl.a
  gnulibc_version='2.20'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -fno-stack-protector -O3 -march=native -mtune=native -L/usr/local/lib -fstack-protector-strong'


@​INC for perl 5.22.0​:
  /home/kent/perl5/perlbrew/perls/5.22.0/lib/site_perl/5.22.0/x86_64-linux
  /home/kent/perl5/perlbrew/perls/5.22.0/lib/site_perl/5.22.0
  /home/kent/perl5/perlbrew/perls/5.22.0/lib/5.22.0/x86_64-linux
  /home/kent/perl5/perlbrew/perls/5.22.0/lib/5.22.0
  .


Environment for perl 5.22.0​:
  HOME=/home/kent
  LANG (unset)
  LANGUAGE (unset)
  LC_CTYPE=en_NZ.UTF8
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/home/kent/perl5/perlbrew/bin​:/home/kent/perl5/perlbrew/perls/5.22.0/bin​:/home/kent/.perl6/2013.04/bin​:/home/kent/.gem/ruby/1.8/bin/​:/home/kent/.rvm/gems/ruby-2.1.2/bin​:/home/kent/.rvm/gems/ruby-2.1.2@​global/bin​:/home/kent/.rvm/rubies/ruby-2.1.2/bin​:/usr/local/bin​:/usr/bin​:/bin​:/opt/bin​:/usr/x86_64-pc-linux-gnu/gcc-bin/5.2.0​:/opt/android-sdk-update-manager/tools​:/opt/android-sdk-update-manager/platform-tools​:/usr/games/bin​:/home/kent/.rvm/bin​:/home/kent/.rvm/bin
  PERLBREW_BASHRC_VERSION=0.72
  PERLBREW_HOME=/home/kent/.perlbrew
  PERLBREW_MANPATH=/home/kent/perl5/perlbrew/perls/5.22.0/man
  PERLBREW_PATH=/home/kent/perl5/perlbrew/bin​:/home/kent/perl5/perlbrew/perls/5.22.0/bin
  PERLBREW_PERL=5.22.0
  PERLBREW_ROOT=/home/kent/perl5/perlbrew
  PERLBREW_VERSION=0.72
  PERL_BADLANG (unset)
  SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Sep 3, 2017

From @jkeenan

On Sat, 05 Dec 2015 01​:24​:40 GMT, kentfredric wrote​:

There's a few features that exist in a few other Dumper tools that are
simple enough that I imagine Data​::Dumper can steal them.

One of these is the trick Data​::Dump does to condense ranges.

Data​::Dump recognises arrays of length >= 3 items that consist of
monotonic integral steps and simplifies them to a range expression​:

For [1,2,3,4,5,6,7,8,9, 10], where Data​::Dumper presently spews 10
lines on indentation levels other than 0, Data​::Dump elegantly emits

[ 1 .. 10 ]

Obviously this is a feature that would be opt-in, as anything
currently relying on string comparison of dumped structures in the
wild randomly changing would be bad.

Why a feature for Data​::Dumper and not "Just use the other module"?

The other module doesn't do sub { } deparsing, which somewhat limits
the ability to get both sub deparsing and simplified lists.

Data​::Dump also only handles forward ranges, and I think we can
possibly do a better job, and maybe support
[ 10, 9, 8, 7,6,5,4,3,2, 1 ]
emitting
[ reverse 1 .. 10 ]

In a perfect world it could simplify any OEIS entry, but I can settle
for a few higher-frequency common cases.

This ticket has received no comments or replies since it was first posted in December 2015. Lack of comment on a new feature request usually means that people have (silently) made the judgment​: "The benefit of this feature does not outweigh the cost of creating and maintaining it."

I suspect that's the case here. Data​::Dumper poses maintenance problems because it has both "pure perl" and XS versions. New features would have to be implemented both ways. The people who can grok the XS code only have the time to keep up with critical bugs. I doubt they have time to implement additional features -- and then maintain them.

In this situation you're more likely to get the new functionality if you do it first as a new CPAN distribution. If that CPAN distribution is well received, then perhaps it could be brought into core in future years.

For that reason I recommend that we close this ticket.

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Sep 3, 2017

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Sep 3, 2017

From @kentfredric

On 3 September 2017 at 12​:16, James E Keenan via RT
<perlbug-followup@​perl.org> wrote​:

In this situation you're more likely to get the new functionality if you do it first as a new CPAN distribution. If that CPAN distribution is well received, then perhaps it could be brought into core in future years.

Like the other Data​::Dumper feature request you replied to with an
identical text​:

- Yes, I considered that
- Yes, it already exists
- I stated this and named it as part of my rationale
- The motivation here is that last part, to bring existing functionality to core

Maybe responses to bugs should be based on reading their content?

--
Kent

KENTNL - https://metacpan.org/author/KENTNL

@p5pRT
Copy link
Author

p5pRT commented Sep 4, 2017

From @iabyn

On Sat, Sep 02, 2017 at 05​:16​:35PM -0700, James E Keenan via RT wrote​:

This ticket has received no comments or replies since it was first posted in December 2015. Lack of comment on a new feature request usually means that people have (silently) made the judgment​: "The benefit of this feature does not outweigh the cost of creating and maintaining it."

As I just said in the other ticket, I think wishlist tickets should be
kept open unless we have explicitly rejected a suggestion.

--
"You're so sadly neglected, and often ignored.
A poor second to Belgium, When going abroad."
  -- Monty Python, "Finland"

@p5pRT
Copy link
Author

p5pRT commented Sep 5, 2017

From @xsawyerx

On 09/04/2017 11​:34 AM, Dave Mitchell wrote​:

On Sat, Sep 02, 2017 at 05​:16​:35PM -0700, James E Keenan via RT wrote​:

This ticket has received no comments or replies since it was first posted in December 2015. Lack of comment on a new feature request usually means that people have (silently) made the judgment​: "The benefit of this feature does not outweigh the cost of creating and maintaining it."
As I just said in the other ticket, I think wishlist tickets should be
kept open unless we have explicitly rejected a suggestion.

I mostly agree.

I think it's not unreasonable after a certain amount of time to revisit
and ask ourselves "Do we really want to do it?" and then if no
responses, maybe considering removing it and only raising it again if
it's worthwhile or demanded more by users.

Having a large queue system is not in our favor because it makes it hard
to sort and prioritize. Declaring feature options that are long
forgotten (allowing them to resurface again with renewed interest) can
help with that.

@p5pRT
Copy link
Author

p5pRT commented Sep 5, 2017

From @kentfredric

Besides, here the delay has served its own purpose.

It allowed me to get around to filing https://rt.perl.org/Public/Bug/Display.html?id=132027 , which should be implemented before this is even considered.

I had meant to file it earlier, I just simply forgot/was distracted.

Otherwise this feature just adds another potential formatting concern people *have* to make sure they've safe-guarded.

@p5pRT
Copy link
Author

p5pRT commented Sep 5, 2017

From @demerphq

Iirc have implemented this feature and/or similar features in
Data​::Dump​::Streamer.

I do not think they are appropriate in Data​::Dumper.

DD is a special beast. It is bundled with perl, it has XS and perl
implementations, it has defaults and features that make it difficult to
extend, it is abused in production systems and it contains many known and
difficult to fix bugs because of the features it supports and because perl
code is a relatively shitty format for representing the full range of data
structures one can create using perl code (which sounds perverse but isn't).

A feature like this is something that will be of value to only a few, will
cost every user or alternatively require significant changes to the
implementation. Given it is available in other dumpers lacking the wide use
of DD I think there is no reason to add it to DD.

As an author of two relatively widely used perl serialisation modules, one
DD like and one Storable like I can say with a high degree of confidence
that serialisation is hard and features like this are much much bigger can
of worms than might seem to someone with more limited use cases and
preferences.

This type of feature for instance is likely to cause headaches to anyone
dealing with aliases or self referential structures, and the existing bugs
in DD that relate to them. Heck even deciding if something should be
serialised as a number or as a string is difficult to impossible.

So I would vote against this ticket, although I might support it in some
kind of DD successor with less hysterical baggage. Note I don't think that
DDS would be such a candidate.

Yves

On 5 Dec 2015 03​:25, "Kent Fredric" <perlbug-followup@​perl.org> wrote​:

# New Ticket Created by Kent Fredric
# Please include the string​: [perl #126812]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=126812 >

There's a few features that exist in a few other Dumper tools that are
simple enough that I imagine Data​::Dumper can steal them.

One of these is the trick Data​::Dump does to condense ranges.

Data​::Dump recognises arrays of length >= 3 items that consist of
monotonic integral steps and simplifies them to a range expression​:

For [1,2,3,4,5,6,7,8,9, 10], where Data​::Dumper presently spews 10 lines
on indentation levels other than 0, Data​::Dump elegantly emits

[ 1 .. 10 ]

Obviously this is a feature that would be opt-in, as anything currently
relying on string comparison of dumped structures in the wild randomly
changing would be bad.

Why a feature for Data​::Dumper and not "Just use the other module"?

The other module doesn't do sub { } deparsing, which somewhat limits the
ability to get both sub deparsing and simplified lists.

Data​::Dump also only handles forward ranges, and I think we can possibly
do a better job, and maybe support
[ 10, 9, 8, 7,6,5,4,3,2, 1 ]
emitting
[ reverse 1 .. 10 ]

In a perfect world it could simplify any OEIS entry, but I can settle for
a few higher-frequency common cases.

---
Flags​:
category=core
severity=low
---
Site configuration information for perl 5.22.0​:

Configured by kent at Fri Jun 19 08​:03​:55 NZST 2015.

Summary of my perl5 (revision 5 version 22 subversion 0) configuration​:

Platform​:
osname=linux, osvers=4.0.0-gentoo, archname=x86_64-linux
uname='linux katipo2 4.0.0-gentoo #23 smp preempt sat apr 25 06​:58​:21
nzst 2015 x86_64 intel(r) core(tm) i5-2410m cpu @​ 2.30ghz genuineintel
gnulinux '
config_args='-de -Dprefix=/home/kent/perl5/perlbrew/perls/5.22.0
-Dusecbacktrace -Doptimize= -fno-stack-protector -O3 -march=native
-mtune=native -Dman1dir=none -Dman3dir=none -Accflags= -fno-stack-protector
-DPERL_HASH_FUNC_SDBM -DUSE_C_BACKTRACE_ON_ERROR -Aldflags=
-fno-stack-protector -lbfd -Aeval​:scriptdir=/home/kent/
perl5/perlbrew/perls/5.22.0/bin'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler​:
cc='cc', ccflags ='-fno-stack-protector -DPERL_HASH_FUNC_SDBM
-DUSE_C_BACKTRACE_ON_ERROR -fwrapv -fno-strict-aliasing -pipe
-fstack-protector-strong -DUSE_C_BACKTRACE -g -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
optimize=' -fno-stack-protector -O3 -march=native -mtune=native',
cppflags='-fno-stack-protector -DPERL_HASH_FUNC_SDBM
-DUSE_C_BACKTRACE_ON_ERROR -fwrapv -fno-strict-aliasing -pipe
-fstack-protector-strong'
ccversion='', gccversion='4.9.2', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678,
doublekind=3
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16,
longdblkind=3
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries​:
ld='cc', ldflags =' -fno-stack-protector -lbfd
-fstack-protector-strong -L/usr/local/lib'
libpth=/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.2/include-fixed /usr/lib
/usr/local/lib /lib/../lib64 /usr/lib/../lib64 /lib /lib64 /usr/lib64
/usr/local/lib64
libs=-lpthread -lnsl -lnm -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
-lgdbm_compat
perllibs=-lpthread -lnsl -lnm -ldl -lm -lcrypt -lutil -lc
libc=libc-2.20.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.20'
Dynamic Linking​:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -fno-stack-protector -O3
-march=native -mtune=native -L/usr/local/lib -fstack-protector-strong'

---
@​INC for perl 5.22.0​:
/home/kent/perl5/perlbrew/perls/5.22.0/lib/site_perl/5.
22.0/x86_64-linux
/home/kent/perl5/perlbrew/perls/5.22.0/lib/site_perl/5.22.0
/home/kent/perl5/perlbrew/perls/5.22.0/lib/5.22.0/x86_64-linux
/home/kent/perl5/perlbrew/perls/5.22.0/lib/5.22.0
.

---
Environment for perl 5.22.0​:
HOME=/home/kent
LANG (unset)
LANGUAGE (unset)
LC_CTYPE=en_NZ.UTF8
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/home/kent/perl5/perlbrew/bin​:/home/kent/perl5/
perlbrew/perls/5.22.0/bin​:/home/kent/.perl6/2013.04/bin​:/
home/kent/.gem/ruby/1.8/bin/​:/home/kent/.rvm/gems/ruby-2.1.
2/bin​:/home/kent/.rvm/gems/ruby-2.1.2@​global/bin​:/home/
kent/.rvm/rubies/ruby-2.1.2/bin​:/usr/local/bin​:/usr/bin​:/
bin​:/opt/bin​:/usr/x86_64-pc-linux-gnu/gcc-bin/5.2.0​:/opt/
android-sdk-update-manager/tools​:/opt/android-sdk-update-
manager/platform-tools​:/usr/games/bin​:/home/kent/.rvm/bin​:
/home/kent/.rvm/bin
PERLBREW_BASHRC_VERSION=0.72
PERLBREW_HOME=/home/kent/.perlbrew
PERLBREW_MANPATH=/home/kent/perl5/perlbrew/perls/5.22.0/man
PERLBREW_PATH=/home/kent/perl5/perlbrew/bin​:/home/kent/
perl5/perlbrew/perls/5.22.0/bin
PERLBREW_PERL=5.22.0
PERLBREW_ROOT=/home/kent/perl5/perlbrew
PERLBREW_VERSION=0.72
PERL_BADLANG (unset)
SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Sep 5, 2017

From @kentfredric

Much of this is motivated by the facts that I've been using the competition for a long time, and there's only a handful of reasons why I'm using the competition.

Data​::Dumper is *mostly* fine, its just got some quirks that make it periodically impractical when dumping output *for humans* to read.

If for example, I was the author of Data​::Dumper​::Concise, or Devel​::Confess , it would be much preferable to just determine that the Data​::Dumper that shipped with Perl had such useful functions for making output more readable, and the motivation to add a dependency on some other Dumper ( one that can't do deparse at that ) for this just wouldn't be worth it.

Producing competition on CPAN is a fine philosophy, but software runs in cycles. You have a divergence phase where people fork off because they want something different, and then a waiting period while they discover all the things wrong with that approach. And then people find the best of what works and come together and re-centralise.

We've spent years with this feature and others floating around on CPAN, its getting around about time for the best of those features to become available.

I accept that perhaps, there are some issues with this feature being added to the DD code, but *shrug*. And I accept there are some people out there who won't have a use for it, and it won't work as a *serialization* strategy and DD will lose some fidelity when the feature is used.

But that's why DD has tuneable features, and some of those features already lose fidelity and are unsuited for people using it as a serializer, But are perfectly fine for people who are just using DD for human readability.

There's plenty of debugging code out there which uses DD which would love to just go "eval { DD->VERSION(3.1415) } ? $instance->CondenseRanges(1) ", because its much preferable than having your screen filled.

And that's what Data​::Dump is *good* at, just sometimes I want to see the source code of functions hiding in my data structures too, and I can't, I have to either have lots of cruft w/ source, or no cruft, and no source.

And on top of that, I have to have useless conversations asking me why I need to depend on a 3rd party dumper library when there's one in core ...

@jkeenan jkeenan added dist-Data-Dumper issues in the dual-life blead-first Data-Dumper distribution Wishlist and removed Severity Low labels Jul 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dist-Data-Dumper issues in the dual-life blead-first Data-Dumper distribution Wishlist
Projects
None yet
Development

No branches or pull requests

2 participants