MAD produces invalid XML #11136

p5pRT · 2011-02-17T11:32:23Z

Migrated from rt.perl.org#84314 (status was 'open')

Searchable as RT84314$

p5pRT · 2011-02-17T11:32:24Z

From h.goebel@goebel-consult.de

Created by h.goebel@goebel-consult.de

This is a bug report for perl from h.goebel@goebel-consult.de,
generated with the help of perlbug 1.39 running under perl 5.10.1.

-----------------------------------------------------------------

MAD XML output is not alway valid: attributes are not always encoded
properly.

Example: When MAD-dumping this script

m!<error>(\d+).*<message>(.+?)</message>!s;

this will give you (snippet):

<pmop
pre="<error>(\d+).*<message>(.+?)</message>"
when="RUN"
pmflags="SCANFIRST"
/>

The value for the `pre` attribute is not encoded properly. According
to XML, angle brackets are to be encodes when used within attribute
values.

Please node: The same attribute value is encoded properly at other places, esp. when used in <mad_sv>-tags:

<mad_sv key="=" val="<error>(\d+).*<message>(.+?)</message>"/>

So here the encoding is correct.

Perl Info


Flags:
    category=core
    severity=low

Site configuration information for perl 5.10.1:

Configured by Mandriva at Mon Jun 14 08:42:32 UTC 2010.

Summary of my perl5 (revision 5 version 10 subversion 1) configuration:
   
  Platform:
    osname=linux, osvers=2.6.22.12-server-1mdv, archname=i386-linux-thread-multi
    uname='linux n4.mandriva.com 2.6.22.12-server-1mdv #1 smp tue nov 20 09:35:09 est 2007 i686 i686 i386 gnulinux '
    config_args='-des -Dinc_version_list=5.10.0 5.10.0/i386-linux-thread-multi 5.8.8 5.8.7 5.8.6 5.8.5 5.8.4 5.8.3 5.8.2 5.8.1 5.8.0 5.6.1 5.6.0 -Darchname=i386-linux -Dcc=i586-mandriva-linux-gnu-gcc -Doptimize=-O2 -g -pipe -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector --param=ssp-buffer-size=4 -fomit-frame-pointer -march=i586 -mtune=generic -fasynchronous-unwind-tables -DDEBUGGING=-g -Dprefix=/usr -Dvendorprefix=/usr -Dsiteprefix=/usr -Dsitebin=/usr/local/bin -Dsiteman1dir=/usr/local/share/man/man1 -Dsiteman3dir=/usr/local/share/man/man3 -Dman3ext=3pm -Dcf_by=Mandriva -Dmyhostname=localhost -Dperladmin=root@localhost -Dcf_email=root@localhost -Dd_dosuid -Ud_csh -Duseshrplib -Duseithreads -Di_db -Di_ndbm -Di_gdbm'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='i586-mandriva-linux-gnu-gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g -pipe -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector --param=ssp-buffer-size=4 -fomit-frame-pointer -march=i586 -mtune=generic -fasynchronous-unwind-tables',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.4.3', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='i586-mandriva-linux-gnu-gcc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.11.1.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.11.1'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/usr/lib/perl5/5.10.1/i386-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector --param=ssp-buffer-size=4 -fomit-frame-pointer -march=i586 -mtune=generic -fasynchronous-unwind-tables -L/usr/local/lib'

Locally applied patches:
    Mandriva Linux patches


@INC for perl 5.10.1:
    /usr/lib/perl5/site_perl/5.10.1/i386-linux-thread-multi
    /usr/lib/perl5/site_perl/5.10.1
    /usr/lib/perl5/vendor_perl/5.10.1/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.10.1
    /usr/lib/perl5/5.10.1/i386-linux-thread-multi
    /usr/lib/perl5/5.10.1
    /usr/lib/perl5/site_perl/5.8.8
    /usr/lib/perl5/site_perl
    /usr/lib/perl5/vendor_perl/5.10.0
    /usr/lib/perl5/vendor_perl/5.10.0/i386-linux-thread-multi
    /usr/lib/perl5/vendor_perl/5.8.8
    /usr/lib/perl5/vendor_perl/5.8.7
    /usr/lib/perl5/vendor_perl/5.8.6
    /usr/lib/perl5/vendor_perl
    .


Environment for perl 5.10.1:
    HOME=/home/hartmut
    LANG=de_DE.UTF-8
    LANGUAGE=de_DE.UTF-8:de
    LC_ADDRESS=de_DE.UTF-8
    LC_COLLATE=de_DE.UTF-8
    LC_CTYPE=de_DE.UTF-8
    LC_IDENTIFICATION=de_DE.UTF-8
    LC_MEASUREMENT=de_DE.UTF-8
    LC_MESSAGES=de_DE.UTF-8
    LC_MONETARY=de_DE.UTF-8
    LC_NAME=de_DE.UTF-8
    LC_NUMERIC=de_DE.UTF-8
    LC_PAPER=de_DE.UTF-8
    LC_SOURCED=1
    LC_TELEPHONE=de_DE.UTF-8
    LC_TIME=de_DE.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/usr/bin:/bin:/usr/local/bin:/usr/X11R6/bin/:/usr/games:/usr/lib/qt4/bin:/home/hartmut/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

p5pRT · 2011-12-19T00:39:33Z

From @jkeenan

On Thu Feb 17 03:32:24 2011, h.goebel@goebel-consult.de wrote:

MAD XML output is not alway valid: attributes are not always encoded
properly.

I'm probably not the only one who has to ask: "What does 'MAD' mean in
this context?"

Thank you very much.
Jim Keenan

p5pRT · 2011-12-19T00:39:34Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2011-12-19T00:49:10Z

From @tonycoz

On Sun, Dec 18, 2011 at 04:39:34PM -0800, James E Keenan via RT wrote:

On Thu Feb 17 03:32:24 2011, h.goebel@goebel-consult.de wrote:

MAD XML output is not alway valid: attributes are not always encoded
properly.

I'm probably not the only one who has to ask: "What does 'MAD' mean in
this context?"

It's a (neglected) tool to dump the perl parse tree into something
that can be processed by external tools.

To use, build with -Dmad:

./Configure -Dmad ...

Then run perl with PERL_XMLDUMP to the name of a file to dump to:

PERL_XMLDUMP=somefilename perl ...

There's a few problems:

a) it's largely undocumented

b) what few tests there are, are not run as part of the normal
regression tests even when we build with -Dmad, since there's no core
XML parser. See the mad/ directory in the perl tree.

There's been some discussion of changing the output to JSON to make it
more testable, but I doubt it's a high priority for anyone.

Tony

p5pRT · 2011-12-19T00:51:46Z

From @jkeenan

On Sun Dec 18 16:49:10 2011, tonyc wrote:

On Sun, Dec 18, 2011 at 04:39:34PM -0800, James E Keenan via RT wrote:

It's a (neglected) tool to dump the perl parse tree into something
that can be processed by external tools.

To use, build with -Dmad:

./Configure -Dmad ...

Then run perl with PERL_XMLDUMP to the name of a file to dump to:

PERL_XMLDUMP=somefilename perl ...

There's a few problems:

a) it's largely undocumented

b) what few tests there are, are not run as part of the normal
regression tests even when we build with -Dmad, since there's no core
XML parser. See the mad/ directory in the perl tree.

There's been some discussion of changing the output to JSON to make it
more testable, but I doubt it's a high priority for anyone.

Thanks for your quick response. It leads me to ask: Should MAD be in
core at all?

Jim Keenan

p5pRT · 2011-12-19T02:14:15Z

From @rjbs

* James E Keenan via RT <perlbug-followup@perl.org> [2011-12-18T19:51:46]

Thanks for your quick response. It leads me to ask: Should MAD be in
core at all?

It is there for $Larry's sake for the sake of testing a future five-to-six
converter. It is my understanding that it is not to be removed.

--
rjbs

p5pRT · 2011-12-19T02:29:29Z

From chromatic@wgz.org

On Sunday, December 18, 2011 at 06:13 PM, Ricardo Signes wrote:

* James E Keenan via RT <perlbug-followup@perl.org> [2011-12-18T19:51:46]

Thanks for your quick response. It leads me to ask: Should MAD be in
core at all?

It is there for $Larry's sake for the sake of testing a future five-to-six
converter. It is my understanding that it is not to be removed.

What it does can only be done in the core. The parser throws away too much
information for any other approach to work.

(If the parser kept such information, the story would be different.)

-- c

p5pRT · 2011-12-19T02:34:42Z

From @tonycoz

On Sun, Dec 18, 2011 at 09:13:42PM -0500, Ricardo Signes wrote:

* James E Keenan via RT <perlbug-followup@perl.org> [2011-12-18T19:51:46]

Thanks for your quick response. It leads me to ask: Should MAD be in
core at all?

It is there for $Larry's sake for the sake of testing a future five-to-six
converter. It is my understanding that it is not to be removed.

From memory, Jesse thought it could become useful.

Tony

p5pRT · 2011-12-19T04:27:36Z

From chris@prather.org

On Sun, Dec 18, 2011 at 9:34 PM, Tony Cook <tony@develop-help.com> wrote:

On Sun, Dec 18, 2011 at 09:13:42PM -0500, Ricardo Signes wrote:

* James E Keenan via RT <perlbug-followup@perl.org> [2011-12-18T19:51:46]

Thanks for your quick response. It leads me to ask: Should MAD be in
core at all?

It is there for $Larry's sake for the sake of testing a future five-to-six
converter. It is my understanding that it is not to be removed.

From memory, Jesse thought it could become useful.

Tony

I actually have experimental code somewhere that parses it into a
Moose object tree. I wasn't sure what I was going to do with it after
that, I found the prospects overwhelming.

-Chris

p5pRT · 2011-12-19T07:07:12Z

From @obra

On Mon, Dec 19, 2011 at 01:34:12PM +1100, Tony Cook wrote:

On Sun, Dec 18, 2011 at 09:13:42PM -0500, Ricardo Signes wrote:

* James E Keenan via RT <perlbug-followup@perl.org> [2011-12-18T19:51:46]

Thanks for your quick response. It leads me to ask: Should MAD be in
core at all?

It is there for $Larry's sake for the sake of testing a future five-to-six
converter. It is my understanding that it is not to be removed.

From memory, Jesse thought it could become useful.

I still think it could be useful. :)

Tony

--

p5pRT · 2011-12-19T09:11:13Z

From acme@astray.com

On Mon, Dec 19, 2011 at 11:48:35AM +1100, Tony Cook wrote:

a) it's largely undocumented

b) what few tests there are, are not run as part of the normal
regression tests even when we build with -Dmad, since there's no core
XML parser. See the mad/ directory in the perl tree.

It "generates" XML by a whole bunch of print statements - it was never
going to be valid XML. It's also a whole lot of code split all over
the place and not easily refactored or tested :-(

At one point I considered replacing it with YAJL:

http://lloyd.github.com/yajl/
https://metacpan.org/module/JSON::YAJL

... but the task was quite huge, so I never quite started it.

HTH, Leon.

richardleach · 2019-10-21T21:49:34Z

The MAD (Misc Attribute Decoration) build option has been gone for a while, according to perl5220delta.

Is this ticket now obsolete and can therefore be closed?

toddr · 2019-10-21T22:42:20Z

Yes

p5pRT added Severity Low distro-Linux type-core labels Oct 19, 2019

toddr closed this as completed Oct 21, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MAD produces invalid XML #11136

MAD produces invalid XML #11136

p5pRT commented Feb 17, 2011

p5pRT commented Feb 17, 2011

p5pRT commented Dec 19, 2011

p5pRT commented Dec 19, 2011

p5pRT commented Dec 19, 2011

p5pRT commented Dec 19, 2011

p5pRT commented Dec 19, 2011

p5pRT commented Dec 19, 2011

p5pRT commented Dec 19, 2011

p5pRT commented Dec 19, 2011

p5pRT commented Dec 19, 2011

p5pRT commented Dec 19, 2011

richardleach commented Oct 21, 2019

toddr commented Oct 21, 2019

MAD produces invalid XML #11136

MAD produces invalid XML #11136

Comments

p5pRT commented Feb 17, 2011

p5pRT commented Feb 17, 2011

From h.goebel@goebel-consult.de

Created by h.goebel@goebel-consult.de

p5pRT commented Dec 19, 2011

From @jkeenan

p5pRT commented Dec 19, 2011

p5pRT commented Dec 19, 2011

From @tonycoz

p5pRT commented Dec 19, 2011

From @jkeenan

p5pRT commented Dec 19, 2011

From @rjbs

p5pRT commented Dec 19, 2011

From chromatic@wgz.org

p5pRT commented Dec 19, 2011

From @tonycoz

p5pRT commented Dec 19, 2011

From chris@prather.org

p5pRT commented Dec 19, 2011

From @obra

p5pRT commented Dec 19, 2011

From acme@astray.com

richardleach commented Oct 21, 2019

toddr commented Oct 21, 2019