Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

utf8::decode turns undef into defined #12475

Closed
p5pRT opened this issue Oct 9, 2012 · 7 comments
Closed

utf8::decode turns undef into defined #12475

p5pRT opened this issue Oct 9, 2012 · 7 comments
Labels
distro-Linux type-utilities

Comments

@p5pRT
Copy link

p5pRT commented Oct 9, 2012

Migrated from rt.perl.org#115196 (status was 'rejected')

Searchable as RT115196$
@p5pRT
Copy link
Author

p5pRT commented Oct 9, 2012

From durandle@gmail.com

Created by durandle@gmail.com

This is a bug report for perl from durandle@​gmail.com,
generated with the help of perlbug 1.39 running under perl 5.16.1.

-----------------------------------------------------------------
Under perl 5.12 (and possibly other versions)​:

my $foo;
utf8​::decode($foo);

The result would be that $foo is still undefined.

We upgraded to 5.16 recently, the result of the above code is that $foo changes from undef to
blank but defined. I can't find anything in the perl changelog to indicate this is a designed
change, but then I could just be missing it.

It did cause issues for us as our form validation checks undef vs blank, although it could be
said that we should be more careful and not decode an undef value!

Perl Info 

Flags:
    category=utilities
    severity=medium

Site configuration information for perl 5.16.1:

Configured by dfj at Mon Oct  1 16:22:17 UTC 2012.

Summary of my perl5 (revision 5 version 16 subversion 1) configuration:
   
  Platform:
    osname=linux, osvers=2.6.35-9-generic-pae, archname=i686-linux-thread-multi
    uname='linux eu-devel.bm.car-rental-world.com 2.6.35-9-generic-pae #14~lucid1 smp thu jan 13 05:55:18 utc 2011 i686 gnulinux '
    config_args='-de -Duserelocatableinc -Dprefix=/home/staff/dfj/perl5/perlbrew/perls/perl-5.16.1 -Dusethreads -Dusemultiplicity -Aeval:scriptdir=/home/staff/dfj/perl5/perlbrew/perls/perl-5.16.1/bin'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.4.3', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/../lib /usr/lib/../lib /lib /usr/lib /usr/lib64
    libs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.11.1.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.11.1'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'

Locally applied patches:
    


@INC for perl 5.16.1:
    /home/staff/dfj/perl5/perlbrew/perls/perl-5.16.1/lib/site_perl/5.16.1/i686-linux-thread-multi
    /home/staff/dfj/perl5/perlbrew/perls/perl-5.16.1/lib/site_perl/5.16.1
    /home/staff/dfj/perl5/perlbrew/perls/perl-5.16.1/lib/5.16.1/i686-linux-thread-multi
    /home/staff/dfj/perl5/perlbrew/perls/perl-5.16.1/lib/5.16.1
    .


Environment for perl 5.16.1:
    HOME=/home/staff/dfj
    LANG=C
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/staff/dfj/perl5/perlbrew/bin:/home/staff/dfj/perl5/perlbrew/perls/perl-5.16.1/bin:/home/staff/dfj/bin/:/home/staff/dfj/bin/:/home/staff/dfj/bin/:/home/staff/dfj/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
    PERLBREW_BASHRC_VERSION=0.49
    PERLBREW_HOME=/home/staff/dfj/.perlbrew
    PERLBREW_MANPATH=/home/staff/dfj/perl5/perlbrew/perls/perl-5.16.1/man
    PERLBREW_PATH=/home/staff/dfj/perl5/perlbrew/bin:/home/staff/dfj/perl5/perlbrew/perls/perl-5.16.1/bin
    PERLBREW_PERL=perl-5.16.1
    PERLBREW_ROOT=/home/staff/dfj/perl5/perlbrew
    PERLBREW_VERSION=0.49
    PERL_BADLANG (unset)
    PERL_CPANM_OPT=--mirror=http://mirror.bytemark.co.uk/cpan/
    SHELL=/bin/bash

@p5pRT
Copy link
Author

p5pRT commented Oct 9, 2012

From @cpansprout

On Tue Oct 09 07​:47​:52 2012, durandle@​gmail.com wrote​:

This is a bug report for perl from durandle@​gmail.com,
generated with the help of perlbug 1.39 running under perl 5.16.1.

-----------------------------------------------------------------
Under perl 5.12 (and possibly other versions)​:

my $foo;
utf8​::decode($foo);

The result would be that $foo is still undefined.

We upgraded to 5.16 recently, the result of the above code is that
$foo changes from undef to
blank but defined. I can't find anything in the perl changelog to
indicate this is a designed
change, but then I could just be missing it.

It did cause issues for us as our form validation checks undef vs
blank, although it could be
said that we should be more careful and not decode an undef value!

This was an intentional change. See also
<https://rt-archive.perl.org/perl5/Ticket/Display.html?id=91852>.

I think it was missed when the changelog was made because I accidentally
squashed two unrelated commits into one​:
http​://perl5.git.perl.org/perl.git/commitdiff/c7102404

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Oct 9, 2012

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Oct 9, 2012

@cpansprout - Status changed from 'open' to 'rejected'

@p5pRT p5pRT closed this as completed Oct 9, 2012
@p5pRT
Copy link
Author

p5pRT commented Oct 9, 2012

From @Leont

On Tue, Oct 9, 2012 at 5​:35 PM, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

This was an intentional change. See also
<https://rt-archive.perl.org/perl5/Ticket/Display.html?id=91852>.

It would make sense to me if undef decoded to undef.

Leon

@p5pRT
Copy link
Author

p5pRT commented Oct 9, 2012

From @cpansprout

On Tue Oct 09 08​:45​:00 2012, LeonT wrote​:

On Tue, Oct 9, 2012 at 5​:35 PM, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

This was an intentional change. See also
<https://rt-archive.perl.org/perl5/Ticket/Display.html?id=91852>.

It would make sense to me if undef decoded to undef.

Encode and decode should be consistent.

$ perl5.14.0 -le '$_ = undef; utf8​::encode $_; print $_//"undef"'

$ perl5.14.0 -le '$_ = undef; utf8​::decode $_; print $_//"undef"'
undef

If we make decode leave the argument undefined, then encode should be
changed likewise.

Is it worth yet another incompatible change?

--

Father Chrysostomos

@p5pRT
Copy link
Author

p5pRT commented Oct 9, 2012

From @ikegami

On Tue Oct 09 08​:45​:00 2012, LeonT wrote​:

On Tue, Oct 9, 2012 at 5​:35 PM, Father Chrysostomos via RT
<perlbug-followup@​perl.org> wrote​:

This was an intentional change. See also
<https://rt-archive.perl.org/perl5/Ticket/Display.html?id=91852>.

It would make sense to me if undef decoded to undef.

Note that utf8​::decode does warn on undef.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
distro-Linux type-utilities
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@p5pRT