Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

t/run/locale.t: Test failures on OpenBSD-6.4 unthreaded builds #16745

Closed
p5pRT opened this issue Nov 11, 2018 · 31 comments
Closed

t/run/locale.t: Test failures on OpenBSD-6.4 unthreaded builds #16745

p5pRT opened this issue Nov 11, 2018 · 31 comments

Comments

@p5pRT
Copy link

p5pRT commented Nov 11, 2018

Migrated from rt.perl.org#133649 (status was 'resolved')

Searchable as RT133649$

@p5pRT
Copy link
Author

p5pRT commented Nov 11, 2018

From @jkeenan

Here is a really weird one​: On OpenBSD-6.4, the last two unit tests in
t/run/locale.t fail -- but only on *unthreaded* builds.

I first noticed this in smoke-test reports submitted by Carlos Guevara.
See, e.g., http​://perl5.test-smoke.org/report/73721.

#####
v5.29.4-36-gb4f5c5b5f6
---------------------- ----------------------------------------------------
F F -Dcc=clang++ (*1)
F F -Dcc=clang++ -Duse64bitall (*1)
O O -Dcc=clang++ -Duseithreads (*1)
O O -Dcc=clang++ -Duseithreads -Duse64bitall (*1)
| |
| +------------------ stdio DEBUGGING
+--------------------- stdio

...

Test failures​:
~~ ../t/run/locale.t ........................................... FAILED 8-9
  [stdio] -Dcc=clang++ -Duse64bitall DEBUGGING
  [stdio] -Dcc=clang++ -Duse64bitall
  [stdio] -Dcc=clang++
  [stdio] -Dcc=clang++ DEBUGGING
#####

Note that t/run/locale.t only failed on unthreaded builds. We don't
have any indication that this was happening on OpenBSD-6.3; see, e.g.,
http​://perl5.test-smoke.org/report/72946.

Over the last week I modified t/run/locale.t to properly account for
tests in SKIP blocks. As a consequence, what were tests 8 and 9 in the
report above will now be reported as tests 38 and 39.

Also over the last week I have been able to set up an OpenBSD-6.4 VM
using one of the "generic/openbsd6" boxes from Hashicorp's
vagrantup.com. (I think that may be the same setup which Carlos is
using for OpenBSD-6.4.) When testing both in a git checkout of blead
and in smoke-tests, I can reproduce this pattern of results. Here is a
threaded build of blead​:

#####
$ sh ./Configure -des -Dusedevel -Duseithreads

$ ./perl -Ilib -V
Summary of my perl5 (revision 5 version 29 subversion 5) configuration​:
  Commit id​: 46814e9
  Platform​:
  osname=openbsd
  osvers=6.4
  archname=OpenBSD.amd64-openbsd-thread-multi
  uname='openbsd perl-reporter-04 6.4 generic.mp#364 amd64 '
  config_args='-des -Dusedevel -Duseithreads'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=define
  usemultiplicity=define
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
...

$ cd t; ./perl harness -v run/locale.t; cd -
...
ok 37 # skip no locale available where LC_NUMERIC radix isn't '.'
ok 38 - In setting complicated invalid LC_ALL, final individ category
doesn't need a ';'
ok 39 - In setting complicated valid LC_ALL, final individ category
doesn't need a ';'
1..39
#####

Next, an *unthreaded* build of blead​:

#####
$ # sh ./Configure -des -Dusedevel

[perl -V output attached]

$ cd t; ./perl harness -v run/locale.t; cd -
ok 1 - /il matching of [bracketed] doesn't skip POSIX class if fails
individ char
...
ok 37 # skip no locale available where LC_NUMERIC radix isn't '.'
not ok 38 - In setting complicated invalid LC_ALL, final individ
category doesn't need a ';'
# Failed test 38 - In setting complicated invalid LC_ALL, final individ
category doesn't need a ';' at t/run/locale.t line 487
# got "65280"
# expected "0"
not ok 39 - In setting complicated valid LC_ALL, final individ category
doesn't need a ';'
# Failed test 39 - In setting complicated valid LC_ALL, final individ
category doesn't need a ';' at t/run/locale.t line 496
# got "65280"
# expected "0"
1..39
#####

Finally, this smoke-test report shows this pattern of failures​:
http​://perl5.test-smoke.org/report/73858.

#####
v5.29.4-38-g46814e9b31
---------------------- ----------------------------------------------------
F F F F (*1)
O F F O -Duseithreads (*1)
| | | |
| | | +------------ perlio DEBUGGING
| | +--------------- stdio DEBUGGING
| +------------------ perlio
+--------------------- stdio

Applied patches​:
  SMOKE46814e9b310dbc8413ddba25028023aa4b8e6236

Skipped tests​:
  # One test name on a line

Test failures​:
~~ ../cpan/Sys-Syslog/t/syslog.t ............................... FAILED
145 Non-zero exit status​: 4
  [perlio] -Duseithreads
~~ ../cpan/Sys-Syslog/t/syslog.t ............................... FAILED
145-147 Non-zero exit status​: 5
  [stdio] -Duseithreads DEBUGGING
~~ ../cpan/Sys-Syslog/t/syslog.t ............................... FAILED
145 Non-zero exit status​: 5
  [perlio] DEBUGGING
~~ ../t/run/locale.t ........................................... FAILED
38-39
  [stdio/perlio]
  [stdio/perlio] DEBUGGING
#####

(Ignore the failures in cpan/Sys-Syslog/t/syslog.t. We're seeing them
in quite a few places. Those are limitations of the test file for which
I have filed https://rt.cpan.org/Ticket/Display.html?id=127454.)

This perhaps can be seen more clearly like this​:

#####
$ grep -n -E '(Configuration​:|locale.*FAILED)' smokecurrent.log
7430​:[2018-11-10 12​:35​:39-0500] Configuration​: -Dusedevel -DDEBUGGING
10302​:[2018-11-10 12​:56​:26-0500]
../t/run/locale.t...........................................FAILED
10343​:[2018-11-10 13​:10​:43-0500]
../t/run/locale.t...........................................FAILED
10352​:[2018-11-10 13​:10​:43-0500] Configuration​: -Dusedevel
13226​:[2018-11-10 13​:29​:19-0500]
../t/run/locale.t...........................................FAILED
13245​:[2018-11-10 13​:42​:10-0500]
../t/run/locale.t...........................................FAILED
13250​:[2018-11-10 13​:42​:10-0500] Configuration​: -Dusedevel -Duseithreads
-DDEBUGGING
16203​:[2018-11-10 14​:26​:20-0500] Configuration​: -Dusedevel -Duseithreads
#####

Any ideas?

Thank you very much.
Jim Keenan

@p5pRT
Copy link
Author

p5pRT commented Nov 11, 2018

From @jkeenan

Summary of my perl5 (revision 5 version 29 subversion 5) configuration​:
  Commit id​: 46814e9
  Platform​:
  osname=openbsd
  osvers=6.4
  archname=OpenBSD.amd64-openbsd
  uname='openbsd perl-reporter-04 6.4 generic.mp#364 amd64 '
  config_args='-des -Dusedevel'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=undef
  usemultiplicity=undef
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_FORTIFY_SOURCE=2'
  optimize='-O2'
  cppflags='-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='4.2.1 Compatible OpenBSD Clang 6.0.0 (tags/RELEASE_600/final)'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags ='-Wl,-E -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/lib /usr/local/lib
  libs=-lpthread -lm -lutil -lc
  perllibs=-lpthread -lm -lutil -lc
  libc=/usr/lib/libc.so.92.5
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags=' '
  cccdlflags='-DPIC -fPIC '
  lddlflags='-shared -fPIC -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​:
  HAS_TIMES
  PERLIO_LAYERS
  PERL_COPY_ON_WRITE
  PERL_DONT_CREATE_GVSV
  PERL_MALLOC_WRAP
  PERL_OP_PARENT
  PERL_PRESERVE_IVUV
  PERL_USE_DEVEL
  USE_64_BIT_ALL
  USE_64_BIT_INT
  USE_LARGE_FILES
  USE_LOCALE
  USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC
  USE_LOCALE_TIME
  USE_PERLIO
  USE_PERL_ATOF
  Built under openbsd
  Compiled at Nov 10 2018 12​:06​:57
  %ENV​:
  PERL_WORKDIR="/home/jkeenan/gitwork/perl"
  @​INC​:
  lib
  /usr/local/lib/perl5/site_perl/5.29.5/OpenBSD.amd64-openbsd
  /usr/local/lib/perl5/site_perl/5.29.5
  /usr/local/lib/perl5/5.29.5/OpenBSD.amd64-openbsd
  /usr/local/lib/perl5/5.29.5

@p5pRT
Copy link
Author

p5pRT commented Nov 11, 2018

From @sisyphus

On Sat, 10 Nov 2018 17​:02​:31 -0800, jkeenan@​pobox.com wrote​:

Here is a really weird one​: On OpenBSD-6.4, the last two unit tests in
t/run/locale.t fail -- but only on *unthreaded* builds.

This issue also cropped up a week ago at​:
https://www.perlmonks.org/?node_id=1225215

Cheers,
Rob

@p5pRT
Copy link
Author

p5pRT commented Nov 11, 2018

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Nov 11, 2018

From @khwilliamson

On 11/10/18 6​:02 PM, James E Keenan wrote​:

Here is a really weird one​:  On OpenBSD-6.4, the last two unit tests in
t/run/locale.t fail -- but only on *unthreaded* builds.

See my response on monks

I first noticed this in smoke-test reports submitted by Carlos Guevara.
See, e.g., http​://perl5.test-smoke.org/report/73721.

#####
v5.29.4-36-gb4f5c5b5f6
---------------------- ----------------------------------------------------
F  F                    -Dcc=clang++ (*1)
F  F                    -Dcc=clang++ -Duse64bitall (*1)
O  O                    -Dcc=clang++ -Duseithreads (*1)
O  O                    -Dcc=clang++ -Duseithreads -Duse64bitall (*1)
|  |
|  +------------------  stdio DEBUGGING
+---------------------  stdio

...

Test failures​:
~~ ../t/run/locale.t ........................................... FAILED 8-9
  [stdio] -Dcc=clang++ -Duse64bitall DEBUGGING
  [stdio] -Dcc=clang++ -Duse64bitall
  [stdio] -Dcc=clang++
  [stdio] -Dcc=clang++ DEBUGGING
#####

Note that t/run/locale.t only failed on unthreaded builds.  We don't
have any indication that this was happening on OpenBSD-6.3; see, e.g.,
http​://perl5.test-smoke.org/report/72946.

Over the last week I modified t/run/locale.t to properly account for
tests in SKIP blocks.  As a consequence, what were tests 8 and 9 in the
report above will now be reported as tests 38 and 39.

Also over the last week I have been able to set up an OpenBSD-6.4 VM
using one of the "generic/openbsd6" boxes from Hashicorp's
vagrantup.com.  (I think that may be the same setup which Carlos is
using for OpenBSD-6.4.)  When testing both in a git checkout of blead
and in smoke-tests, I can reproduce this pattern of results.  Here is a
threaded build of blead​:

#####
$ sh ./Configure -des -Dusedevel -Duseithreads

$ ./perl -Ilib -V
Summary of my perl5 (revision 5 version 29 subversion 5) configuration​:
  Commit id​: 46814e9
  Platform​:
    osname=openbsd
    osvers=6.4
    archname=OpenBSD.amd64-openbsd-thread-multi
    uname='openbsd perl-reporter-04 6.4 generic.mp#364 amd64 '
    config_args='-des -Dusedevel -Duseithreads'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
...

$ cd t; ./perl harness -v run/locale.t; cd -
...
ok 37 # skip no locale available where LC_NUMERIC radix isn't '.'
ok 38 - In setting complicated invalid LC_ALL, final individ category
doesn't need a ';'
ok 39 - In setting complicated valid LC_ALL, final individ category
doesn't need a ';'
1..39
#####

Next, an *unthreaded* build of blead​:

#####
$ # sh ./Configure -des -Dusedevel

[perl -V output attached]

$ cd t; ./perl harness -v run/locale.t; cd -
ok 1 - /il matching of [bracketed] doesn't skip POSIX class if fails
individ char
...
ok 37 # skip no locale available where LC_NUMERIC radix isn't '.'
not ok 38 - In setting complicated invalid LC_ALL, final individ
category doesn't need a ';'
# Failed test 38 - In setting complicated invalid LC_ALL, final individ
category doesn't need a ';' at t/run/locale.t line 487
#      got "65280"
# expected "0"
not ok 39 - In setting complicated valid LC_ALL, final individ category
doesn't need a ';'
# Failed test 39 - In setting complicated valid LC_ALL, final individ
category doesn't need a ';' at t/run/locale.t line 496
#      got "65280"
# expected "0"
1..39
#####

Finally, this smoke-test report shows this pattern of failures​:
http​://perl5.test-smoke.org/report/73858.

#####
v5.29.4-38-g46814e9b31
---------------------- ----------------------------------------------------
F  F  F  F               (*1)
O  F  F  O              -Duseithreads (*1)
|  |  |  |
|  |  |  +------------  perlio DEBUGGING
|  |  +---------------  stdio DEBUGGING
|  +------------------  perlio
+---------------------  stdio

Applied patches​:
  SMOKE46814e9b310dbc8413ddba25028023aa4b8e6236

Skipped tests​:
      # One test name on a line

Test failures​:
~~ ../cpan/Sys-Syslog/t/syslog.t ............................... FAILED
145 Non-zero exit status​: 4
  [perlio] -Duseithreads
~~ ../cpan/Sys-Syslog/t/syslog.t ............................... FAILED
145-147 Non-zero exit status​: 5
  [stdio] -Duseithreads DEBUGGING
~~ ../cpan/Sys-Syslog/t/syslog.t ............................... FAILED
145 Non-zero exit status​: 5
  [perlio]  DEBUGGING
~~ ../t/run/locale.t ........................................... FAILED
38-39
  [stdio/perlio]
  [stdio/perlio]  DEBUGGING
#####

(Ignore the failures in cpan/Sys-Syslog/t/syslog.t.  We're seeing them
in quite a few places.  Those are limitations of the test file for which
I have filed https://rt.cpan.org/Ticket/Display.html?id=127454.)

This perhaps can be seen more clearly like this​:

#####
$ grep -n -E '(Configuration​:|locale.*FAILED)' smokecurrent.log
7430​:[2018-11-10 12​:35​:39-0500] Configuration​: -Dusedevel -DDEBUGGING
10302​:[2018-11-10 12​:56​:26-0500]
../t/run/locale.t...........................................FAILED
10343​:[2018-11-10 13​:10​:43-0500]
../t/run/locale.t...........................................FAILED
10352​:[2018-11-10 13​:10​:43-0500] Configuration​: -Dusedevel
13226​:[2018-11-10 13​:29​:19-0500]
../t/run/locale.t...........................................FAILED
13245​:[2018-11-10 13​:42​:10-0500]
../t/run/locale.t...........................................FAILED
13250​:[2018-11-10 13​:42​:10-0500] Configuration​: -Dusedevel -Duseithreads
-DDEBUGGING
16203​:[2018-11-10 14​:26​:20-0500] Configuration​: -Dusedevel -Duseithreads
#####

Any ideas?

Thank you very much.
Jim Keenan

@p5pRT
Copy link
Author

p5pRT commented Nov 11, 2018

From @khwilliamson

On 11/10/18 6​:02 PM, James E Keenan (via RT) wrote​:

# New Ticket Created by James E Keenan
# Please include the string​: [perl #133649]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133649 >

Here is a really weird one​: On OpenBSD-6.4, the last two unit tests in
t/run/locale.t fail -- but only on *unthreaded* builds.

I first noticed this in smoke-test reports submitted by Carlos Guevara.
See, e.g., http​://perl5.test-smoke.org/report/73721.

#####
v5.29.4-36-gb4f5c5b5f6
---------------------- ----------------------------------------------------
F F -Dcc=clang++ (*1)
F F -Dcc=clang++ -Duse64bitall (*1)
O O -Dcc=clang++ -Duseithreads (*1)
O O -Dcc=clang++ -Duseithreads -Duse64bitall (*1)
| |
| +------------------ stdio DEBUGGING
+--------------------- stdio

...

Test failures​:
~~ ../t/run/locale.t ........................................... FAILED 8-9
[stdio] -Dcc=clang++ -Duse64bitall DEBUGGING
[stdio] -Dcc=clang++ -Duse64bitall
[stdio] -Dcc=clang++
[stdio] -Dcc=clang++ DEBUGGING
#####

Note that t/run/locale.t only failed on unthreaded builds. We don't
have any indication that this was happening on OpenBSD-6.3; see, e.g.,
http​://perl5.test-smoke.org/report/72946.

Over the last week I modified t/run/locale.t to properly account for
tests in SKIP blocks. As a consequence, what were tests 8 and 9 in the
report above will now be reported as tests 38 and 39.

Also over the last week I have been able to set up an OpenBSD-6.4 VM
using one of the "generic/openbsd6" boxes from Hashicorp's
vagrantup.com. (I think that may be the same setup which Carlos is
using for OpenBSD-6.4.) When testing both in a git checkout of blead
and in smoke-tests, I can reproduce this pattern of results. Here is a
threaded build of blead​:

#####
$ sh ./Configure -des -Dusedevel -Duseithreads

$ ./perl -Ilib -V
Summary of my perl5 (revision 5 version 29 subversion 5) configuration​:
Commit id​: 46814e9
Platform​:
osname=openbsd
osvers=6.4
archname=OpenBSD.amd64-openbsd-thread-multi
uname='openbsd perl-reporter-04 6.4 generic.mp#364 amd64 '
config_args='-des -Dusedevel -Duseithreads'
hint=recommended
useposix=true
d_sigaction=define
useithreads=define
usemultiplicity=define
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
...

$ cd t; ./perl harness -v run/locale.t; cd -
...
ok 37 # skip no locale available where LC_NUMERIC radix isn't '.'
ok 38 - In setting complicated invalid LC_ALL, final individ category
doesn't need a ';'
ok 39 - In setting complicated valid LC_ALL, final individ category
doesn't need a ';'
1..39
#####

Next, an *unthreaded* build of blead​:

#####
$ # sh ./Configure -des -Dusedevel

[perl -V output attached]

$ cd t; ./perl harness -v run/locale.t; cd -
ok 1 - /il matching of [bracketed] doesn't skip POSIX class if fails
individ char
...
ok 37 # skip no locale available where LC_NUMERIC radix isn't '.'
not ok 38 - In setting complicated invalid LC_ALL, final individ
category doesn't need a ';'
# Failed test 38 - In setting complicated invalid LC_ALL, final individ
category doesn't need a ';' at t/run/locale.t line 487
# got "65280"
# expected "0"
not ok 39 - In setting complicated valid LC_ALL, final individ category
doesn't need a ';'
# Failed test 39 - In setting complicated valid LC_ALL, final individ
category doesn't need a ';' at t/run/locale.t line 496
# got "65280"
# expected "0"
1..39
#####

Finally, this smoke-test report shows this pattern of failures​:
http​://perl5.test-smoke.org/report/73858.

#####
v5.29.4-38-g46814e9b31
---------------------- ----------------------------------------------------
F F F F (*1)
O F F O -Duseithreads (*1)
| | | |
| | | +------------ perlio DEBUGGING
| | +--------------- stdio DEBUGGING
| +------------------ perlio
+--------------------- stdio

Applied patches​:
SMOKE46814e9b310dbc8413ddba25028023aa4b8e6236

Skipped tests​:
# One test name on a line

Test failures​:
~~ ../cpan/Sys-Syslog/t/syslog.t ............................... FAILED
145 Non-zero exit status​: 4
[perlio] -Duseithreads
~~ ../cpan/Sys-Syslog/t/syslog.t ............................... FAILED
145-147 Non-zero exit status​: 5
[stdio] -Duseithreads DEBUGGING
~~ ../cpan/Sys-Syslog/t/syslog.t ............................... FAILED
145 Non-zero exit status​: 5
[perlio] DEBUGGING
~~ ../t/run/locale.t ........................................... FAILED
38-39
[stdio/perlio]
[stdio/perlio] DEBUGGING
#####

(Ignore the failures in cpan/Sys-Syslog/t/syslog.t. We're seeing them
in quite a few places. Those are limitations of the test file for which
I have filed https://rt.cpan.org/Ticket/Display.html?id=127454.)

This perhaps can be seen more clearly like this​:

#####
$ grep -n -E '(Configuration​:|locale.*FAILED)' smokecurrent.log
7430​:[2018-11-10 12​:35​:39-0500] Configuration​: -Dusedevel -DDEBUGGING
10302​:[2018-11-10 12​:56​:26-0500]
../t/run/locale.t...........................................FAILED
10343​:[2018-11-10 13​:10​:43-0500]
../t/run/locale.t...........................................FAILED
10352​:[2018-11-10 13​:10​:43-0500] Configuration​: -Dusedevel
13226​:[2018-11-10 13​:29​:19-0500]
../t/run/locale.t...........................................FAILED
13245​:[2018-11-10 13​:42​:10-0500]
../t/run/locale.t...........................................FAILED
13250​:[2018-11-10 13​:42​:10-0500] Configuration​: -Dusedevel -Duseithreads
-DDEBUGGING
16203​:[2018-11-10 14​:26​:20-0500] Configuration​: -Dusedevel -Duseithreads
#####

Any ideas?

Thank you very much.
Jim Keenan

Run the attached script on a DEBUGGING build with runtime switch -DLv
and post the output

@p5pRT
Copy link
Author

p5pRT commented Nov 11, 2018

From @khwilliamson

133649.pl

@p5pRT
Copy link
Author

p5pRT commented Nov 11, 2018

From @jkeenan

On Sun, 11 Nov 2018 18​:00​:27 GMT, public@​khwilliamson.com wrote​:

On 11/10/18 6​:02 PM, James E Keenan (via RT) wrote​:

# New Ticket Created by James E Keenan
# Please include the string​: [perl #133649]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133649 >

Here is a really weird one​: On OpenBSD-6.4, the last two unit tests
in
t/run/locale.t fail -- but only on *unthreaded* builds.

I first noticed this in smoke-test reports submitted by Carlos
Guevara.
See, e.g., http​://perl5.test-smoke.org/report/73721.

#####
v5.29.4-36-gb4f5c5b5f6
----------------------
----------------------------------------------------
F F -Dcc=clang++ (*1)
F F -Dcc=clang++ -Duse64bitall (*1)
O O -Dcc=clang++ -Duseithreads (*1)
O O -Dcc=clang++ -Duseithreads -Duse64bitall (*1)
| |
| +------------------ stdio DEBUGGING
+--------------------- stdio

...

Test failures​:
~~ ../t/run/locale.t ...........................................
FAILED 8-9
[stdio] -Dcc=clang++ -Duse64bitall DEBUGGING
[stdio] -Dcc=clang++ -Duse64bitall
[stdio] -Dcc=clang++
[stdio] -Dcc=clang++ DEBUGGING
#####

Note that t/run/locale.t only failed on unthreaded builds. We don't
have any indication that this was happening on OpenBSD-6.3; see,
e.g.,
http​://perl5.test-smoke.org/report/72946.

Over the last week I modified t/run/locale.t to properly account for
tests in SKIP blocks. As a consequence, what were tests 8 and 9 in
the
report above will now be reported as tests 38 and 39.

Also over the last week I have been able to set up an OpenBSD-6.4 VM
using one of the "generic/openbsd6" boxes from Hashicorp's
vagrantup.com. (I think that may be the same setup which Carlos is
using for OpenBSD-6.4.) When testing both in a git checkout of blead
and in smoke-tests, I can reproduce this pattern of results. Here is
a
threaded build of blead​:

#####
$ sh ./Configure -des -Dusedevel -Duseithreads

$ ./perl -Ilib -V
Summary of my perl5 (revision 5 version 29 subversion 5)
configuration​:
Commit id​: 46814e9
Platform​:
osname=openbsd
osvers=6.4
archname=OpenBSD.amd64-openbsd-thread-multi
uname='openbsd perl-reporter-04 6.4 generic.mp#364 amd64 '
config_args='-des -Dusedevel -Duseithreads'
hint=recommended
useposix=true
d_sigaction=define
useithreads=define
usemultiplicity=define
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
...

$ cd t; ./perl harness -v run/locale.t; cd -
...
ok 37 # skip no locale available where LC_NUMERIC radix isn't '.'
ok 38 - In setting complicated invalid LC_ALL, final individ category
doesn't need a ';'
ok 39 - In setting complicated valid LC_ALL, final individ category
doesn't need a ';'
1..39
#####

Next, an *unthreaded* build of blead​:

#####
$ # sh ./Configure -des -Dusedevel

[perl -V output attached]

$ cd t; ./perl harness -v run/locale.t; cd -
ok 1 - /il matching of [bracketed] doesn't skip POSIX class if fails
individ char
...
ok 37 # skip no locale available where LC_NUMERIC radix isn't '.'
not ok 38 - In setting complicated invalid LC_ALL, final individ
category doesn't need a ';'
# Failed test 38 - In setting complicated invalid LC_ALL, final
individ
category doesn't need a ';' at t/run/locale.t line 487
# got "65280"
# expected "0"
not ok 39 - In setting complicated valid LC_ALL, final individ
category
doesn't need a ';'
# Failed test 39 - In setting complicated valid LC_ALL, final individ
category doesn't need a ';' at t/run/locale.t line 496
# got "65280"
# expected "0"
1..39
#####

Finally, this smoke-test report shows this pattern of failures​:
http​://perl5.test-smoke.org/report/73858.

#####
v5.29.4-38-g46814e9b31
----------------------
----------------------------------------------------
F F F F (*1)
O F F O -Duseithreads (*1)
| | | |
| | | +------------ perlio DEBUGGING
| | +--------------- stdio DEBUGGING
| +------------------ perlio
+--------------------- stdio

Applied patches​:
SMOKE46814e9b310dbc8413ddba25028023aa4b8e6236

Skipped tests​:
# One test name on a line

Test failures​:
~~ ../cpan/Sys-Syslog/t/syslog.t ...............................
FAILED
145 Non-zero exit status​: 4
[perlio] -Duseithreads
~~ ../cpan/Sys-Syslog/t/syslog.t ...............................
FAILED
145-147 Non-zero exit status​: 5
[stdio] -Duseithreads DEBUGGING
~~ ../cpan/Sys-Syslog/t/syslog.t ...............................
FAILED
145 Non-zero exit status​: 5
[perlio] DEBUGGING
~~ ../t/run/locale.t ...........................................
FAILED
38-39
[stdio/perlio]
[stdio/perlio] DEBUGGING
#####

(Ignore the failures in cpan/Sys-Syslog/t/syslog.t. We're seeing
them
in quite a few places. Those are limitations of the test file for
which
I have filed https://rt.cpan.org/Ticket/Display.html?id=127454.)

This perhaps can be seen more clearly like this​:

#####
$ grep -n -E '(Configuration​:|locale.*FAILED)' smokecurrent.log
7430​:[2018-11-10 12​:35​:39-0500] Configuration​: -Dusedevel -DDEBUGGING
10302​:[2018-11-10 12​:56​:26-0500]
../t/run/locale.t...........................................FAILED
10343​:[2018-11-10 13​:10​:43-0500]
../t/run/locale.t...........................................FAILED
10352​:[2018-11-10 13​:10​:43-0500] Configuration​: -Dusedevel
13226​:[2018-11-10 13​:29​:19-0500]
../t/run/locale.t...........................................FAILED
13245​:[2018-11-10 13​:42​:10-0500]
../t/run/locale.t...........................................FAILED
13250​:[2018-11-10 13​:42​:10-0500] Configuration​: -Dusedevel
-Duseithreads
-DDEBUGGING
16203​:[2018-11-10 14​:26​:20-0500] Configuration​: -Dusedevel
-Duseithreads
#####

Any ideas?

Thank you very much.
Jim Keenan

Run the attached script on a DEBUGGING build with runtime switch -DLv
and post the output

I had to add '-MPOSIX' to the command-line invocation. See attached.

Thank you very much.
--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Nov 11, 2018

From @jkeenan

[perl] 7 $ ./perl -Ilib -V​:config_args
config_args='-des -Dusedevel -DDEBUGGING';

[perl] 8 $ ./perl -Ilib -DLv /home/jkeenan/learn/perl/133649.pl

EXECUTING...

Undefined subroutine &POSIX​::setlocale called at /home/jkeenan/learn/perl/133649.pl line 3.

[perl] 9 $ ./perl -Ilib -MPOSIX -DLv /home/jkeenan/learn/perl/133649.pl
./vutil.c​: 691​: lock lc_numeric_standard​: new depth=2
./vutil.c​: 707​: lc_numeric_standard decrement lock, new depth=1
./vutil.c​: 691​: lock lc_numeric_standard​: new depth=2
./vutil.c​: 707​: lc_numeric_standard decrement lock, new depth=1
./vutil.c​: 691​: lock lc_numeric_standard​: new depth=2
./vutil.c​: 707​: lc_numeric_standard decrement lock, new depth=1
./vutil.c​: 691​: lock lc_numeric_standard​: new depth=2
./vutil.c​: 707​: lc_numeric_standard decrement lock, new depth=1
./vutil.c​: 691​: lock lc_numeric_standard​: new depth=2
./vutil.c​: 707​: lc_numeric_standard decrement lock, new depth=1
./vutil.c​: 691​: lock lc_numeric_standard​: new depth=2
./vutil.c​: 707​: lc_numeric_standard decrement lock, new depth=1
./vutil.c​: 691​: lock lc_numeric_standard​: new depth=2
./vutil.c​: 707​: lc_numeric_standard decrement lock, new depth=1
./vutil.c​: 691​: lock lc_numeric_standard​: new depth=2
./vutil.c​: 707​: lc_numeric_standard decrement lock, new depth=1
./vutil.c​: 691​: lock lc_numeric_standard​: new depth=2
./vutil.c​: 707​: lc_numeric_standard decrement lock, new depth=1
./vutil.c​: 691​: lock lc_numeric_standard​: new depth=2
./vutil.c​: 707​: lc_numeric_standard decrement lock, new depth=1

EXECUTING...

locale.c​:2203​: setlocale(LC_ALL, "LC_COLLATE=foo_BAR;LC_CTYPE=foo_BAR;LC_MESSAGES=foo_BAR;LC_MONETARY=foo_BAR;LC_NUMERIC=foo_BAR") returned "LC_COLLATE=foo_BAR;LC_CTYPE=foo_BAR;LC_MESSAGES=foo_BAR;LC_MONETARY=foo_BAR;LC_NUMERIC=foo_BAR"
Can't fix broken locale name "LC_COLLATE=foo_BAR;LC_CTYPE=foo_BAR;LC_MESSAGES=foo_BAR;LC_MONETARY=foo_BAR;LC_NUMERIC=foo_BAR" at /home/jkeenan/learn/perl/133649.pl line 3.

@p5pRT
Copy link
Author

p5pRT commented Nov 11, 2018

From @jkeenan

On Sun, 11 Nov 2018 18​:51​:32 GMT, jkeenan wrote​:

On Sun, 11 Nov 2018 18​:00​:27 GMT, public@​khwilliamson.com wrote​:

On 11/10/18 6​:02 PM, James E Keenan (via RT) wrote​:

# New Ticket Created by James E Keenan
# Please include the string​: [perl #133649]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133649 >

Here is a really weird one​: On OpenBSD-6.4, the last two unit tests
in
t/run/locale.t fail -- but only on *unthreaded* builds.

I first noticed this in smoke-test reports submitted by Carlos
Guevara.
See, e.g., http​://perl5.test-smoke.org/report/73721.

#####
v5.29.4-36-gb4f5c5b5f6
----------------------
----------------------------------------------------
F F -Dcc=clang++ (*1)
F F -Dcc=clang++ -Duse64bitall (*1)
O O -Dcc=clang++ -Duseithreads (*1)
O O -Dcc=clang++ -Duseithreads -Duse64bitall (*1)
| |
| +------------------ stdio DEBUGGING
+--------------------- stdio

...

Test failures​:
~~ ../t/run/locale.t ...........................................
FAILED 8-9
[stdio] -Dcc=clang++ -Duse64bitall DEBUGGING
[stdio] -Dcc=clang++ -Duse64bitall
[stdio] -Dcc=clang++
[stdio] -Dcc=clang++ DEBUGGING
#####

Note that t/run/locale.t only failed on unthreaded builds. We don't
have any indication that this was happening on OpenBSD-6.3; see,
e.g.,
http​://perl5.test-smoke.org/report/72946.

Over the last week I modified t/run/locale.t to properly account for
tests in SKIP blocks. As a consequence, what were tests 8 and 9 in
the
report above will now be reported as tests 38 and 39.

Also over the last week I have been able to set up an OpenBSD-6.4 VM
using one of the "generic/openbsd6" boxes from Hashicorp's
vagrantup.com. (I think that may be the same setup which Carlos is
using for OpenBSD-6.4.) When testing both in a git checkout of blead
and in smoke-tests, I can reproduce this pattern of results. Here is
a
threaded build of blead​:

#####
$ sh ./Configure -des -Dusedevel -Duseithreads

$ ./perl -Ilib -V
Summary of my perl5 (revision 5 version 29 subversion 5)
configuration​:
Commit id​: 46814e9
Platform​:
osname=openbsd
osvers=6.4
archname=OpenBSD.amd64-openbsd-thread-multi
uname='openbsd perl-reporter-04 6.4 generic.mp#364 amd64 '
config_args='-des -Dusedevel -Duseithreads'
hint=recommended
useposix=true
d_sigaction=define
useithreads=define
usemultiplicity=define
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
...

$ cd t; ./perl harness -v run/locale.t; cd -
...
ok 37 # skip no locale available where LC_NUMERIC radix isn't '.'
ok 38 - In setting complicated invalid LC_ALL, final individ category
doesn't need a ';'
ok 39 - In setting complicated valid LC_ALL, final individ category
doesn't need a ';'
1..39
#####

Next, an *unthreaded* build of blead​:

#####
$ # sh ./Configure -des -Dusedevel

[perl -V output attached]

$ cd t; ./perl harness -v run/locale.t; cd -
ok 1 - /il matching of [bracketed] doesn't skip POSIX class if fails
individ char
...
ok 37 # skip no locale available where LC_NUMERIC radix isn't '.'
not ok 38 - In setting complicated invalid LC_ALL, final individ
category doesn't need a ';'
# Failed test 38 - In setting complicated invalid LC_ALL, final
individ
category doesn't need a ';' at t/run/locale.t line 487
# got "65280"
# expected "0"
not ok 39 - In setting complicated valid LC_ALL, final individ
category
doesn't need a ';'
# Failed test 39 - In setting complicated valid LC_ALL, final individ
category doesn't need a ';' at t/run/locale.t line 496
# got "65280"
# expected "0"
1..39
#####

Finally, this smoke-test report shows this pattern of failures​:
http​://perl5.test-smoke.org/report/73858.

#####
v5.29.4-38-g46814e9b31
----------------------
----------------------------------------------------
F F F F (*1)
O F F O -Duseithreads (*1)
| | | |
| | | +------------ perlio DEBUGGING
| | +--------------- stdio DEBUGGING
| +------------------ perlio
+--------------------- stdio

Applied patches​:
SMOKE46814e9b310dbc8413ddba25028023aa4b8e6236

Skipped tests​:
# One test name on a line

Test failures​:
~~ ../cpan/Sys-Syslog/t/syslog.t ...............................
FAILED
145 Non-zero exit status​: 4
[perlio] -Duseithreads
~~ ../cpan/Sys-Syslog/t/syslog.t ...............................
FAILED
145-147 Non-zero exit status​: 5
[stdio] -Duseithreads DEBUGGING
~~ ../cpan/Sys-Syslog/t/syslog.t ...............................
FAILED
145 Non-zero exit status​: 5
[perlio] DEBUGGING
~~ ../t/run/locale.t ...........................................
FAILED
38-39
[stdio/perlio]
[stdio/perlio] DEBUGGING
#####

(Ignore the failures in cpan/Sys-Syslog/t/syslog.t. We're seeing
them
in quite a few places. Those are limitations of the test file for
which
I have filed https://rt.cpan.org/Ticket/Display.html?id=127454.)

This perhaps can be seen more clearly like this​:

#####
$ grep -n -E '(Configuration​:|locale.*FAILED)' smokecurrent.log
7430​:[2018-11-10 12​:35​:39-0500] Configuration​: -Dusedevel -DDEBUGGING
10302​:[2018-11-10 12​:56​:26-0500]
../t/run/locale.t...........................................FAILED
10343​:[2018-11-10 13​:10​:43-0500]
../t/run/locale.t...........................................FAILED
10352​:[2018-11-10 13​:10​:43-0500] Configuration​: -Dusedevel
13226​:[2018-11-10 13​:29​:19-0500]
../t/run/locale.t...........................................FAILED
13245​:[2018-11-10 13​:42​:10-0500]
../t/run/locale.t...........................................FAILED
13250​:[2018-11-10 13​:42​:10-0500] Configuration​: -Dusedevel
-Duseithreads
-DDEBUGGING
16203​:[2018-11-10 14​:26​:20-0500] Configuration​: -Dusedevel
-Duseithreads
#####

Any ideas?

Thank you very much.
Jim Keenan

Run the attached script on a DEBUGGING build with runtime switch -DLv
and post the output

I had to add '-MPOSIX' to the command-line invocation. See attached.

Thank you very much.

Note​: I tried switching the 2 lines of code around. I also tried with 'C' instead of 'C.UTF-8'. I got the same "Can't fix broken locale name" error in all cases.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Nov 11, 2018

From @khwilliamson

On 11/11/18 11​:58 AM, James E Keenan via RT wrote​:

On Sun, 11 Nov 2018 18​:51​:32 GMT, jkeenan wrote​:

On Sun, 11 Nov 2018 18​:00​:27 GMT, public@​khwilliamson.com wrote​:

On 11/10/18 6​:02 PM, James E Keenan (via RT) wrote​:

# New Ticket Created by James E Keenan
# Please include the string​: [perl #133649]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133649 >

Here is a really weird one​: On OpenBSD-6.4, the last two unit tests
in
t/run/locale.t fail -- but only on *unthreaded* builds.

[snip]

Run the attached script on a DEBUGGING build with runtime switch -DLv
and post the output

I had to add '-MPOSIX' to the command-line invocation. See attached.

Thank you very much.

Note​: I tried switching the 2 lines of code around. I also tried with 'C' instead of 'C.UTF-8'. I got the same "Can't fix broken locale name" error in all cases.

Try compiling the attached C program, and run it with a single argument,
  a legal non-C locale, perhaps en_US.UTF-8

like

./a.out en_US.UTF-8

and post the results

@p5pRT
Copy link
Author

p5pRT commented Nov 11, 2018

From @khwilliamson

#include <stdio.h>
#include <locale.h>
#include <stdlib.h>

int main(int argc, char * argv[]) {
    char * result;

    fprintf(stdout, "setlocale(LC_ALL, \"C\") returns ");
    result = setlocale(LC_ALL, "C");
    if (result) {
        fprintf(stdout, "'%s'\n", result);
    }
    else {
        fprintf(stdout, "NULL\n");
    }

    fprintf(stdout, "setlocale(LC_CTYPE, \"%s\") returns ",  argv[1]);
    result = setlocale(LC_CTYPE, argv[1]);
    if (result) {
        fprintf(stdout, "'%s'\n", result);
    }
    else {
        fprintf(stdout, "NULL\n");
    }

    fprintf(stdout, "setlocale(LC_ALL, NULL) returns ");
    result = setlocale(LC_ALL, NULL);
    if (result) {
        fprintf(stdout, "'%s'\n", result);
    }
    else {
        fprintf(stdout, "NULL\n");
    }

    exit(0);
}

@p5pRT
Copy link
Author

p5pRT commented Nov 11, 2018

From @jkeenan

On Sun, 11 Nov 2018 22​:01​:56 GMT, public@​khwilliamson.com wrote​:

On 11/11/18 11​:58 AM, James E Keenan via RT wrote​:

On Sun, 11 Nov 2018 18​:51​:32 GMT, jkeenan wrote​:

On Sun, 11 Nov 2018 18​:00​:27 GMT, public@​khwilliamson.com wrote​:

On 11/10/18 6​:02 PM, James E Keenan (via RT) wrote​:

# New Ticket Created by James E Keenan
# Please include the string​: [perl #133649]
# in the subject line of all future correspondence about this
issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133649 >

Here is a really weird one​: On OpenBSD-6.4, the last two unit
tests
in
t/run/locale.t fail -- but only on *unthreaded* builds.

[snip]

Run the attached script on a DEBUGGING build with runtime switch
-DLv
and post the output

I had to add '-MPOSIX' to the command-line invocation. See
attached.

Thank you very much.

Note​: I tried switching the 2 lines of code around. I also tried
with 'C' instead of 'C.UTF-8'. I got the same "Can't fix broken
locale name" error in all cases.

Try compiling the attached C program, and run it with a single
argument,
a legal non-C locale, perhaps en_US.UTF-8

like

./a.out en_US.UTF-8

and post the results

Here's a sampler​:

#####
[perl] 17 $ ~/learn/perl/p5p/khw-testlocale-20181111 en_US.UTF
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "en_US.UTF") returns NULL
setlocale(LC_ALL, NULL) returns 'C'
[perl] 18 $ ~/learn/perl/p5p/khw-testlocale-20181111 iso8859-1
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "iso8859-1") returns 'iso8859-1'
setlocale(LC_ALL, NULL) returns 'C/iso8859-1/C/C/C/C'
[perl] 19 $ ~/learn/perl/p5p/khw-testlocale-20181111 C.UTF-8
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "C.UTF-8") returns 'C.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/C.UTF-8/C/C/C/C'
[perl] 20 $ ~/learn/perl/p5p/khw-testlocale-20181111 C
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "C") returns 'C'
setlocale(LC_ALL, NULL) returns 'C'
[perl] 21 $ ~/learn/perl/p5p/khw-testlocale-20181111 es_US.UTF
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_US.UTF") returns NULL
setlocale(LC_ALL, NULL) returns 'C'
#####

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Nov 11, 2018

From @jkeenan

On Sun, 11 Nov 2018 22​:59​:57 GMT, jkeenan wrote​:

On Sun, 11 Nov 2018 22​:01​:56 GMT, public@​khwilliamson.com wrote​:

On 11/11/18 11​:58 AM, James E Keenan via RT wrote​:

On Sun, 11 Nov 2018 18​:51​:32 GMT, jkeenan wrote​:

On Sun, 11 Nov 2018 18​:00​:27 GMT, public@​khwilliamson.com wrote​:

On 11/10/18 6​:02 PM, James E Keenan (via RT) wrote​:

# New Ticket Created by James E Keenan
# Please include the string​: [perl #133649]
# in the subject line of all future correspondence about this
issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133649 >

Here is a really weird one​: On OpenBSD-6.4, the last two unit
tests
in
t/run/locale.t fail -- but only on *unthreaded* builds.

[snip]

Run the attached script on a DEBUGGING build with runtime switch
-DLv
and post the output

I had to add '-MPOSIX' to the command-line invocation. See
attached.

Thank you very much.

Note​: I tried switching the 2 lines of code around. I also tried
with 'C' instead of 'C.UTF-8'. I got the same "Can't fix broken
locale name" error in all cases.

Try compiling the attached C program, and run it with a single
argument,
a legal non-C locale, perhaps en_US.UTF-8

like

./a.out en_US.UTF-8

and post the results

Here's a sampler​:

#####
[perl] 17 $ ~/learn/perl/p5p/khw-testlocale-20181111 en_US.UTF
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "en_US.UTF") returns NULL
setlocale(LC_ALL, NULL) returns 'C'
[perl] 18 $ ~/learn/perl/p5p/khw-testlocale-20181111 iso8859-1
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "iso8859-1") returns 'iso8859-1'
setlocale(LC_ALL, NULL) returns 'C/iso8859-1/C/C/C/C'
[perl] 19 $ ~/learn/perl/p5p/khw-testlocale-20181111 C.UTF-8
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "C.UTF-8") returns 'C.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/C.UTF-8/C/C/C/C'
[perl] 20 $ ~/learn/perl/p5p/khw-testlocale-20181111 C
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "C") returns 'C'
setlocale(LC_ALL, NULL) returns 'C'
[perl] 21 $ ~/learn/perl/p5p/khw-testlocale-20181111 es_US.UTF
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_US.UTF") returns NULL
setlocale(LC_ALL, NULL) returns 'C'
#####

Thank you very much.

I just realized I failed to include '-8' in some of those names. So here's a different sample​:

#####
$ cat locales_some.txt en_AU.UTF-8
en_CA.UTF-8
en_GB.UTF-8
en_US.UTF-8
es_AR.UTF-8
es_BO.UTF-8
es_CH.UTF-8
es_CO.UTF-8
es_CR.UTF-8
es_CU.UTF-8
es_DO.UTF-8

$ for l in `cat locales_some.txt`; do ./khw-testlocale-20181111 $l;done

setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "en_AU.UTF-8") returns 'en_AU.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/en_AU.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "en_CA.UTF-8") returns 'en_CA.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/en_CA.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "en_GB.UTF-8") returns 'en_GB.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/en_GB.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "en_US.UTF-8") returns 'en_US.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/en_US.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_AR.UTF-8") returns 'es_AR.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/es_AR.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_BO.UTF-8") returns 'es_BO.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/es_BO.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_CH.UTF-8") returns 'es_CH.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/es_CH.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_CO.UTF-8") returns 'es_CO.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/es_CO.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_CR.UTF-8") returns 'es_CR.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/es_CR.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_CU.UTF-8") returns 'es_CU.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/es_CU.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_DO.UTF-8") returns 'es_DO.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/es_DO.UTF-8/C/C/C/C'
#####
--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Nov 11, 2018

From @jkeenan

On Sun, 11 Nov 2018 01​:02​:31 GMT, jkeenan@​pobox.com wrote​:

Here is a really weird one​: On OpenBSD-6.4, the last two unit tests in
t/run/locale.t fail -- but only on *unthreaded* builds.

I first noticed this in smoke-test reports submitted by Carlos Guevara.
See, e.g., http​://perl5.test-smoke.org/report/73721.

An additional instance of this from the smoker​:

http​://perl5.test-smoke.org/report/73918

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Nov 12, 2018

From @khwilliamson

On 11/11/18 4​:08 PM, James E Keenan via RT wrote​:

On Sun, 11 Nov 2018 22​:59​:57 GMT, jkeenan wrote​:

On Sun, 11 Nov 2018 22​:01​:56 GMT, public@​khwilliamson.com wrote​:

On 11/11/18 11​:58 AM, James E Keenan via RT wrote​:

On Sun, 11 Nov 2018 18​:51​:32 GMT, jkeenan wrote​:

On Sun, 11 Nov 2018 18​:00​:27 GMT, public@​khwilliamson.com wrote​:

On 11/10/18 6​:02 PM, James E Keenan (via RT) wrote​:

# New Ticket Created by James E Keenan
# Please include the string​: [perl #133649]
# in the subject line of all future correspondence about this
issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133649 >

Here is a really weird one​: On OpenBSD-6.4, the last two unit
tests
in
t/run/locale.t fail -- but only on *unthreaded* builds.

[snip]

Run the attached script on a DEBUGGING build with runtime switch
-DLv
and post the output

I had to add '-MPOSIX' to the command-line invocation. See
attached.

Thank you very much.

Note​: I tried switching the 2 lines of code around. I also tried
with 'C' instead of 'C.UTF-8'. I got the same "Can't fix broken
locale name" error in all cases.

Try compiling the attached C program, and run it with a single
argument,
a legal non-C locale, perhaps en_US.UTF-8

like

./a.out en_US.UTF-8

and post the results

Here's a sampler​:

#####
[perl] 17 $ ~/learn/perl/p5p/khw-testlocale-20181111 en_US.UTF
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "en_US.UTF") returns NULL
setlocale(LC_ALL, NULL) returns 'C'
[perl] 18 $ ~/learn/perl/p5p/khw-testlocale-20181111 iso8859-1
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "iso8859-1") returns 'iso8859-1'
setlocale(LC_ALL, NULL) returns 'C/iso8859-1/C/C/C/C'
[perl] 19 $ ~/learn/perl/p5p/khw-testlocale-20181111 C.UTF-8
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "C.UTF-8") returns 'C.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/C.UTF-8/C/C/C/C'
[perl] 20 $ ~/learn/perl/p5p/khw-testlocale-20181111 C
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "C") returns 'C'
setlocale(LC_ALL, NULL) returns 'C'
[perl] 21 $ ~/learn/perl/p5p/khw-testlocale-20181111 es_US.UTF
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_US.UTF") returns NULL
setlocale(LC_ALL, NULL) returns 'C'
#####

Thank you very much.

I just realized I failed to include '-8' in some of those names. So here's a different sample​:

#####
$ cat locales_some.txt en_AU.UTF-8
en_CA.UTF-8
en_GB.UTF-8
en_US.UTF-8
es_AR.UTF-8
es_BO.UTF-8
es_CH.UTF-8
es_CO.UTF-8
es_CR.UTF-8
es_CU.UTF-8
es_DO.UTF-8

$ for l in `cat locales_some.txt`; do ./khw-testlocale-20181111 $l;done

setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "en_AU.UTF-8") returns 'en_AU.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/en_AU.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "en_CA.UTF-8") returns 'en_CA.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/en_CA.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "en_GB.UTF-8") returns 'en_GB.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/en_GB.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "en_US.UTF-8") returns 'en_US.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/en_US.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_AR.UTF-8") returns 'es_AR.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/es_AR.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_BO.UTF-8") returns 'es_BO.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/es_BO.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_CH.UTF-8") returns 'es_CH.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/es_CH.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_CO.UTF-8") returns 'es_CO.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/es_CO.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_CR.UTF-8") returns 'es_CR.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/es_CR.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_CU.UTF-8") returns 'es_CU.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/es_CU.UTF-8/C/C/C/C'
setlocale(LC_ALL, "C") returns 'C'
setlocale(LC_CTYPE, "es_DO.UTF-8") returns 'es_DO.UTF-8'
setlocale(LC_ALL, NULL) returns 'C/es_DO.UTF-8/C/C/C/C'
#####

It looks like this is a result of this OS using a different syntax for
complex LC_ALL definitions than most other OS's. I thought I had taken
care of this possibility, but will look at it in the next few days.

@p5pRT
Copy link
Author

p5pRT commented Nov 12, 2018

From @khwilliamson

On 11/11/18 4​:12 PM, James E Keenan via RT wrote​:

On Sun, 11 Nov 2018 01​:02​:31 GMT, jkeenan@​pobox.com wrote​:

Here is a really weird one​: On OpenBSD-6.4, the last two unit tests in
t/run/locale.t fail -- but only on *unthreaded* builds.

I first noticed this in smoke-test reports submitted by Carlos Guevara.
See, e.g., http​://perl5.test-smoke.org/report/73721.

An additional instance of this from the smoker​:

http​://perl5.test-smoke.org/report/73918

I'm wondering why 6.3 worked, but not 6.4. I have a suspicion. Please
post the config.h files for each version.

@p5pRT
Copy link
Author

p5pRT commented Nov 12, 2018

From @jkeenan

On Mon, 12 Nov 2018 16​:48​:57 GMT, public@​khwilliamson.com wrote​:

On 11/11/18 4​:12 PM, James E Keenan via RT wrote​:

On Sun, 11 Nov 2018 01​:02​:31 GMT, jkeenan@​pobox.com wrote​:

Here is a really weird one​: On OpenBSD-6.4, the last two unit tests in
t/run/locale.t fail -- but only on *unthreaded* builds.

I first noticed this in smoke-test reports submitted by Carlos Guevara.
See, e.g., http​://perl5.test-smoke.org/report/73721.

An additional instance of this from the smoker​:

http​://perl5.test-smoke.org/report/73918

I'm wondering why 6.3 worked, but not 6.4. I have a suspicion. Please
post the config.h files for each version.

Unfortunately, our openbsd-6.3 smoker met an early demise and it would take a fair amount of work to revive it. Attaching the config.h for openbsd-6.4 (from the nonthreaded DEBUGGING build previously mentioned).

Thank you very much.
--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Nov 12, 2018

From @jkeenan

config.h.openbsd-6.4

@p5pRT
Copy link
Author

p5pRT commented Nov 13, 2018

From @jkeenan

On Mon, 12 Nov 2018 17​:49​:53 GMT, jkeenan wrote​:

On Mon, 12 Nov 2018 16​:48​:57 GMT, public@​khwilliamson.com wrote​:

[snip]

I'm wondering why 6.3 worked, but not 6.4. I have a suspicion.
Please
post the config.h files for each version.

Unfortunately, our openbsd-6.3 smoker met an early demise and it would
take a fair amount of work to revive it. Attaching the config.h for
openbsd-6.4 (from the nonthreaded DEBUGGING build previously
mentioned).

Okay, I managed to revive that OpenBSD-6.3 VM. Please find 3 files attached. But note ...

Although this VM reports having the same number of locales available as our OpenBSD-6.4 VM, only 2 of them -- C and POSIX -- were found while running t/run/locale.t. I don't know enough about this OS to speculate as to the reason why.

As a consequence, only 2 unit tests in that file were run, so I can't meaningfully compare the results with those we're getting on OpenBSD-6.4.

Also note that there were no significant differences between the config.h files generated by building blead on the two OS versions.

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Nov 13, 2018

From @jkeenan

config.h.openbsd-6.3

@p5pRT
Copy link
Author

p5pRT commented Nov 13, 2018

From @jkeenan

C
C.UTF-8
POSIX
POSIX.UTF-8
Pig.UTF-8
ar_SD.UTF-8
ar_SY.UTF-8
ca_ES.UTF-8
cs_CZ.UTF-8
da_DK.UTF-8
de_AT.UTF-8
de_CH.UTF-8
de_DE.UTF-8
el_GR.UTF-8
en_AU.UTF-8
en_CA.UTF-8
en_GB.UTF-8
en_US.UTF-8
es_AR.UTF-8
es_BO.UTF-8
es_CH.UTF-8
es_CO.UTF-8
es_CR.UTF-8
es_CU.UTF-8
es_DO.UTF-8
es_EC.UTF-8
es_ES.UTF-8
es_GQ.UTF-8
es_GT.UTF-8
es_HN.UTF-8
es_MX.UTF-8
es_NI.UTF-8
es_PA.UTF-8
es_PE.UTF-8
es_PR.UTF-8
es_PY.UTF-8
es_SV.UTF-8
es_US.UTF-8
es_UY.UTF-8
es_VE.UTF-8
fa_IR.UTF-8
fi_FI.UTF-8
fr_BE.UTF-8
fr_CA.UTF-8
fr_CH.UTF-8
fr_FR.UTF-8
hu_HU.UTF-8
hy_AM.UTF-8
is_IS.UTF-8
it_CH.UTF-8
it_IT.UTF-8
ja_JP.UTF-8
ko_KR.UTF-8
lt_LT.UTF-8
nl_BE.UTF-8
nl_NL.UTF-8
no_NO.UTF-8
pl_PL.UTF-8
pt_PT.UTF-8
ro_RO.UTF-8
ru_RU.UTF-8
sk_SK.UTF-8
sl_SI.UTF-8
sv_SE.UTF-8
tr_TR.UTF-8
uk_UA.UTF-8
zh_CN.UTF-8
zh_TW.UTF-8

@p5pRT
Copy link
Author

p5pRT commented Nov 13, 2018

From @jkeenan

# locales available​: C POSIX
ok 1 - /il matching of [bracketed] doesn't skip POSIX class if fails individ char
ok 2 - /l matching of [bracketed] doesn't skip non-first POSIX class
1..2

@p5pRT
Copy link
Author

p5pRT commented Nov 19, 2018

From @jkeenan

On Mon, 12 Nov 2018 16​:48​:57 GMT, public@​khwilliamson.com wrote​:

On 11/11/18 4​:12 PM, James E Keenan via RT wrote​:

On Sun, 11 Nov 2018 01​:02​:31 GMT, jkeenan@​pobox.com wrote​:

Here is a really weird one​: On OpenBSD-6.4, the last two unit tests in
t/run/locale.t fail -- but only on *unthreaded* builds.

I first noticed this in smoke-test reports submitted by Carlos Guevara.
See, e.g., http​://perl5.test-smoke.org/report/73721.

An additional instance of this from the smoker​:

http​://perl5.test-smoke.org/report/73918

I'm wondering why 6.3 worked, but not 6.4. I have a suspicion. Please
post the config.h files for each version.

Karl,

Your commit 9f77c4a in your current smoke-me/khw-locale branch mitigates the problem. In the following two smoke test reports there are no failures in t/run/locale.t on OpenBSD-6.4​:

http​://perl5.test-smoke.org/report/74334
http​://perl5.test-smoke.org/report/74337

Feel free to merge.

Thank you very much.
--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

p5pRT commented Nov 19, 2018

From @khwilliamson

This has been fixed in blead by
8f6aacb

Commit e1895ad
allows using Configure to use the thread-safe functions even on unthreaded builds, which is another work around for this problem
--
Karl Williamson

@p5pRT
Copy link
Author

p5pRT commented Nov 19, 2018

@khwilliamson - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

p5pRT commented Nov 26, 2018

From andrew@afresh1.com

On Mon, Nov 19, 2018 at 02​:29​:05PM -0800, Karl Williamson via RT wrote​:

This has been fixed in blead by
8f6aacb

Commit e1895ad
allows using Configure to use the thread-safe functions even on unthreaded builds, which is another work around for this problem

I applied this to a 5.28 release (minus the perldelta changes) and still
failed run/locale. Even adding -Dccflags='-DUSE_THREAD_SAFE_LOCALE' to
the configure arguments didn't help me any. Is there something I'm
missing?

@p5pRT
Copy link
Author

p5pRT commented Nov 26, 2018

From @jkeenan

On 11/25/18 8​:04 PM, Andrew Fresh via RT wrote​:

On Mon, Nov 19, 2018 at 02​:29​:05PM -0800, Karl Williamson via RT wrote​:

This has been fixed in blead by
8f6aacb

Commit e1895ad
allows using Configure to use the thread-safe functions even on unthreaded builds, which is another work around for this problem

I applied this to a 5.28 release (minus the perldelta changes) and still
failed run/locale. Even adding -Dccflags='-DUSE_THREAD_SAFE_LOCALE' to
the configure arguments didn't help me any. Is there something I'm
missing?

Can you post either a smoke-test result or report of failure plus perl -V?

Also, ISTR your mentioning that the way you build the perl package for
OpenBSD differs from the way we typically work in blead and in smoke
tests. I don't remember the details -- but could that be a factor here?

jimk

@p5pRT
Copy link
Author

p5pRT commented Nov 26, 2018

From andrew@afresh1.com

On Sun, Nov 25, 2018 at 08​:47​:38PM -0500, James E Keenan wrote​:

On 11/25/18 8​:04 PM, Andrew Fresh via RT wrote​:

On Mon, Nov 19, 2018 at 02​:29​:05PM -0800, Karl Williamson via RT wrote​:

This has been fixed in blead by
8f6aacb

Commit e1895ad
allows using Configure to use the thread-safe functions even on unthreaded builds, which is another work around for this problem

I applied this to a 5.28 release (minus the perldelta changes) and still
failed run/locale. Even adding -Dccflags='-DUSE_THREAD_SAFE_LOCALE' to
the configure arguments didn't help me any. Is there something I'm
missing?

Can you post either a smoke-test result or report of failure plus perl -V?

Also, ISTR your mentioning that the way you build the perl package for
OpenBSD differs from the way we typically work in blead and in smoke tests.
I don't remember the details -- but could that be a factor here?

It seems I did something wrong the last time, not sure what yet, but
trying again the tests pass. If I figure out what I did wrong and it
seems important I'll let you know.

l8rZ,
--
andrew - http​://afresh1.com

I wish life had an UNDO function.

@p5pRT
Copy link
Author

p5pRT commented May 22, 2019

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.30.0, this and 160 other issues have been
resolved.

Perl 5.30.0 may be downloaded via​:
https://metacpan.org/release/XSAWYERX/perl-5.30.0

If you find that the problem persists, feel free to reopen this ticket.

@p5pRT
Copy link
Author

p5pRT commented May 22, 2019

@khwilliamson - Status changed from 'pending release' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant