From tchrist@perl.com

On the systems I tested it on, the manpages whose pods contain non-ASCII
are botched, some to the point of complete illegibility.

This is wrong​:

  $ man perlhist| grep var
  Steve Hay, Matt S Trout, David Golden, Florian Ragwitz, Tatsuhiko Miyagawa, Chris "BinGOs" Williams, Zefram and var Arnfjoer` Bjarmason.
  Chip Salzenberg gets credit for that, with a nod to his cow orker, David Croy. We had passed around various names (baton, token, hot potato) but none caught on.
  var 5.13.10 2011-Feb-20

And this is intolerable​:

  $ man perlcn
  PERLCN(1) Perl Programmers Reference Guide PERLCN(1)

  NAME
  perlcn - XXXX Perl XX

  DESCRIPTION
  XXXX Perl XXX!

  X 5.8.0 XXX, Perl XXXXXX Unicode (XXX) XX, XXXXXXXXXXXXXXXXXXX; CJK (XXX) XXXXXXXX. Unicode XXXXXXX, XXXXXXXXXXXX​: XXXX, XXXX,
  XXXXXXXX (XXX, XXXX, XXXX, XXXX, XXX, XXXX, XX). XXXXXXXXXXXXXX (X PC XXXX).

  Perl XXX Unicode XXXX. XXX Perl XXXXXXXXXX Unicode XX; Perl XXXXXX (XXXXXXXXX) XXX Unicode XXXX. XXXXXXX, XXXXX Unicode
  XXXXXXXXXXXX, Perl XXX Encode XXXX, XXXXXXXXXXXXXXXXXXX.

The bug is that while you are allowed to and encouraged to declare
your podfiles' encoding, this is severely disrespected.

The following podpages show the problem​:

  perlebcdic.pod
  perlgit.pod
  perlhist.pod
  perlpodspec.pod
  perlthrtut.pod

  perl588delta.pod
  perl5100delta.pod
  perl5120delta.pod
  perl5121delta.pod
  perl5122delta.pod
  perl5123delta.pod
  perl5124delta.pod
  perl5140delta.pod
  perl5141delta.pod
  perl5142delta.pod
  perl5150delta.pod
  perl5151delta.pod
  perl5152delta.pod
  perl5153delta.pod
  perl5154delta.pod
  perl5156delta.pod
  perl5157delta.pod
  perl5158delta.pod

  perlcn.pod
  perljp.pod
  perlko.pod
  perltw.pod

--tom

Summary of my perl5 (revision 5 version 14 subversion 0) configuration​:
 
  Platform​:
  osname=openbsd, osvers=4.4, archname=OpenBSD.i386-openbsd
  uname='openbsd chthon 4.4 generic#0 i386 '
  config_args='-des'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=undef, usemultiplicity=undef
  useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
  use64bitint=undef, use64bitall=undef, uselongdouble=undef
  usemymalloc=y, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include',
  optimize='-O2',
  cppflags='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
  ccversion='', gccversion='3.3.5 (propolice)', gccosandvers='openbsd4.4'
  intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
  ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=4, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags ='-Wl,-E -fstack-protector -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib
  libs=-lgdbm -lm -lutil -lc
  perllibs=-lm -lutil -lc
  libc=/usr/lib/libc.so.48.0, so=so, useshrplib=false, libperl=libperl.a
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' '
  cccdlflags='-DPIC -fPIC ', lddlflags='-shared -fPIC -L/usr/local/lib -fstack-protector'

Characteristics of this binary (from libperl)​:
  Compile-time options​: MYMALLOC PERL_DONT_CREATE_GVSV PERL_MALLOC_WRAP
  PERL_PRESERVE_IVUV USE_LARGE_FILES USE_PERLIO
  USE_PERL_ATOF
  Built under openbsd
  Compiled at Jun 11 2011 11​:48​:28
  %ENV​:
  PERL_UNICODE="SA"
  @​INC​:
  /usr/local/lib/perl5/site_perl/5.14.0/OpenBSD.i386-openbsd
  /usr/local/lib/perl5/site_perl/5.14.0
  /usr/local/lib/perl5/5.14.0/OpenBSD.i386-openbsd
  /usr/local/lib/perl5/5.14.0
  /usr/local/lib/perl5/site_perl/5.12.3
  /usr/local/lib/perl5/site_perl/5.11.3
  /usr/local/lib/perl5/site_perl/5.10.1
  /usr/local/lib/perl5/site_perl/5.10.0
  /usr/local/lib/perl5/site_perl/5.8.7
  /usr/local/lib/perl5/site_perl/5.8.0
  /usr/local/lib/perl5/site_perl/5.6.0
  /usr/local/lib/perl5/site_perl/5.005
  /usr/local/lib/perl5/site_perl
  .

From @jkeenan

On Wed Feb 29 13​:04​:43 2012, tom christiansen wrote​:

On the systems I tested it on, the manpages whose pods contain non-
ASCII
are botched, some to the point of complete illegibility.

This is wrong​:

$ man perlhist| grep var
   Steve Hay\, Matt S Trout\, David Golden\, Florian Ragwitz\,

Tatsuhiko Miyagawa, Chris "BinGOs" Williams, Zefram and var Arnfjoer`
Bjarmason.
Chip Salzenberg gets credit for that, with a nod to his cow
orker, David Croy. We had passed around various names (baton, token,
hot potato) but none caught on.
var 5.13.10 2011-Feb-20

And this is intolerable​:

$ man perlcn
PERLCN\(1\)                                            Perl

Programmers Reference Guide
PERLCN(1)

NAME
   perlcn \- XXXX Perl XX

DESCRIPTION
   XXXX Perl XXX\!

   X 5\.8\.0 XXX\, Perl XXXXXX Unicode \(XXX\) XX\, XXXXXXXXXXXXXXXXXXX;

CJK (XXX) XXXXXXXX. Unicode XXXXXXX, XXXXXXXXXXXX​: XXXX, XXXX,
XXXXXXXX (XXX, XXXX, XXXX, XXXX, XXX, XXXX, XX). XXXXXXXXXXXXXX (X
PC XXXX).

   Perl XXX Unicode XXXX\. XXX Perl XXXXXXXXXX Unicode XX; Perl XXXXXX

(XXXXXXXXX) XXX Unicode XXXX. XXXXXXX, XXXXX Unicode
XXXXXXXXXXXX, Perl XXX Encode XXXX, XXXXXXXXXXXXXXXXXXX.

The bug is that while you are allowed to and encouraged to declare
your podfiles' encoding, this is severely disrespected.

The following podpages show the problem​:

perlebcdic\.pod
perlgit\.pod
perlhist\.pod
perlpodspec\.pod
perlthrtut\.pod

perl588delta\.pod
perl5100delta\.pod
perl5120delta\.pod
perl5121delta\.pod
perl5122delta\.pod
perl5123delta\.pod
perl5124delta\.pod
perl5140delta\.pod
perl5141delta\.pod
perl5142delta\.pod
perl5150delta\.pod
perl5151delta\.pod
perl5152delta\.pod
perl5153delta\.pod
perl5154delta\.pod
perl5156delta\.pod
perl5157delta\.pod
perl5158delta\.pod

perlcn\.pod
perljp\.pod
perlko\.pod
perltw\.pod

--tom

Is this a problem that can be fixed in the 'pod2man' utility?

Or does it require corrections of individual .pod files?

Or both?

Or neither?

Thank you very much.
Jim Keenan

The RT System itself - Status changed from 'new' to 'open'

From @mrallen1

On Wed Sep 26 19​:07​:49 2012, jkeenan wrote​:

On Wed Feb 29 13​:04​:43 2012, tom christiansen wrote​:

On the systems I tested it on, the manpages whose pods contain non-
ASCII
are botched, some to the point of complete illegibility.

Is this a problem that can be fixed in the 'pod2man' utility?

Or does it require corrections of individual .pod files?

Or both?

Or neither?

Hi. I'm the current maintainer of Pod​::Perldoc. In 90% of the cases where the output from
pod2man is displayed as "X" instead of an proper glyph, it is typically groff itself which botches
the handling non-ASCII characters.

Getting clean non-ASCII characters typically requires a groff >= 1.20.1 and within the context
of perldoc (at least) a recent version (>=3.18). As far as pod2man is concerned, I believe it
handles non-ASCII characters as well as it can given the various points where encoding matters
to display which are (among others), the shell itself, perl, groff, the pager program, and
whatever terminal one is using.

So I would say this report is inconclusive. It may well be that pod2man is generating proper roff
output, and groff is mangling it when it tries to format and display it.

Mark