Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PATCH] ExtUtils::Manifest may mix different types of eol #11559

Closed
p5pRT opened this issue Aug 6, 2011 · 5 comments
Closed

[PATCH] ExtUtils::Manifest may mix different types of eol #11559

p5pRT opened this issue Aug 6, 2011 · 5 comments

Comments

@p5pRT
Copy link

p5pRT commented Aug 6, 2011

Migrated from rt.perl.org#96408 (status was 'resolved')

Searchable as RT96408$

@p5pRT
Copy link
Author

p5pRT commented Aug 6, 2011

From @andk

I've seen MANIFEST files on CPAN with a mish-mash of end of line types.
That is not only ugly, it also has impact on signature checking. See
https://rt.cpan.org/Ticket/Display.html?id=69897 or
MANWAR/Test-Excel-1.23.tar.gz.

Those manifests were apparently written by ExtUtils​::Manifest​::maniadd.
Appended patch fixes this bug by rewriting non-native manifest files
before maniadd() appends a line. A test is included that fails with
current ExtUtils​::Manifest.

Enjoy,
--
andreas

@p5pRT
Copy link
Author

p5pRT commented Aug 6, 2011

From @andk

0001-prevent-mixing-non-native-and-native-newlines-in-man.patch
From 1848c521c5c0147a336b2560a819a91971596c80 Mon Sep 17 00:00:00 2001
From: Sandy Andy <andk@cpan.org>
Date: Sat, 6 Aug 2011 08:09:22 +0200
Subject: [PATCH] prevent mixing non-native and native newlines in manifest
 files

---
 dist/ExtUtils-Manifest/lib/ExtUtils/Manifest.pm |   34 ++++++++++++-----
 dist/ExtUtils-Manifest/t/Manifest.t             |   44 ++++++++++++++++++++++-
 2 files changed, 67 insertions(+), 11 deletions(-)

diff --git a/dist/ExtUtils-Manifest/lib/ExtUtils/Manifest.pm b/dist/ExtUtils-Manifest/lib/ExtUtils/Manifest.pm
index ce4314c..62e9877 100644
--- a/dist/ExtUtils-Manifest/lib/ExtUtils/Manifest.pm
+++ b/dist/ExtUtils-Manifest/lib/ExtUtils/Manifest.pm
@@ -13,7 +13,7 @@ use vars qw($VERSION @ISA @EXPORT_OK
           $Is_MacOS $Is_VMS $Is_VMS_mode $Is_VMS_lc $Is_VMS_nodot
           $Debug $Verbose $Quiet $MANIFEST $DEFAULT_MSKIP);
 
-$VERSION = '1.58';
+$VERSION = '1.59';
 @ISA=('Exporter');
 @EXPORT_OK = qw(mkmanifest
                 manicheck  filecheck  fullcheck  skipcheck
@@ -706,21 +706,35 @@ sub maniadd {
 }
 
 
-# Sometimes MANIFESTs are missing a trailing newline.  Fix this.
+# Make sure this MANIFEST is consistently written with native
+# newlines and has a terminal newline.
 sub _fix_manifest {
     my $manifest_file = shift;
 
     open MANIFEST, $MANIFEST or die "Could not open $MANIFEST: $!";
-
-    # Yes, we should be using seek(), but I'd like to avoid loading POSIX
-    # to get SEEK_*
-    my @manifest = <MANIFEST>;
+    local $/;
+    my @manifest = split /(\015\012|\012|\015)/, <MANIFEST>, -1;
     close MANIFEST;
+    my $must_rewrite = "";
+    if ($manifest[-1] eq ""){
+        # sane case: last line had a terminal newline
+        pop @manifest;
+        for (my $i=1; $i<=$#manifest; $i+=2) {
+            unless ($manifest[$i] eq "\n") {
+                $must_rewrite = "not a newline at pos $i";
+                last;
+            }
+        }
+    } else {
+        $must_rewrite = "last line without newline";
+    }
 
-    unless( $manifest[-1] =~ /\n\z/ ) {
-        open MANIFEST, ">>$MANIFEST" or die "Could not open $MANIFEST: $!";
-        print MANIFEST "\n";
-        close MANIFEST;
+    if ( $must_rewrite ) {
+        open MANIFEST, ">", $MANIFEST or die "(must_rewrite=$must_rewrite) Could not open >$MANIFEST: $!";
+        for (my $i=0; $i<=$#manifest; $i+=2) {
+            print MANIFEST "$manifest[$i]\n";
+        }
+        close MANIFEST or die "could not write $MANIFEST: $!";
     }
 }
 
diff --git a/dist/ExtUtils-Manifest/t/Manifest.t b/dist/ExtUtils-Manifest/t/Manifest.t
index 91b126b..8d2ff8b 100644
--- a/dist/ExtUtils-Manifest/t/Manifest.t
+++ b/dist/ExtUtils-Manifest/t/Manifest.t
@@ -13,7 +13,7 @@ chdir 't';
 
 use strict;
 
-use Test::More tests => 94;
+use Test::More tests => 96;
 use Cwd;
 
 use File::Spec;
@@ -231,6 +231,48 @@ is( $files->{wibble}, '',    'maniadd() with undef comment' );
 is( $files->{yarrow}, 'hock','          with comment' );
 is( $files->{foobar}, '',    '          preserved old entries' );
 
+{
+    # EOL normalization in maniadd()
+
+    # move manifest away:
+    rename "MANIFEST", "MANIFEST.bak" or die "Could not rename MANIFEST to MANIFEST.bak: $!";
+    my $prev_maniaddresult;
+    my @eol = ("\012","\015","\015\012");
+    # for all line-endings:
+    for my $i (0..$#eol) {
+        my $eol = $eol[$i];
+        #   cp the backup of the manifest to MANIFEST, line-endings adjusted
+        my $content = do { local $/; open my $fh, "MANIFEST.bak" or die; <$fh> };
+    SPLITTER: for my $eol2 (@eol) {
+            if ( index($content, $eol2) > -1 ) {
+                my @lines = split /$eol2/, $content;
+                pop @lines while $lines[-1] eq "";
+                open my $fh, ">", "MANIFEST" or die "Could not open >MANIFEST: $!";
+                print $fh map { "$_$eol" } @lines;
+                close $fh or die "Could not close: $!";
+                last SPLITTER;
+            }
+        }
+        #   try maniadd
+        maniadd({eoltest => "end of line normalization test"});
+        #   slurp result and compare to previous result
+        my $maniaddresult = do { local $/; open my $fh, "MANIFEST" or die; <$fh> };
+        if ($prev_maniaddresult) {
+            if ( $maniaddresult eq $prev_maniaddresult ) {
+                pass "normalization success with i=$i";
+            } else {
+                require Data::Dumper;
+                local $Data::Dumper::Useqq = 1;
+                local $Data::Dumper::Terse = 1;
+                is Data::Dumper::Dumper($maniaddresult), Data::Dumper::Dumper($prev_maniaddresult), "eol normalization failed with i=$i";
+            }
+        }
+        $prev_maniaddresult = $maniaddresult;
+    }
+    # move backup over MANIFEST
+    rename "MANIFEST.bak", "MANIFEST" or die "Could not rename MANIFEST.bak to MANIFEST: $!";
+}
+
 my %funky_files;
 # test including a filename with a space
 SKIP: {
-- 
1.7.5.4

@p5pRT
Copy link
Author

p5pRT commented Aug 6, 2011

From @cpansprout

On Sat Aug 06 00​:12​:22 2011, andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

I've seen MANIFEST files on CPAN with a mish-mash of end of line types.
That is not only ugly, it also has impact on signature checking. See
https://rt.cpan.org/Ticket/Display.html?id=69897 or
MANWAR/Test-Excel-1.23.tar.gz.

Those manifests were apparently written by ExtUtils​::Manifest​::maniadd.
Appended patch fixes this bug by rewriting non-native manifest files
before maniadd() appends a line. A test is included that fails with
current ExtUtils​::Manifest.

Enjoy,

Thank you. Applied as e4ac890.

@p5pRT
Copy link
Author

p5pRT commented Aug 6, 2011

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Aug 6, 2011

@cpansprout - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant