Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strawberry Perl fails to truncate large file #14737

Closed
p5pRT opened this issue Jun 7, 2015 · 12 comments
Closed

Strawberry Perl fails to truncate large file #14737

p5pRT opened this issue Jun 7, 2015 · 12 comments

Comments

@p5pRT
Copy link

p5pRT commented Jun 7, 2015

Migrated from rt.perl.org#125347 (status was 'resolved')

Searchable as RT125347$

@p5pRT
Copy link
Author

p5pRT commented Jun 7, 2015

From ntysdd@gmail.com

The 64-bit Portable version of Strawberry Perl fail to `truncate'
large files (much larger than 4GB) on Windows 7.

It should handle large file correctly, or it should be documented in
perldoc -f truncate and perldoc perlport.

@p5pRT
Copy link
Author

p5pRT commented Jun 8, 2015

From @tonycoz

On Sun Jun 07 03​:12​:34 2015, ntysdd@​gmail.com wrote​:

The 64-bit Portable version of Strawberry Perl fail to `truncate'
large files (much larger than 4GB) on Windows 7.

It should handle large file correctly, or it should be documented in
perldoc -f truncate and perldoc perlport.

How much larger? It seems to work truncating an 8GB file down to 6GB​:

C​:\Users\tony>perl -e "$size = 0x200000000; open my $fh, '+<', 'testfile' or die
qq/open $!/; truncate $fh, $size or die qq/extend​: $!/; close $fh;"

C​:\Users\tony>dir testfile
Volume in drive C has no label.
Volume Serial Number is 10A0-5EC5

Directory of C​:\Users\tony

08/06/2015 11​:19 AM 8,589,934,592 testfile
  1 File(s) 8,589,934,592 bytes
  0 Dir(s) 806,087,630,848 bytes free

C​:\Users\tony>perl -e "$size = 0x180000000; open my $fh, '+<', 'testfile' or die
qq/open $!/; truncate $fh, $size or die qq/trunc​: $!/; close $fh;"

C​:\Users\tony>dir testfile
Volume in drive C has no label.
Volume Serial Number is 10A0-5EC5

Directory of C​:\Users\tony

08/06/2015 11​:19 AM 6,442,450,944 testfile
  1 File(s) 6,442,450,944 bytes
  0 Dir(s) 808,235,110,400 bytes free

Tony

@p5pRT
Copy link
Author

p5pRT commented Jun 8, 2015

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

p5pRT commented Jun 8, 2015

From ntysdd@gmail.com

Sorry so late.
I didn't test the FILEHANDLE version. It seems it's OK.
The EXPR version fails to truncate this testfile.

2015-06-08 9​:20 GMT+08​:00 Tony Cook via RT <perlbug-followup@​perl.org>​:

On Sun Jun 07 03​:12​:34 2015, ntysdd@​gmail.com wrote​:

The 64-bit Portable version of Strawberry Perl fail to `truncate'
large files (much larger than 4GB) on Windows 7.

It should handle large file correctly, or it should be documented in
perldoc -f truncate and perldoc perlport.

How much larger? It seems to work truncating an 8GB file down to 6GB​:

C​:\Users\tony>perl -e "$size = 0x200000000; open my $fh, '+<', 'testfile' or die
qq/open $!/; truncate $fh, $size or die qq/extend​: $!/; close $fh;"

C​:\Users\tony>dir testfile
Volume in drive C has no label.
Volume Serial Number is 10A0-5EC5

Directory of C​:\Users\tony

08/06/2015 11​:19 AM 8,589,934,592 testfile
1 File(s) 8,589,934,592 bytes
0 Dir(s) 806,087,630,848 bytes free

C​:\Users\tony>perl -e "$size = 0x180000000; open my $fh, '+<', 'testfile' or die
qq/open $!/; truncate $fh, $size or die qq/trunc​: $!/; close $fh;"

C​:\Users\tony>dir testfile
Volume in drive C has no label.
Volume Serial Number is 10A0-5EC5

Directory of C​:\Users\tony

08/06/2015 11​:19 AM 6,442,450,944 testfile
1 File(s) 6,442,450,944 bytes
0 Dir(s) 808,235,110,400 bytes free

Tony

@p5pRT
Copy link
Author

p5pRT commented Jun 9, 2015

From @tonycoz

On Mon Jun 08 04​:58​:25 2015, ntysdd@​gmail.com wrote​:

Sorry so late.
I didn't test the FILEHANDLE version. It seems it's OK.
The EXPR version fails to truncate this testfile.

Thanks, now I can reproduce it.

The PerlLIO_open() was failing due to MSVCRT failing to seek on
text files.

The attached fixes it for me.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jun 9, 2015

From @tonycoz

0001-perl-125347-allow-truncate-to-work-on-large-files-on.patch
From e48b3f83b9e08c0987b4486e07c511ee9ea3d229 Mon Sep 17 00:00:00 2001
From: Tony Cook <tony@develop-help.com>
Date: Tue, 9 Jun 2015 10:48:28 +1000
Subject: [PATCH] [perl #125347] allow truncate to work on large files on Win32

truncate($filename, $size) was using a simple PerlIO_open() to open
the file, which on Win32 defaults to a text mode open.

Unfortunately, on a text mode open(), MSVCRT attempts to seek to the
end of file using only 32-bit offsets, which fails.

For good measure, add in O_LARGEFILE if it's available, which may
prevent similar issues on other platforms.

Also, remove the erroneous SETERRNO() added by 375ed12a to the open
failure branch, PerlLIO_open() should already set errno on failure, so
we get sane error messages when the open fails.
---
 pp_sys.c | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/pp_sys.c b/pp_sys.c
index 7c20f52..922b897 100644
--- a/pp_sys.c
+++ b/pp_sys.c
@@ -2304,10 +2304,22 @@ PP(pp_truncate)
 	        result = 0;
 #else
 	    {
-		const int tmpfd = PerlLIO_open(name, O_RDWR);
+                int mode = O_RDWR;
+                int tmpfd = PerlLIO_open(name, O_RDWR);
+
+#if defined(USE_64_BIT_RAWIO) && defined(O_LARGEFILE)
+                mode |= O_LARGEFILE;	/* Transparently largefiley. */
+#endif
+#ifdef O_BINARY
+                /* On open(), the Win32 CRT tries to seek around text
+                 * files using 32-bit offsets, which causes the open()
+                 * to fail on large files, so open in binary mode.
+                 */
+                mode |= O_BINARY;
+#endif
+                tmpfd = PerlLIO_open(name, mode);
 
 		if (tmpfd < 0) {
-                    SETERRNO(EBADF,RMS_IFI);
 		    result = 0;
 		} else {
 		    if (my_chsize(tmpfd, len) < 0)
-- 
1.9.5.msysgit.0

@p5pRT
Copy link
Author

p5pRT commented Jun 9, 2015

From ntysdd@gmail.com

Why is the function PerlLIO_open called twice?

2015-06-09 8​:51 GMT+08​:00, Tony Cook via RT <perlbug-followup@​perl.org>​:

On Mon Jun 08 04​:58​:25 2015, ntysdd@​gmail.com wrote​:

Sorry so late.
I didn't test the FILEHANDLE version. It seems it's OK.
The EXPR version fails to truncate this testfile.

Thanks, now I can reproduce it.

The PerlLIO_open() was failing due to MSVCRT failing to seek on
text files.

The attached fixes it for me.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jun 9, 2015

From @tonycoz

On Tue, Jun 09, 2015 at 06​:42​:55PM +0800, ntysdd wrote​:

Why is the function PerlLIO_open called twice?

Sigh, you're right.

I remember typing that correctly, but I guess it was in the wrong
window.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jun 10, 2015

From @tonycoz

On Tue Jun 09 03​:43​:40 2015, ntysdd@​gmail.com wrote​:

Why is the function PerlLIO_open called twice?

Thanks, I've fixed that and applied the result as d484df6.

Tony

@p5pRT
Copy link
Author

p5pRT commented Jun 16, 2015

@tonycoz - Status changed from 'open' to 'pending release'

@p5pRT
Copy link
Author

p5pRT commented May 13, 2016

From @khwilliamson

Thank you for submitting this report. You have helped make Perl better.
 
With the release of Perl 5.24.0 on May 9, 2016, this and 149 other issues have been resolved.

Perl 5.24.0 may be downloaded via https://metacpan.org/release/RJBS/perl-5.24.0

@p5pRT
Copy link
Author

p5pRT commented May 13, 2016

@khwilliamson - Status changed from 'pending release' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant