Skip Menu |
Report information
Id: 125347
Status: resolved
Priority: 0/
Queue: perl5

Owner: Nobody
Requestors: ntysdd [at] gmail.com
Cc:
AdminCc:

Operating System: (no value)
PatchStatus: (no value)
Severity: low
Type: unknown
Perl Version: (no value)
Fixed In: (no value)

Attachments
0001-perl-125347-allow-truncate-to-work-on-large-files-on.patch



From: ntysdd <ntysdd [...] gmail.com>
Subject: Strawberry Perl fails to truncate large file
To: perlbug [...] perl.org
Date: Sun, 7 Jun 2015 18:11:50 +0800
Download (untitled) / with headers
text/plain 227b
The 64-bit Portable version of Strawberry Perl fail to `truncate' large files (much larger than 4GB) on Windows 7. It should handle large file correctly, or it should be documented in perldoc -f truncate and perldoc perlport.
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 1.1k
On Sun Jun 07 03:12:34 2015, ntysdd@gmail.com wrote: Show quoted text
> The 64-bit Portable version of Strawberry Perl fail to `truncate' > large files (much larger than 4GB) on Windows 7. > > It should handle large file correctly, or it should be documented in > perldoc -f truncate and perldoc perlport.
How much larger? It seems to work truncating an 8GB file down to 6GB: C:\Users\tony>perl -e "$size = 0x200000000; open my $fh, '+<', 'testfile' or die qq/open $!/; truncate $fh, $size or die qq/extend: $!/; close $fh;" C:\Users\tony>dir testfile Volume in drive C has no label. Volume Serial Number is 10A0-5EC5 Directory of C:\Users\tony 08/06/2015 11:19 AM 8,589,934,592 testfile 1 File(s) 8,589,934,592 bytes 0 Dir(s) 806,087,630,848 bytes free C:\Users\tony>perl -e "$size = 0x180000000; open my $fh, '+<', 'testfile' or die qq/open $!/; truncate $fh, $size or die qq/trunc: $!/; close $fh;" C:\Users\tony>dir testfile Volume in drive C has no label. Volume Serial Number is 10A0-5EC5 Directory of C:\Users\tony 08/06/2015 11:19 AM 6,442,450,944 testfile 1 File(s) 6,442,450,944 bytes 0 Dir(s) 808,235,110,400 bytes free Tony
Date: Mon, 8 Jun 2015 19:57:24 +0800
From: ntysdd <ntysdd [...] gmail.com>
Subject: Re: [perl #125347] Strawberry Perl fails to truncate large file
To: perlbug-followup [...] perl.org
Download (untitled) / with headers
text/plain 1.4k
Sorry so late. I didn't test the FILEHANDLE version. It seems it's OK. The EXPR version fails to truncate this testfile. 2015-06-08 9:20 GMT+08:00 Tony Cook via RT <perlbug-followup@perl.org>: Show quoted text
> On Sun Jun 07 03:12:34 2015, ntysdd@gmail.com wrote:
>> The 64-bit Portable version of Strawberry Perl fail to `truncate' >> large files (much larger than 4GB) on Windows 7. >> >> It should handle large file correctly, or it should be documented in >> perldoc -f truncate and perldoc perlport.
> > How much larger? It seems to work truncating an 8GB file down to 6GB: > > C:\Users\tony>perl -e "$size = 0x200000000; open my $fh, '+<', 'testfile' or die > qq/open $!/; truncate $fh, $size or die qq/extend: $!/; close $fh;" > > C:\Users\tony>dir testfile > Volume in drive C has no label. > Volume Serial Number is 10A0-5EC5 > > Directory of C:\Users\tony > > 08/06/2015 11:19 AM 8,589,934,592 testfile > 1 File(s) 8,589,934,592 bytes > 0 Dir(s) 806,087,630,848 bytes free > > C:\Users\tony>perl -e "$size = 0x180000000; open my $fh, '+<', 'testfile' or die > qq/open $!/; truncate $fh, $size or die qq/trunc: $!/; close $fh;" > > C:\Users\tony>dir testfile > Volume in drive C has no label. > Volume Serial Number is 10A0-5EC5 > > Directory of C:\Users\tony > > 08/06/2015 11:19 AM 6,442,450,944 testfile > 1 File(s) 6,442,450,944 bytes > 0 Dir(s) 808,235,110,400 bytes free > > Tony
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 326b
On Mon Jun 08 04:58:25 2015, ntysdd@gmail.com wrote: Show quoted text
> Sorry so late. > I didn't test the FILEHANDLE version. It seems it's OK. > The EXPR version fails to truncate this testfile.
Thanks, now I can reproduce it. The PerlLIO_open() was failing due to MSVCRT failing to seek on text files. The attached fixes it for me. Tony
Subject: 0001-perl-125347-allow-truncate-to-work-on-large-files-on.patch
From e48b3f83b9e08c0987b4486e07c511ee9ea3d229 Mon Sep 17 00:00:00 2001 From: Tony Cook <tony@develop-help.com> Date: Tue, 9 Jun 2015 10:48:28 +1000 Subject: [PATCH] [perl #125347] allow truncate to work on large files on Win32 truncate($filename, $size) was using a simple PerlIO_open() to open the file, which on Win32 defaults to a text mode open. Unfortunately, on a text mode open(), MSVCRT attempts to seek to the end of file using only 32-bit offsets, which fails. For good measure, add in O_LARGEFILE if it's available, which may prevent similar issues on other platforms. Also, remove the erroneous SETERRNO() added by 375ed12a to the open failure branch, PerlLIO_open() should already set errno on failure, so we get sane error messages when the open fails. --- pp_sys.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/pp_sys.c b/pp_sys.c index 7c20f52..922b897 100644 --- a/pp_sys.c +++ b/pp_sys.c @@ -2304,10 +2304,22 @@ PP(pp_truncate) result = 0; #else { - const int tmpfd = PerlLIO_open(name, O_RDWR); + int mode = O_RDWR; + int tmpfd = PerlLIO_open(name, O_RDWR); + +#if defined(USE_64_BIT_RAWIO) && defined(O_LARGEFILE) + mode |= O_LARGEFILE; /* Transparently largefiley. */ +#endif +#ifdef O_BINARY + /* On open(), the Win32 CRT tries to seek around text + * files using 32-bit offsets, which causes the open() + * to fail on large files, so open in binary mode. + */ + mode |= O_BINARY; +#endif + tmpfd = PerlLIO_open(name, mode); if (tmpfd < 0) { - SETERRNO(EBADF,RMS_IFI); result = 0; } else { if (my_chsize(tmpfd, len) < 0) -- 1.9.5.msysgit.0
Date: Tue, 9 Jun 2015 18:42:55 +0800
From: ntysdd <ntysdd [...] gmail.com>
Subject: Re: [perl #125347] Strawberry Perl fails to truncate large file
To: perlbug-followup [...] perl.org
Download (untitled) / with headers
text/plain 470b
Why is the function PerlLIO_open called twice? 2015-06-09 8:51 GMT+08:00, Tony Cook via RT <perlbug-followup@perl.org>: Show quoted text
> On Mon Jun 08 04:58:25 2015, ntysdd@gmail.com wrote:
>> Sorry so late. >> I didn't test the FILEHANDLE version. It seems it's OK. >> The EXPR version fails to truncate this testfile.
> > Thanks, now I can reproduce it. > > The PerlLIO_open() was failing due to MSVCRT failing to seek on > text files. > > The attached fixes it for me. > > Tony >
To: ntysdd <ntysdd [...] gmail.com>
CC: perlbug-followup [...] perl.org
Subject: Re: [perl #125347] Strawberry Perl fails to truncate large file
From: Tony Cook <tony [...] develop-help.com>
Date: Tue, 9 Jun 2015 21:19:31 +1000
Download (untitled) / with headers
text/plain 209b
On Tue, Jun 09, 2015 at 06:42:55PM +0800, ntysdd wrote: Show quoted text
> Why is the function PerlLIO_open called twice?
Sigh, you're right. I remember typing that correctly, but I guess it was in the wrong window. Tony
RT-Send-CC: perl5-porters [...] perl.org
Download (untitled) / with headers
text/plain 201b
On Tue Jun 09 03:43:40 2015, ntysdd@gmail.com wrote: Show quoted text
> Why is the function PerlLIO_open called twice?
Thanks, I've fixed that and applied the result as d484df69ed26120321cdca989748cd4636b276bb. Tony
Download (untitled) / with headers
text/plain 252b
Thank you for submitting this report. You have helped make Perl better. With the release of Perl 5.24.0 on May 9, 2016, this and 149 other issues have been resolved. Perl 5.24.0 may be downloaded via https://metacpan.org/release/RJBS/perl-5.24.0


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

For issues related to this RT instance (aka "perlbug"), please contact perlbug-admin at perl.org