stat dies on valid filename not existing #13557
Comments
From perl-diddler@tlinx.orgCreated by perl-diddler@tlinx.org# next line uses ^V^J after 'test' to create file w/NL @ end of name
## ^^ perl dies at this point.. not a valid response to a Ishtar:/tmp> perl -MP -w -e 'use strict; Perl should not die on a stat call failure. (FWIW, the file name was downloaded from a website that had The file downloaded correctly though I think the 'nl' was Note that it's existance is verified if the file exists -- it Perl Info |
From victor@vsespb.rubut this is documented behaviour - see perldiag and this answer 2014/1/26 Linda Walsh <perlbug-followup@perl.org>:
|
|
The RT System itself - Status changed from 'new' to 'open' |
From perl-diddler@tlinx.orgOn Sun Jan 26 03:57:38 2014, vsespb wrote:
you might find File::stat's object representation to be more convenient than the list returned by CORE::stat. So you are saying that CORE::stat is broken, but it's ok because it is documented and there is a workaround? Um... You know what they say about "assume". You make an ass out of U and me? Following the specifications is far preferable to broken CORE functions. The fact of the matter was that I was replicating a valid file name, and perl can't test for that file names existence w/o dying. It's also the case that the treatment of the file name is "passed" if the file exists, but breaking a core function to put in a ***faulty***, fatal error is just ridiculous. If a user wants their filenames sanitized, a pragma might be a valid solution, but invalidating legal paths is still broken -- documented or not -- perl can't safely work with filenames because someone might get a bug up their bonnet about some OTHER character. I've seen some POSIX folks suggest that any non-printing character should be illegal (including space). If someone *wants* to santize their names, they can -- but forcing things on people because someone else thinks it is "good for them", has resulted in alot of different kinds of abuse over human history. Carving a special exception for perl to allow forcing someone's idea of right is no better than any other such situation. |
From @rjbs~$ perl -wE 'use strict; Unsuccessful stat on filename containing newline at -e line 2. Perl does not die. It warns. This is documented and not a bug. -- |
|
@rjbs - Status changed from 'open' to 'rejected' |
From perl-diddler@tlinx.orgOn Sun Jan 26 04:28:48 2014, rjbs wrote:
In many programs, warnings are not allowed. Any warnings are considered errors. If a developer or user wants a pragma to check for such things, then "buy-in" would be the way to go... then it doesn't disturb valid usage. |
From @karenetheridgeOn Sun, Jan 26, 2014 at 04:35:52AM -0800, Linda Walsh via RT wrote:
Developers must understand the ramifications of 'use warnings FATAL => "all"'
See perldoc perldiag - it describes how to suppress the warning for this |
From @demerphqOn 26 January 2014 22:46, Karen Etheridge <perl@froods.org> wrote:
Indeed. One of the ramifications of warnings FATAL is that programs Yves -- |
From perl-diddler@tlinx.orgOn Sun Jan 26 13:47:19 2014, perl@froods.org wrote:
Indeed. One must look at reasons why they do things as well. My background is kernel and security programming. It is common practice to ensure the kernel compiles with no warnings. This has roots in the security "best practices" as recommended by CERT @ I refer specifically to this section: ***Heed compiler warnings.*** *** Compile code using the ___highest warning level available for your In an update to these notes, @ * MSC00-C. Compile cleanly at high warning levels .... MSC00-EX1: Compilers can produce diagnostic messages for correct code, ... *** Do not simply quiet warnings*** ...Instead, understand the reason for the warning and consider As a result of security "best practice advice" for the past 10-15 years, it Not doing so would go against "best practices" for secure software. That many in the perl community disagree with enabling warnings and treating Willful adherence to ignorance and bad practice is a 1st order danger to the success of a project or product. Best practices specifically advise to compile with the highest level of warnings turned on, treat warnings as ERRORS, and only ship production code that runs without errors or warnings. The effect of such a policy is to ensure that the shipped code in a language that can generate new warnings on any major update, is to dissuade best security practices. Those responsible for such policies are ultimate responsible for a generally lowering of good security practice in those who use such code. To advise that treating "warnings-as-errors", and to design and develop software that encourages bad security practice is contributing to to a lowering of security practice in the users of such. Indeed, as evidence of this , one look at how many CPAN projects build and install w/no warnings. As a core language developer wrote: In other words, if one tries to follow best security practices - just to the Incompatible programming changes were supposed to be preceeded by The current maintenance team is obviously free to continue along their path I would point out that a more egregious example of turning off warnings and This makes for a more toxic environment -- affecting not just a current |
From @demerphqOn 27 January 2014 11:36, Linda Walsh via RT <perlbug-followup@perl.org> wrote:
Yes, and this is compliant with the CERT references you made. You asked for fatal warnings. You upgraded. We started warning about something that we consider to be a likely problem. Your code died. Thus exactly what was supposed to happen happened. You cant say "I want to be secure and treat all warnings as fatal" AND
Nonsense. Where does it say that?
You *vulneteered* for this paternalism by enabling fatal warnings. You specifically asked for it. And you did it for the same reason we started producing warnings. Ergo you are whining about getting what you asked for. That isn't nice.
Your position is totally inconsistent. You want to be secure, but you You simply cannot have it both ways. And your attempt to turn this into an attack on the Perl development Yves |
From @epaLinda Walsh via RT <perlbug-followup <at> perl.org> writes:
That is certainly a good idea. But Perl's warnings are a mixture of In Perl, you can make many compile-time warnings fatal using use warnings FATAL => 'syntax'; -- |
From perl-diddler@tlinx.orgOn Sun Jan 26 20:10:59 2014, demerphq wrote:
Elsewhere, though it didn't make my final draft, more than one source
When I move up from successive version of my code in gcc, I don't Part of the problem here -- that makes responsibilities in perl
Without giving advance warning as procedures regarding non-compatible
Well, this is were communication has broken down. 1) lexical $_ While some experiemental features are new -- some of the features that It was my believe that if one introduced an experimental feature in What is the point of releasing new features, but keeping them as experimental -- If I'd been aware such new features -- often released with much fanfare as Some of the features that others don't like about those features are ones
If a security relevent bug is found I'd expect an error, not a warning. Second guessing what previously working code does at run time is, IMO,
http://perldoc.perl.org/perlpolicy.html: Any language change which breaks backward-compatibility should be able to be Whether or not the new experimental warnings would be inplicit with Regardless, that would only apply to new features made automatic/implicit That you and others don't know this is **obviously**, part of the
Nope. I wasn't aware perl monitored what what appropriate data
I ask for warnings that happen when I run and develop the program --
I don't ask for warnings about *valid* data that some find Warning me about my file names or data formats are not something In my case, a rather sophisticated and cool library (Mojolicious You have NO browsers that I know of that will flag such a pathname The stat call has never had such a bug. Again-- a new 'feature' has
Neither is turning on undocumented data-checks at run time. Perl DOES document data checks on file names in the open Just because Intel documented wrong behavior of their math
You are repeating yourself. The differences are: 1) warning me about code vs.warning me about data that may never be hit, but
See above.
An attack? please. I was very careful to NOT make it one. That you |
From @epaLinda Walsh via RT <perlbug-followup <at> perl.org> writes:
I think it would be a good idea to find sources for this advice, which in -- |
From @iabynOn Mon, Jan 27, 2014 at 03:24:54AM -0800, Linda Walsh via RT wrote:
I hope you realise that gcc and perl are two *completely* different Upgrading gcc can quite possibly introduce new compile-time warnings on
The "Unsuccessful %s on filename containing newline" warning has been
Then only turn on those categories of warnings.
Because perl isn't C. You are criticising something that is in the very
Perl isn't a browser.
Then you'll know that for 19 years perl has warned on stats and similar while (my $file = <>) { without the necessary chomp, then wonder why their code doesn't work.
Its not a new feature, and it doesn't throw an error unless you ask it to.
If you don't want warnings on wishy-washy features, then don't enable
It's documented in perldiag. Which of course you've perused in fine -- |
From @rjbs* Linda Walsh via RT <perlbug-followup@perl.org> [2014-01-27T06:24:54]
I do not consider introducing new warnings to be breaking backward Turning on fatal warnings should be done selectively, in Perl, both to make At any rate, we have now left the realm of the ticket's initial report.
p5p is not closed. It is an open-subscription mailing list with public
You are, to the best of my knowledge, the only person who has been more or less
This is a bizarre criticism. Perl is not C. The same restrictions and design
Again, not so. -- |
From perl-diddler@tlinx.orgOn Mon Jan 27 07:31:11 2014, perl.p5p@rjbs.manxome.org wrote:
New warnings are an incompatible code change --- they change the
Fine... show me where the ***behavior*** is documented. I assert the behavior is undocumented -- the error is documented, I would assert that hidden and undocumented behaviors who's only
Your subscription and posting process is filtered. That
Interesting. I only started trying to use perl as a programming
Yes, again, you point out the error in my thinking -- trying to use
Maybe not, but show me the documentation that says perl will have such |
From @demerphqOn 28 January 2014 19:52, Linda Walsh via RT <perlbug-followup@perl.org> wrote:
No. This has been debated before and we decided to error on the side We might one day change this, but it wont be because of "use warnings 'FATAL'".
We document the behavior that can trigger a warning in perldiag. Here Unsuccessful %s on filename containing newline If you have not reviewed the complete list of warnings in perldiag and
In perldiag.
Sure it does. We filter spam. And because some people refuse to play The default however is to allow people to post to the list. And anything posted to the list is public. This is what "open" means.
There is no such distinction. A program is a program, be it compiled,
As I said above there is no such distinction. There are many other
So basically you are saying that you learned language X and then used Well *duh*, what did you think would happen? Do you think all C Every language, computer and human, has its quirks. You cannot expect
In perldiag. Every warnings and error that Perl can throw is If there is a real bug worth talking about in this thread is that we I will look into doing a patch to make it absolutely clear that FATAL Yves -- |
From perl-diddler@tlinx.orgOn Tue Jan 28 09:07:35 2014, demerphq wrote:
Here's where your logic is flawed. When a distro is upgraded, it is When you do want it is when the *author* (programmer) is *in* the code But whatever the new feature, enabling it when on a new installation when it If you want the warnings directed to programmers, as you claim -- then turn on That's not what you are doing though -- you aren't toggling it on when the Reallistically, you believe it best to turn on new warnings that can cause code Why wouldn't you do it when you know an author is making changes to use new The 1st way causes maximum disruption to unrelated users and unsuspecting authors. Why is causing potential harm to unrelated people considered preferable over |
From @demerphqOn 29 January 2014 01:29, Linda Walsh via RT <perlbug-followup@perl.org> wrote:
If sysadmin are running their scripts with warnings FATAL then And if you dont want to be affected by this then install a second perl The latter is actually the general recommendation for how to run
You seem like a clever person, albeit somewhat misguided at times. I
Yes I do. I/we do not recommend people write critical business logic Yves -- |
From perl5-porters@perl.orgYves Orton wrote:
I find that statement surprising, considering you are the author of |
From @demerphqOn 29 January 2014 04:25, Father Chrysostomos <sprout@cpan.org> wrote:
Bah. I thought I had done that too, but when I couldn't find it Anyway, thanks. Yves -- |
From perl-diddler@tlinx.orgOn Tue Jan 28 09:38:22 2014, demerphq wrote:
I wouldn't want my log-rotations, my file serving, my email ... and I want to be able to have a "grace period" where I can run the new Open suse, if you remember (which used to be the best, but have They also -- depending on the module, hard code the library paths for So all perl rpms, minimally will have deps on perl-5.18.2 -- where an 1.5-2 years ago with the advent of 12.1, I found I could no longer That being the case, I was rather at the mercy of whenever my distro While I do alot of my own automation with perl, -- not being able to Understand, that unlike MOST users, I am able to, *usually* rebuild
Considering I need the default perl for myself and root and some Too many programs have /bin/bash and/or /usr/bin/perl in their Anything that perl developers can do to make things easier on users That isn't what computers were created for -- they were supposed to
Fortunately, I figured out the build problem and worked around it
??? I think you missed my clever idea. Perl wouldn't change the When authors/designers/programmers get into the code and want It's not perl that makes the decision. By being "clever", you That's why you only add new features on an opt-in basis -- so Thinking of the user is of prime importance for me. I'm shocked The president **gets** to serve. His satisfaction comes from
Sadly, Ricardo agrees with you -- pointing very differing priorities I would have preferred perl be upgraded to being less quirky and |
From @demerphqOn 29 January 2014 10:53, Linda Walsh via RT <perlbug-followup@perl.org> wrote:
Thanks Linda. Your position is noted. We have already documented the following in perllexwarn: +B<NOTE:> Users of FATAL warnings, especially those using C<FATAL => 'all'> We will update that note with a mention that all warnings and errors And with that, we will close this ticket as "wont fix". Thanks for your report. Yves -- |
From perl-diddler@tlinx.orgOn Tue Jan 28 19:03:36 2014, demerphq wrote:
Doesn't anyone prune what they are responding to anymore?
perllexwarn *** feels like *** an appendix to me -- not My question to ricardo went un-answered. Where is this *behavior* of checking file names against The fact that "open" will prune spaces in the 2 arg form is If this was supposed to be a feature, why isn't it documented Only documenting warnings and not the tests themselves makes Usually "undocumented behavior" is considered a bug. That I.e. if you want to keep this wart in perl, it should be documented Are there other formats that also trigger warnings? What other commands have undocumented behaviors and I mean Or is documenting these quirks with each command they are implemented in
Actually, if you don't want warnings spewed out all over the place, If I see a warning out of code, I think something is broken. If perl will randomly cause warning messages to appear in new I.e. "no warnings" needs to be at the front of programs to protect FWIW, I keep warnings on and fatal, as I don't want the program to (sorry for the reply, but I didn't see the question about |
From @demerphqOn 29 January 2014 14:34, Linda Walsh via RT <perlbug-followup@perl.org> wrote:
perllexwarn is the *standard* documentation for warnings. I dont know
I answered this already. It is documented in perldiag. Unsuccessful %s on filename containing newline That specifies that there is a warning "Unsuccessful %s on filename If you don't like this format of documentation then you can post a However you cannot keep ignoring that this IS DOCUMENTED, and asking We owe you nothing. Nothing at all. We are volunteers working on something we love, and we do not have any Repeatedly demanding that we do fundamentally mistakes the nature of You are a consumer of something we do for free. We owe you nothing for being a consumer of our work. If anything it is the contrary, by using our efforts for your own That is both disrespectful and plain old rude.
So what?
Because this warning can fire in many circumstances, specifically any That includes many different operations, and were we to document it in
You keep ignoring that we document the behaviour. That you don't like
I dont know what a "errata" section is. perldiag is the place we If you chose to use warnings FATAL without thoroughly reading perldiag
No, Sorry. As I said before, if we took that approach then our docs On the other hand if you wished to file a documentation patch we would
I dont know. I have never had the need to read perldiag and see. I
Im not here to help you avoid reading the docs. I am here to improve Your rejection of our documentation is completely irrelevant and irrational. Furthermore as I have said before WE OWE YOU NOTHING. Demanding that
Yes it is too much to ask. It is inefficient of the most precious
If you think that is a good policy then please go ahead. I personally
Ok. Thanks for letting us know.
You are free to take whatever actions you wish, that includes other
I am unfamilar with what you refer and have no interest in
I am unaware of any requirement or corruption. If you can produce a
Your opinion is noted. Thanks for sharing.
See perldiag. This is the last post I will make in this thread. Yves -- |
From @doughera88On Tue, Jan 28, 2014 at 06:53:16PM -0800, "Linda Walsh via RT" wrote:
I recall quite a bit of back and forth where folks on this list provided
It is a gdbm bug, so directing the bug report there was appropriate. -- |
From @epaLinda Walsh via RT <perlbug-followup <at> perl.org> writes:
I feel the same way, which is why I run my programs with warnings enabled.
Well maybe... or perhaps better to arrange for warnings to be sent to the
The problem of mixing up program output with diagnostic messages is an old
In that case I would suggest use warnings FATAL => 'uninitialized'; You may also want to use use warnings FATAL => 'syntax'; to make all syntax (compilation) warnings fatal: this is the rough I would not recommend making *all* warnings fatal; the distinction between -- |
From perl-diddler@tlinx.orgOn Wed Jan 29 05:22:11 2014, doughera wrote:
My memory isn't what it used to be, but I don't recall any help from While I mentioned I had problems building perl -- no advice from the I submitted a patch to the gnu folks and while the opensuse people I do remember advice on a coredump problem in #78728 where people pointed use mro 'C3'; After that was removed, no more core dumps to this day. But if people hadn't convinced me how unreliably CPAN was viewed, I A year after the gnu-red-herring, I found another problem in a dedup I think I've worked around that one by using sysopen/read and not pointing claiming some perl-core-dump was was in
I seem to remember that, but discarded it working, sicne both the
If it would fix the whole problem, that wouldn't be a bad idea, but It's not that I didn't think it through. I DID submit a patch to the (only responding to your note at his time..) |
From @karenetheridgeOn Thu, Jan 30, 2014 at 08:30:29PM -0800, Linda Walsh via RT wrote:
The words you are missing here are "thank you". |
From @ysthYves wrote:
"the filename contained a newline", and the "operation failed, |
From perl-diddler@tlinx.orgOn Fri Jan 31 00:36:41 2014, sthoenna@gmail.com wrote:
|
From perl-diddler@tlinx.orgOn Fri Jan 31 00:36:41 2014, sthoenna@gmail.com wrote:
But that is not the case. The filename isn't wrong and has a newline. It's only test-actions (including open/read on non-existent filename) Notice one only gets warned on the test operation and then only if it ==== my $fname="name"."\n"; P "file created"; 1st time output:
2nd time output:
-rw-rw-r-- 1 25 Jan 31 04:51 name Note that the Camel book makes no mention of such behavior -- |
From @kentfredricOn Fri Jan 31 04:57:43 2014, LAWalsh wrote:
Yes, thats because the linefeed character is a legal character in many platforms and filename. Thus, if you stat a file with a trailing "\n", and it exists, then there is a clear indication that you meant to do that, and no warning is emitted. However, if you stat a file with a "\n", and no such file exists, there are 2 possibilities: 1. You meant to find a file with a trailing \n, and it does not exist. A warning exists, because case #2 is the most probable. But in the event you wanted case one, then thats a legitimate use for { In essence, when you get a warning, it is not a _certain_ indication of something that is wrong. Its just a 50/50 "you might want to pay attention". And legitimate solutions include: - Solving the problem that caused the warning to be emitted. |
From perl-diddler@tlinx.orgOn Fri Jan 31 05:16:05 2014, kentfredric wrote:
I don't know if I would call it a problem or not. It was the name of a file passed to me by Mojolicious that was parsing HTML that had it in something like: My code was checking to see if it was in a cache, and if not, then caching it -- so one way or the other, my code didn't care -- except that --- because it was a long running program that dump status of what it was doing ... had dumped ~100K lines and not sure when it would have stopped -- any warning spit out wouldn't be seen among the rest of the messages UNLESS it stops the program -- like an 'undef', that got returned for content in a few cases. Wouldn't have stopped the program -- w/o dying on warnings. The undef was a result of it literally reading in 'nothing' for content due a link getting a 404 or pointing to a non-existent site. But a warning for valid data as returned from the site... and the warning not being documented in with the affected calls -- only able to be found in the error text? Other than developers of perl5, I would be money that the number of people who read perldiag for each release from start to finish would fall under 10% if you count casual users of perl -- probably < 1%. Providing documentation for such behavior in a place where it won't be seen by most people is bad design. Example ---- say I want people to do "X" -- like turn off a light in the garage. I can install a camera. I can tell folks or put up a sign that not turning off the light will entail a warning -- because it will be caught. Or I can not tell them and berate them after the fact. Can you tell which I think is the better option? I opted for neither approach -- I installed it on a 5 press timer, where each press gives 10 minutes more of light, pressing it 6 times and holding it for 10 seconds will disable the timer. Problem solved -- I don't have to play 'heavy', they don't have to be 'wrong', light goes out all the time (no one has ever accidently left it on)... For warnings -- if I don't give a traceback, I usually have no clue where they happened, and if I don't stop the execution, I often will have no idea that anything noteworthy happened. It's a case of something crying wolf that wasn't warned of up front that I thought was bad (not to mention halting execution -- which it has to do or be missed). Along those lines -- it would be more useful if perl used a system-log that could be reviewed periodically like many other system installed programs. |
From @iabynOn Fri, Jan 31, 2014 at 08:29:13AM -0800, Linda Walsh via RT wrote:
Why on earth would you want to re-read perldiag for each new release? Any
So for example, the "Use of uninitialized value" warning. Where would you
If you want that, then use a logging module, and use $SIG{__WARN__} -- |
From @kentfredricOn Fri Jan 31 08:29:13 2014, LAWalsh wrote:
For this specific case, this is an example of why warnings should _not_ be fatalised, but only fatalised in development. And in this case, I'd be inclined to opt for the lexical `no warnings "newline"` approach. In either case, the user uploads a file with invalid filenames encoded in XML, its their responsibility to get it right. Communicating the effect of this warning to the user however is up to you "somehow". You can either - Ignore this warning entirely and simply tell the user "that filename does not exist", and potentially confuse them because they don't know XML strings keep whitespace. I'd probably stick with #1. You'll have users come to you with "Um, this file doesn't work", and you'll then have to work out why, and it will be in wherever the warnings get piped to.
Just the thing is, in terms of the functionality it warns about, understanding the root cause is really something that aught to be documented, not in Perl, If you specify a fixed length string to an IO system that takes fixed length strings and uses them verbatim for file mechanics, you will get exactly what you asked for. Even if it is not what you intend. As such, the warning is not a chastisement, or an instruction to the user as to how they may or may not behave, but a heuristic, a marker that "hey, this is a bit weird, are you sure this is what you want"? If that is what you want, suppress or simply ignore the warning. ( And its only made worse for you by the fact you made the warnings so you cant simply ignore them due to fatalization ) A better analogy is not your lighting problem, but a smoke alarm. Smoke alarms are pesky things. They don't take any action on their own, they exist If you're near one, and consuming some smoked substance such as tobacco, or cooking something particularly smoky, you're likely to trigger such a sensor, even though there is no real threat present. You have 2 such options when such a sensor alarm goes off. You observe, and respond to the sensors indication ( ie: in the case of the fire breaking out, ... you want to stop that ) Or you can ignore/repress the sensors indication ( some senors have a "temporarily disable" feature , or you can rip the battery out, or you can get really heavy duty earmuffs, or you can just sleep through them screaming if you're really lucky ) Fatalised warnings however, is hooking up your smoke alarms to a fire suppression system. In some cases it calls for it, usually when you're testing that the fire suppression system in fact works... but not in general cases. Would be bad to have an entire office destroyed by mere water damage because somebody didn't realise they couldn't have a smoke within 10m of some doorway.
There are alternatives here. Warnings don't normally give much traceback, because a very large amount of Perl is cli oriented, so a 500 line traceback in the middle of execution for something that can be ignored is pretty bad. However, if this is what you want, there are ways of making that happen: https://metacpan.org/pod/Devel::Confess ^ Now, each and every warning has a complete backtrace. Each and every exception does too, even if it was caught!
This at very most means an improvement to the documentation of perldoc -f stat and any other applicable functions that /may/ invoke that warning/ But even there, nobody would likely see it. Mostly, because people are not looking for things until they need to know, and you never expect you needed to know about this case. So no obvious entry point would have prepared you. Most people only respond to warning messages when they see them unexpectedly, and they encounter the warning because they were not able to anticipate the scenario that occurred to make the warning happen. Which is why people use FATAL warnings when _testing_, so that they may find any warning sources and eliminate them where possible, and then use non-fatal warnings in production, in conjunction with reading messages sent to STDERR
There are CPAN modules that provide such a feature, and some propagate warnings / errors to the various logging systems, but the complexity of that is a little too much to be a language-level construct. Even in heavyweight languages like Java, printing to STDERR is still rather common for CLI things, and proper logging tech like log4j is not employed by everyone. (Heck, some of the time you're lucky to get it in STDERR, some people log to STDOUT :/ ) For web-services, many web toolkits have facilities to flow errors/warnings into server log files out of the box. But if you're writing something yourself, or just enhancing existing code, a quick scan reveals things like https://metacpan.org/pod/Log::WarnDie , which proxies native warn/die handlers to a sophisticated logging tool of some kind. And if you're doing anything web dev using the Plack stack, there's this middleware : https://metacpan.org/pod/Plack::Middleware::LogWarn And if you just want warn/die to flow into Syslog, there's this https://metacpan.org/pod/Carp::Syslog So you can do exactly as you request right now. Just not with out-of-the-box components without some glue. The closest you'll get to what you want out of the box without CPAN deps is either a) funneling log output to STDERR to a file with shell redirection |
From @AbigailOn Fri, Jan 31, 2014 at 05:16:05AM -0800, Kent Fredric via RT wrote:
Perl could increase the chance of 2. happening instead of 1. If a file Abigail |
From perl-diddler@tlinx.orgOn Fri Jan 31 08:58:47 2014, davem wrote:
There have been changes that were documented in perldelta's for minor
Perlop or perlvar. Though in that situation, almost any use of an The filename passed to the one- and two-argument forms of open() Yet ... lets see...
What do ya know... This IS a bug. FWIW, it doesn't give the warning in the case of "+<" nor ">". But regardless of the op, the white space is stripped. But in the case Conversely, in the 3 arg format where whitespace is not stripped: perl -we'use strict; open(my $fh, "<", "mytest There is no warning. (and the white space really isn't stripped That's doubly broken. It gives a warning where it is
As was wrongly pointed out, I might add -- it's only trigged in But the warning in 'open' is broken. My druthers: since open is broken in 2 ways -- the test would have Given that and their being only 1 other place it is used: it would Given that no one has missed it on the 3-arg open (though it is Should this bug be reopened, or should 2 new bugs be opened? -- i.e. |
From @rjbs* Linda Walsh via RT <perlbug-followup@perl.org> [2014-01-31T16:45:07]
No, not 5.8 (2002), but 5.000 (1994). -- |
From @b2gillsI wanted to discuss the points brought up in this post as On Sun, Jan 26, 2014 at 9:36 PM, Linda Walsh via RT
The problem with this is that C errors are similar to Perl syntax errors,
The reason they don't recommend disabling any warning in C is that you Actually reading that document, C doesn't even have a way to make sure Most Perl warnings are more akin to static and dynamic analysis tools, Perl also helps in security practice number 7 by warning you when you Perl can't help with number 8, as it says to use It would be difficult for Perl to help with the rest of the top 10 secure
That's best practices for secure C software. The best practice for secure Perl software is to to only fatalize some warnings
The success of Perl software is sometimes how well it responds to For example I have a piece of software that parses free-form html That warning in this case has NOTHING to do with security as it is caused by
The only C-like warnings that Perl can detect at compile time are the ones What they meant with "treat warnings as errors" is that you should fix Actually sometimes the warnings only happen in production, so how would you fix
The two big categories of warnings we have added recently We currently have no other way to programmatically tell programmers That is we are "warning" them about a future problem, so we add it to the
Again you're mistaking static language security practices with Perl
Actually since adding `use v5.12` to your code enables strict mode;
One of the warnings categories is `deprecated` so by fatalizing every warning
Incompatible changes are preceded with announcements and a deprecation Again Perl warnings are indicators of POSSIBLE problems, whether
We do NOT ignore ANY input, we just sometimes decide to do something other Actually if we ignored "input contrary to what we wanted to hear"; there
You don't have to turn off warnings, you just have to not fatalize them all.
We have made missteps in the past, and most assuredly will continue to
The ramifications of not fatalizing absolutely every warning; are far I'm sure that almost every expert level Perl programmer would agree with me And actually Perl warnings are fundamentally different than C warnings, |
From @ap* Linda Walsh via RT <perlbug-followup@perl.org> [2014-01-31 22:50]:
“Ricardo says”? Here’s the 5.000 commit in perl’s repository: Here’s what you’d found in line 1848 of perldiag if you opened it 1994: (I went with the repository because I couldn’t find a full series of It took me 5 minutes of looking in the most immediately obvious places It doesn’t matter what Ricardo says, or that it’s Ricardo who is saying Regards, |
From @craigberryOn Fri, Jan 31, 2014 at 5:02 PM, Brad Gilbert <b2gills@gmail.com> wrote:
Not disagreeing with the overall thrust of your argument but there are |
From perl-diddler@tlinx.orgOn Fri Jan 31 15:53:07 2014, aristotle wrote:
The point was not who said it, or if it was in 5.8 or 5.0 -- the point was that it's been buggy since it was implemented. The check in open is wrong. The check in stat is 'questionable' -- since it is not documented as "behavior" -- something that can guide you as to what will happen if you do "X", but as a "diagnostic" message -- something that tells you something **after** you have encountered the problem. Behavior *predicts*. Diagnostics explain. The difference in meaning is why I said and say this is undocumented *behavior*. Example. I can read about alcohol and it's effect on behavior and know that it can cause problems. A breathalyzer, blood or urine test are examples of diagnostics. To figure out how someone is going to behave if I know they might be drinking at a party would be to read about the effects of alcohol on behavior. No one in their right mind would start with a breathalyzer test, before someone started showing problematic symptoms. Reading diagnostics don't help avoid the issue in the first place. As doctors how well reading a clinical diagnostic manual is in preventing health problems. You don't read diagnostics to prevent problems, you look for documentation on implemented behavior. In any event... |
From @ikegamiOn Fri, Jan 31, 2014 at 8:04 PM, Linda Walsh via RT <
perldoc perldiag
The tests are in open, stat, etc. Using use warnings is a decision to use tests tests. The string emitted to STDERR is the test results. |
From perl5-porters@perl.orgKent Fredric wrote:
Sounds like my cooking!
Not even to one's lungs? |
From perl-diddler@tlinx.orgOn Fri Jan 31 15:03:03 2014, brad wrote:
Perl is lucky in that it has a canary 'undef' value that can flag
I remember pragma's 20+ years back to do just that. They are usually
Sort a "unsanitary": the code was trying to replicate an existing The input in question is NOT *INVALID* -- from the standpoint that
The warnings keep changing -- code that works today w/o warnings may If you are the CIA or NSA, you want fail "closed". I am not thinking of security as much in terms of violations of If I am writing programs that need to run as root because they are
And one that crashes losing all user data doesn't make much at all. Wasn't it Target, that just recently had a score-million credit Discover used handle all the time costs of switching a lost When the onus for contacting merchants (many online) reverted If merchants w/insufficient security had to pay out of pocket for all §§ With my home server, I've had an opposite stance on the issue of Generally, I have tended toward the side of caution, but foolish A simple dedup program "ate" a terabyte or two of data -- that One of the programs that fails w/5.18 (and a reason I haven't That issue still needs to be addressed: If experimental features Of course that still doesn't address issues when previously
In my programs, having die at the first point of failure with
Depends on the SW, if it is critical enough, it might be best So, inherently, looking to CPAN for effects of unanticipated
I thought deprecation warnings previously existed (?)...
Actually it points out the uselessness of having code that
That's why to use the new features, one had to say "use 'feature'",
1) a future that may never come, and 2) when they use the
That's part of the problem -- thinking that that what passes for In that vein -- just as "use version" allows specifying a minimum
Am aware of that and started using that as a shorthand -- then
This was one of my complaints -- *I* DO read the deltas when I
That's why I asked for a site-wide switch and for such to ~/bin> find . -xdev -name RCS -prune -o -iname \*.orig -prune -o -iname \*.bak -prune -o -type f -print0 |xargs -0 grep '^#!/usr/bin/perl' -n |wc -l And that's just my home-bin directory. Divide that in half
Just to clarify -- I can't post to the list -- I can only file Anyway I find I am going too far off topic (several paragraphs Thanks for your comments. Yours, at least seem to be |
From @b2gillsOn Fri, Jan 31, 2014 at 11:57 PM, Linda Walsh via RT
Ok you got me; I don't know a lot about C. If I had to pick a low-level language, it would be D.
It's unlikely for there to be a file with a newline in its name. At no point did I say it was invalid. Normally the only time stat would get a filename with a newline That it warns is perhaps the best thing it could do. If you are certain that you don't have a problem in that code,
Imagine a some software that modifies a file only to have it stop Imagine a three letter organization is using Perl to find a person of interest. print $LOG '$some_scalar: ',$some_scalar if rand > 0.99; Now that person got away. Imagine a web site that has an API request that only accepts numbers Sometime later they have a meeting and determine that if that API So they just leave the line that was added, and call it done. Now they have to report the incident because they were "hacked". If only they had just logged warnings instead of blindly fatalizing them all,
If your server always goes down shortly after a hacker attacks it in Security through obscurity is better than no security at all. By fatalizing all warnings you have removed the obscurity. So everywhere your security wasn't absolutely perfect,
That was because they were using unpatched Windows XP If it were from Perl code, and you fatalized then the POS device If instead you logged all of the warnings, and sent them to a central
Even if they did have to pay, it still might make economical sense to Back in the day Ford did a cost-benefit analysis of adding a couple They determined that they could save more than enough money to They only saved $11 from each Pinto they produced. ( search for "ford pinto case study" )
I wrote a simple dedup program once, I didn't have a problem I've also written quite a few little programs that just modify a file
Actually popular opinion is that given/when should be changed, not removed.
I think I would rather have it shut itself down cleanly than Also it may be easier to find out what went wrong if you can
Actually we only look for CPAN modules that are broken,
I was remembering wrong, I was probably remembering something
If it wasn't in a stable version of Perl, then no one would use it. Basically almost every added feature from now on is going to be experimental ( At least that is my impression of the situation )
That serves a completely different purpose. For example if someone wrote a subroutine named "say" Any code that doesn't ask for a feature will continue to work as it always had.
I'm not saying they don't generally apply, only that how you apply the What they tell you to do is eliminate all of the warnings before
Actually there is a way to do that now no v5.16.4;
I'm just saying that we want to surprise as few people as possible Also if we did wait a release before we warn, that would delay
"exports PERL5OPT=-M-warnings=deprecated" I actually have more than one version of Perl installed so that if I want perl-5.14.1 -E'...' perl-5.14.1 is a symlink in /opt/perl/bin to /opt/perl-5.14.1/bin/perl It does mean that I have to run cpanp at least once for every I've been thinking about writing some code to do this for me, So if you did that you could just change the shebang line to point
I was aware of that, I was referring to the posts on RT and As to the openness of the list, they have listened to what I have
I don't think we are ever going to see eye to eye on this, so |
From perl-diddler@tlinx.orgOn Sat Feb 01 03:29:49 2014, brad wrote: Let me start by saying I probably won't reply to each point --
I'm sure there are some But Linux and NT aren't them It's a great way [sic] for both malware and DRM (actually,
I'm certain it is properly lookintg for and would create a
Now that's another discussion -- I try to write my sofware
That's why you limit the # of transactions 1 instance of the prog
The needs of external security programs are different than those
That would be bad, but if it goes down more than once due to the When my mail filter worked better -- (it's more held together For some reason it doesn't kill itself like it used to - but in
Some, but not all.
Maybe, maybe not -- if the old system of killing
When I first went from XP to Win7, Win7 ate my disk at least
In my case, not. The defective SW falls back to fail-safe
I got warnings coming out my ... pores! I almost never have time
(Ford-Pinto bombs noted!) Human life is not valued very highly -- but the bank and
The actual linking and deleting takes most of the time:
So spaning the tree took 3.2s, Just the 1st 2 steps on my home dir: --- (program still 'under constrction')
Ditto. to link a->b, it creates hard-link tmp name in dir where 'b' In debug mode, the files are compared twice -- once with my
I was refering more to "my $_" -- given/when were listed as
Well, middle in my case is in writing to a tmp file, so
Um... getting there: Ishtar:law> ll /home/perl Not quite fleshed out fully, but just started doing that
(or my path or symlinks in /usr/bin...)
Considering the similarities in handling many things, Forgive any braino's -- I was sleep typing most of |
From @kentfredricOn Fri Jan 31 21:57:47 2014, LAWalsh wrote:
Actually, the defacto behaviour for Browsers is the same as the defacto behaviour for perl. 1. When there is an error in the HTML parse, a warning _is_ emitted. Sometimes the warning is serious enough that you get output on the terminal you opened your browser from. Other times you need to find the "developer console", which will for most web pages, log any number of possible warning conditions that could be indications of runtime problems. 2. Like Perl, browsers do not fatal. You can augment it somewhat I think to make it fatal, but nobody really does this. 3. Like Perl, browsers keep going as best they can, because fatal exits are entirely unwanted. A "popup" and a "warning" are 2 very very different behaviours and you really aught not to conflate the two. A popup is a blocking interface that _demands_ response. A warning is simply a message designed in such a way that you may ignore it if you so desire. Fatalising this warning is user error if dying on petty things that might not be a problem is unacceptable. You're not going to complain "Well, those browsers should be adjusted so the warnings don't happen", the warnings exist for a reason, and they're there for the utility of people who care to look. You're not going to successfully argue that those warnings should be fatal, because 90% of the internet would spontaneously be off limits to you. And you're not going to successfully argue that those warnings should not be warnings, but should instead be documented behaviours, and the warnings themselves should be documented, for a great plethora of reasons. And worse, and this is important: Even if it was documented to behave as such, the odds of you reading the documentation and seeing the notes about the possible warning in advance of the warning occur are so low, its negligible. And if you documented it absolutely everywhere it was relevant to do so, the documentation would become obscene and unreadable due to the substantially increased context one would have to learn to read and read selectively. For instance, methods like 'require' in perldoc perlfunc are already 196 lines of text for that one function. Are you telling me you read and understood every line of that prior to ever using require? `use` is another 124 lines, have you read and fully understood each and every line? Have you read all 417 lines of the documentation for `open` ? Additionally, have you read all the documentation in perlport about the `open` function, something that is suggested at the bottom of the `open` documentation? I'm not saying you most certainly haven't, but I would imagine most people haven't, because many/most people are only interested in what is relevant to them, and will skim for context that matches their mental keyword scanner. And that mentality is very useful, but its somewhat blind to seeing things that they don't yet know is relevant to them. Which is why warnings occur: Because we accept that you're not able to know everything, and we accept that being human, you are fallible, and some of the time your failings look like intent, and vice versa. Hence, we have heuristics for known, common problems that people face, so that they can avoid these problems. In essence, there is literally _nothing_ you can do to improve the documentation to prepare people for these issues. If they don't read the documentation, or the updates to the documentation, they can't be helped. Yet, they will still encounter side effects of what they do, things the never expected would be a problem, and the heuristic lets them know "Hey, this could be a problem", and the diagnostic data gives more in-depth to the kind of problem expressed. And worse, even people who _know_ about the problems they can face, _still_ make the mistakes that these warnings exist to guard against. These people will realize, upon seeing the warning "whoops, I made that mistake again, time to fix". And again, for these people, no documentation enhancement will help. They know its a problem. Just they don't know they're causing the problem at the time they cause it. I don't mean to say "no documentation at all" here, just I'm rather aware that sometimes, far too much documentation is identically problematic as not enough. You can improve on this with formatting and layout sometimes, but its _really_ _really_ _hard_. If one was to document this specific warning case in every user facing function that could trigger, I'd suggest that instead of every function documenting it individually, the functions should say : "NB. This functions internals are defined in terms of X, Y and Z, and anything that is applicable there is applicable here too", and then document the propensity for the warning at the inner most callpoint. At least, you could say then "Its documented", just the documentation would be a little hard to read. I'd really love to have some sort of inheritance aware documentation viewer that rendered perldoc for things /including/ their deeply nested implementation documentation in a close proximity, but thats something we just simply do not have yet. |
From perl-diddler@tlinx.orgOn Sat Feb 01 06:58:10 2014, kentfredric wrote:
There is a difference. The warnings in browsers goto a brower-specific, separate console. That's the crux of the problem w/warnings in perl. If you don't Note -- browser errors, like page not found, are sent to a Apart from security -- just NOTICING the message -- or being
Aside from my browser crashing on some fatal problems (1/2 ;-), It used to be the case (don't know about now), that IE would,
Um... browsers just don't work in the face of errors. I click FF -- when something gets stuck in an infinite loop for more than
Maybe -- if the popup grabs focus, then yes -- but if it doesn't,
Not always with browsers. They have popups that require responses -- "Although this page is encrypted, the information you have entered is Used to see that all the time on some microsoft web pages where it
People filed a bug on that behavior, because unlike many similar
Much of it is. I run w/NoScript AND Request Policy (allows If I determine that something a warning, warns me about, isn't For perl, a parallel mechanism, might be to allow whatever I.e. the policy of adding new warning features unless the user
Ignoring warnings in javascript, will, more often than not, cause How often (or how many examples) can you think of where each program
To operate a browser, I agree. To program javascript to operate with
Much of the require and use documentation is new. However, I would The perlport stuff -- if I am looking to make something portable,
Agreed.
Agreed, BUT I was aware of open striping blanks from ages ago --
There are only 2 areas or functions that have this warning Under stat, you would have a warning that embedded newlines in Under lstat, you would have a similar warning (see warning about Under the -X file calls: since all of these use the stat function, If you look at all the cases documented for the "-X" calls, that If you really think that is confusing. Compare it to the length Second -- if you are talking heuristics... it should likely my $fname="test. looks less accidental than my $fname="test Third -- the check on the open call is just wrong. It
It's called HTML. But for line-oriented manpages, you could go for |
From @AbigailOn Sun, Feb 02, 2014 at 01:17:49PM -0800, Linda Walsh via RT wrote:
Program output goes to STDOUT. Warnings go to STDERR. Those were different channels long before Tim Berners-Lee invented the web. Abigail |
From perl-diddler@tlinx.orgOn Mon Feb 03 01:43:56 2014, abigail@abigail.be wrote:
Eh? Different channels? As we've noted in this discussion, in And perl's runtime (because perl is a compiler, linker and a runtime), |
From @iabynOn Fri, Jan 31, 2014 at 01:45:07PM -0800, Linda Walsh via RT wrote:
Agreed, I don't think it should warn in this case.
That's because it only warns on read-only file access.
Which I think is a bug.
Which I hope we both agree is the correct behaviour. -- |
From perl-diddler@tlinx.orgOn Tue Feb 04 04:29:51 2014, davem wrote:
Right -- just noting, RO & file not existing.
Given that the 2 arg for says to use the 3 arg to avoid white space I would also point out that inserting sach a warning at this point
Yup Which leaves the 3 2-3 lines comments I added avove documenting However, given that it is only tested in 1 place and only when the file Wouldn't it be a sign of wisdom to take this opportunity to remove this |
Migrated from rt.perl.org#121085 (status was 'rejected')
Searchable as RT121085$
The text was updated successfully, but these errors were encountered: