Q: Why Perl warn() or other STDERR output is not shown/logged/saved/flushed into my log file?
A: You may have encountered the well-known feature of stream buffering which is enabled by default.
An excerpt from the perlvar documentation says that “…STDOUT will typically be line buffered if output is to the terminal and block buffered otherwise”. Thus it is always buffered, also for STDERR.
Usually people remember to set STDOUT as auto-flush, but you should enable this for STDERR as well, or else your messages to STDERR may not appear in your log file immediately, if you are redirecting STDERR to a file.
The following piece of code sets an auto-flush for both STDOUT and STDERR:
select(STDERR); $| = 1; select(STDOUT); # default $| = 1;
The select() function and the $| variable are built-in for Perl and require no additional libraries to be included.
Alternatively, you can also use IO::Handle to achieve the same result:
use IO::Handle; STDERR->autoflush(1); STDOUT->autoflush(1);
I never realized why stream buffering for both STDOUT and STDERR is enabled by default for most scripting languages… But that’s just me.
References:
- Flushing Output
- search Google for “perl flush stdout” or “perl flush output” or just “perl flush“
September 29, 2014 at 3:37 am
Nice tips.
Regarding “you never realized why buffering is enabled by default”
When the output is to a terminal, they think you want to see the action immideately.
When the output is to a file or piped to another program, they think you want maximum performance. Flushing costs.
October 2, 2014 at 9:50 pm
Hi David. You don’t really see the output even in the terminal immediately. It’s “line-buffered” which means that you see it when a new-line is encountered, or you eventually reached the maximum block buffer size.
While this behavior is rather standard for STDOUT, it isn’t for STDERR. You can review the man page notes at the very bottom: http://man7.org/linux/man-pages/man3/stdout.3.html
Still, Perl decided that STDERR should also be buffered.
Hmm. Now that I test this with Perl 5.14, I can no longer reproduce this non-standard behavior. Currently, STDERR is unbuffered, as expected. Maybe my tests when I wrote the article weren’t correct, or Perl fixed something.