Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deamon exits on ftp timeout #818

Closed
p5pRT opened this issue Nov 5, 1999 · 4 comments
Closed

deamon exits on ftp timeout #818

p5pRT opened this issue Nov 5, 1999 · 4 comments

Comments

@p5pRT
Copy link

p5pRT commented Nov 5, 1999

Migrated from rt.perl.org#1746 (status was 'resolved')

Searchable as RT1746$

@p5pRT
Copy link
Author

p5pRT commented Nov 5, 1999

From rah@atc.ll.mit.edu

I have written a perl daemon that does ftp on an hourly basis, downloading
files to my computer. I am using the GBAR ftp module. Every time I get
the following message the daemon exits without crashing or leaving any trace.

Timeout at /opt/perl/lib/site_perl/5.005/Net/FTP.pm line 422

I have contacted GBAR, and he says that this timeout is due to the ftp
site not responding, and is not a problem with FTP.pm. I think that any
error in a module called from a daemon should not cause its exit, and
should be recoverable. Included are the scripts to ftp and start the daemon.
These should easily runnable from your site, but it downloads a lot of data
every day. Sometimes it is several days before an exit.

#!/bin/ksh

set -v

ftp_ruc --h "140.90.6.103" --d4 "/ncepe/pggrib" --r4 ".*$" --d8 "/ncepe/p211grib" --r8 ".*$"

#!/opt/perl/bin/perl -w

#
# File​:
# ftp_ruc
#
# Description​:
# ftp login to remote host and download files matching PERL regular expressions.
#
# Options​:
#
# Example​:
# ftp_ruc --h "140.90.6.103" --d4 "/ncepe/nggrib" --r4 ".*$" --d8 "/ncepe/p211grib" --r8 ".*$"
#

$, = " ";

use Getopt​::Long;
use Net​::FTP;
use POSIX;

@​mon = (Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec);
open STDOUT, ">>ftp_ruc.log"; $| = 1;
open STDERR, ">>ftp_ruc.log"; $| = 1;

GetOptions(
  'h=s' => \$host,
  'd4=s' => \$dir_40km,
  'd8=s' => \$dir_80km,
  'r4=s' => \$pre_40km,
  'r8=s' => \$pre_80km);

# strip trailing slash
$dir_40km =~ s{/$}{};
$dir_80km =~ s{/$}{};

# set up program as daemon
$pid = fork;
exit if $pid;
die "Couldn't fork​: $!" unless defined($pid);
POSIX​::setsid() or die "Can't start a new session​: $!";

# set up handler to use INT, TERM, or HUP to kill program
$time_to_die = 0;
sub sig_handler
{
  $time_to_die = 1;
}
$SIG{INT} = $SIG{TERM} = $SIG{HUP} = \&sig_handler;
$SIG{PIPE} = 'IGNORE';

# execute daemon loop, waiting approx. one hour between ftp's
$t1 = time();
until ( $time_to_die )
{
  $ct = ctime(time());
  chomp($ct);
  print "$ct​: ftp $host\n";
  if ( $ftp = Net​::FTP->new($host, Timeout => 600) )
  {
  $ftp->login() or do {print "$ct​: login​: $ftp->message\n"; goto QUIT;};

  $ftp->binary() or do {print "$ct​: binary​: $ftp->message\n"; goto QUIT;};

  @​t = gmtime();
  $d = sprintf "ruc2a.%02d%02d%02d", $t[5], $t[4]+1, $t[3];

  # build remote dir path
  $rdp = $dir_40km . "/" . $d;
  $ftp->cwd($rdp) or do {print "$ct​: cwd​: $ftp->message\n"; goto QUIT;};

  # build local dir path
  $ldp = sprintf "/ldata/wolf/19%02d/%s/%02d/40km/$d", $t[5], $mon[$t[4]], $t[3], $d;

  # mkdir the local directory to hold ftp'ed data
  not `mkdir -p $ldp` or die "mkdir​: 40km - unable to make dir.";

  # ls remote dir, match against $pre_40km, and get files
  @​ls = $ftp->ls or do {print "$ct​: ls​: 40km​: $ftp->message\n"; goto QUIT;};
  # print @​ls, "\n";
  if ( @​ls )
  {
  for $file ( grep { /$pre_40km$/o } @​ls )
  {
  # match files that do not currently exist
  $f = $ldp . "/" . $file;
  if ( !(-e $f) )
  {
  $ftp->get($file, $f) or do {print "$ct​: get​: 40km​: $ftp->message\n"; goto QUIT;};
  print " $f\n";
  }
  }
  }

  # build remote dir path
  $rdp = $dir_80km . "/" . $d;
  $ftp->cwd($rdp) or do {print "$ct​: 80km​: cwd​: $ftp->message\n"; goto QUIT;};

  # build local dir path
  $ldp = sprintf "/ldata/wolf/19%02d/%s/%02d/80km/$d", $t[5], $mon[$t[4]], $t[3], $d;

  # mkdir the local directory to hold ftp'ed data
  not `mkdir -p $ldp` or die "80km​: mkdir​: unable to make dir.";

  # ls remote dir, match against $pre_80km, and get files
  @​ls = $ftp->ls or do {print "$ct​: 80km​: ls​: $ftp->message\n"; goto QUIT;};
  # print @​ls, "\n";
  if ( @​ls )
  {
  for $file ( grep { /$pre_80km$/o } @​ls )
  {
  # match files that do not currently exist
  $f = $ldp . "/" . $file;
  if ( !(-e $f) )
  {
  $ftp->get($file, $f) or do {print "$ct​: 80km​: get​: $ftp->message\n"; goto QUIT;};
  print " $f\n";
  }
  }
  }

  $ftp->quit;

  QUIT​:
  }
  else
  {
  print "$ct​: new​: $@​\n";
  }

  # calculate time in secs to sleep till next hourly ftp
  sleep(3600 - (time() - $t1)%3600);
}

Perl Info


Site configuration information for perl 5.00503:

Configured by rah at Tue Sep 14 10:33:42 EDT 1999.

Summary of my perl5 (5.0 patchlevel 5 subversion 3) configuration:
  Platform:
    osname=solaris, osvers=2.7, archname=sun4-solaris
    uname='sunos wolf 5.7 generic_106541-07 sun4m sparc sunw,sparcstation-20 '
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef useperlio=undef d_sfio=undef
  Compiler:
    cc='cc', optimize='-xO4', gccversion=
    cppflags='-I/usr/local/include -I/opt/local/include'
    ccflags ='-I/usr/local/include -I/opt/local/include'
    stdchar='char', d_stdstdio=define, usevfork=false
    intsize=4, longsize=4, ptrsize=4, doublesize=8
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    alignbytes=8, usemymalloc=y, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib -L/opt/local/lib'
    libpth=/usr/local/lib /opt/local/lib /lib /usr/lib /usr/ccs/lib
    libs=-lsocket -lnsl -ldl -lm -lc -lcrypt
    libc=/lib/libc.so, so=so, useshrplib=false, libperl=libperl.a
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' '
    cccdlflags='-KPIC', lddlflags='-G -L/usr/local/lib -L/opt/local/lib'

Locally applied patches:
    


@INC for perl 5.00503:
    /opt/perl/lib/5.00503/sun4-solaris
    /opt/perl/lib/5.00503
    /opt/perl/lib/site_perl/5.005/sun4-solaris
    /opt/perl/lib/site_perl/5.005
    .


Environment for perl 5.00503:
    HOME=/home/wolf/rah
    LANG=C
    LANGUAGE (unset)
    LD_LIBRARY_PATH=/opt/SUNWspro/lib:/usr/local/lib:/opt/SUNWmotif/lib:/usr/openwin/lib
    LOGDIR (unset)
    PATH=.:/etc:/usr/etc:/usr/ccs/bin:/usr/ccs/lib:/usr/dt/bin:/usr/openwin/bin:/usr/bin:/usr/sbin:/usr/lib/lp/postscript:/usr/openwin/demo:/opt/SUNWspro/bin:/opt/Acrobat3/bin:/opt/local/bin:/opt/perl/bin:/opt/perl/lib:/opt/netscape:/public/matlab/matlab/bin:/pub/bin:/home/wolf/rah/bin:/home/wolf/rah/tools/staroffice/bin:/home/wolf/rah/tools/perl/bin:/home/wolf/rah/tools/netscape:/home/wolf/rah/tools/scripts:/home/wolf/rah/tools/xmt212/clients:/home/wolf/rah/tools/nutshell/ksh/kshdb:/home/wolf/rah/tools/sh/shbook:/home/wolf/rah/proj/aatt/data:/home/wolf/rah/proj/aatt/util_progs/wgrib:/home/wolf/rah/proj/aatt/util_progs/cg236:/home/wolf/rah/proj/aatt/util_progs/pgrib:/home/wolf/rah/proj/aatt/util_progs/car:/home/wolf/rah/proj/aatt/g2gp
    PERL_BADLANG (unset)
    SHELL=/bin/ksh

@p5pRT
Copy link
Author

p5pRT commented Nov 5, 1999

From [Unknown Contact. See original ticket]

I have written a perl daemon that does ftp on an hourly basis, downloading
files to my computer. I am using the GBAR ftp module. Every time I get
the following message the daemon exits without crashing or leaving any trace.

Timeout at /opt/perl/lib/site_perl/5.005/Net/FTP.pm line 422

I have contacted GBAR, and he says that this timeout is due to the ftp
site not responding, and is not a problem with FTP.pm. I think that any
error in a module called from a daemon should not cause its exit, and
should be recoverable. Included are the scripts to ftp and start the daemon.
These should easily runnable from your site, but it downloads a lot of data
every day. Sometimes it is several days before an exit.

[beautiful script snipped]

What's wrong with C<eval> ?

François Désarménien

@p5pRT
Copy link
Author

p5pRT commented Nov 5, 1999

From @gsar

On Fri, 05 Nov 1999 16​:30​:52 +0100, =?iso-8859-1?Q?Fran=E7ois=20D=E9sarm=E9nien
?= wrote​:

Timeout at /opt/perl/lib/site_perl/5.005/Net/FTP.pm line 422

I have contacted GBAR, and he says that this timeout is due to the ftp
site not responding, and is not a problem with FTP.pm. I think that any
error in a module called from a daemon should not cause its exit, and
should be recoverable. Included are the scripts to ftp and start the daemon
[beautiful script snipped]

What's wrong with C<eval> ?

And why was the report sent to perlbug? Net​::FTP is not maintained as
part of the Perl sources...

Sarathy
gsar@​ActiveState.com

@p5pRT
Copy link
Author

p5pRT commented Nov 5, 1999

From [Unknown Contact. See original ticket]

Gurusamy Sarathy wrote​:

On Fri, 05 Nov 1999 16​:30​:52 +0100, =?iso-8859-1?Q?Fran=E7ois=20D=E9sarm=E9nien
?= wrote​:

Timeout at /opt/perl/lib/site_perl/5.005/Net/FTP.pm line 422

I have contacted GBAR, and he says that this timeout is due to the ftp
site not responding, and is not a problem with FTP.pm. I think that any
error in a module called from a daemon should not cause its exit, and
should be recoverable. Included are the scripts to ftp and start the daemon
[beautiful script snipped]

What's wrong with C<eval> ?

And why was the report sent to perlbug? Net​::FTP is not maintained as
part of the Perl sources...

Sarathy
gsar@​ActiveState.com

I think this is because Graham saying it is not a FTP.pm problem, so he thoughts
it was a generic Perl problem.

Email and misunderstanding are often flame sources :-(

François Désarménien

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant