Saturday, September 29, 2007

A Hella of a box - Settingup a Black Box NZB Leecher.

Install and Maintain HellaNZB

PRE REQ's
Ubuntu Plain Server install
open ssh
Samba
Smartmontools
HellaNZB
- Config lives in /usr/etc/hellanzb.conf


TECHNICAL OVERVIEW
  1. bootup - Run HellaNZB as a daemon
  2. Every hour between 9am and 1am - run pause_hellanzb
  3. Every hour between 1am and 9am - run continue_hellanzb
  4. At 1am run main_hellanzb_manager - to retrieve NZB files from server and store them locally

ISSUES

Issue 1 - Permissions on usenet directory
# hellanzb inherits the umask from the current user's environment (unless it's
# running in daemon mode). The umask can be forced with this option
#Hellanzb.UMASK = 0022

Issue 2 - Webmin refusing to start at boot.

Needed t0 create a symbolic link in the boot up run state directory
/etc/rcS.d$ sudo ln -s S99webmin /etc/init.d/webmin

UPDATE 1 - Setup HellaNZB Scheduling

1) ProFTP allows anonymous FTP access to /home/ftp, so use this as the Transit Area.
Provide a share for this so that I can save NZB to /10.1.1.100/nzb
Webmin Create a File share. - /home/ftp is the directory to share.
2) So we have an FTP server and a Winfs share- Fix permissions so Windows can write to home/ftp
sgoggin@Orangutang:/home$ chmod 777 ftp
chmod: changing permissions of `ftp': Operation not permitted
sgoggin@Orangutang:/home$ sudo chmod 777 ftp
Password:
sgoggin@Orangutang:/home$

3) Now we need a script to run on Leecher that ftp's the files from this directory at 1am.
AutoFTP does this - http://www.csh.rit.edu/~adam/Progs/autoftp-2.4.1.tar.bz2

Change the first line in "autoftp" to reflect where Perl is installed on your system.
#!/usr/bin/perl
Copy the autoftp program to your favorite bin directory (i.e. /usr/local/bin).
cd /usr/local/bin
sgoggin@Leecher:/usr/local/bin$ sudo vi autoftp
Copy the man page to your favorite manl directory and rename it as appropriate for that directory (i.e. /usr/local/man/manl/autoftp.l)

sgoggin@Leecher:/usr/share/man/man1$ vi autoftp.1
sgoggin@Leecher:/usr/share/man/man1$ sudo vi autoftp.1
sgoggin@Leecher:/usr/share/man/man1$ man autoftp
Issues - Setuid/gid script is writable by world.
chmod 4755 *
Script to Get files
** List the files
sudo autoftp -v -l '10.1.1.100;;a;*nzb'

** Get the files
sudo autoftp -v '10.1.1.100;;a;*nzb'

** Delete the files
sudo autoftp -v -k '10.1.1.100;;a;*nzb'

Script Structure

Create entry in log file
autoftp - Show list of NZB's on main server in log file
autoftp - Transfer nzb's to nzb-transferred.
autoftp - Transfer nzb's to hellanzb autoimport folder.
autoftp - delete source nzb's

If count of files in autoimpart folder is >0 then
start hellanzb
1: go to sleep for 10 mins
Is hellanzb completed? (Check status)
Is it time to shut down
LOOP

Transfer files that have been caught to server.
Initiate shutdown.

sudo ./main_hellanzb_manager
Script is located in /usr/local/bin/main_hellanzb_manager and is setup in webmin
# Directory to copy to on the destination machine.
FETCHDESTDIR="/home/sgoggin/nzb/daemon.queue "

# May be needed if run by cron?
export PATH=$PATH:/bin:/usr/bin:/usr/local/bin

# STEP 1 - Pre Startup

LOGFILE=/var/log/hellanzb-report-`date +%F`
touch $LOGFILE && chmod 600 $LOGFILE
echo "START -`date +%F`\n" >> $LOGFILE
logger "HELLANZB - Fetch Files"
autoftp -v -l '10.1.1.100;;a;*nzb' >> $LOGFILE


cd $FETCHDESTDIR
echo "HELLANZB: Placing files in `pwd`">>$LOGFILE

# autoftp - Transfer nzb's to nzb-transferred.
# autoftp - Transfer nzb's to hellanzb autoimport folder.
autoftp -v '10.1.1.100;;a;*nzb' >> $LOGFILE


# autoftp - delete source nzb's
autoftp -k '10.1.1.100;;a;*nzb' >> $LOGFILE



Now let's use webmin to do this script at 9am..

dule Config
Scheduled Cron Jobs

Select all. | Invert selection. | Create a new scheduled cron job. | Create a new environment variable. | Control user access to cron jobs.
User
Active? Command Move
root Yes [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxli ...
Yes /etc/webmin/cron/tempdelete.pl
Yes /usr/local/bin/main_hellanzb_manager
Yes /usr/local/bin/pause_hellanzb
Yes /usr/local/bin/continue_hellanzb
Select all. | Invert selection. | Create a new scheduled cron job. | Create a new environment variable. | Control user access to cron jobs.


Now let's make sure downloads only happen 1am to 9am - This script is called in webmin cropn job every hour from 1am to 9am.

sgoggin@Leecher:/usr/local/bin$ ls
autoftp continue_hellanzb main_hellanzb_manager pause_hellanzb
sgoggin@Leecher:/usr/local/bin$ cat continue_hellanzb
#!/bin/sh
# /etc/cron.daily/standard: standard daily maintenance script

# May be needed if run by cron?
export PATH=$PATH:/bin:/usr/bin:/usr/local/bin

# STEP 1 - Pre Startup
logger "HELLANZB - Continue"
/usr/bin/python /usr/bin/hellanzb.py continue


This script is called from a cron job that runs every hour from 9am to 1am.
sgoggin@Leecher:/usr/local/bin$ cat pause_hellanzb
#!/bin/sh
# /etc/cron.daily/standard: standard daily maintenance script
# Written by Ian A. Murdock
# Modified by Ian Jackson
# Modified by Steve Greenland

# May be needed if run by cron?
export PATH=$PATH:/bin:/usr/bin:/usr/local/bin

# STEP 1 - Pre Startup
logger "HELLANZB - Pause"
/usr/bin/python /usr/bin/hellanzb.py pause

And I think that might be it...







No comments: