Calling SSH from Netbackup bpstart_notify

Calling a script at the beginning of a backup job is a fairly common task.
The convention for this when using Symantec (Veritas) Netbackup is to place your commands in a script called 'bpstart_notify', chmodding it to 555 and placing it in /usr/openv/netbackup/bin. If your script can complete in under (the default) 300 seconds you should be home-free.

That is unless your script calls anything on another computer over ssh.

Oracle Full Restore

I just wrapped up a 3 day full restore of Oracle and in doing so, gained a much better handle on wrangling my backup software. To important points. When restoring a backup job whose files were archived over more than 1 day, restore the job by its session object, not by using a date selection. The second important note is to restore filesystems concurrently. Another recommendation after I was finished was to select 'restore sparse files'. Now it's time for some sleep.

X11 forwarding with sshd on AIX

AIX now ships with sshd but is not configured to forward X11 by default. In addition it looks like the person who packaged OpenSSH at IBM did not have xauth in the default AIX location.

To enable X11 forwarding on AIX.
#1 Edit /etc/ssh/sshd_config and uncomment:

X11Forwarding yes
X11DisplayOffset 10
X11UseLocalhost yes

Then add an entry pointing to the correct xauth location.

XauthLocation /usr/bin/X11/xauth

#2 Login to your console using your HMC or with telnet - in other words NOT with ssh

Omitting Directories from Unix find results

Syntax of the find utility is made more complex by the need to escape some of its switches for use in a shell, but sooner or later you will run into a situation where you have too many files that need an operation for any other tool to manage.

The first time I ran into this situation was with some software that produced a log file with each event that was processed. When I noticed the file system was getting full, there were already millions of files that needed deleting. They each had a date stamp in their names but the normal way of 'wildcarding' for the oldest files:

Fixing JFS2, Tivoli Storage Manager

Today I found out a single critical file was needed from one of the JFS2 filesystems I was working with the other day. I was able to recover the file and things are working again.

Following a fiber-channel card failure I was asked to help recover some corrupt JFS2 filesystems. They couldn't be fsck-ed but I was able to run fsdb to modify the filesystem meta-data. First I tried replacing their superblock structures which didn't help.

Attempting to fix an IBM JFS2 filesystem.

# fsck /dev/stgA
The current volume is: /dev/stgA
Primary superblock is valid.
J2_LOGREDO:log redo processing for /dev/stgA
Primary superblock is valid.
Invalid data detected in aggregate inode 2.
Invalid data detected in aggregate inode 2.
fsck: 0507-278 Cannot continue.
File system is dirty.

You can check the primary and secondary superblocks on your volume with the following commands.

# lquerypv -h /dev/stgA 8000 100 # try primary superblock
# lquerypv -h /dev/stgA F000 100 # try secondary superblock

On my system the output looks like this.

Slowing Dictionary Attacks with NetFilter

iptables v1.3 and newer come with a module 'recent' which allows users
to limit access based on time.

To allow only 3 connect attempts from any single IP address in any 2 minute window

# rule to accept but also log new ssh valid connections
iptables -N acceptlog
iptables -A acceptlog -j LOG --log-prefix "*** ACCEPT LOG ***"
iptables -A acceptlog -j ACCEPT

# rule runs against all new ssh connections, and drops repeat offenders
iptables -N SSH_CHECK
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -j SSH_CHECK
iptables -A SSH_CHECK -m recent --set --name SSHDROP

Rows of Hashes in vi

A few days ago I spent half of my lunch break on google looking for a good way to make a row of 80
hash marks in the vi editor. Trust me, its a hard one to find in a search engine.

I did manage to figure it out today when one my my typos produced something similar.

To create a row of 80 hash marks in vi, press esc a few times to enter command mode and enter:
80i#[esc]

Logging root sessions

Here is an addition to /etc/profile to log 'root' login sessions on AIX using the command 'script' using ksh syntax.

# Script log each session to file
# jshunter
# friday 13 - 2008

if [[ ${USER} = "root" ]]; then

# Format todays date
DTE=$(date '+%Y-%m-%d_%H.%M.%S')

# Find the pty root is logging in on
MYTERM=$(ps $$ |grep $$ |awk '{print $2}')

# Use information in who command to get hostname
MYHOST=$(who |grep ${MYTERM}|awk '{print $6}'|sed 's/[\(\)]//g')

# Fork script shell and log info into a file named with date and host

Mounting /bin/echo

I stumbled across this trick getting a printer to work on AIX. AIX uses a command called enq to submit jobs to its printing subsystem. For compatibility reasons it also includes commands like 'lp' and 'lpr' for submitting print jobs.

Typically one might send 'report.txt' to printer called 'mylp1' with the command:

lpr -P mylp1 report.txt

To find out what the underlying enq command receives mount it to /bin/echo

mount /bin/echo /bin/enq

Now running the lpw command above sends the enq parameter to /bin/echo which displays:

-P mylp1 -c -Bgn report.txt

Syndicate content