Back in April (April 22nd to be precise) I handed in two copies of my 72 page document which outlines everything about my project.  I wanted to blog about my dissertation while I was doing it, but the thought was always pushed back by other thoughts of actually doing the project.

The project investigated how one could use virtualisation techniques of today and use them to virtualise every application on a machine.  Here’s a copy of my abstract:

In computing, occasionally an application can cause a whole system to be brought down. This wastes valuable time for users and has the potential to lose unsaved data. This project sets out to create a concept whereby applications are virtualised for overall system stability and which appears completely transparent to the
end user. The project attempts to create a system based on this concept using OpenSolaris 2009.06 and a number of its technologies, most notably: Zones.

The project discovers that the choice of technologies used are not yet mature enough to implement the concept, though they are very close to maturing.

As the abstract says, I used OpenSolaris Zones to implement the concept.  I created a script which wraps around the OpenSolaris `pkg` command which is used to install applications.  My script creates a new Zone and installs the new application into that Zone.  The application is now basically in it’s own operating system, to better describe it, it’s as if the application has been installed onto a whole new machine.  The application is accessed using SSH and tunneling X11 over it, this worked quite well after I had set up the infrastructure, but the user had to input their password when they wanted to launch an application.

The project was really fun, I really enjoyed it and learnt so much.  Now it’s finished I’m rather lost as to what to do with myself…

If anyone would like to read my Dissertation or try implementing my project (with is licensed under the BSD licence), please don’t hesitate to contact me.  At the end of it all, the documentation consisted of 72 pages in PDF form (that was 1086 lines of LaTeX, excluding comments and blank lines), and the project consisted of 1494 lines of Bash script and 1393 comments!

This post describes how you can use traps to help your code fail more gracefully.

Sometimes you write a script, it fails, and leaves a shed load of temporary files over your file system.  Or perhaps (like in part two) some stuff gets set up, your script fails and leaves a network half set up or a naming service half completed.  Whatever the problem, we can trap them (well, almost all of them…).

For my example, I’m going to use a section of my dissertation.  I have a script which is called “INSTALL.sh”, this sets some stuff up itself, and also delegates some tasks to other scripts.  Here is the general gist:

Check for root permissions – If no permissions, exit with status 1
Move some files about
Create some directories and move files to those directories
Call a script to set up the network
Install NIS (A.K.A. YP)
Set up NIS
Set up ability to share home directories
Call a script to set up a Solaris Zone (virtual machine)
Move a customised file over a system file
Call a cleanup function

Now, if any part of that script goes wrong, the script will fail (it will because I’ve used the “set -e” option).  If the script exists half of the configuration is done, and if you try to run the script again, it will fail because some of the setup is already complete.

To get out of this horrible situation, I have used a trap.  If the script exists with an exit status of 1 or greater, or if it is interrupted by a HUP, INT, QUIT or TERM signal, it will be trapped and run a function called “abort”.  Here is my trap statement:

trap ‘abort’ 1 2 3 15

My “about” method looks something like this:

abort(){
echo "ABORTING!  Please take note of any warnings!"
#If pkg-manage exists, rename it to pkg
if [[ -a /usr/bin/pkg-manage ]]; then
rm /usr/bin/pkg
mv /usr/bin/pkg-manage /usr/bin/pkg
fi
#Call cleanup to clear away temp files
cleanup
#Remove contents of /var/vaes
rm -rf /var/vaes
#Remove config file
rm $CONFIG_FILE
#Uninstall NIS & related things
pkg uninstall SUNWyp
domainname ""
rm /etc/defaultdomain
cp /etc/nsswitch.files /etc/nsswitch.conf
zfs set sharenfs=off rpool/export/home
cat /etc/auto_home | grep -v "`hostname`:/export/home" > /tmp/auto_home
mv /tmp/auto_home /etc/auto_home
#Delete zones dir
POOL=`zfs list | awk '{ print $1 }' | \
grep "export" | sed 's/\([a-zA-Z]*\)\/.*/\1/g' | head -1`
zfs destroy -Rf $POOL/export/vaes-zones
#Undo zones
bash /tmp/zone_setup.sh abort
#Undo networking
bash /tmp/network_setup.sh abort
}

By using the trap statement, you can clean up after your failed script in the majority of cases.  A word of warning though: You cannot trap the KILL (or 9) signal, if your script fails because it has received a KILL signal, it will just exit.

This concludes the series on robust Bash scripting.  I hope it has been of some help to someone.

This post offers some advice on how to Bash script defensively.

Be Prepared is the old Scout motto, and never is it more relevant then when scripting in Bash (or any language come to think of it).

Missing files and directories

If you are going to be working with files and directories in Bash (and you probably will be 99.9% of the time) you should test to see if they exist.

You can do a very generic test like this:

MYFILE=/tmp/test
if [[ -e $MYFILE ]]; then
#Do something
else
#Do something else
#Perhaps create file/directory
fi

The “-e” will test for the existence of ANY file, including directories.  You will probably be better off using “-d” for directories though, and “-f” for regular files, see the man page for more information about test (use the command “man test” or “help test”).

If you are creating a directory somewhere like “/opt/local/share/lib/foo/bar”, you should either test for the existence of each folder, or you can simple used the “-p” flag on the “mkdir” command, this will create any folders that aren’t created.

Spaces

Sometimes your script will deal with text that has spaces in it, and you need to be prepared for that, even if you are sure it will never need to handle spaces, there will always be one occasion somewhere down the line.

If we take this example:

for EACH in $@;
do
echo $EACH
done

Say the input to the script reads something like this:
bash myscript.sh hashbang0 “Ben Lavery” Aberystwyth
The for loop shown previously would print out the output like this:
hashbang0
Ben
Lavery
Aberystwyth

If we quote the $@ like this:

for EACH in "$@";
do
echo $EACH
done

Thou output now becomes:

hashbang0
Ben Lavery
Aberystwyth

Failing Gracefully

If you are updating a lot of files in a directory, what happens if your script fails halfway through?  Half of your files are unmodified, and the other half are modified.  This could be disastrous, especially in something like a directory full of web documents.

One solution is to copy the contents of the folder before you start working on them, do the work, move the copied files over the old files.  Something a bit like this:

cp -a /tmp/mydir /tmp/mydir-temp
#modify files in /tmp/mydir-temp
mv /tmp/mydir /tmp/mydir-BAK #backup
mv /tmp/mydir-temp /tmp/mydir

As long as you do the relevant testing and exit the script if anything goes wrong, the script will never overwrite /tmp/mydir, and if it does, you have a backup!

This post shows you how to use Bash’s built-in error checking.

Sometimes you might write a script where one line of the script may fail, this failure could cause the rest of your script to fail horribly!

For example, you have two scripts: One which sets up the network on a machine, and another which sets up a naming service based on details given in the network script.  The second script calls the network setup script before it sets up the naming service.

If the network setup fails for some reason, the script shouldn’t attempt to set up the naming service as something has gone wrong.

You might try something like this:

bash setup_network.sh
if [ "$?"-ne 0]; then
echo "command failed"
exit 1
fi
#start setting up naming service

Although this is fine, what happends if you forget to add it?  Or assume that the network will always be set up so decide not to add it?  If the network doesn’t get setup, you’ll be in a bit of a pickle!

Bash provides its own, built-in, error checking for you in the form of the “errexit" option, or "set -e".  If a command fails in a script with this option enabled, the script will automatically stop with an error code 1.

If you need to turn off error checking (and you may need to if you are using something like grep which may fail, but you know youve handled it), then you could do this:

command1
command2
set +e
command_that-may-fail
set -e
command4

“set +e” will turn off error checking.

In a later post I will look at the `trap` statement to catch the error this option throws.

This post shows you how you can guard against unset variables.

Bash doesn’t provide any sort of checking for unset variables by default.  You may have something similar to the following piece of code:

#!/bin/bash
#Script is called delete.sh
DEL=$1
rm -rf ~/$DEL

You would use it similar to this: `bash delete.sh tmp`, which would delete ~/tmp.  Imagine if you forgot to add the “tmp” on the end…POOF!  That’s your home directory gone!

You should get into the habit of using the Bash option “nounset”, otherwise known as “set -u”.  If you put that at the top of your script and run the command without any arguments, you’ll get something like this:

$ bash delete.sh
delete.sh: line 3: $1: unbound variable

This is somewhat similar to a runtime error in Python or Perl and could be very annoying, but at least you’ve saved your home directory, or another critical location on your computer!

As a side note, if “set -u” or “set -o nounset” is specified at the top of the script and the script exists because of an error, it will exit with an exit status of 1.  In a later post I will explain how to use the `trap` statement to catch this error.

# search for all instances of an executable in $PATH
function whence {
for i in `echo $PATH | sed "s/:/ /g"`
do
/usr/bin/ls $i/$@ 2>/dev/null
done
}

It’s nice and straighforward, nothing complicated about it at all.  In fact, to let everyone on your system use it, you could just stick “#!/bin/sh” at the top and stick in a file in /usr/bin !

Today, I wanted to find a man page, the sysidcfg man page to be precise.  Instead of doing the usual trick (`find / | grep sysidcfg”), I thought I’d modify `whence` to look for it for me, and seeing as it’s no longer `whence`, I called it `whenceman`:

# search for all instances of a man page in $MANPATH
function whenceman {
for i in `echo $MANPATH | sed "s/:/ /g"`
do
/usr/bin/find $i/ 2>/dev/null | grep -i $@
done
}

As you can see, it’s very similar, and you could do the same thing by putting it in /usr/bin so everyone could use it.

Wrote a small Tetris on sed’e: sedtris.sed and wrapper on Bashe sedtris.sh

Download both scripts to the same directory, on the command line do:

bash sedtris.sh

It’s actually very good!