Bash Script Behaves Differently When Called From Cron

Unix users and administrators will be familiar with the cron, unix’s built in job scheduler. It is a good way of running regular jobs eg backups, system monitoring programs or housekeeping scripts. The configuration of cron is quite particular and care is needed when setting up a new job. Your well tested script can behave differently when it is called from cron. Sometimes the differences won’t matter. But sometimes they do, and finding the cause can be tricky.

This brief article describes how many such problems can be tracked down simply by capturing the standard error output properly. In short, make sure your troublesome cron job is not quietly discarding the very information you need to fix it. Continue reading

An Example of Parallel Processing

This post shows how to use parallel processing to get a CPU intensive job done faster in Unix/Linux. By splitting a large task into several parts, it is quite easy to give each part to a separate CPU, and complete the task many times faster than it would on a single processor.

These days, even small PCs and other devices often come equipped with several CPU cores. But some tasks will use only one core, sometimes using 100% of it, while other cores stand by idle. Sometimes this is a waste of resources. Continue reading

Process Substitution and Pipes

Command substitution is a widely used feature of the Bash and Korn shells, allowing the output of one command to be captured and used in another. Like this:

$ echo "Backup started at $(date)"
Backup started at Fri Mar 16 15:35:14 GMT 2012

Command substitution is not to be confused with that less well known (and, to be honest, less useful) shell feature, process substitution. Despite being rarely used, process substitution is worth knowing about, if only because it illuminates other fundamental unix features – the shell, sub processes, named and unnamed pipes.

This post discusses process substitution, command substitution and the vertical bar (|). Three very different shell features, but all making use of unnamed pipes, and so not as different as they first appear. The examples are from Linux but also work on Solaris 10 and, due to the ubiquity of pipes, are likely to work on other unixes too. Continue reading