Bash Script Behaves Differently When Called From Cron

Unix users and administrators will be familiar with the cron, unix’s built in job scheduler. It is a good way of running regular jobs eg backups, system monitoring programs or housekeeping scripts. The configuration of cron is quite particular and care is needed when setting up a new job. Your well tested script can behave differently when it is called from cron. Sometimes the differences won’t matter. But sometimes they do, and finding the cause can be tricky.

This brief article describes how many such problems can be tracked down simply by capturing the standard error output properly. In short, make sure your troublesome cron job is not quietly discarding the very information you need to fix it. Continue reading

/tmp/sortXXXXXX files

A backup script that runs on several Linux systems recently produced the following error:

ls: cannot access '/tmp/sortrq9hq8': No such file or directory

It happens every time the backup runs. Other than the above message, there seems to be no ill effect on the backup, which completes successfully. The ls command did not find a file that it expected to be there.

This article explains how the missing file was created by the Linux sort command as a temporary storage area, how error messages about these files are likely to crop up from time to time, how to reproduce the error, and some background about the behaviour of the sort command on Linux and Solaris. Continue reading

Ansible: Match Special Characters Without Escaping

Ansible provides a rich pattern matching ability. Modules like lineinfile can match strings based on regular expressions. Similar expressions are used in Python, Perl and older tools such as egrep, grep, sed and awk.

When attempting to match a string containing awkward characters, an escape mechanism can be used. For example, the dollar character ($) has a special meaning within in a regular expression, being the match for end-of-line. So to match a literal dollar, an escape character, usually a backslash (\), is needed. For example, the regular expression “\$1.65” will successfully match $1.65, without treating $ as end of line.

When processing a string that contains many special characters, the escape syntax can become onerous. One solution is to just “blanket” match the special character, rather than trying to match it precisely. In other words: just use a dot. Continue reading

Automatic Nextcloud Installation on Raspberry Pi

Nextcloud is an open source software package providing remote file sharing services. It is similar to Dropbox. But with Nextcloud, you retain ownership, security and control of the shared data. This procedure describes how to build a working Nexcloud service using just 3 commands.

Note: If you would rather do the installation manually, step-by-step, without the help of a script, please see my previous article Simple Nextcloud Installation on Raspberry Pi. It explains how to do the installation in detail, and provides more background information on Nextcloud. Both procedures achieve the same overall result, however. Continue reading

Set Up Your Own Link Shortening Service with a Raspberry Pi

“Link shortening” happens when a short URL, such as http://bit.ly/2bo3XYY, points to the same web page as a longer link, such as https://en.wikipedia.org/wiki/BBC.  Short links are often used where there are a limited number of characters available, such as an SMS text or a Twitter post.  Short links are also quicker to type and neater than the associated full length links.

Two of the main providers of short links are Bitly and Google (Goo.gl).  For example, I used Bitly to create the short link in the above paragraph.  However, if you have a Raspberry Pi (or any kind of Linux server), you don’t need to use a provider.  You can create your own short links.  This article explains how. Continue reading

Perl: Sort Hash Values by Key with a Hash Slice

In Perl, hash (associative array) sorting is a common and easy practice. Sorting values by key is easy. And so is sorting by value. But how do you sort the values of a hash by key? One answer is to use a hash slice. Continue reading

List Virtual Machines on ESXi

ESXi is popular hypervisor product from VMware. It comes with several management GUIs including Vsphere and VCenter. Command line tools are also bundled, though they are used more rarely. This article describes a short script to list all virtual machines on the system.

The script is below. It is intended to run under the Busybox shell, the default environment when you ssh directly into the system hosting ESXi. Continue reading

Simple Picture Gallery on Raspberry Pi

The Raspberry Pi is a small Linux computer designed to help children learn programming. Being a full Linux System, it can also be used as a server or as the basis for various projects. The Pi’s low power consumption makes it particularly suited to the role of always-on web server.

This post describes how to create a simple photo gallery on the Pi, which can be shared over the internet with or without password protection. While not as polished as Flickr, Smugmug or similar services, it allows you to retain ownetship, control and security of the shared images. Continue reading

Quick Script to Find Duplicate Files

Here’s a quick script to show duplicate files on Linux. It should cope with arbitrary spaces in file names, and to save time and CPU resources, it will checksum only files of the same size.

Usage: Save the script to dups.sh or whatever, then run it with no arguments. A list of duplicated files is output. Continue reading

Perl – OR Pattern Match Slow, Use Two Patterns Instead

A handy feature of regular expressions is their ability to “or” match. Searching for two strings in a file is easy with a construct like “egrep ‘root|uucp’ /etc/passwd”. The vertical bar (“|”) acts as an “or” operator. Perl supports the vertical bar too, and the same match could be achieved in Perl thus:

bash-4.2$ perl -n -e 'print $_ if /root|uucp/' < /etc/passwd root:x:0:0:root:/root:/bin/bash uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin

Weirdly though, this construction is up to 10 times slower than 2 separate matches performing the same search, as can be shown with a quick demonstration. Continue reading