Sendmail Authentication for Local Mail Delivery

Posted on June 2, 2012 by Jim

These days, service providers like British Telecom have tightened up their requirements on email delivery. Most ISPs now accept email only from servers that can authenticate through SMTP. Sendmail has authentication built in, allowing administrators to use an “authinfo” file, for example. The server then authenticates every time it sends mail out to the ISP.

That’s great for reducing spam. But it can have a side-effect of killing local mail on the server. Here is an recent example from Debian Squeeze. The authentication features that allowed this server to successfully send mail through the ISP were preventing local mail from working: Continue reading →

Large Directory Causes ls to "Hang"

Posted on May 20, 2012 by Jim

So you have a directory with millions of files, and ls just hangs ?

Use ls -1 -f to show the files immediately. To delete the files, if you want to remove ALL files in the current directory, use something like

ls -1 -f | xargs rm

After cleaning up very many unwanted files, you are likely to be left with a huge and sparse directory object. Three million files in one directory, for example, apart from taking up space in themselves, will likely push the directory object to occupy over 100 Mb of space. Continue reading →

Perl – OR Pattern Match Slow, Use Two Patterns Instead

Posted on April 22, 2012 by Jim

A handy feature of regular expressions is their ability to “or” match. Searching for two strings in a file is easy with a construct like “egrep ‘root|uucp’ /etc/passwd”. The vertical bar (“|”) acts as an “or” operator. Perl supports the vertical bar too, and the same match could be achieved in Perl thus:

bash-4.2$ perl -n -e 'print $_ if /root|uucp/' < /etc/passwd root:x:0:0:root:/root:/bin/bash uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin

Weirdly though, this construction is up to 10 times slower than 2 separate matches performing the same search, as can be shown with a quick demonstration. Continue reading →

Repairing the MySQL WordPress Database

Posted on March 27, 2012 by Jim

You can recover a running MySQL database with the mysqlcheck command.

After a power cut, I found these errors in /var/log/daemon.log. This was on Debian Squeeze running WordPress 3.3.1 and MySQL 5.1.61.

Mar 22 11:22:16 debian /etc/mysql/debian-start[2153]: Triggering myisam-recover for all MyISAM tables Mar 22 11:22:17 debian mysqld: 120322 11:22:17 [ERROR] /usr/sbin/mysqld: Table './wordpress/wp_commentmeta' is marked as crashed and should be repaired Continue reading →

Process Substitution and Pipes

Posted on March 22, 2012 by Jim

Command substitution is a widely used feature of the Bash and Korn shells, allowing the output of one command to be captured and used in another. Like this:

$ echo "Backup started at $(date)" Backup started at Fri Mar 16 15:35:14 GMT 2012

Command substitution is not to be confused with that less well known (and, to be honest, less useful) shell feature, process substitution. Despite being rarely used, process substitution is worth knowing about, if only because it illuminates other fundamental unix features – the shell, sub processes, named and unnamed pipes.

This post discusses process substitution, command substitution and the vertical bar (|). Three very different shell features, but all making use of unnamed pipes, and so not as different as they first appear. The examples are from Linux but also work on Solaris 10 and, due to the ubiquity of pipes, are likely to work on other unixes too. Continue reading →

Large files / small backups on Debian LAMP

Posted on February 21, 2012 by Jim

Backups on a LAMP server were much smaller than expected, and the cause was excessively large log files. This post explains the apparent paradox and the steps taken to trace and fix the problem.

The server in question runs the usual set of grandfather-father-son backups. Before each backup, the software prints the expected size of the backup into a log file. On other servers the estimate is usually bang-on. On this server, the backup turned out to be about 12 times smaller than the estimate. Not a problem in itself, but puzzling and indicative of something untoward on the system. Continue reading →

ZFS Corruption Persists in Unlinked Files

Posted on January 22, 2012 by Jim

This article explains how zfs errors can sometimes persist even after the files containing the offending blocks have been deleted, and presents an easy way of removing corruption.

Most solaris sysadmins will be familiar with the following situation. A zpool gives errors about failed checksums and enters a degraded state: Continue reading →

Fedora Dual Boot Convoluted Boot Path

Posted on December 18, 2011 by Jim

This post describes a problem with receiving Linux kernel updates after installing Fedora 16 in a dual boot environment. The Fedora installation process allows the user to rewrite Grub in the master boot record (MBR) or to leave the MBR untouched. I chose to leave it untouched, leading to a later kernel update problem which was fairly easily solved with some Grub 2 commands. Continue reading →

Open Source Software and GNU/Linux

Posted on December 18, 2011 by Jim

Introduction

This article discusses some of the workings of open source software with particular reference to the Linux kernel and GNU. It will explain some of the processes involved for those of us who know Linux but are not software engineers. Perhaps you are an Linux administrator or enthusiast, but find yourself unsure of terms such as upstream, downstream, mainline, patch, merge and other words developers love to bandy ? Read on.

Open Source Software

If you are a Linux user, you might be familiar with installing software using yum or apt-get. For example Continue reading →

Simple Locking Wrapper for VI

Posted on November 5, 2011 by Jim

This post presents a quick way to add file locking to the vi editor.

The vi editor is found on pretty much every unix system, which means it is often used by system administrators to update configuration files. Unfortunately vi does not lock the file being edited. This can lead to two or more people inadvertently editing the same file at the same time. Edits can become duplicated, confused or lost entirely, perhaps leaving the file in a non-working state.

Editing Critical Files

It is bad enough in the case of small files such as yum.conf or ntp.conf. More serious are files like /etc/fstab, where a simple syntax error could lead to an unbootable system. Worst of all are network-wide resources like DNS zone files. A mistake here could mess up more than one machine. Continue reading →

Unix etc.

Unix, Linux and related technologies.