The Linux packet filter provides an easy way to protect against unwanted network intrusions. Often referred to simply as “iptables“, it is a basic firewall built into the Linux kernel. Iptables is most useful, perhaps, on those servers most susceptible to attack, such as LAMP systems, content management servers and blogging platforms like WordPress, especially where they are Internet facing.

Ipset is a fairly recent addition to Linux, having been introduced into kernel version 2.6.32. This means it is supported in Debian 7 and 8, as well as Red Hat 6 onwards. In short, ipset allows a large number of IP addresses to be blocked in an efficient way, as demonstrated below.

Install Ipset

Easy. On Red Hat and related systems, yum install ipset should do it. On Debian and its descendants, apt-get install ipset.

Create a List

The purpose of ipset is to create and manage lists of IP addresses in the kernel, using efficient structures to allow quick lookup. An ipset list can contain other items, but this article will focus on IP addresses.

Create a list as follows. Let’s call it blacklist1

# ipset create blacklist1 hash:ip

The “hash:ip” bit tells the ipset command to use a hash table to store items, and that the items will be ip addresses. The ipset list command can be used to view our new list:

# ipset list
Name: blacklist1
Type: hash:ip
Revision: 4
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 8272
References: 0
Members:

“Members” is blank because so far the list is empty.

Add IP Addresses

Add a couple of IP addresses to the list. These two were responsible for an (unsuccessful) dictionary attack on one of our WordPress platforms in March 2016. The attack was over after a few hours, but let’s block them anyway.

# ipset add blacklist1 107.170.73.100
# ipset add blacklist1 198.11.241.82

Now print the list again:

# ipset list
Name: blacklist1
Type: hash:ip
Revision: 4
Header: family inet hashsize 1024 maxelem 65536
Size in memory: 8304
References: 0
Members:
107.170.73.100
198.11.241.82

I feel better already. We could add many more addresses, but just leave it at these two for now.

Incorporating an Ipset List into the Firewall

To make use of the new list, iptables needs to be reconfigured. Here are two commands to make the firewall block all IP addresses in the list.

# iptables -A INPUT -m set --match-set blacklist1 src -j LOG --log-level 4 --log-prefix "Blocked by IPset: "
# iptables -A INPUT -m set --match-set blacklist1 src -j DROP

Note: the block is performed by the second command.  The first command just creates a message in the logs, recording the block.

List the firewall rules, and the new entries are shown.

# iptables -L
...
LOG        all  --  anywhere             anywhere             match-set blacklist1 src LOG level warning prefix "Blocked by IPset: "
DROP       all  --  anywhere             anywhere             match-set blacklist1 src

Very good, everything is set. To check that the list is working, we can either wait until some traffic happens to come in from those excluded IP addresses, or do a quick test. Let’s do a test.

Testing Iptables

To demonstrate that the iptables list is working, the address of a local test system will be added to the list blacklist1. First though, let’s see what a legitimate web hit looks like.

Monitoring the Apache access log and the syslog file (where iptables logs to) in the current shell will cause messages of interest to appear immediately:

# tail -f /var/log/apache2/access.log &
# tail -f /var/log/syslog &

My test box has the IP address 10.0.1.4. A wget command will be issued on that system, directed at the system with the new ipset blacklist. A simple fetch of the Apache front page was enough. Here is what was recorded in the local Apache access log:

10.0.1.4 - - [18/Apr/2016:22:42:33 +0100] "GET / HTTP/1.1" 200 54124 "-" "Wget/1.16 (linux-gnueabihf)"

The web page was successfully served. Nothing was recorded in the syslog file, because no action has (yet) been taken by iptables.

Iptables Blocks IP Address via Ipset

The test system is now added to the blacklist1 list:

# ipset add blacklist1 10.0.1.4

And the wget was repeated.

This time, nothing was recorded in the Apache log. Instead, this appeared in the syslog:

Apr 18 22:45:47 pluto kernel: [384444.211715] Blocked by IPset: IN=eth0 OUT= MAC=c5:21:dd:d7:42:f9:88:43:a1:22:73:8e:01:00:35:00:00:33:b6:d7:00:03:41:05:15:96 SRC=10.0.1.4 DST=123.123.123.123 LEN=48 TOS=0x00 PREC=0x00 TTL=64 ID=42710 DF PROTO=TCP SPT=37977 DPT=80 WINDOW=29200 RES=0x00 SYN URGP=0

The Linux netfilter has successfully blocked the test address 10.0.1.4 by virtue of it being on the ipset list. This is proved by the absence of activity from Apache (nothing in access.log, no hit, no page served). Meanwhile, the entry in the syslog file shows that iptables blocked the test IP address.

On the Fly Changes

The test above demonstrates an interesting point about ipset and iptables. The ipset list can be changed without reloading the firewall. As soon as the test address 10.0.1.4 was added to the set (with ipset add), the firewall immediately started blocking it. There was no need to reload the firewall, or to re-activate in any way the firewall rule referring to the blacklist1 list. The ipset list is dynamic. This might be convenient for systems administrators.

Server Usage

So much for the test. But what would be the best way to use ipset on a production LAMP server ? Well it partly depends on your environment and web infrastructure. IP address blacklists may already be managed by other network elements at your site. However, in very simple terms, a single server could be protected as follows. What follows is just an example and not intended as a production design.

Example of Blacklist Management

1. Put all blacklisted IP addresses into a simple text file, for example, /root/blacklist1.txt.

2. Add something like this to your firewall stop/start script. I am assuming here that the firewall is started by a shell script which contains “iptables -A” commands for all rules, in the appropriate order.

BLACKLIST="/root/blacklist1.txt"
...
# Initialise ipset containing blacklisted addresses.
ipset create blacklist1 hash:ip

cat $BLACKLIST | awk '/^[0-9]/ {print "ipset add blacklist1 "$0}' | sh

iptables -A INPUT -m set --match-set blacklist1 src -j LOG --log-level 4 --log-prefix "Blocked by IPset: "
iptables -A INPUT -m set --match-set blacklist1 src -j DROP
...
ipset destroy blacklist1

– The awk pipeline just adds every IP address in the $BLACKLIST text file to the set blacklist1. The awk pattern will ignore any line not starting with a number, so you can have comments and blank lines in the file if desired.

– The two iptables commands log and block traffic from blacklisted IPs.

– The ipset destroy command simply removes the list, and should be placed in the part of the script that turns off the firewall. The “ipset destroy” command should come last, after any “iptables” commands.

– Yes the leading cat is not strictly necessary.

This is just one approach. Alternatives include using the ipset “save” and “restore” options to keep a permanent record of blocked IPs.

Notes

• Incidentally, if you implement ipset and then find that too many messages are being written to the logs about blocked IP addresses, please check my procedure for redirecting firewall messages in Linux.
• If you want to block or allow a contiguous range of IP addresses, rather than a list of unrelated addresses, please see my related article on ranges of address and ports.

Conclusion

I hope this article has been of use and that ipset is useful for you. If you need to block a set of unrelated IP addresses, using ipset is neater and more efficient than writing separate iptables rules for each IP address. It is also faster because the kernel will perform a single hash lookup on each incoming IP address, rather than a linear search through many firewall rules.

Epilogue

By coincidence, while I was writing this article, and so monitoring the Apache log file, a dictionary attack was initiated from IP address 91.200.12.114. It was moderately aggressive, making about 3 requests every second. Seemingly, 91.200.12.114 is a well known nuisance address based in Kiev, now added to ipset and blocked.

UPDATE. Twelve hours later the attack was still in progress, having also increased to ten requests per second on the WordPress login script. This was measured by briefly removing the rogue IP from the set (with ipset del), checking the Apache log (now registering the unblocked address), then once again blocking the address with ipset add. The dynamic nature of ipset list is useful for this sort of thing.