An AWS hosted website went offline at 02:00 this morning. It was running on a t2.nano Debian 9 instance. I was unable to log into the affected server, and a reboot was the only available course of action. Logging in and checking the logs afterwards revealed thousands of errors like this in the kernel log file, from 2:00 AM onward:
nf_conntrack: nf_conntrack: table full, dropping packet
The cause was a denial of service attack, coming from a couple of IP addresses seemingly in Iran. However, it was a little unusual for a couple of reasons. This article explains more about the attack vector and presents a solution to guard against future attacks. (In summary: block IP addresses, tune the kernel).
Continue reading