How to Watch and Record Live TV with a Raspberry Pi

This procedure explains how to turn a Raspberry Pi into a DVR, or “digital video recorder”. A DVR is a box that sits under your television, allowing you to watch digital television and record it to a hard disk. TV and recordings can also be “streamed” to other devices on your network, such as phones, tablets and PCs.

Procedure last updated 8th Jan 2022 for OSMC 19 / Kodi “Matrix” / Pi 4.

Commercial examples of DVRs are available from manufacturers like Humax and Panasonic, providing access to free digital TV services, which in the UK are called Freeview and Freesat. Other brand names are used in other countries, but the underlying technology is the same. By following the article, you will be able to view, record and stream free digital TV. (You won’t be able to watch encrypted services, like those broadcast by Sky and BT).

Why turn a Pi into a DVR? You might be just curious about the possibilities. You might be looking to move from a paid-for service (eg. BT or Sky) to a free one. You might want to replace older equipment or just get access to more modern services, such as HDTV (high definition TV). I was looking to replace my old Topfield DVR, and the procedure below worked for me.

Continue reading

Raspberry Pi 4 Real World Tests

The Raspberry Pi 4 was launched on 24th June and has been well received, to say the least. The spec is a big step up on previous models. It has 4 CPU cores like the Pi 2, a gigabit port like the Pi 3, plus USB 3, a better SoC, a separated bus architecture, faster memory and more of it.

Over the years, many “home” devices have been launched with Gigabit Ethernet, promising lightning fast network speeds, only to disappoint due to their lack of overall grunt. The Linkstation Live, the Sheevaplug and, to a lesser extent the Pi 3 are all on that category, unable to push their gigabit ports to more than about 14, 8 and 12 megabytes/sec respectively, due to the limitations of the CPU and the board. Is the Pi 4 the same, or can it operate as a serious NAS ?

Short answer: Yes. The Pi 4 is a *serious* NAS contender. Sustained write speeds of over 68 MB/s were obtained, and over 105 MB/s for reading, including saturation of the Gigabit network. Yes, the Pi 4 can push even a 1000 MB/s network to 100%.

Continue reading

How to Protect a LAMP Server Against nf_conntrack Flood Attacks

An AWS hosted website went offline at 02:00 this morning. It was running on a t2.nano Debian 9 instance. I was unable to log into the affected server, and a reboot was the only available course of action. Logging in and checking the logs afterwards revealed thousands of errors like this in the kernel log file, from 2:00 AM onward:

nf_conntrack: nf_conntrack: table full, dropping packet

The cause was a denial of service attack, coming from a couple of IP addresses seemingly in Iran. However, it was a little unusual for a couple of reasons. This article explains more about the attack vector and presents a solution to guard against future attacks. (In summary: block IP addresses, tune the kernel).

Continue reading

Create a Recovery USB Stick in Linux

A recovery USB stick is a bootable USB drive that can be used to rescue a system or perform critical maintenance. It’s a useful tool to have around. Typically, a system of interest is booted from the USB stick, maintenance is performed, then the repaired system is rebooted from its own disk. Below is a simple guide to creating a number of rescue USB sticks.

A Rescue stick can help fix many system problems, such as a system that won’t boot, a broken GRUB configuration, a disk or other hardware problem. By allowing the whole operating system to be taken offline, a rescue USB allows maintenance of a kind that cannot be performed any other way.

It is easy to create a bootable USB stick in Linux. In these examples, I used a Raspberry Pi, but any Linux PC would do equally well.

Continue reading

How to Set Local Search Provider in Android Firefox

In Android Firefox, you can perform a search by typing directly into the address bar (aka the “awesome bar“). Results are provided by the default search engine, usually google.com. That’s fine, but you might prefer the results to come from a more local source, such as google.co.uk, or a completely different provider, such as bing.com.

Use the following procedure to change the default search engine in Android Firefox. “Awesome bar” searches will then be performed by your provider of choice.

Continue reading

Fixing a Corrupted Apache Log File

The Apache access.log file is a good place to look for evidence of hacking activity. Code injections, brute force attacks and excessive crawling all show up in there, along with legitimate hits. While searching recently, I was surprised to see that Linux had started to regard the file as binary data:

$ grep something access.log
Binary file access.log matches
Continue reading

Bash: Unescaped Left Brace in Regex

A strange message was recently received from a Bash script running under Linux Mint 18:

Unescaped left brace in regex is deprecated, passed through in regex; marked by <-- HERE in m/%{ <-- HERE (.*?)}/ at /usr/bin/print line 528.
Error: no such file "test\n"

Slightly confusing, as it reads like a Perl error, rather than bash. Below is another variety of the same thing

Continue reading

Redirecting Firewall Messages in Linux

The Linux firewall is a great way to secure a server, especially one that is Internet facing. Together with ipset and an appropriate blacklist, it can protect your server from the worst the Internet can throw at it. However, netfilter (aka iptables), can generate a lot of messages. By default they go to the kernel logging channel, flooding out log files such as messages, syslog and kern.log.

It is important to keep log files clear so that system issues are not missed. For example, a hardware or memory error message might be written to kern.log, but could be difficult to notice due to many thousands of firewall messages. Worse, over time, the important message will be moved into a historical log files due to the action of logrotate.

This article explains how to send firewall messages to their own log file, using the example of a Raspberry Pi running Raspbian 9 (Debian Stretch). After a small configuration change, netfilter messages go to their own file instead of clogging up the general logs.

Continue reading

Messing Around with Graphics.py

This article explains how to create some simple mathematical shapes with graphics.py, a popular graphics library for Python written by John Zelle. Graphics.py is a single file containing graphics functions such as Point, Line, Circle and Rectangle. In this article though, we are just going to use it to plot single points.

At the top of the page is a blancmange like shape. The program that drew it is at the bottom of the article, if you want to jump straight there.  Otherwise, a couple of simpler plots will be demonstrated first, just to show a couple of underlying principles. Continue reading

Complex Data Structures in Python

Most programming languages offer the facility for making large, compound data structures. For example C, Pascal, Perl and Python. A few simple data types are provided, out of which larger structures can be built. A programmer can store data in a whatever way is most suitable for the application.

Often, a simple list or dictionary will be enough. Read the data in, process it, and print the results out. Perfect. But for a larger or more useful application, more data, and more kinds of data, will need to be stored and processed at the same time.

This article demonstrates the building of a complex data structure in Python. Note: it is not about classes, or object oriented programming, just the syntax for handling complex data structures, made up of lists, dictionaries and simple strings and integers. Continue reading