There was quite a long while where I really trusted Debian and if I wanted to implement some kind of web related services on for example a VPS I wouldn’t hesitate to recommend it. Sadly although people will disingenuously claim Debian provides more init freedom, as you can choose your init provider, oddly they don’t seem to mention that the official repository doesn’t offer for example udev alternatives like eudev, mdev and others… and that just looking at one of the many parts of the infrastructure systemd has subsumed.
So there I was sitting on a VPS that had been through some dist-upgrade’s and been bodged to keep it morphing into just another systemdOS variant. I upgraded to a version of Devuan but some hidden linkage to libsystemd in an application really hosed an upgrade to Devuan ASCII. At the risk of another
rant aside Devuan has been a disappointment, development being rather slow, and even in other simple setups I’ve experienced a number of APT dependency foobars, maybe in a few more years it will be worth looking at again…. a real shame….
So I was in possession of my backup data and both pieces of a borken VPS, not being entirely happy with the previous VPS provider I decided to park the domain and take stock.
Not all VPS are created equal (that’s for sure!) often on the backend you might be lucky to see what is in effect a HTML version of an SSH client, let’s face it that’s about as much use as a chocolate tea pot and not even as tasty! Unless you’re very careful about experimenting with iptables rules or networking settings, you have no way to fix your machine, then you’re at the mercy of some under staffed and harassed support staff and probably waiting hours for an initial ticket contact with probably a bit of procrastination to give them breathing space, a simple mistake could mean your site is offline for hours if you’re lucky…. Obviously not ideal…
Of course the biggest issue when looking for a VPS is what OS you can run on it, usually its a choice of systemdOS variants with *BSD being the only Unix like OS on offer. It had been quite some time since I last trawled around looking at different VPS suppliers and only a very few offered any custom ISO option. Often a custom ISO used to mean posting a ticket with an ISO URL and probably spending a bit extra too… Oh and waiting….
Thankfully technology is always moving forward and if you look you can find affordable VPS services, that not only offer automated custom ISO’s and a proper virtual console that doesn’t rely on networking… If you’ve read this far you’re probably enough of a geek that you’ve run some kind of VM on your own machine, something like VirtualBox, a decent VPS can now offer you a service very similar to running a VM on your local machine – the virtual console won’t break when you do something dumb with networking so you can always access it, and if you do something monumentally comical you can always reboot with an ISO like SystemRescueCd this leaves you much more free to
play learn by actually doing in a real life environment.
I fully expected that Void Linux would do a competent job in this role, but really pleasantly surprised me was just how lean and fast Void can be, its far from a slouch on the desktop, but even for a decent LAMP stack with the usual extra bits and bobs excluding the required data for my services the foot print was well less than a few hundred packages, looking at the running processes too gives you a clear indication that there is as little bloat going on as possible. From a system that while it lasted longer than some systemdOS trials I’ve done, felt was creaking under the weight of historical choices and uncertain future I now have a slick sleek system that’s fast and bang up to date, that literally took less that 10 mins to install from CD and reboot, of course the seemingly endless tweaking and testing often takes much longer. I have to say that Void Linux is definitely a very well engineered distribution you can rely on. BTW if you like my previous posts (and this one!) and you’re considering getting a VPS you could do me a small favour by using this link to Vultr ….
It’s not unusual when going through a process like this you might find some new tip, if you are going to allow ssh passwords instead of keys (sometimes leaving a key on a mobile device isn’t an option…) then its very useful to limit the number of login attempts that can be made over time. A common solution to this is to use fail2ban which monitors logs and typically responds with iptables rules (although you can have it use route instead) A more efficient method I found on serverfault is to use just iptables… it bares looking at in a little detail
at some point in your iptables rules you’re going to have a rule something like this to deal with established traffic
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
I’m showing lines from an iptables-save file as its probably a little more readable… Somewhere AFTER this rule you can start marking and counting SSH connections
-A INPUT -p tcp -m tcp --dport 22 -m recent --update --seconds 60 --hitcount 3 --name SSH --mask 255.255.255.255 --rsource -j DROP -A INPUT -p tcp -m tcp --dport 22 -m recent --set --name SSH --mask 255.255.255.255 --rsource -j ACCEPT
because in rule order we’re after the established rule, we should only see new connections when we hit these two, if within 60 seconds there are 3 connections from the same source the traffic is dropped (you could log first as well if you really wanted) otherwise we accept the traffic.
As you can see iptables is very powerful, so powerful in fact you could easily blow your own foot off! All this is going on in the kernel before any userland application has to scan logs and fire off new rules to iptables. Additionally this doesn’t unduly punish legitimate users by the time you’ve found your little black book of passwords plenty of time has elapsed for you to be able to try again, but someone brute forcing things is going to find things painful…
I’ve previously seen variations on rules like this, but care needs to be taken with the volume of rules and routes through them carefully designed, you could slow traffic and cause excessive CPU use if you really tried hard enough I’m sure….!