Updates to my HomeLab

I've made a bunch of changes to my HomeLab recently, mostly around migrating some older workloads, some rework and some new hardware. This is mostly me just rambling about what I've been up to.

Hardware

Turns out I needed a lot more RAM than I had. I run a bunch of services internally and believe it or not, they all love to eat some RAM. I previously had 32GB, with the bulk of it going to TrueNAS and my Kubernetes cluster. My metrics tell me it was sitting at around 29GB of RAM usage and 5GB of swap at 'idle'.

I had a bit of a dilemma around this, do I invest in 2x32GB DDR4 RAM sticks which are quite pricey or do I get some 'new' (used) hardware. I decided to stick with the current hardware and just grab the 64GB of RAM. Turns out yep, that made a huge difference and everything became more stable.

I also finally outgrew my 2x2TB used enterprise SSDs. I archive my twitch streams as well as my friends so 2TB fills up quickly. I've been offloading it to a cloud provider for a while but that's a single source. I bought 2x12TB Ironwolf drives which have been treating me well.

Backups

I intend on writing a blog post about my move away from American cloud services and towards EU based services. To put it short, I've moved my data backups from Backblaze B2 over to Scaleway Glacier Tier. This has both saved me money and moved my data to an EU owned company and I couldn't be happier with Scaleway so far.

Kubernetes

I managed to break my Talos cluster to the point where I had no idea where to go for recovery (yes, I should have taken VM backups). I tried to upgrade to the latest version of Talos which was overdue, ran talosctl upgrade without specifying an image and it rebooted to a version without secure boot and my config would no longer boot. Heck. So I took this opportunity to move back to a system I know how to use, K3s on top of Flatcar Linux which feels a lot nicer.

I've also moved my backup job away from a crontab on a random VM into Kubernetes which was surprisingly painless 😄

That's a quick recap of what I've been up to, at least what I remember