r/devops • u/root0ps • 15h ago

Career / learning Built a small 3-node Kubernetes homelab

Post image

I recently set up a small 3-node Kubernetes homelab using Dell OptiPlex Micro PCs running Talos Linux.

The main goal was to have a local cluster where I can learn, experiment, break things, and test tools without relying on cloud infrastructure every time. I’m planning to use it for Kubernetes networking, storage, observability, GitOps, security, upgrades, and general experimentation.

Small setup, but already a very useful playground.

Curious what others here are running for their Kubernetes homelabs.

337 Upvotes

97% Upvoted

56 comments sorted by

92

u/HugeRoof 15h ago

I suspect you may have some overheating issues with that setup as is.

40

u/ryeguy 15h ago

no problem, just wrap the machines in a wool blanket to cancel out the plastic wrap, it's basic thermodynamics

2

u/Thoughtulism 13h ago

It's good because wool is a breathable fabric so you get better overall thermal regulation than plastic

2

u/walkdaddydawg 12h ago

Nah he’s good. None of them are plugged in

3

u/coffee-loop 15h ago

If the thing was even powered on.

9

u/root0ps 15h ago

no problem with heating yet, the cluster has been up for more than 3 days, and it is running 24/7

11

u/Michal_F 14h ago

Nice node naming :)

Heating was a joke, related to the plastic wrap ;)

3

u/root0ps 14h ago

Thanks🙌🏼, got you😅

1

u/RevolutionaryWorry87 15h ago

I have the same setup, how would you check for overheating? I have worries

3

u/sylvester_0 13h ago

Complete whoosh. This thread began with a joke about the PCs being encased in plastic.

1

u/RevolutionaryWorry87 13h ago

Fair one didn't realise. I have the same setup and actually thinking about overheating in a mini rack, but take the upvote sir

29

u/OkPumpkin7799 15h ago

Better than therapy lol

11

u/TooManyBison 15h ago

Unless you need to start going to anger management because you can’t get it working.

9

u/Liquid_G 15h ago

Do you have anything that can provide things like DNS/DHCP/Docker registry/NTP/NFS? I have a small Synology NAS device that does all these things, and can somewhat mimic real services that an enterprise environment might have.

3

u/root0ps 15h ago

not at the moment, and that's why I called it a small homelab, but I'll add more hardware gradually

3

u/SysBadmin 15h ago

Can’t you run pfsense or vyatta or whatever it’s called now lol

4

u/Gesha24 15h ago

I just used raspberry pi's - works just fine for smaller workloads, you have to sometimes handle rebuilding images to ARM and some other small annoyances that IMO help with learning.

5

u/mangocrysis 13h ago

What are their specs?

9

u/Readdeo 15h ago

For learning, experimenting and especially breaking things, I recommend using proxmox on them, so you can just spin up a vm, snapshot it before trying out something so you can retry. Also you can host other vms for other puropses so you can switch them around for the workload you currently need.

Will do the same thing this week with SFF versions of these.

I run gitops infrastructure, arr stack, other self hosting stuff i regularly use. Also, a test cluster for the new marketplace I'm working on.

3

u/gigium1 11h ago

Yes, you will save so much time recreating VMs instead of flashing Talos for the nth time

-4

u/R10t-- 11h ago

Everyone and their mom advertises Proxmox…

Did you read the post? OP said they are running TalOS Linux w/ Kubernetes. You cannot run Proxmox on Kubernetes

5

u/Readdeo 10h ago

Did you read my comment? No, not really...

0

u/R10t-- 8h ago

Yes I did. The OP would have to completely reinstall their OS just to be able to run proxmox just to do what they are already doing without it

1

u/Readdeo 1h ago

Please never work in IT

2

u/b1urbro 2h ago

But you can run Kubernetes on top of Proxmox and it's actually very good advice for a lab where you want to break things, rather then keeping uptime high.

2

u/redfusion Global SRE Lead 11h ago

Literally built this last week.

K3s in HA mode so each instance is a controller and a worker.

I'm using flux pointed to a git repo as a mono repo with a bunch of applications defined as helm or kustomisation projects.

I'm moving from a single hp gen8 running nomad.

I'll be honest, it's been a pain to reset my expectations for storage in particular.

Apps wanting to share paths or nfs targets, bungling around with sqllite over nfs is horrible.

Dealing with GPU attachments for transcoding and machine learning hasn't been too bad though.

Still an ongoing project. K3s made it simple, flux isn't as nice as ArgoCD but it's lighter to run for my use case

1

u/TruckeeAviator91 5h ago

Sqlite over nfs is terrible. I had to run it for a while and had so many issues.

With your setup, it could be worth while to use longhorn or ceph rook. Works best with 10gb+ networking but, I've gotten away with 1gb for home lab testing.

•

u/redfusion Global SRE Lead 1m ago

Yeah, I've been considering options, but my current might end up being local path to nvme with a litespeed sidecar syncing the data off box. If a migration/failure occurs the storage gets populated from the backup but I keep the high performance drives.

I've only got 1gbit on the 5090 optiplex, a m2.e card to 2.5g is possible but I'm not pulling that thread just yet

6

u/IIGrudge DevOps 15h ago

These days, why do you need all this? Install Kind or K3s and you can mimic as many nodes as you want. Easier to sandbox too cause of quick setup/teardown.

11

u/redskelly 15h ago edited 14h ago

What do you mean by “all this”? 3 machines instead of 1?

Because you learn way more using 3 physical computers running K3s in Proxmox instead of having all the K3s nodes in Proxmox on one physical computer.

- quorum is actually breakable, by pulling power on 1/3 machines, teaches corosync behavior and etcd quorum loss. On one machine, the hypervisor never dies independently.

- live migration, HA groups, fencing. Non existent with one host.

- storage replication with real failure domains. Longhorn replicas across 3 hosts fail differently than 3 replicas sharing one disk. Same for ZFS or Ceph.

- kube-vip failover across physical layer 2, VLAN trunk config

- draining a node, patching proxmox, rejoining, with workloads still up.

- network switch configuration

3

u/pydood 12h ago

What are the chances this person is going to be working with physical nodes or just needs to know how to diagnose things with Kubectl against a GKE/EKS cluster? You’re not wrong about depth, but the point here is that you can learn most of the day to day shit needed with a much lighter setup. For someone just coming into the space they likely should start simple and dive deep, not start at the bottom of the iceberg.

1

u/engineerL 11h ago

I'd argue OP is not going deep enough. If you can't improvise a power cable from spare copper wire and solder a broken motherboard, are you even hireable in 2026?

1

u/Commercial-Virus2627 3h ago

Bruh can’t even quoth the raven in binary smh /s

12

u/Liquid_G 15h ago

because sometimes its a good learning opportunity to have multiple nodes, test HA, etcd recovery.

Way back when I was learning k8s, I did roughly the same with a bunch of intel nuc's running VMWare. It was a great resource to have at home to learn / try shit that I couldn't do at work.

2

u/cobolfoo 14h ago

Yup, nothing beat shutting down VMs, unplugging network cards or power cable and seeing live what happens in HA.

1

u/Readdeo 13h ago

Even K3S needs 3 nodes if it is a proper, professionally mature one, especially with etcd. Also, you will need nvme ssd with etcd if you want workloads on the control nodes too.

1

u/titi_prosop 14h ago

Nice setup there, mate! Have fun playing around. I am wondering if you have a recommendation for a guide or some sorts of tutorials regarding achieving your goals for somebody less experienced?

1

u/Dratir System Engineer 13h ago

Nice, what hardware did you pick?

1

u/UpdateInProgress 13h ago

I just built my own cluster with multipass from within my ubuntu rig. It works awesome and it’s coming in handy for the CKA prep

1

u/throwawaystopper20 12h ago

Where did you order these from ?

1

u/Character-Athlete847 7h ago

How much these costs?

1

u/veritable_squandry 4h ago

did it live outside? i can't even keep one mac mini cool for my lab.

1

u/b1urbro 2h ago

https://github.com/kristiangogov/homelab

It's a bit outdated as I simply haven't had the time to tinker this summer between work and holidays. Will probably be rebuilding it from the ground up soon as there may have been some architectural mistakes along the way. Currently just running a compose/ansible setup because there are some services I heavily rely on (Frigate/HA/Silverbullet etc.).

1

u/cobolfoo 14h ago

You should use a promox cluster installed on your baremetal then deploy your kubernetes cluster in it (with like 3 cp, 3 workers spread across your proxmox cluster). I think its a waste of resource to have a whole cp per machine.

I have something very similar at home, but I added some spicy stuff like a CEPH cluster with HA (extra 2.5gb network card + sata disk).

1

u/BuyerEnvironmental69 13h ago

How did it perform? I am considering buying 2.5 NICs for the same setup, but I am worried it might be too slow. Especially with consumer (NVMe) SSDs.

1

u/cobolfoo 12h ago

For homelab it's just fine, I test my opentofu / sensible stuff on it without much issues but it will not saturate the nvme capacities. A 10gb network is another option but cost a bit more.

But keep in mind that having hardware bottlenecks is not a bad thing when you want to learn to deploy robust technology.

1

u/BuyerEnvironmental69 12h ago

Not sure if I understand the last part. Would you mind elaborating on that a bit?

1

u/cobolfoo 3h ago

I meant that having hard restrictions make you a better engineer overall because you have to move around to find a solution. For instance, a CEPH cluster on slow network could create issues that make you learn a lot about how to make CEPH resilient, something you might never encounter with a fast network.

0

u/sylvester_0 15h ago

Was running k3s on some Pi4s before I gave up due to hardware challenges. Much happier with my single node home server on NixOS now. I get to play with k8s plenty at work.