I’m looking for experiences and opinions on kubernetes storage.
I want to create a highly available homelab that spans 3 locations where the pods have a preferred locations but can move if necessary.
I’ve looked at linstore or seaweedfs/garage with juicefs but I’m not sure how well the performance of those options is across the internet and how well they last in long term operation. Is anyone else hosting k3s across the internet in their homelab?
Edit: fixed wording
Those are block storage services, not Storage backends. Your options for SCi Provisioned are these: https://kubernetes.io/docs/concepts/storage/storage-classes/#provisioner
I would say if you’re not truly familiar with K8s, or DEEPLY familiar with Ops, don’t fuck with K8s internals. You’re gonna have a bad time.
I mean storage backends as in the provisioner, I will use local storage on the nodes with either lvm or just storage on a filesystem.
I already set up a cluster and tried linstore, I’m searching for experiences with the options because I don’t want to test them all.
I currently manage all the servers with a NixOS repository but am looking for better failover.
You won’t find any benchmarks for distributed filesystems, because they don’t apply to any one setup. Nobody knows your network situation, disk speed, availability across clusters…etc.
Well, if that is the case then I will have to try them all but I’m hoping at least general behaviour would be similar to others so that I can start with a good option.