Using Longhorn with K3s


estimated read time: 2 minutes

What is a Persistent Volume?

A Persistent Volume (PV) is a piece of storage in the cluster that has been provisioned statically or dynamically using Storage Classes. It’s a cluster resource, much like a node, and has a lifecycle independent of any individual Pod that uses it.

What is Longhorn?

Longhorn is a lightweight, reliable, and easy-to-use distributed block storage system for Kubernetes.

Why Do You Need It?

Because we’re running our cluster on physical hardware rather than a cloud provider, we don’t have access to the quality-of-life features you’d normally get from the cloud—such as high availability and automated backup tools. When using our own hardware, we need to take on this responsibility ourselves.

Containers should be stateless, but sometimes we need to maintain state. Examples include:

Features of Longhorn

High Availability

Longhorn works as a DaemonSet, so by default storage is replicated across cluster nodes. The advantage is that if one drive becomes unavailable, one of the replicas takes over as the active storage node. The disadvantage is that this duplication multiplies storage requirements by the number of nodes.

In my setup with four nodes, each 1GB of storage requires 4GB in total.

Setting Up Longhorn

Installing Longhorn

Using kubectl

kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.3.2/deploy/longhorn.yaml

This can take a while. You can watch the status of the pods with:

watch kubectl get pods --namespace longhorn-system

Once all pods are running, create an Ingress rule for the service. Create a file called ingress.yaml, replacing <master_node> with the IP of one of your master nodes:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: longhorn-ingress
  namespace: longhorn-system
  annotations:
    kubernetes.io/ingress.class: "traefik"
spec:
  rules:
    - host: "longhorn.<master_node>.nip.io"
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: longhorn-frontend
                port:
                  number: 80

Apply the configuration:

kubectl apply -f ingress.yaml

Longhorn dashboard

Pi Cluster Series