Skip to content

K3S Traefik Setup

Brief

Enabling traefik access to dashboard and metrics for traefik ingress controller in k3s kubernetes cluster

  • by rskntroot on 2024-07-01

Assumptions

$ k3s --version
k3s version v1.29.5+k3s1 (4e53a323)
go version go1.21.9

$ kubectl version
Client Version: v1.29.5+k3s1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.5+k3s1

Traefik Dashboards

K3S comes packaged with Traefik Dashboard and Prometheus Metrics which are disabled by default.

Preparation

Set DNS record traefik.your.domain.com in a non-public DNS

Alternatively, you can just edit your workstations hosts file.

/etc/hosts
10.0.0.1    traefik.your.domain.com

This example does not include authentication. Exposing these dashboards is a security risk.

Update Manifest

On host with kubectl access.

Add the following to spec.valuesContent in:

vim /var/lib/rancher/k3s/server/manifests/traefik.yaml 

    dashboard:
      enabled: true
    metrics:
      prometheus: true

spec:
  chart: https://%{KUBERNETES_API}%/static/charts/traefik-25.0.3+up25.0.0.tgz
  set:
    global.systemDefaultRegistry: ""
  valuesContent: |-
    deployment:
      podAnnotations:
        prometheus.io/port: "8082"
        prometheus.io/scrape: "true"
    dashboard:
      enabled: true
    metrics:
      prometheus: true

Restart Ingress Controller

kubectl -n kube-system scale deployment traefik --replicas=0
# wait a few seconds
kubectl -n kube-system get deployment traefik
kubectl -n kube-system scale deployment traefik --replicas=1

$ kubectls scale deployment traefik --replicas=0
deployment.apps/traefik scaled
$ kubectls get deployment traefik
NAME      READY   UP-TO-DATE   AVAILABLE   AGE
traefik   0/0     0            0           3d1h
$ kubectls scale deployment traefik --replicas=1
deployment.apps/traefik scaled

Create Resource Definition YAML

Save the following to traefik-dashboard.yml in your workspace.

traefik-dashboard.yml
apiVersion: v1
kind: Service
metadata:
  name: traefik-dashboard
  namespace: kube-system
  labels:
    app.kubernetes.io/instance: traefik
    app.kubernetes.io/name: traefik-dashboard
spec:
  type: ClusterIP
  ports:
  - name: traefik
    port: 9000
    targetPort: 9000
    protocol: TCP
  selector:
    app.kubernetes.io/instance: traefik-kube-system
    app.kubernetes.io/name: traefik

---

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: traefik-ingress
  namespace: kube-system
  annotations:
    spec.ingressClassName: traefik
spec:
  rules:
    - host: traefik.${DOMAIN}
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: traefik-dashboard
                port:
                  number: 9000
traefik-dashboard.yml
apiVersion: v1
kind: Service
metadata:
  name: traefik-metrics
  namespace: kube-system
  labels:
    app.kubernetes.io/instance: traefik
    app.kubernetes.io/name: traefik-metrics
spec:
  type: ClusterIP
  ports:
  - name: traefik
    port: 9100
    targetPort: 9100
    protocol: TCP
  selector:
    app.kubernetes.io/instance: traefik-kube-system
    app.kubernetes.io/name: traefik

---

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: traefik-ingress
  namespace: kube-system
  annotations:
    spec.ingressClassName: traefik
spec:
  rules:
    - host: traefik.${DOMAIN}
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: traefik-dashboard
                port:
                  number: 9000
          - path: /metrics
            pathType: Prefix
            backend:
              service:
                name: traefik-metrics
                port:
                  number: 9100
traefik-dashboard.yml
apiVersion: v1
kind: Service
metadata:
  name: traefik-dashboard
  namespace: kube-system
  labels:
    app.kubernetes.io/instance: traefik
    app.kubernetes.io/name: traefik-dashboard
spec:
  type: ClusterIP
  ports:
  - name: traefik
    port: 9000
    targetPort: 9000
    protocol: TCP
  selector:
    app.kubernetes.io/instance: traefik-kube-system
    app.kubernetes.io/name: traefik

---

apiVersion: v1
kind: Service
metadata:
  name: traefik-metrics
  namespace: kube-system
  labels:
    app.kubernetes.io/instance: traefik
    app.kubernetes.io/name: traefik-metrics
spec:
  type: ClusterIP
  ports:
  - name: traefik
    port: 9100
    targetPort: 9100
    protocol: TCP
  selector:
    app.kubernetes.io/instance: traefik-kube-system
    app.kubernetes.io/name: traefik

---

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: traefik-ingress
  namespace: kube-system
  annotations:
    spec.ingressClassName: traefik
spec:
  rules:
    - host: traefik.${DOMAIN}
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: traefik-dashboard
                port:
                  number: 9000
          - path: /metrics
            pathType: Prefix
            backend:
              service:
                name: traefik-metrics
                port:
                  number: 9100

Create Service & Ingress Resources

First, set the environment variable for to your domain.

export DOMAIN=your.domain.com

envsubst < traefik-dashboard.yml | kubectl apply -f -

$ envsubst < traefik-dashboards.yml | kubectl apply -f -
service/traefik-dashboard created
service/traefik-metrics created
ingress.networking.k8s.io/traefik-ingress created
$ kubectls get svc | grep traefik-
traefik-dashboard   ClusterIP      10.43.157.54    <none>    9000/TCP    25s
traefik-metrics     ClusterIP      10.43.189.128   <none>    9100/TCP    25s

Why are passing the yaml file into envsubst? (1)

  1. envsubst - gnu - enables code-reuse by providing environment variable substituion as demonstrated above.

Access Dashboards

That's it. You should now be able to access the Traefik Ingress Controller Dashboard and metrics remotely.

Don't forget to include the appropriate uri paths:

https://traefik.your.domain.com/dashboard/

When navigating to the traefik dashboard the / at the end is necessary. /dashboard will not work. 

https://traefik.your.domain.com/metrics

Disable Dashboards

envsubst < traefik-dashboard.yml | kubectl delete -f -

$ envsubst < traefik-dashboards.yml | kubectl delete -f -
service "traefik-dashboard" deleted
service "traefik-metrics" deleted
ingress.networking.k8s.io "traefik-ingress" deleted

Shortcuts

alias kubectls

When using an alias to substitute kubectl command completion will not work.

echo 'alias kubectls="kubectl -n kube-system"' >> ~/.bashrc
source ~/.bashrc

$ echo 'alias kubectls="kubectl -n kube-system"' >> ~/.bashrc
$ source ~/.bashrc
$ kubectls get deployments
NAME                     READY   UP-TO-DATE   AVAILABLE   AGE
coredns                  1/1     1            1           3d2h
local-path-provisioner   1/1     1            1           3d2h
metrics-server           1/1     1            1           3d2h
traefik                  1/1     1            1           3d2h

Alternatives

  • skubectl means you can hit [up-arrow] [ctrl]+[a] [s] [enter] when you inevitably forget to include -n kube-system
  • kubectls just adds [alt]+[right-arrow] into the above before [s]
  • kubesctl makes sense because all of these are really kube-system-ctl, but that adds 4x [right-arrow], ewww.

References