Secure k3s Cluster with cert-manager using arkade

· 4 min read

In previous post I described how you could setup your own k3s cluster using a few Raspberry Pis. In this post I will show you how to secure the cluster by adding TLS to the ingress routes.

This will be achieved by setting up our own certificate authority and using cert-manager to issue certificates for the ingress routes.

Step 1, Install cert-manager

cert-manager is a tool that automates certificate management for Kubernetes clusters. It supports CA, Hashicorp Vault, Venafi and ACME based signing authorities. We will be using the CA issuer.

There are a bunch of different ways to install cert-manager, basic manifests and helm charts to name a couple. We will be using arkade, a super-simple utility written by Alex Ellis for installing helm charts and applications to Kubernetes clusters.

arkade can be installed using the following command, beware it uses sudo to move the binary to /usr/local/bin. If you don’t like to run arbitrary scripts from the internet as root then you can run it unprivileged and move the binary manually yourself.

curl -sLS https://dl.get-arkade.dev | sudo sh

Then install cert-manager as follows:

arkade install cert-manager

Step 2, Create a Certificate Authority

To make this as simple as possible we will be using a utility described as a “PKI/TLS swiss army knife”, cfssl.

cfssl releases can be found on Github. Below are some instructions on installing version 1.4.1 (the most up to date as of writing) cfssl for Linux.

sudo curl hhttps://github.com/cloudflare/cfssl/releases/download/v1.4.1/cfssl_1.4.1_linux_amd64 -o /usr/local/bin/cfssl
sudo curl https://github.com/cloudflare/cfssl/releases/download/v1.4.1/cfssljson_1.4.1_linux_amd64 -o /usr/local/bin/cfssljson
sudo chmod +x /usr/local/bin/cfssl /usr/local/bin/cfssljson

The first step is to create a ca.json file which will provide the specification for our CA. Here is what I am using.

{
  "CN": "Cluster CA",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
  {
    "C": "UK",
    "O": "Cluster",
    "OU": "Cluster Root CA"
  }
 ]
}

We can then generate our CA’s certificate, key and certificate signing request using cfssl:

cfssl gencert -initca ca.json | cfssljson -bare ca

Since this is for an internal development cluster we are not going to generate an intermediate CA.

Step 3, Create ClusterIssuer

The next step is to install the CA in to the k3s cluster as a secret. The ca-certificate.yml for this is as follows:

apiVersion: v1
kind: Secret
metadata:
  name: ca-key-pair
  namespace: cert-manager
data:
  tls.crt: # base64 encoded contents of ca.pem
  tls.key: # base64 encoded contents of ca-key.pem

This can then be installed in to the k3s cluster using kubectl apply -f ca-certificate.yml.

This secret can then be used by cert-manager to issue certificates. This can be done in two ways, using an Issuer or a ClusterIssuer. An Issuer is a namespaced resource and will only issue certificates to resources in the namespace it is deployed in. A ClusterIssuer is not namespaced, it can issue certificates to resources in any namespace. Since we are the only users of this cluster we will deploy a ClusterIssuer.

The cluster-issuer.yml for this is as follows:

apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: ca-issuer
  namespace: cert-manager
spec:
  ca:
    secretName: ca-key-pair

This can then be installed in to the k3s cluster using kubectl apply -f cluster-issuer.yml.

Step 4, Deploy an Application with TLS Ingress

The below yaml will create a simple hello world HTTP application.

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-world-http-server
  namespace: example
  labels:
    app: hello-world-http-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello-world-http-server
  template:
    metadata:
      labels:
        app: hello-world-http-server
    spec:
      containers:
        - name: hello-world-http-server
          image: markdashcs/hello-world-http-server:arm32v7-1.0.0
          imagePullPolicy: Always
          ports:
          - containerPort: 8080
            protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  name: hello-world-http-server
  namespace: example
  labels:
    app: hello-world-http-server
spec:
  ports:
  - port: 8080
    targetPort: 8080
    name: tcp
  selector:
    app: hello-world-http-server
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: hello-world-http-server
  namespace: example
  annotations:
    kubernetes.io/ingress.class: "traefik"
    cert-manager.io/cluster-issuer: ca-issuer
spec:
  rules:
  - host: hello.home
    http:
      paths:
      - path:
        backend:
          serviceName: hello-world-http-server
          servicePort: 8080
  tls:
  - hosts:
    - hello.home
    secretName: hello-home-cert

The interesting part is in the Ingress definition.

The cert-manager.io/cluster-issuer annotation tells cert-manager which ClusterIssuer should be used to issue a certificate.

The below fragment are directions for cert-manager on how to issue a certificate. cert-manager will issue a certificate for hello.home and store the certificate and key in a secret called hello-home-cert.

...
  tls:
  - hosts:
    - hello.home
    secretName: hello-home-cert

Once this has been deployed, and you have edited your hosts file to direct hello.home to your k3s cluster you should be able to navigate to https://hello.home. If all is well you should be greeted with “Hello, world!”.

I hope you found this useful.

Comments

comments powered by Disqus