Navigate back to the homepage

Secure HA Kubernetes on bare metal using k3s

Claas Störtenbecker
December 21st, 2020 · 1 min read

This walkthrough will leave you with a kubeconfig to a fully functional, secure cluster.

What is k3s?

Kubernetes is was too hard for the solo developer. Using k3s you can deploy Kubernetes in a couple of commands per node. The Kubernetes distribution is:

  • fully conformant
  • production-ready
  • lightweight
  • packaged in a single binary

I personally run 3 SKVM-4G from MaxKVM for 18$/month with student discount. Each node has 2 EPYC cores, 4GB ECC, 3TB traffic @ 1GBs, 75GB NVME; unmatched for that price.

1. Setup node iptables

To isolate network communication we need to setup some firewall rules on each node. This tutorial will use ufw to configure iptables but you may use any other tool. I assume that each node will have two distinct network interfaces:

  • eth0 assigned with a public IP used for internet communication
  • eth1 assigned with a private IP used for intranet communication

We will start by blocking incoming and allowing outgoing traffic.

1sudo ufw default allow outgoing
2sudo ufw default deny incoming

Further we need to enable ssh access to the node.

1sudo ufw allow ssh

We allow communication through the intranet. You can also add each node IP seperately. In this example eth1 is part of the subnet 172.16.0.0/12.

1sudo ufw allow from 172.16.0.0/12

We allow communication to the CNI (Container Network Interface) from the Pod CIDR.

1sudo ufw allow in on cni0 from 10.42.0.0/16

We allow communication to the Kubernetes API.

1sudo ufw allow 6443/tcp

2. Install Kubernetes

To join nodes k3s needs a shared secret. We can generate one with:

1openssl rand -base64 32

2.1 First node

We configure necassary installation parameters on the server:

  • K3S_TOKEN the secret we generated in step 2
  • INSTALL_K3S_EXEC arguments the k3s installer is called with
    • --cluster-init this node initializes a completely new cluster
    • -i {eth1_ip} this node uses the internal IP
    • --flannel-iface eth1 the cluster network is build on the intranet
    • --disable traefik k3s ships with outdated traefik as ingress
1export K3S_TOKEN="{generated_secret}" &
2export INSTALL_K3S_EXEC="server \
3 --cluster-init \
4 -i {eth1_ip} \
5 --flannel-iface eth1 \
6 --disable traefik"

Now we execute the install script.

1curl -sfL https://get.k3s.io | sh -

2.2 Other nodes

Configuration is similar to the first node. Now we do not initialize a cluster, but join the cluster that was created in 2.1.

1export K3S_TOKEN="{generated_secret}" &
2export INSTALL_K3S_EXEC="server \
3 --server https://{node1_internal_ip}:6443 \
4 -i {eth1_ip} \
5 --flannel-iface eth1 \
6 --disable traefik"

Also execute the install script.

1curl -sfL https://get.k3s.io | sh -

2.3 Finalize

On any node run kubectl get nodes to check if your cluster was build sucessfully. Now remove from /etc/systemd/system/k3s.service on every node:

  • --cluster-init if it is the first node
  • --server else

at the bottom of the file.

Reload and check the node after editing.

1systemctl daemon-reload
2service k3s restart
3kubectl get nodes

Copy /etc/rancher/k3s/k3s.yaml to your machine. After that replace localhost with a node or loadbalancer IP and you are done.

Join our email list and get notified about new content

Be the first to receive our latest content with the ability to opt-out at anytime. We promise to not spam your inbox or share your email with any third parties.

More articles from clstb.codes

Managing data models in Go

How I avoid duplicating data structures in Go services.

March 4th, 2021 · 1 min read

Secure HA Kubernetes on bare metal using k3s

Walkthrough deploying secure HA Kubernetes on bare metal using k3s and embedded etcd.

December 21st, 2020 · 1 min read
© 2020–2021 clstb.codes
Link to $https://github.com/clstbLink to $https://linkedin.com/in/clstbLink to $mailto:[email protected]