Introduction

In my homelab, I wanted to take automation further by deploying a complete Kubernetes cluster on Proxmox, relying on infrastructure as code and automation tools such as Ansible. The goal was to be able to quickly provision a reproducible k8s cluster, ready to host workloads, while keeping control over every step of the deployment through code. This progression naturally led me to Kubespray.

Context

After first installing a Kubernetes cluster manually with kubeadm, I found the process particularly tedious. That experience convinced me of the need to automate and standardise the deployment, to gain reproducibility and efficiency.

This is where Kubespray comes in: this open source project provides ready-to-use Ansible playbooks to install, configure and maintain a Kubernetes cluster with a 100% as code approach. Unlike kubeadm, it offers fine-grained control over the cluster components (CRI, CNI, DNS, etc.) while remaining automated and reproducible. In this article, I describe the approach I followed, the technical choices I made and the tools I used to reach this goal.

Prerequisites

Before starting the deployment, you need the following resources:

Infrastructure:

• At least 24 GB of RAM available on the Proxmox hypervisor
• 150 GB of free storage
• Network access with static IP assignment
• An SSH key to access the VMs

Components

• Proxmox 8.X+: provides the infrastructure to run and manage virtual machines.
• Ansible 2.16+: the automation tool used by Kubespray to deploy and configure Kubernetes.
• Terraform 1.10+ & Packer 1.12+: to automate the creation and configuration of the VMs.

Target cluster architecture

The cluster is made of three nodes with the following sizing:

Creating the VMs

For my homelab, I went for a simple architecture made of three VMs: one control plane and two worker nodes.

There are several reasons for this choice:

• Simplicity for learning: with three nodes, the cluster stays simple to manage while still allowing you to try most Kubernetes features (load balancing, pod affinity, fault tolerance, etc.).
• Worker fault tolerance: two workers make it possible to test node failure scenarios and observe how Kubernetes behaves when rescheduling pods.
• Resource control: this sizing stays reasonable in terms of resources consumed (4 vCPU, 8 GB RAM, 50 GB storage per VM), which matters in a homelab.

📝 Note

High availability: this architecture has a single control plane. For real high availability in production, you would need at least 3 control planes (an odd number for the etcd quorum) with a load balancer in front of the API servers, or dedicated etcd instances outside the cluster. For this homelab, a single control plane is enough for learning and experimenting.

With the sizing defined, let’s create the VMs. For this, I used Terraform and Packer, which lets me deploy my infrastructure quickly and in a fully automated way. I explain this approach in detail, in particular building Proxmox templates with Packer and deploying them with Terraform, in my article on Proxmox templates with Packer.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

############################
# VMs for Kubernetes Cluster
############################
resource "proxmox_virtual_environment_vm" "k8s_cluster" {
  for_each = {
    k8s-master = {
      description = "K8S Control Plane"
      vm_id       = 110
      ip_address  = "192.168.1.110/24"
      tags        = ["k8s_cluster", "kube_control_plane", "etcd"]
    }
    k8s-node1 = {
      description = "K8S Worker Node 1"
      vm_id       = 111
      ip_address  = "192.168.1.111/24"
      tags        = ["k8s_cluster", "kube_node"]
    }
    k8s-node2 = {
      description = "K8S Worker Node 2"
      vm_id       = 112
      ip_address  = "192.168.1.112/24"
      tags        = ["k8s_cluster", "kube_node"]
    }
  }

  name                = each.key
  description         = each.value.description
  tags                = concat(["debian12", "terraform"], each.value.tags)
  node_name           = "pve"
  vm_id               = each.value.vm_id
  pool_id             = "PROD"
  reboot_after_update = false

  ...

  clone {
    vm_id        = 1101
    datastore_id = "local-lvm"
    full         = true
    node_name    = "pve"
  }
}

Once the Terraform code is applied, the three virtual machines are deployed on Proxmox. Each VM gets one or more of these tags: (k8s_cluster, kube_control_plane, etcd, kube_node). These tags make it easier to generate the Kubespray inventory, which I describe further down in the article.

Network configuration and SSH access

Thanks to Cloud-Init during the VM creation process (through the Packer template and the initialization argument of the proxmox_virtual_environment_vm resource of the Terraform provider), the network configuration and SSH access are automated:

• Static IP address: each VM gets its static IP directly at deployment, defined in the Terraform ip_config section. This avoids any manual configuration after deployment.
• SSH key: the public SSH key is injected automatically into each VM through Cloud-Init, which allows an immediate connection without copying the key by hand.

This process makes it possible to manage all the VMs with Ansible as soon as they are created, without any further manual step. In my case, I can therefore use the Kubespray playbooks directly.

⚠️ Warning

In a production environment, it is recommended to manage secrets (Proxmox API tokens, SSH keys, etc.) with dedicated tools such as Ansible Vault or external secret managers.

The resources are now ready and configured, we can move on to setting up the Kubernetes cluster with Kubespray.

Setting up Kubespray

Cloning and configuring the Kubespray repository

I added Kubespray to my project as a git submodule. This lets you include another repository while keeping its history separate. It makes dependency management easier and lets you track a specific version of Kubespray, update it easily, and follow the project’s changes without mixing source code.

1

git submodule add https://github.com/kubernetes-sigs/kubespray.git kubespray

To pin the submodule to a specific tag or commit, go into the submodule directory and check out the desired commit or tag. Then go back to the main project directory, commit the changes and push them.

1
2
3
4

cd kubespray/
git checkout v2.27.0  # Version used for this deployment
cd ../
git commit -m "Pinned submodule to v2.27.0"

Pinning the Kubespray version is important to guarantee reproducible deployments and avoid any instability caused by uncontrolled changes in the upstream project.

Once the Kubespray project is integrated and configured in the Ansible repository, we can move on to generating the Kubespray inventory.

Creating the Kubespray inventory

For the Ansible inventory used by Kubespray, I chose the Proxmox dynamic inventory. This gives an inventory that is always up to date, since it is loaded at every playbook run, which avoids maintaining the inventory file by hand.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

plugin: community.general.proxmox
url: https://<proxmox_host>:8006
user: "<user>"
token_id: "<token_id>"
token_secret: "<token_secret>"
validate_certs: false # Only in a homelab
want_facts: true
keyed_groups:
  - key: proxmox_tags_parsed
    separator: ""
compose:
  ansible_host: proxmox_agent_interfaces[1]["ip-addresses"][0].split('/')[0]

The inventory above retrieves all the VMs in Proxmox, groups them by their proxmox_tags_parsed tags and sets the ansible_host variable of each inventory entry to the IP of each VM.

For Kubespray, the tags must follow the naming described in the documentation, because that is what runs the playbooks on the right hosts.

To generate the inventory, I use the command below, where the --graph option makes the output a bit more visual, to clearly tell apart the groups and hosts of the Ansible inventory.

1

ansible-inventory -i inventories/proxmox.yml --graph

In my case, this gives the following result:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

@all:
  ...
  ...
  |--@debian12:
  |  |--k8s-node1
  |  |--k8s-master
  |  |--k8s-node2
  |--@k8s_cluster:
  |  |--k8s-node1
  |  |--k8s-master
  |  |--k8s-node2
  |--@kube_node:
  |  |--k8s-node1
  |  |--k8s-node2
  |--@terraform:
  |  |--k8s-node1
  |  |--k8s-master
  |  |--k8s-node2
  |--@etcd:
  |  |--k8s-master
  |--@kube_control_plane:
  |  |--k8s-master
  ...
  ...

Once the inventory is in place, I can test the connectivity of the VMs with the ad hoc command below, which uses the Ansible ping module:

1

ansible -i inventories/proxmox.yml k8s_cluster -m ping -u <username>

Which gives the following result:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

k8s-node1 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
k8s-master | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
k8s-node2 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

Now that I have the Proxmox dynamic inventory and reachable virtual machines, I can move on to customising my K8s cluster.

Customising the variables

To customise the Kubernetes cluster, you go through Ansible group_vars. These variables let us configure various aspects of the cluster, such as:

• Kubernetes settings (kube version, internal network, DNS, …)
• Container Runtime Interface (CRI)
• Container Network Interface (CNI)
• Container Storage Interface (CSI)
• ETCD configuration
• Addons and optional components (ArgoCD, Helm, cert-manager, MetalLB, …)
• And much more…

These variables are spread across several inventory files to better structure the configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

inventories/group_vars/
├── etcd
│   ├── all.yml
│   └── etcd.yml
└── k8s_cluster
    ├── addons.yml
    ├── all.yml
    ├── cri-o.yml
    ├── k8s-cluster.yml
    └── k8s-net-flannel.yml

For more information about the possible cluster configuration and the available components, refer to the Kubespray documentation.

📝 Note

In this article, I mostly use the default configurations provided by Kubespray. The goal is to show the deployment process rather than justify specific technical choices for each component (CNI, CRI, etc.). These choices can be adapted to the specific needs of each project.

The target configuration of the Kubernetes cluster is defined, the next step is the deployment!

Deploying the Kubernetes cluster

Before running the deployment with Ansible, I need to set up my environment. To do this, I use a Python venv to pin the version of Ansible and of my dependencies, so that everything is isolated and potential conflicts with other projects are avoided. The venv also gives a reproducible environment, which is crucial to run deployments identically.

1
2
3

python3 -m venv venv
source venv/bin/activate
pip install -r kubespray/requirements.txt

Once the environment is ready, it is finally time to deploy the cluster, by running the cluster.yml playbook with the Proxmox dynamic inventory configured earlier.

1

ansible-playbook -i inventories/proxmox.yml -b -u <username> kubespray/cluster.yml

Once the playbook has finished, the Kubernetes cluster is ready to use with the configuration defined above.

1
2
3
4

PLAY RECAP *************************************************************************
k8s-master: ok=599 changed=39 unreachable=0 failed=0 skipped=965 rescued=0 ignored=1
k8s-node1 : ok=421 changed=25 unreachable=0 failed=0 skipped=618 rescued=0 ignored=1
k8s-node2 : ok=443 changed=28 unreachable=0 failed=0 skipped=681 rescued=0 ignored=1

Checking the cluster

Once the deployment is complete, it is essential to check that the Kubernetes cluster is operational and that all components work correctly. Several kubectl commands let you validate the state of the nodes and pods.

To check the state of the cluster nodes:

1

kubectl get nodes

Result:

1
2
3
4

NAME         STATUS   ROLES           AGE   VERSION
k8s-master   Ready    control-plane   1d    v1.31.4
k8s-node1    Ready    worker          1d    v1.31.4
k8s-node2    Ready    worker          1d    v1.31.4

All nodes must be in the Ready state, which means they are ready to run workloads.

To make sure all pods are running, use the command:

1

kubectl get pods -A

Result:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

NAMESPACE              NAME                                         READY   STATUS      RESTARTS      AGE
external-dns           external-dns-754bcb687f-cqncz                1/1     Running     0             24h
ingress-nginx          ingress-nginx-controller-6d7ff9f49f-2b9dq    1/1     Running     0             24h
kube-system            coredns-d665d669-7h6nf                       1/1     Running     0             24h
kube-system            coredns-d665d669-fklwz                       1/1     Running     0             24h
kube-system            dns-autoscaler-5cb4578f5f-zkr67              1/1     Running     0             24h
kube-system            kube-apiserver-k8s-master                    1/1     Running     0             24h
kube-system            kube-controller-manager-k8s-master           1/1     Running     0             24h
kube-system            kube-flannel-9n6zs                           1/1     Running     0             24h
kube-system            kube-flannel-m5xtr                           1/1     Running     0             24h
kube-system            kube-flannel-qchzm                           1/1     Running     0             24h
kube-system            kube-proxy-54k4k                             1/1     Running     0             24h
kube-system            kube-proxy-qpb4m                             1/1     Running     0             24h
kube-system            kube-proxy-zq7ww                             1/1     Running     0             24h
kube-system            kube-scheduler-k8s-master                    1/1     Running     0             24h
kube-system            nginx-proxy-k8s-node1                        1/1     Running     0             24h
kube-system            nginx-proxy-k8s-node2                        1/1     Running     0             24h
kube-system            nodelocaldns-l2nc4                           1/1     Running     0             24h
kube-system            nodelocaldns-pp8ng                           1/1     Running     0             24h
kube-system            nodelocaldns-x522v                           1/1     Running     0             24h
kubernetes-dashboard   kubernetes-dashboard-8984b9757-2pcmz         1/1     Running     0             24h
kubernetes-dashboard   kubernetes-metrics-scraper-6d4c5d99f9-z6g6b  1/1     Running     0             24h
metallb-system         controller-7f649565d4-6dcd8                  1/1     Running     0             24h
metallb-system         speaker-6pj99                                1/1     Running     0             24h
metallb-system         speaker-v44zd                                1/1     Running     0             24h
metallb-system         speaker-w8wzp                                1/1     Running     0             24h

All pods must be in the Running state. This guarantees that the cluster components are correctly deployed and working.

These checks confirm that the Kubernetes cluster is ready to host workloads and that the essential services are up.

Result

Post-deployment operations

Lifecycle management

Kubespray is not limited to the initial deployment, it also provides playbooks to manage the full lifecycle of the cluster:

Adding nodes:

1
2

# To add a new worker node to the cluster
ansible-playbook -i inventories/proxmox.yml -b -u <username> kubespray/scale.yml

Upgrading the cluster:

1
2

# To upgrade the Kubernetes version or the components
ansible-playbook -i inventories/proxmox.yml -b -u <username> kubespray/upgrade-cluster.yml

Full reset:

1
2

# To remove the cluster completely
ansible-playbook -i inventories/proxmox.yml -b -u <username> kubespray/reset.yml

Conclusion

Automating the deployment of a Kubernetes cluster with Ansible through the Kubespray project brings reliability, speed and reproducibility, which is essential for me in my homelab.

Thanks to the dynamic inventory and the Ansible variables, it is easy to adapt the cluster configuration to my needs, while keeping control over each component. This approach also makes maintaining and evolving the k8s cluster easier, since everything is done as code and Kubespray also provides playbooks to add a node to the cluster (scale.yml) or even reset the cluster (reset.yml).

Workflow

Useful links

• Kubespray documentation
• Ansible inventory for Proxmox documentation
• Example Kubespray group_vars