Kubernetes Node Swap Is GA: Set Up LimitedSwap Safely

Kubernetes nodes usually run out of RAM long before they run out of CPU, and agent-style AI workloads widen that gap. They often need a large memory footprint to start, then sit idle waiting for the next request. Swap gives the node a way to move that idle memory to disk instead of hitting the eviction limit straight away. Node swap support reached general availability in Kubernetes v1.34, so it is now a supported option for production clusters. This post explains the two swap behaviours, which pods may use swap, and what to check before you switch it on.

What changed in v1.34

The kubelet controls how workloads use swap through one setting, memorySwap.swapBehavior. It accepts two values. NoSwap is the default and means pods cannot use swap at all. LimitedSwap lets some pods use a share of the swap space on the node.

Adding swap to the operating system is not enough on its own. By default the kubelet will not start on a Linux node that has swap enabled, so you also need failSwapOn: false. Note that even with NoSwap, processes outside Kubernetes-managed containers, such as systemd services and the kubelet itself, can still use swap.

Which pods get swap under LimitedSwap

LimitedSwap is restrictive by design. Only pods in the Burstable QoS class may use swap. BestEffort and Guaranteed pods are not allowed to, and high-priority pods are kept out of swap so their memory stays in RAM. A container inside a Burstable pod can opt out by setting its memory request equal to its memory limit.

The swap a container may use is proportional to its memory request:

container swap limit = (containerMemoryRequest / nodeTotalMemory) x totalPodsSwapAvailable

A pod with a small request therefore gets only a small slice of swap. The Kubernetes documentation also points out that part of the swap space usually stays unavailable to workloads because of this design.

Preparing a node

Test on one non-production node first. Work through these steps in order.

1. Confirm the node uses cgroup v2. Run stat -fc %T /sys/fs/cgroup/ on the node and check that the answer is cgroup2fs. The kubelet sets memory.swap.max through the cgroup v2 path.
2. Create swap space on the host. The example below makes a 4 GiB swap file and makes it persistent across reboots:

sudo fallocate -l 4G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

3. Set the kubelet configuration. Add these fields to your KubeletConfiguration file and keep any existing settings you need:

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
failSwapOn: false
memorySwap:
  swapBehavior: LimitedSwap

4. Restart the kubelet with sudo systemctl restart kubelet, then check that the node is Ready.
5. Verify from the cluster side. Run kubectl top nodes --show-swap and confirm that swap capacity appears for the node. Kubelet metrics at /metrics/resource also expose node_swap_usage_bytes and container_swap_usage_bytes for dashboards and alerts.

Eviction thresholds and risks

Swap makes memory behaviour harder to predict. Moving data back into RAM can be many times slower than normal memory access, which may cause performance drops that look unrelated to the workload. Swap also increases the chance of noisy neighbours, where a pod that uses a lot of RAM forces other pods to swap.

The documentation recommends setting kubelet eviction thresholds slightly below the vm.min_free_kbytes value on the host. Check the current value with sysctl vm.min_free_kbytes. This lets the node begin swapping before the kubelet starts evicting pods. Memory-backed volumes, such as secrets and emptyDir volumes with medium: Memory, use a tmpfs mount with the noswap option, so their contents should stay in RAM.

How TechProvidence can help

Changing memory behaviour on a live cluster needs careful testing, especially when swap, eviction thresholds and QoS classes all interact. TechProvidence runs Kubernetes upgrades, node configuration changes and managed server operations for teams that do not want to test these changes alone. If you are planning a move to v1.34 or want swap tested on your nodes, contact us.

Source: Kubernetes Blog