
Air gapped cloud upgrades with Atmosphere release manifests
Air gapped cloud on OpenStack: how Atmosphere's digest-pinned release manifests move a release across the boundary, and what the manifest leaves out.
Lire la noteNotes de terrain / Dernières nouvelles
Des notes d’ingénierie issues de l’exploitation d’infrastructures ouvertes : pannes, décisions de conception et travail upstream qui améliorent l’infrastructure ouverte.
Parcourir toutes les notes
Air gapped cloud on OpenStack: how Atmosphere's digest-pinned release manifests move a release across the boundary, and what the manifest leaves out.
Lire la note
Learn when your team needs OpenStack consulting services and how expert support can help with deployments, migrations, upgrades, and operations.
Lire la note
Assess enterprise AI readiness before investing in GPUs. Explore 12 key factors covering data, models, infrastructure, governance, operations, and costs.
Lire la noteAdd the magnum-cluster-api driver to an OpenStack cloud you already run: services it needs, the management cluster, Kolla-Ansible and manual installs, templates and existing networks.
Your OpenStack cloud was installed with Kolla-Ansible or by hand, and your users now want Kubernetes clusters from it on demand. OpenStack has a component for that job. Magnum takes a cluster request from the openstack coe commands or the Horizon dashboard and builds the cluster out of your cloud's own servers, networks and volumes.
Magnum hands the actual building to a plug-in it calls a driver. magnum-cluster-api is the driver we wrote to pass that work to Cluster API, a Kubernetes community project that writes down every cluster as Kubernetes objects and keeps the real servers in line with them. Those objects live on a separate Kubernetes cluster, the management cluster, which you run next to your cloud.
This walkthrough adds the driver to a cloud that doesn't have it yet. By the end, Magnum will build clusters through Cluster API, and one test cluster will be running on a network your users already have. People have asked for this guide in our issue tracker since January 2023, and it follows release 0.38.2 of the driver, from August. On Atmosphere, our OpenStack distribution, Magnum already runs on the management cluster and the driver's proxy is deployed with the cloud, so most of this work is done for you.
Some mistakes here don't name their cause. A missing kubeconfig on a Kolla-Ansible cloud quietly switches off the Cluster API drivers for Ubuntu, Flatcar and Rocky Linux images. A wrong image property fails template creation with "Cluster type ... not supported", which names the image's OS and leaves out the fix. Both have been reported to us, and each is fixed with one change.
Start with the services. Keystone, Nova, Neutron, Glance and Cinder are the base. The driver package declares Magnum 15.0.0 or later. For release 0.38.2, our CI builds a test cloud on Ubuntu 24.04 from Magnum's master branch. That cloud also runs Barbican, Manila and Octavia with the OVN provider. CI then runs the Kubernetes conformance tests on 1.33 through 1.36 with both Calico and Cilium.
Octavia is a hard requirement. Each cluster puts its Kubernetes API behind an Octavia load balancer, and our getting-started guide says the driver needs that load balancer enabled to work properly. Our maintainers have also said there is no supported highly available way to run these clusters without Octavia.
In every cluster, the driver also installs the Kubernetes cloud provider for OpenStack through its cloud controller manager. By default, it adds the Cinder CSI plugin too, as long as your Keystone catalog lists Cinder. With those two in place, a LoadBalancer Service gets an Octavia load balancer and a volume claim gets a Cinder volume. You pick the Octavia provider per cluster with the octavia_provider label. With OVN, the only balancing algorithm is SOURCE_IP_PORT. The trade-offs are covered in choosing between amphora and OVN for Kubernetes.
Change two Magnum settings early. Magnum's own default network driver is flannel, and the driver accepts only Calico or Cilium, so set the allowed and default drivers the way our CI does. Our CI also points Magnum at Nova API microversion 2.15. That microversion added the soft-anti-affinity server group policy, and the driver applies it to every node group by default.
[cluster_template]
kubernetes_allowed_network_drivers = calico,cilium
kubernetes_default_network_driver = calico
[nova_client]
api_version = 2.15One Cinder detail: our CI adds a volumev3 entry to the Keystone catalog before each test run, as a workaround for the Cinder CSI plugin. If your catalog has no volumev3 entry, test a volume claim on your first cluster.
The management cluster is a Kubernetes cluster where the Cluster API controllers run, separate from the clusters Magnum creates. Cluster API v1.13 supports Kubernetes 1.32 through 1.36 for it. Our maintainers have said a kind cluster is enough for a test. For production, the Cluster API docs ask for a real cluster with backup and disaster recovery in place, because kind isn't designed for production use.
Install the controllers with clusterctl. Our test setup for 0.38.2 first installs the OpenStack Resource Controller (ORC) v2.2.0, then runs:
export CLUSTER_TOPOLOGY=true EXP_CLUSTER_RESOURCE_SET=true
clusterctl init --core cluster-api:v1.13.2 \
--bootstrap kubeadm:v1.13.2 --control-plane kubeadm:v1.13.2 \
--infrastructure openstack:v0.14.4Next, plan the network paths. Four connections have to work:
[capi_client] endpoint_type in magnum.conf, which defaults to public. If those endpoints use a private CA, set [capi_client] ca_file. Without it, the driver falls back to a bundle of public CAs.[drivers] openstack_ca_file.From here on, the management cluster is part of your control plane, so decide where it runs and how it is backed up before it carries production clusters.
![Map of the four connections magnum-cluster-api needs: Magnum hosts reach the management cluster's API port with a kubeconfig, the management cluster reaches OpenStack services with an application credential under [capi_client], each new cluster calls the public Keystone endpoint ([drivers] openstack_ca_file), and both reach each cluster's API on a floating IP by default.](/_next/image/?url=https%3A%2F%2Fstatic.vexxhost.com%2Fmedia%2Fmagnum-cluster-api-existing-openstack-infographic-1.webp&w=3840&q=75)
On Kolla-Ansible, Kolla's Magnum images already include the driver. An older image may carry an older driver; a 2024 report from a Bobcat cloud found exactly that. Run pip show magnum-cluster-api in the magnum_conductor container to see which release you have, and rebuild the image with a newer one if you need it.
Then give Kolla-Ansible the kubeconfig. Put it at /etc/kolla/config/magnum/kubeconfig on the deploy host, add the settings above to /etc/kolla/config/magnum.conf, and run the Magnum deploy again. In the 2025.2 release, Kolla-Ansible copies that file into magnum-api and magnum-conductor as /var/lib/magnum/.kube/config. If the file is missing, Kolla-Ansible lists the Ubuntu, Flatcar and Rocky Linux Cluster API drivers under disabled_drivers in magnum.conf. The 2024 Bobcat report ended there: its drivers were off because the kubeconfig was missing from the Kolla-Ansible configuration.
On a manual install, pip install magnum-cluster-api==0.38.2 into the same Python environment as Magnum, which needs Python 3.10 or later, on every host that runs magnum-api or magnum-conductor. The driver also runs the helm command, so install Helm where magnum-conductor runs, as Kolla does in its conductor image. Put the management cluster's kubeconfig at ~/.kube/config for the account those services run as, then restart both. Our DevStack script installs the package the same way and then restarts both services.
Magnum picks the driver from the image's os_distro property. The driver answers for ubuntu, debian, flatcar and rockylinux, plus the older ubuntu-focal. One user set ubuntu-jammy and got "Cluster type (vm, ubuntu-jammy, kubernetes) not supported" until they changed it to ubuntu.
Each Kubernetes version needs its own node image. We publish prebuilt Ubuntu 24.04 images in the capo-image-elements releases. The 0.38.2 docs use release 2026.05-7, with images for v1.33.12, v1.34.8, v1.35.5 and v1.36.1. Upload each one to Glance with --property os_distro=ubuntu.
Our docs put the average image at about 4 GB. Nova caches images locally, so your first cluster takes longer to build than the ones after it. If nodes pull container images slowly, set the container_infra_prefix label to point them at your own registry mirror.
Then create one template per Kubernetes version:
openstack coe cluster template create k8s-v1.35.5 \
--coe kubernetes --image ubuntu-24.04-v1.35.5 \
--external-network public --dns-nameserver 8.8.8.8 \
--master-lb-enabled --network-driver calico \
--flavor m1.medium --master-flavor m1.medium \
--label kube_tag=v1.35.5Always set kube_tag to the image's version. Without it, a known bug fails the create with KeyError: 'kube_tag', and the failed cluster then won't delete cleanly. When users add labels at create time, have them pass --merge-labels. One reporter's cluster, created without it, had no kube_tag label, and Horizon's upgrade menu stayed empty. Control plane counts must be odd, so the driver rejects 2 or 4.
Yes. Pass --fixed-network and --fixed-subnet when creating the cluster or template, or clear "Create New Network" in Horizon. The driver checks that the network exists and isn't external, then attaches the nodes to it. It creates no subnet of its own, so the fixed_subnet_cidr label only matters for networks the driver creates.
For a cluster whose API is public, the steps one of our engineers wrote up in issue #181 give the network a DNS server and a router to the external network. Horizon also offers "Accessible on private network only" (master_lb_floating_ip_enabled=false). That option needs magnum-cluster-api-proxy running on all your Neutron network nodes. When magnum-conductor runs outside Kubernetes, as it does on Kolla-Ansible and manual installs, it can't yet resolve the proxy's address. We opened an issue for that in February 2026. Until it is fixed, this walkthrough keeps cluster APIs on floating IPs.
In late 2024, creating clusters on existing networks failed with driver 0.24.0. Our maintainers marked that bug fixed in February 2025. If you run 0.24.0 or another release from before that fix, upgrade to 0.38.2 before you test this path.
Create a cluster from the template with one control plane node and one worker, and follow status_reason in openstack coe cluster show <name>. Our docs put a build at under five minutes, and the first one can be slower while nodes download the image.
At CREATE_COMPLETE, fetch a kubeconfig with openstack coe cluster config <name> and confirm three things. kubectl get nodes shows every node Ready, a LoadBalancer Service gets an external address from Octavia, and a PersistentVolumeClaim binds to a Cinder volume. On the management cluster, kubectl -n magnum-system get clusters should show the cluster as Provisioned. If Magnum still reports CREATE_IN_PROGRESS, start with finding why a Magnum cluster is stuck. Passing all three confirms that nodes come up and that load balancers and volumes get provisioned.
After the first cluster, the work becomes upkeep. When you adopt a new driver release, update the package or your Magnum images and test it with your Cluster API versions. Each Kubernetes version your users want needs its own node image and template, and running clusters move up one minor version at a time, as the magnum-cluster-api upgrade guide counts out. On Kolla-Ansible and manual installs, this walkthrough also keeps cluster APIs on floating IPs while the proxy issue is open. When the operators doing this also run the rest of the cloud, that upkeep competes with everything else they own.
For that upkeep, it helps to know who is behind the driver. We wrote magnum-cluster-api and maintain it in the open, and we published 11 releases of it on GitHub between January and August 2026. A Kolla-Ansible user reported in August that resizing autoscaled clusters failed on 0.38.1. The fix had already merged and shipped in 0.38.2 later that month.
Atmosphere 7.8.1, released on September 2, installs Magnum with the driver already in its image, gives Magnum direct access to the Kubernetes cluster it runs on, and runs the driver's proxy for you. Magnum runs inside Kubernetes there, so the external-conductor limit in that proxy issue doesn't apply.
Once your first cluster passes its checks, offer templates for 1.35 and 1.36 only, since upstream fixes for 1.34 end on October 27. If you are weighing whether to keep running the driver yourself or move to Atmosphere, contact the engineers who maintain the driver.
Choose from Atmosphere Cloud, Hosted, or On-Premise.
Simplify your cloud operations with our intuitive dashboard.
Run it yourself, tap our expert support, or opt for full remote operations.
Leverage Terraform, Ansible or APIs directly powered by OpenStack & Kubernetes