GPUs get the headlines but storage, networking, and scheduling determine real AI performance. Learn what actually matters and how open infrastructure helps.
Everyone is talking about GPUs. It makes sense. They are the most visible and often the most expensive part of any AI stack.
But GPUs do not operate on their own. They rely on storage to supply data, networking to connect workloads across nodes, and orchestration to keep everything running efficiently. When any of these layers fall short, GPUs sit idle and costs rise without corresponding performance.
In fact, this is not an edge case. Research shows that unoptimized AI training environments can experience 30 to 50 percent GPU idle time while waiting for data, meaning a large portion of expensive compute is not being used effectively.
This is something many teams discover after deployment. The cluster is running, but training jobs slow down because storage cannot keep up. Inference latency is affected by network performance rather than model complexity. Utilization remains low because workloads are not scheduled efficiently.
The challenge in AI infrastructure is not just getting access to GPUs. It is building the system around them so that storage, networking, orchestration, and operations work together reliably.
This is what the post explores. Not just the GPU layer, but everything that supports it and makes it effective.
And why platforms built on OpenStack and Kubernetes deliver the full stack control that GPU focused thinking often misses.
§1 The Storage Bottleneck Nobody Talks About
GPUs can only work as fast as the data reaching them. In many AI environments, storage is where performance starts to degrade.
Training pipelines process large volumes of data such as images, text, sensor data, and embeddings, often at the scale of hundreds of terabytes. This data must be read, processed, and continuously delivered to GPUs. When the storage layer cannot keep up, GPUs remain underutilized, and costs increase without corresponding output.
The challenge continues after training begins. Checkpoints accumulate, intermediate outputs grow, and experiment logs expand across multiple runs. Model artifacts are stored and versioned for reproducibility. Over time, what starts as a dataset becomes a large and complex storage footprint. If this data sits in a proprietary object store, it becomes increasingly difficult and expensive to move.
Data locality also plays a key role. Training performance improves when storage is physically close to compute, within the same rack, availability zone, or network segment. In many managed environments, there is limited visibility into how storage is placed relative to GPUs. In contrast, open infrastructure allows direct control over placement and performance.
This is where platforms like Ceph, which is integrated with our OpenStack deployment tool, Atmosphere make a measurable difference. Ceph provides scalable block and object storage built on open standards, deployed alongside compute, with full visibility into performance and placement. No egress penalties for accessing your own data. No proprietary formats locking artifacts in place..
Storage isn't a supporting feature of AI infrastructure. It's a performance-critical layer and when it's overlooked, GPUs pay the price.
If you’d like to learn more about Atmosphere’s approach to storage and third-party integrations, check out our post Deploying Atmosphere: A Guide to Storage Integration.
§2 Networking Is The Silent Performance Constraint
AI workloads, especially distributed training, depend heavily on networking. When a model runs across multiple GPUs or nodes, those systems need to exchange gradients, synchronize parameters, and stay in constant coordination. The speed of that communication directly affects how quickly training completes.
If the network cannot keep up, GPUs end up waiting on each other. A single slow connection between nodes can slow down an entire training job. In many cases, network limitations have a bigger impact on performance than the number of GPUs available.
Inference brings its own requirements. Serving models at scale involves handling large volumes of requests with consistent low latency. When response times increase, the issue is often related to network congestion or inefficient routing rather than the model itself.
The specifics matter. High-bandwidth fabrics, low-latency interconnects, and hardware-level acceleration like SR-IOV and DPDK aren't nice-to-haves, they're requirements for AI workloads running at any meaningful scale. On hyperscaler platforms, networking is largely abstracted away. That simplifies setup but removes control. You can't tune what you can't see.
On open infrastructure, networking is a configurable layer. Atmosphere supports SR-IOV for near-bare-metal network performance, DPDK for accelerated packet processing, and speeds up to 100Gbps. Teams can design network topologies that match their workload requirements placing GPUs, storage, and endpoints where performance demands, not where a provider's defaults allow.
Networking rarely gets top billing in AI infrastructure discussions. But when it underperforms, everything else slows down with it.
§3 Scheduling and Orchestration: Where Efficiency Lives or Dies
A GPU sitting idle costs the same as one running at full load. The difference is whether you are getting value from it.
Scheduling determines that. AI workloads are bursty and varied, from large training jobs to smaller fine tuning tasks, inference endpoints, and batch processing. Without proper scheduling, expensive hardware sits underused.
Kubernetes is the standard orchestration layer for AI workloads. It manages scheduling, resource allocation, autoscaling, and jobs. But it can only work well if the infrastructure provides the right inputs, such as GPU visibility, topology awareness, and resource granularity.
This is where infrastructure matters. GPU passthrough, fractional allocation through MIG, and NUMA aware placement allow Kubernetes to schedule workloads efficiently. Without them, utilization drops.
Atmosphere provides this foundation. OpenStack exposes GPU topology and hardware details to Kubernetes through open APIs, enabling scheduling based on the actual environment.
The difference between 35 percent and 70 percent GPU utilization is not more hardware. It is better scheduling on infrastructure that exposes the right information.
§4 Cost and Control: The Full Stack Problem
The GPU hourly rate is the number most people focus on. It is also the least representative of total cost.
AI infrastructure costs span every layer. Storage grows with each training run as datasets, checkpoints, logs, and model artifacts accumulate. Networking costs increase with data movement between nodes, across zones, and outside a provider’s network. Idle GPU capacity continues to incur cost even when not in use. Managed services for ML platforms, monitoring, and orchestration add another layer.
In hyperscaler environments, these costs are difficult to isolate and optimize. Pricing structures are complex, egress fees discourage data movement, and discounts often depend on long term commitments. Over time, costs increase while control becomes more limited.
This is a full stack issue. When storage, networking, scheduling, and compute are all controlled by a single provider, optimization becomes constrained. Infrastructure decisions are shaped by available features and pricing models rather than actual requirements. For a deeper look at how this dynamic plays out, read The GPU Cloud Trap: AI Infrastructure and the Open Alternative .
Open infrastructure changes this dynamic. Each layer is visible and configurable. Storage can scale without egress penalties, networking can be designed around workload needs, and scheduling can operate with full awareness of the underlying hardware. This leads to more predictable costs and better resource utilization.
Cost control in AI infrastructure is not just a financial exercise. It is a result of how the system is designed and who controls it.
§5 How VEXXHOST Delivers Full Stack AI Infrastructure
Atmosphere by VEXXHOST is built for this problem, AI infrastructure where every layer works together, not just the GPU.
OpenStack manages the foundation: compute provisioning, GPU allocation, storage placement, networking, and identity, all through open, auditable APIs. You control where resources live and how they are configured.
Kubernetes handles the workloads: scheduling, scaling, and orchestration through upstream, CNCF certified APIs. Workloads remain portable across environments because nothing underneath is proprietary. To understand why this combination is gaining momentum, read OpenStack, Kubernetes, and AI: What 2025 Taught Us About the Future of Cloud.
Ceph provides scalable block and object storage deployed alongside compute, with no egress fees, no proprietary formats, and no data movement constraints.
On top of that, Atmosphere supports GPU passthrough, MIG, and vGPU configurations. High performance networking with SR IO V and DPDK. NUMA aware placement for optimized GPU scheduling. Deployment on premise, in colocation, or hosted, depending on workload requirements.
Every layer is open. Every layer is controllable. Every layer works together because it is designed that way. For organizations in regulated industries, we also cover how this architecture supports compliance in Sovereign by Architecture: Building AI Infrastructure for the EU AI Act.
Conclusion
GPUs get the attention. But storage, networking, scheduling, and control determine whether those GPUs actually deliver value.
The organizations running AI efficiently aren't just buying more compute. They're building infrastructure where every layer is visible, optimized, and under their control.
That's what open infrastructure delivers. That's what Atmosphere is built for.
🔗 Explore Atmosphere and build AI infrastructure that works, not just at the GPU layer, but all the way through.
