The AI Compute Gap: Enterprises Accelerate Spending While Struggling to Measure and Optimize Infrastructure

A new VentureBeat Pulse Research report reveals a significant "compute gap" across 107 enterprises, where aggressive AI infrastructure spending is outpacing organizations’ ability to understand, steer, and optimize its economics. While most companies currently leverage hyperscalers and model-provider APIs for their AI operations, a dramatic shift is on the horizon, with the next wave of investment targeting specialized compute providers that few currently use. This impending re-platforming is compounded by a striking lack of visibility into existing infrastructure costs and utilization, leading to substantial inefficiencies and a critical need for improved financial oversight.

The research highlights that only about one in five (21%) enterprises are running AI in production at scale, yet their spending intentions are already far ahead of this maturity curve. The largest planned area for evaluation over the next year is AI-specialized clouds (45%), a segment almost entirely absent from current deployments. Simultaneously, existing GPU infrastructure often lies underutilized, with 83% of enterprises reporting GPU utilization of 50% or less. Furthermore, fewer than half (44%) can rigorously track their AI compute costs, illustrating a scenario where enterprises are acquiring more infrastructure faster than they can account for their current assets. This foundational instability is further underscored by high churn intent, with 64% of organizations planning to switch or add an infrastructure provider within 12 months, and 38% within the next quarter alone.

Methodology: A Snapshot of Enterprise AI Infrastructure

VentureBeat conducted this survey as part of its ongoing Pulse Research series, specifically focusing on enterprise AI infrastructure, compute, and inference economics. The dataset comprises responses from 107 qualified organizations, each with more than 100 employees, collected during a single Q2 2026 (June) wave. To ensure relevance, the smallest size band (1-100 employees) was excluded. As a single-wave, cross-sectional study, the report provides a directional signal rather than month-over-month trends, and its self-selected nature means it should be interpreted as a view from organizations actively building out AI infrastructure, particularly those in the mid-market and earlier stages of adoption, rather than a precise measurement of the broader market or the largest hyperscale operators.

The sample concentrates significantly in the mid-market, with 36% from organizations of 101-250 employees and 27% from 251-1,000 employees. Larger enterprises also contributed, with 22% from 1,001-5,000 employees, 8% from 5,001-10,000, and 7% from 10,001+. Respondents spanned various roles, including managers (38%), individual contributors (28%), VPs and directors (19%), and the C-suite (13%). Crucially, 45% identified as final decision-makers for AI solutions, with another 30% serving as recommenders or influencers, lending credibility to the purchasing authority represented. Technology/Software was the largest industry represented at 26%, followed by Healthcare/Life Sciences (15%), Financial Services (13%), and Retail/E-commerce (12%). Several questions allowed for multiple selections, meaning some shares may sum to more than 100%, indicating presence across multiple platforms or strategies.

Finding 1: Ambition Outpaces Production

The survey reveals a stark reality: despite significant investment, most enterprises are still in the nascent stages of AI deployment. Only 21% of organizations reported running AI in production at scale, representing a mature minority. The vast majority (76%) are either experimenting with proofs of concept (38%) or have some workloads in production but not across the entire organization (37%). A small fraction (4%) has not yet initiated AI workloads. This front-loaded maturity curve implies that the infrastructure decisions and evaluations discussed in the report are largely being made by organizations whose compute footprint and associated costs are poised for substantial growth. The impending evaluation and switching intentions are thus indicative of a build-out phase, not settled preferences.

Finding 2: Enterprises Run on Hyperscalers and Model APIs Today

Currently, the AI infrastructure landscape is dominated by established players. Google Cloud leads as the most-used platform overall at 48%, with Microsoft Azure (29%), AWS (22%), and Oracle Cloud (22%) also holding significant positions. Similarly, major model APIs like Google’s Gemini (41%) and OpenAI (40%) are widely adopted, followed by Anthropic at 12%. Strikingly, the specialized "neocloud" GPU providers, such as CoreWeave, Lambda, Crusoe, Nebius, Together, and Fireworks, which frequently capture AI-infrastructure headlines, register at less than 2% adoption among these enterprises today. Only 6% run their own on-prem or co-located GPU clusters, and 4% use a custom open-source self-managed stack. This indicates that, for now, enterprises are largely leveraging their existing general-purpose cloud providers for AI, making future evaluation plans even more noteworthy. It’s important to note that these figures measure presence in the stack rather than spending or primary status, reflecting the self-selected, mid-market skewed sample.

Finding 3: The Next Dollar Targets Unused Infrastructure

Despite current reliance on hyperscalers, enterprises are signaling a significant departure for future investments. The single most-cited planned evaluation area over the next 12 months is AI-specialized clouds (45%), a category virtually unused by these organizations today. This represents a sharp tension between current operational reality and future strategic intent. Nearly a third (32%) also plan to evaluate non-NVIDIA accelerators (e.g., AWS Trainium, Google TPU, AMD Instinct, Intel Gaudi, in-house ASICs), and 28% intend to assess NVIDIA Blackwell (GB300) and other next-generation GPUs. Even decentralized or distributed compute networks (16%) and sovereign or region-specific compute (11%) are drawing meaningful interest. This isn’t incremental adjustment; it’s the leading edge of a potential re-platforming, with specialized AI clouds showing the highest net momentum (+24), surpassing even hyperscalers (+22) in terms of planned expansion. This trend is consistent with findings from previous survey waves, reinforcing the eagerness to explore new compute options.

Finding 4: A Switching Wave is Building

The foundational nature of compute infrastructure usually suggests vendor stickiness, yet the survey reveals an unusually high churn intent. A clear majority (64%) of enterprises plan to switch or add an infrastructure provider within the next 12 months, with a substantial 38% intending to make changes within the next quarter alone. Only 36% have no plans to change. While the immediate switching consideration often involves incumbents like Microsoft Azure and Google Cloud (33% each) and major model APIs like OpenAI (30%) and Gemini (22%)—suggesting a near-term reshuffling and consolidation among existing players—the longer-term evaluation thesis points towards the specialized AI clouds identified in Finding 3. This indicates a two-tiered movement: immediate adjustments within the current ecosystem, followed by a more fundamental re-evaluation of infrastructure types.

Finding 5: Nobody Buys on Token Price

Contrary to common vendor marketing, headline token price is the least influential factor in enterprise AI infrastructure buying decisions. Only 8% of respondents cited cost per million tokens as the deciding factor. Instead, enterprises prioritize integration with their existing cloud and data stack (41%) and total cost of ownership (TCO) (35%). Performance (latency and throughput) was cited by 24%, while security/compliance, autoscaling for spiky workloads, and GPU access/availability each garnered 19%. This pattern indicates that buyers are optimizing for operational fit and true economic impact over advertised unit rates. This finding also foreshadows the challenge highlighted in Finding 7, where the ability to rigorously measure TCO often lags behind its stated importance.

Finding 6: Expensive GPUs, Idle Most of the Time

A significant inefficiency plagues current AI deployments: existing GPU capacity is largely underutilized. A staggering 83% of enterprises that operate GPUs report utilization of 50% or less. Specifically, 37% run at 26-50% utilization, 34% at 10-25%, and 15% under 10%. Only a small minority (12%) achieves utilization rates over 50%. A further 8% do not measure utilization at all, and 7% consume via API, operating no GPUs of their own. This represents a substantial "compute gap" where enterprises are planning future investments in more specialized compute (Finding 3) while their current, expensive accelerators sit largely idle. The efficiency headroom within the existing fleet is immense, yet often unmeasured, underscoring a critical waste of resources.

Finding 7: Spending Fast, Measuring Slowly

Despite TCO being a key buying criterion (Finding 5), enterprises largely lack rigorous financial visibility into their AI infrastructure. Fewer than half (44%) rigorously track the cost and return on investment (ROI) of their AI compute. The majority either track it only partially (39%), cannot quantify it yet (20%), or do not consider it a priority (6%). This significant measurement gap directly impacts their ability to make informed purchasing decisions based on TCO. While overall satisfaction with current infrastructure is moderately positive, averaging 4.0 on a five-point scale, "ease of implementation" (3.8) and "value for money" (3.9) trail slightly, with cost being the notable area of softness. This suggests that enterprises are rapidly expanding their AI infrastructure without the robust accounting practices needed to ensure optimal value.

Finding 8: The Next Bottleneck Few Are Watching

The report also identifies an emerging frontier constraint in large-scale inference that most enterprises are not yet adequately addressing: the shift from GPU compute to memory bandwidth, specifically KV-cache capacity. When asked how they would address this, responses were scattered, indicating an early and unsettled market. Dell (PowerScale / Project Lightning) was the leading single answer at 31%, followed by Nvidia (Dynamo / ICMSP) at 16%. Other solutions, including Hammerspace (10%), DDN (9%), open-source tooling, model-level efficiency techniques, VAST Data, and WEKA, garnered smaller shares. Most tellingly, nearly one in five enterprises (18%) are either unaware of this constraint (9%) or have not yet begun to address inference-memory limits (8%). This lack of awareness and preparedness for a shift that will fundamentally reshape inference cost and architecture represents the next chapter of the compute gap, arriving before most have closed the current one.

The Bottom Line: A Compute Gap That Faster Spending Will Widen, Not Close

In summary, organizations with over 100 employees are rapidly investing in AI infrastructure, but their spending velocity is far outpacing their ability to measure and optimize these critical assets. Most are still early in their deployment journey, yet their future spending intentions point towards a significant re-platforming to specialized clouds and alternative accelerators that are barely used today. A clear majority also plans to switch or add providers within the year, driven by factors like integration and total cost of ownership, rather than headline prices.

However, the efficacy of these decisions is undermined by a profound visibility gap. The overwhelming majority of enterprises are underutilizing their expensive GPUs, and fewer than half can rigorously track the true costs and returns of their AI compute. Satisfaction with current infrastructure, while decent, is notably soft on "value for money"—the very dimension that is hardest to judge without proper measurement. Adding to this challenge, the next major architectural constraint, the shift from compute to memory in large-scale inference, is emerging while most enterprises remain unaware or unprepared.

Based on 107 directional responses from a Q2 2026 wave, skewed towards mid-market and earlier-stage adopters, the message is consistent: the appetite for AI investment is running significantly ahead of the instrumentation needed to spend wisely. The compute gap is not merely a capacity issue solvable by more hardware; it is fundamentally a problem of seeing and understanding what the hardware already costs. The critical question for future waves of this research is whether enterprises will build this essential visibility before the next wave of re-platforming arrives, or if they will continue to acquire infrastructure as blind to its true economics as they are today.