Architecting Your Future: Unlocking Scalable Enterprise AI With Robust Architecture

Enabling Scalable, Trusted AI Begins With The Right Architecture Strategy

A Forrester Consulting Thought Leadership Study Commissioned By LTIMindtree, July 2025

As organizations race to adopt AI and unlock its transformative potential, many decision-makers find their organizations at an inflection point where ambition outpaces operational readiness. Implementing AI innovation at scale demands a closer look at infrastructure modernization across capacity needs and cloud agility to support evolving workloads and the growing complexity of integrating diverse data and technologies. As AI workloads scale and grow in complexity, it is increasingly critical for organizations to ensure that their infrastructure can stay abreast with the evolving demands of this emerging technology.

In March 2025, LTIMindtree commissioned Forrester Consulting to evaluate organizations’ AI infrastructure needs, gaps, and challenges to help enterprise leaders future-proof their cloud and IT architecture for AI. Forrester conducted an online survey with 576 respondents across North America, Europe, and the Nordics with responsibility over their organization’s AI and cloud infrastructure strategy to explore this topic. We found that organizations face significant challenges in scaling AI that range from complex cloud environments and key AI infrastructure limitations across compute, storage, network and data, to a persistent trust barrier. Organizations must reassess their AI and infrastructure strategy to enable scalable, cost-effective, responsible, and impactful AI deployment.

Key Findings

  • AI adoption is evolving rapidly, but infrastructure readiness varies across organizations. Many organizations are aggressively advancing into AI, but this momentum often outpaces foundational capabilities. Only 18% report having adequate support for data preparation, undermining model performance and reliability.
  • Trust and integration are major pain points that limit AI’s full potential. Only 30% of respondents note that their organization has strong AI literacy or training programs in place to address the lack of trust organizations face in unlocking greater value from AI. Cloud integration challenges and data silos also hinder progress, especially in hybrid and multicloud environments.
  • Infrastructure modernization is essential to meet rising AI workload demands. Modern technologies like specialized chipsets and data lakehouses present key opportunities alongside appropriate application of modern architectures like compute and storage disaggregation. This is especially important as AI workloads become increasingly complex and shift from training to real-time inferencing.
  • It is critical to engage a partner ecosystem that establishes a strong foundation across infrastructure, governance, strategy, and mindset. AI governance remains a crucial yet often overlooked component of the AI roadmap. Strategic partners bring more than technical expertise and domain knowledge — they are also co-creators in shaping long term roadmaps for executives to align AI initiatives with business goals

Consulting Team:

Alicia Choo, Market Impact Consultant

Xin Tao, Market Impact Consultant

Aneesh Ahuja, Market Impact Consultant

Contributing Research:

Forrester’s Technology research group

LTIMindtree logo

Table Of Contents

Estimate your ROI

Registration Required To Continue

Register for Digital Asset

Enterprises are rapidly accelerating AI adoption to transform their business strategies. Yet, many face critical gaps in their foundational infrastructure capabilities, making it crucial to address these challenges to fully unlock AI’s value.

Through our interviews with decision-makers at enterprises to understand their current AI and cloud infrastructure strategy, we found that:

AI ADOPTION IS SURGING, WITH AN APPARENT FOCUS ON OUTCOME-DRIVEN PRIORITIES

Decision-makers are confident in their organization’s AI strategies, with 89% of them affirming that their organization has defined AI-driven outcomes, KPIs, and ROI goals. While agentic AI has been generating significant interest, its full scale deployment is still in the early stages (27%). Almost one in two organizations are focusing on data management (45%) and operational use cases such as observability models (41%) to drive tangible business results.

  • Decision-makers are highly optimistic about the impact of their organization’s AI initiatives. Decision-makers are looking to AI as a means of driving efficiency (44%), a decision-making culture (42%), and enhancing business strategy (41%) (see Figure 1). Larger enterprises (i.e., more than 20,000 employees) also look at more focused drivers, such as using AI to improve software development (43%).
  • However, this optimism often rests on uneven foundations — many organizations still struggle with inadequate data preparation capabilities within their infrastructure. Organizations are focusing on building and applying models, with AI workloads largely focused on training (36%) and inferencing (34%) models. However, data preparation, which serves as the foundation to power the AI engine, represents the smallest portion of AI workload distribution at 30%. This gap is further underscored by the fact that only 18% of decision-makers agree that their organization’s current infrastructure provides a strong support to data preparation (see Figure 2). While organizations appear more ready to run and train models, weak foundations in data readiness undermine the quality of their model. Current abilities to support data inferencing (55%) and training (51%) workloads, though higher than data preparation (18%), still reveals significant room for improvement.
Figure 1

Top Drivers For AI Adoption

Optimize operational efficiency Improve the use of data and insights in business decision-making Improve business strategy Reduce operational cost Increase revenue

Base: 576 decision-makers with responsibility over their organization’s AI and cloud infrastructure strategy
Note: Showing sum of responses who ranked these drivers within their organization’s top five drivers.
Source: Forrester’s Q2 2025 AI Infrastructure Survey [E-63587]

Figure 2

“Please rate how well your current infrastructure is able to support your organization’s current AI workloads."

Click to see data by AI adoption stage


[CONTENT]

Data preparation, which involves collection, discovery, exploration, cleansing, validation, and labeling Training, which involves feeding AI models with (prepared) data to teach them how to correctly interpret the data and generate accurate decisions Inferencing, which is the process of using live data with an AI model to make predictions

Base: 276 decision-makers with responsibility over their organization’s AI and cloud infrastructure strategy
Note: Showing responses for “Strong support”.
Source: Forrester’s Q2 2025 AI Infrastructure Survey [E-63587]

Base: 300 decision-makers with responsibility over their organization’s AI and cloud infrastructure strategy
Note: Showing responses for “Strong support”.
Source: Forrester’s Q2 2025 AI Infrastructure Survey [E-63587]

  • Hybrid cloud preferences are coming to the forefront to balance infrastructure needs. To meet growing AI workload demands, organizations are under pressure to modernize their infrastructure across compute, network, storage, and data management. With respect to cloud deployment, enterprises recognize that public cloud environments offer the necessary scalability and elasticity, especially for peak AI model training cycles and proof-of-concept (PoC) experimentation. This helps to avoid costly over-provisioning of on-premises infrastructure. On-premises and private cloud environments on the other hand, are optimized for data sensitivity, latency needs, and critical inferencing workloads for proximity and control. While IT leaders recognize these attributes due to the diversity in maturity and use cases, enterprises are adopting a hybrid cloud environment to strike the right balance between performance, scalability, and flexibility. As organizations progress in their AI maturity, many are also adopting AI platform-as-a-service (PaaS) models to reduce the operational burden of managing complex tooling and infrastructure.

    Within these cloud architectures, most continue to rely heavily on CPUs (85%) and parallel-processing GPUs (82%) as their compute instances, with GPUs being well-suited for training workloads. However, as AI initiatives mature and shift toward inferencing applications, there is a growing need to adopt specialized processors like data processing units (DPU) (19%), tensor processing units (TPU) (31%), and field programmable gate arrays (FPGA) (21%) that better support real-time and edge inference.

  • As AI workloads grow in complexity, networking capabilities must evolve in parallel. AI- or machine learning (ML)-optimized networking protocols (28%) are expected to see increased adoption over time, complementing the widespread use of GPUs. In terms of storage, most organizations are adopting cloud block storage (65%) in line with their hybrid cloud and multicloud architectures. While object storage (54%) remains the go-to for scalable, cost-effective storage of unstructured or semi-structured training, block storage plays a critical role as the high-performance engine that powers databases, metadata systems, caching layers, virtual machine (VM)/container operations and the high-speed parallel file systems (23%) that feed data into AI training clusters. Solid state drives (SSD) (60%) are also widely used, particularly in performance-optimized environments supporting AI workloads. However, non-volatile memory express (NVMe) (18%), with its ability to deliver substantially higher throughput than traditional SSDs via peripheral component interconnect express (PCIe) interfaces, represents a key area of opportunity.
  • Organizations must look beyond traditional data architectures to future-proof their data infrastructure and support advanced analytics and AI. While traditional databases (80%), data warehouses (62%), and data lakes (42%) are widely used, they are struggling to meet growing business demands due to limitations in agility, performance, concurrency, integration, and governance.1 Data lakehouses are designed to unify the capabilities of both data lakes and warehouses, which can address the above shortcomings. Though adoption is still gaining momentum (31%), data lakehouses offer a more agile, scalable, and AI-ready foundation to support modern workloads (e.g., AI/ML, business intelligence, and data science). To strengthen gaps in their data foundation, organizations should actively evaluate these next-generation architectures as part of their modernization strategy.
  • Many organizations are already seeing tangible value from their AI initiatives, but unlocking deeper strategic impact depends on reinforcing the necessary infrastructure to power AI at scale. AI is delivering on core value propositions that most organizations care about, namely: better decision-making (78%) and optimized operational efficiency (74%) (see Figure 3). At the same time, it is driving profound, long-term digital transformation (75%), an outcome that was not initially top of mind as a driver for AI adoption. While 89% of decision-makers expressed strong confidence that their organization’s AI initiatives are aligned with business goals, the relatively lower realization of improved business strategy (69%) suggests a gap between ambition and impact. Despite strategic intent, more focused effort is needed to ensure AI drives tangible improvements in long-term business direction and competitiveness.
Figure 3

Top Five Realized Benefits From AI Initiatives In The Past 12 Months

Improved the use of data and insights in business decision-making Enhanced CX and customer engagement Accelerated the pace of digital transformation Optimized operational efficiency Increased automation

Base: 576 decision-makers with responsibility over their organization’s AI and cloud infrastructure strategy
Note: Showing sum of responses for “Mostly realized” and “Fully realized”
Source: Forrester’s Q2 2025 AI Infrastructure Survey [E-63587]

Organizations are realizing that scaling AI hinges on trust, readiness, and the ability to operationalize it across complex environments. Technical limitations, regulatory complexities, and internal capability gaps are no longer distant concerns but immediate obstacles to scaling AI successfully. To move beyond experimentation to realizing enterprise-wide impact, decision-makers must address these structural and organizational hurdles head on:

  • AI cannot scale without trust. Fifty-nine percent of decision-makers cite a lack of trust in AI systems as the top challenge to AI adoption, which is closely linked to concerns around data privacy and security (52%) (see Figure 4). Forrester defines “trusted AI” as AI that is designed, developed, deployed, and governed to meet diverse stakeholder needs for accountability, competence, consistency, dependability, empathy, integrity, and transparency.2 A lack of trust in AI systems can stifle adoption: From an internal perspective, many AI projects languish because stakeholders do not trust AI enough to use it in their workflows.3
Figure 4

Top Five Challenges To AI Adoption

Lack of trust in AI systems Data privacy and security concerns Cloud infrastructure concerns Lack of dear AI strategy Governance and risk concerns

Base: 576 decision-makers with responsibility over their organization’s AI and cloud infrastructure strategy
Note: Showing top five sum of responses who ranked these challenges as part of their organization’s top five challenges.
Source: Forrester’s Q2 2025 AI Infrastructure Survey [E-63587]

  • Organizations can leverage AI training and literacy programs to enhance understanding and build confidence in AI systems. However, only 30% of decision-makers strongly agree that their organization has such programs in place, indicating that this is the weakest part of their AI strategy.
  • Cloud infrastructure is a critical enabler or blocker or AI success. Cloud infrastructure not only supports the scalability and flexibility required for AI workloads but also determines the readiness of organizations to innovate and deploy AI solutions. Cloud infrastructure concerns are significant, with 51% of decision-makers noting that it is among the top challenges to AI adoption. This is primarily due to difficulties with integrating AI into existing IT environments. Additionally, 54% of decision-makers have identified siloed data across systems or clouds as a key barrier to expanding AI workload (see Figure 5). As hybrid and multicloud strategies become the norm, organizations face increasing complexity in managing data across diverse environments.
Figure 5

“Top Five Barriers To Expanding AI Workload With Current AI Infrastructure"

Click to see data by Company size


[CONTENT]

The need for high-power processers, conflicting with the organization’s sustainability goals High cost of compute, storage, and network resources Data centers face space, power, and/or cooling constraints Compliance concerns around data security and residency Complexity of managing data across data lakes and warehouses Siloed data across systems or clouds Bandwidth and latency issues that affect application performance Limited supply of GPUs/TPUs or specialized AI hardware

Base: 308 decision-makers with responsibility over their organization’s AI and cloud infrastructure strategy
Note: Showing top five responses with total sum of % ranked.
Source: Forrester’s Q2 2025 AI Infrastructure Survey [E-63587]

Base: 172 decision-makers with responsibility over their organization’s AI and cloud infrastructure strategy
Note: Showing top five responses with total sum of % ranked.
Source: Forrester’s Q2 2025 AI Infrastructure Survey [E-63587]

Base: 96 decision-makers with responsibility over their organization’s AI and cloud infrastructure strategy
Note: Showing top five responses with total sum of % ranked.
Source: Forrester’s Q2 2025 AI Infrastructure Survey [E-63587]

  • Organizations face challenges across the entire AI infrastructure (i.e., compute, network, storage, and data) when expanding their AI workloads. The high costs of managing AI infrastructure ranks as the foremost barrier, with nearly 58% of decision-makers citing it as a top concern for their organization. These costs are intensified by the need for high-performance processors, which can be at odds with sustainability goals (53%). Enterprises must carefully examine the costs and benefits of running workloads on cloud AI infrastructure, acquiring on-premises AI infrastructure, or combining the two into a hybrid model. The right AI infrastructure strategy can help tech leaders control costs while meeting sustainability goals.4

In response to mounting challenges of optimizing and orchestrating compute, storage, and networking resources, organizations are ramping up investments in their infrastructure. The future of AI rests on the ability to drive performance through more flexible, scalable, and secure architectures — motivating many enterprises to increase AI-related spending in the next 12 months. In addition, executives are turning to trusted third parties to accelerate AI maturity by supporting technical priorities and providing strategic guidance. These partnerships play a critical role in future-proofing AI initiatives:

  • As AI is increasingly expected to support more business processes and workflows, organizations are actively investing in AI platforms and infrastructure. As a next step in their AI roadmap, over 50% of decision-makers plan to explore new AI-enabled products (52%) and use cases (51%) within the next 12 months. This effort goes hand-in-hand with plans to develop (40%) or expand existing (18%) AI applications. To accelerate AI transformation on a more stable and flexible foundation, around half of the respondents highlighted building cloud-native AI platforms (50%) and modernizing data infrastructure (47%) as key initiatives their organization will be focusing on in the next year, while 41% are aiming to improve their AI cloud infrastructure. These efforts are backed by increasing adoption in AI model development (i.e., 22% note they are planning to deploy and 41% are planning to expand adoption) and AI data infrastructure (i.e., 22% are planning to deploy and 42% are planning to expand adoption).

  • While organizations are making great strides in prioritizing AI architecture to address core technology challenges, AI governance — particularly around policy and ethics — remains a blind spot. We observe that there is clear alignment between AI plans and infrastructure pain points. Driven by concerns around data privacy and security (52%), 75% of decision-makers report having implemented technologies such as identity and access management (IAM), role-based access control (RBAC), model security, and supply chain security, among which 60% note that their organization continues to expand AI security capabilities. Similarly, 70% of decision-makers say their organization is tackling obstacles in data silos (54%) and compliance (53%) through AI data management, and 43% are looking to increase their organization’s efforts in data integration, preparation, analytics, and visualization.

    However, when it comes to AI policy and ethics, organizations seem to be falling behind. Despite acknowledging the gap in their strategic frameworks, less than 34% say their organization has implemented AI governance practices, and only 16% plan to expand them. This disconnect suggests that governance — especially those around policy management and ethical AI — remains an area for improvement, highlighting potential risk that could undermine progress across data and AI efforts, especially as these domains converge. Forrester’s research predicts that 40% of highly regulated companies will combine their data and AI governance programs, which will require talent and investment in access control, policy management, quality control, and risk mitigation.5 Successful enterprises will soon realize the importance of building a cohesive framework that governs AI and data together.

  • AI spending is surging, with genAI leading the charge. A key reflection of organizations’ commitment to their AI roadmap is their growing budget allocation. More than 70% of decision-makers expect that their organization will increase their spending on AI technologies, with genAI on top of the list: 86% anticipate higher investment, and 45% expect a notable increase of more than 10%. There is also strong momentum in spending on AI compute, with 78% noting their plans on budget increases and 39% projecting growth of over 10% (see Figure 6).

    Investment in predictive AI is also on the rise (74%), though it appears to be falling slightly behind generative AI (genAI). Forrester anticipates predictive AI to remain in the spotlight, especially as organizations encounter roadblocks in optimizing genAI outcomes. Proven use cases such as predictive maintenance, customer personalization, supply chain optimization, and demand forecasting are still expected to draw significant investment. Forward-looking enterprises should recognize that predictive AI and genAI are not mutually exclusive. In fact, predictive capabilities can enhance genAI outputs, leading to more use cases that integrate both technologies.6

Figure 6

“How do you expect your organization’s spending on the following AI technologies and capabilities to change over the next 12 months?"

Click to see data by Region


[CONTENT]

Agentic AI Generative AI AI network AI storage AI compute Physical AI Discriminative/predictive AI Increase by 1% to 5% Increase by 5% to 10% Increase by more than 10%

Base: 234 decision-makers with responsibility over their organization’s AI and cloud infrastructure strategy
Note: Showing responses that indicated an increase in spending.
Source: Forrester’s Q2 2025 AI Infrastructure Survey [E-63587]

Base: 234 decision-makers with responsibility over their organization’s AI and cloud infrastructure strategy
Note: Showing responses that indicated an increase in spending.
Source: Forrester’s Q2 2025 AI Infrastructure Survey [E-63587]

Base: 231 decision-makers with responsibility over their organization’s AI and cloud infrastructure strategy
Note: Showing responses that indicated an increase in spending.
Source: Forrester’s Q2 2025 AI Infrastructure Survey [E-63587]

Base: 111 decision-makers with responsibility over their organization’s AI and cloud infrastructure strategy
Note: Showing responses that indicated an increase in spending.
Source: Forrester’s Q2 2025 AI Infrastructure Survey [E-63587]

  • With the uprising of genAI, GPUs are becoming the top priority for executives, and demands for other specialized chipsets are also expected to grow. Rapid adoption of genAI has made GPUs a top priority (97%) for future AI growth, with specialized processors slowly gaining attention. This shift highlights the urgent need for a more secure and flexible architecture to support evolving AI workloads (see Figure 7).

    While CPUs and GPUs remain the most mature and widely adopted technologies that dominate both current usage and future plans, specialized chipsets such as TPUs, application-specific integrated circuits (ASICs), FPGAs, and DPUs are also on the rise, particularly as organizations become more reliant on inferencing workloads. Take DPUs for example, 32% of decision-makers already rate their adoption as a high or critical priority, even though only 19% have deployed them at this stage. While DPU adoption is currently constrained by cost, complexity, and vendor capabilities, they are playing an increasingly important role as AI workloads shift from training to inferencing. Specifically, DPUs are optimized for data movement, infrastructure management, energy-efficiency, security, and cost-effectiveness, causing major hyperscalers to accelerate their development of homegrown DPUs.

Figure 7

“Current Adoption On Cloud Infrastructure Vs. Future Priorities To Support AI Growth "

Click to see data by Timeframe


[CONTENT]

Central processing units (CPUs) Graphic processing units (GPUs) Tensor processing units (TPUs) Application-specific integrated circuits (ASICs) Field-programmable gate arrays (FPGAs) Data processing units (DPUs)

Base: 576 decision-makers with responsibility over their organization’s AI and cloud infrastructure strategy
Showing sum of responses for “High priority” and ”Critical priority”.
Source: Forrester’s Q2 2025 AI Infrastructure Survey [E-63587]

  • Organizations are seeking strategic partners — not just one-off solution providers — to co-design their long-term AI transformation. Decision-makers are looking for partners who can act as architects that bring more than technology tools, but also proven blueprints, governance playbooks, and phased roadmap guidance. To further advance their AI agendas, organizations require support in delivering high-quality, responsible AI solutions tailored to the specific demands of their verticals. Executives recognize that AI transformation is an organization-wide effort and that it extends well beyond IT. Data shows that support for AI strategy (51%) is ranked as the top criterion when selecting external partners, closely echoing the fact that 50% of decision-makers cite the lack of a clear AI strategy as one of their organization’s top four challenges. In addition to high-quality algorithms (51%), decision-makers prefer industry-specific solutions (50%) that are easy to implement and customize (46%). Industry expertise is particularly valued in sectors such as manufacturing (64%), transportation and logistics (59%), and banking, financial services, and insurance (56%). At the same time, 46% of decision-makers note that their organizations require services that are easy and intuitive to implement without introducing additional complexity or burden to their existing workflows.

While AI adoption is accelerating, the success of organizations’ AI initiatives hinges on the ability to scale and sustain AI across increasingly complex environments. To realize the transformative potential of AI, organizations must first address foundational infrastructure challenges limiting its scale.

Forrester’s in-depth survey of 576 decision-makers about their organization’s AI and cloud infrastructure strategy yielded several important recommendations:

Appendix A: Methodology

In this study, Forrester conducted an online survey of 576 IT and business decision-makers of AI and cloud infrastructure strategy in North America, Europe, and the Nordics to evaluate their organizations’ AI infrastructure readiness for AI initiatives. Survey participants included decision-makers who are C-level executives, vice presidents, directors, and senior managers. Questions provided to the participants asked about their organizations’ AI infrastructure, challenges, and future plans. Respondents were offered a small incentive as a thank you for time spent on the survey. The study began in April 2025 and was completed in May 2025.

Appendix B: Demographics/Data

Region

North America 41%
Europe 40%
Nordics 19%

Number of employees

2,500 to 4,999 53%
5,000 to 19,999 30%
20,000 or more 17%

Revenue

$2B to just under $3B 46%
$3B to just under $4B 27%
$4B to just under $5B 18%
More than $5B 8%

Industry

Banking, financial services, and/or insurance 9%
Business or professional services 9%
Construction 9%
Energy and utilities 9%
Life sciences and healthcare 9%
Manufacturing 9%
Media and entertainment 9%
Retail and consumer goods 9%
Technology and/or technology services 12%
Transportation and logistics 9%
Travel and hospitality 9%

Position

C-level 25%
Vice president 36%
Director/senior manager 39%

Business function

Business strategy and operations 31%
Data science and analytics/AI center of excellence 19%
Digital strategy/transformation/innovation 15%
IT 16%

Responsibility over organization’s AI strategy

Final decision-maker 52%
Part of a team making decisions 26%
Influence decisions 23%

Responsibility over organization’s cloud infrastructure strategy

Final decision-maker 51%
Part of a team making decisions 30%
Influence decisions 19%

AI usage

Experimentation stage 48%
Production stage 52%

Note: Percentages may not total 100 due to rounding

Appendix C: Endnotes

1 Source: Data Lakehouse Is The New Data Warehouse And Data Lake, Forrester Research, Inc., August 31, 2023.

2 Source: Build Stakeholder Trust In Artificial Intelligence, Forrester Research, Inc., August 12, 2022.

3 Source: Top Emerging Tech Overview: Explainable Artificial Intelligence, Forrester Research, Inc., October 28, 2022.

4 Source: The AI Infrastructure Solutions Landscape, Q4 2023, Forrester Research, Inc., December 12, 2023.

5 Source: Predictions 2025: Artificial Intelligence, Forrester Research, Inc., September 9, 2024.

6 Source: Predictions 2025: Artificial Intelligence, Forrester Research, Inc., September 9, 2024.

7 Source: Architecting Your Infrastructure For AI, Forrester Research, Inc., December 11, 2023.

8 Source: Build Stakeholder Trust In Artificial Intelligence, Forrester Research, Inc., August 12, 2022.

This study was conducted by Forrester’s Custom Content Consulting Practice. Get in touch to discuss your custom content needs and goals. Learn more about all of Forrester's consulting capabilities.

ABOUT FORRESTER CONSULTING

Forrester provides independent and objective research-based consulting to help leaders deliver key transformation outcomes. Fueled by our customer-obsessed research, Forrester’s seasoned consultants partner with leaders to execute on their priorities using a unique engagement model that tailors to diverse needs and ensures lasting impact. For more information, visit forrester.com/consulting.

© Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change. Forrester®, Technographics®, Forrester Wave, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies.

Cookie Settings
M
K

https://mainstayadvisor.com/go/mainstay/gdpr/policy.html