Mainframe Application Development Maturity Model

Organizations have invested heavily in modern DevOps tools but usage is inconsistent, with advanced capabilities like AI assistance underused and tool sprawl fragmenting workflows. Operationally, teams rely on a mix of automated and manual processes, leading to slow, inconsistent release cycles and limited performance visibility. Culturally, developers seek autonomy, modern tools, and less toil, while leadership prioritizes predictability and control, creating ongoing tension. Inconsistent training investment also leaves some teams unprepared to adopt emerging technologies effectively. The reassessment of enablement strategies is forcing a candid look at existing delivery models, governance structures, and skill sets, as teams confront the reality that today’s “doing things right” may not translate into tomorrow’s competitive advantage.

In surveying 390 app dev practice leaders at global enterprises, we found the following:

• Effectiveness. Respondents are generally satisfied with the quality of their teams’ output; however, 44% are dissatisfied with the length of software delivery cycles and the time required to introduce new features or operability improvements. Nearly two-thirds of respondents indicate that deploying software changes into production takes five weeks or longer, highlighting a persistent gap between quality outcomes and delivery speed.
  Historically, organizations measured mainframe development success by stability, predictability, cost containment, and throughput. While these metrics remain important, the data shows they no longer suffice on their own. In the AI era, leading organizations are questioning whether legacy KPIs adequately reflect value in an environment where adaptability, data accessibility, experimentation speed, and intelligent automation are increasingly critical. Organizations that lag continue to prioritize uptime over responsiveness, resulting in longer lead times and slower improvement cycles (see Figure 1).

• Metrics strategy. More than 80% of respondents report using Flow, SPACE, DORA, or DX Core 4 metrics to measure development effectiveness. Among these, they most commonly track metrics for efficiency and flow, lead time to changes, and performance indicators (e.g., code quality, failure rate). Respondents less frequently emphasize developer satisfaction and retention metrics, despite long onboarding times and ongoing skills challenges. This pattern suggests that while respondents have widely adopted measurement frameworks, application varies. Higher-maturity organizations use metrics to guide investment and change, while their lower-maturity peers track metrics primarily for visibility rather than action (see Figure 2).

• Developer experience related to the tools that are provided and used for development. Most organizations provide core development tooling, including build pipeline automation, code quality and security tools, test automation, and deployment pipelines. Self-service infrastructure, database schema tooling, and APM or observability capabilities are less common.
  Tool usage highlights sharper maturity differences. Mainframe developers most frequently use modern version control systems, IDEs such as Eclipse and VS Code, merge and pull requests for code review, and CLI/API-driven tooling like Zowe. In contrast, 33% or fewer report regular use of code coverage tools, systematic load or performance testing, testing harnesses with synthetic data, or AI-assisted development tools. Organizations with higher maturity are more consistently adopting these advanced capabilities, reducing manual effort and improving feedback loops (see Figure 3).

• DevOps, testing, and AI usage. Hybrid and cloud-based environments, containerized development, and automated dev environment provisioning remain below majority adoption.
  Testing practices also remain largely manual for respondents. Manual unit testing is most common, followed by automated unit testing in the same source language with mocking and stubbing. AI-assisted testing is still limited. Respondents note that their organizations primarily govern AI use overall via higher-level corporate and regulatory mandates, with less emphasis on team-level policies, monitoring, or metrics. While adoption is strongest for use cases such as code assistance, application understanding, and document generation, more advanced approaches — including agentic workflows and vibe coding — remain aspirational and are most closely associated with more mature development organizations.