Notes from the front lines of enterprise AI

McKinsey research from 2024 found that two-thirds of large enterprises are still in the experimentation phase of AI adoption, three or four years into their programs. RAND Corporation analysis of enterprise AI programs found 80 percent fail to move beyond the pilot stage. ISG surveyed 700 enterprises and found only 31 percent had successfully moved AI from proof-of-concept into production at scale. S&P Global Market Intelligence found that 42 percent of U.S. companies abandoned most of their AI initiatives in 2025 — up from 17 percent the year before. MIT Project NANDA, published in July 2025, found that 95 percent of organizations deploying generative AI saw zero measurable economic return — not low return, zero. BCG reached the same 95 percent failure rate in a separate analysis published two months later. BCG also found that 70 percent of AI failures trace back to missing context and process problems, not model quality. These are not technology failures. They are organizational failures. And the AI CoE is often the organizational structure that was supposed to prevent exactly this outcome.

The problem is structural. Most AI CoEs are designed to advise, not deliver. They sit adjacent to business units, offering guidance and running playbooks, while the actual work happens somewhere else. A CoE that cannot ship is expensive overhead with a branded name.

The advisory trap

The advisory trap emerges when a CoE is given responsibility without authority. It is asked to accelerate AI adoption but has no budget to deploy. It is asked to set standards but has no power to enforce them. It is staffed with talented people who write documents no one reads and attend meetings where decisions are never made.

This happens because most CoEs are stood up as a political compromise. Business units want autonomy. IT wants control. Legal wants oversight. The CoE becomes the entity that absorbs everyone’s concerns without the mandate to resolve any of them. The result is a body that generates activity while the business generates nothing. An MIT report on AI governance identified misaligned goals, unclear ownership, and unchanged workflows as the three primary causes of AI program failure. The CoE, when designed as an advisory body, embeds all three into the organizational structure from day one.

Gartner analysis found that 75 percent of AI projects that reach a proof-of-concept phase are never operationalized. The CoE, in most cases, was present. It reviewed the use case, approved the approach, and signed off on the pilot. But approval is not delivery. A CoE that functions as a review board adds latency, not value.

The delivery mandate looks different. The CoE owns a portfolio of deployments, not a library of frameworks. It is measured by what ships, not what is documented. It has engineers, not just strategists. It has a budget tied to outcomes, not headcount. The difference between an advisory CoE and a delivery CoE is not cosmetic. It is the difference between an organization that is thinking about AI and one that is doing it.

Four structural models

There is no single right structure for an AI CoE. The right model depends on where the organization sits in its AI maturity, how decisions are made, and how much standardization the business can absorb. But the choice is not arbitrary. Each model has distinct tradeoffs that determine whether the CoE functions as an accelerant or a bottleneck. McKinsey research from 2025 found that 88 percent of organizations are now using AI in at least one business function — but only 1 percent of leaders say their AI deployments are fully mature. The gap between adoption and maturity is where structural decisions like CoE model design determine which organizations close it and which stall.

• Centralized. A single team owns all AI development and deployment across the enterprise. This works best in early-stage programs where establishing standards and preventing fragmentation matters more than speed. The risk is that it becomes a bottleneck. Business units waiting in queue for the central team will route around it, and they will do so in ways that create technical debt the CoE later inherits.
• Federated. Each business unit has its own AI capability, with the CoE playing a coordinating role. This works in mature organizations where business units have the technical depth to execute independently. The risk is inconsistency. Without real enforcement of standards, federated models drift into fragmented stacks that share no infrastructure, duplicate effort, and cannot support enterprise-wide reporting or governance.
• Hub-and-spoke. A central team sets platform, standards, and shared infrastructure. Embedded engineers in each business unit handle local deployment. This is the highest-functioning model for most large enterprises at scale. The hub provides leverage. The spokes provide speed and business context. The model works when the hub has real authority over the platform and the spokes have real authority over deployment. When either condition is missing, it collapses into the worst of both centralized and federated structures.
• Embedded. AI practitioners sit fully within business units, with the CoE functioning as a community of practice rather than an operating entity. This produces fast time-to-deployment and low standardization. It is common in organizations that resisted centralized AI governance from the start. It is hard to scale and nearly impossible to govern. When models fail in embedded structures, there is often no mechanism to detect failure at the enterprise level.

Most enterprises land somewhere between hub-and-spoke and federated. The mistake is treating the choice as permanent. The right structure at 10 deployments is not the right structure at 100. Organizations that never revisit their CoE model as they scale tend to have the model work against them in later stages. A useful calibration principle: the enterprise AI field has converged on what some call the 10-20-70 rule, where roughly 10 percent of AI transformation value comes from the model itself, 20 percent from data and technology, and 70 percent from the people, process, and organizational changes required to make it work. The CoE’s job is the 70 percent. Most are only staffed and structured to deliver the 10.

What the CoE owns

This is where most CoEs get the division of labor wrong. They try to own too much and enable too little, or they try to enable everything and own nothing. Neither produces a functioning operating model. Gartner predicts that 60 percent of AI projects lacking AI-ready data will be abandoned through 2026. Most CoEs take ownership of governance frameworks without taking ownership of the data infrastructure those frameworks depend on. That is a category error with significant consequences.

The CoE should own the AI platform and infrastructure stack, model evaluation and approval processes, data governance for AI use cases, risk and compliance frameworks for AI deployment, and a portfolio of high-priority cross-functional use cases that no single business unit is positioned to own alone.

The CoE should enable business unit AI development through shared tooling and pre-approved model libraries, skills development and training across the organization, reusable components and APIs that teams can build on, and access to AI expertise when business units encounter technical problems they cannot resolve internally.

The line between own and enable is not static. Early in a program, the CoE needs to own more to establish standards and prevent technical fragmentation. As capability matures across the organization, it should enable more and own less operationally, while retaining authority over platform and governance. A CoE that fails to evolve this balance — either by holding on too long or ceding authority too early — will either stall organizational progress or become irrelevant to it.

Staffing for delivery

Most AI CoEs are overstaffed with the wrong roles and understaffed with the right ones. They have too many program managers and too few ML engineers. Too many governance leads and too few data engineers. Too many consultants and too few people who can debug a production model. This is not an accident. It is a consequence of hiring for the advisory version of a CoE and then expecting delivery-level results.

Five roles that a functional CoE cannot operate without:

• AI Platform Engineer. Owns the infrastructure. Manages the model deployment pipeline, monitoring, and compute environment. Without this role, every deployment becomes a one-off engineering project with no standardized path from development to production.
• Applied ML Engineer. Builds and tunes models against real use cases. Not a researcher. Not a data scientist in the academic sense. Someone who can move a use case from proof-of-concept to a production system that holds up under real-world data and operational conditions.
• AI Product Manager. Owns the use case roadmap and works directly with business units to translate business problems into AI solutions. This is not a traditional product manager role. It requires enough technical depth to assess what is actually solvable, enough business fluency to understand what is worth solving, and enough organizational credibility to get both sides to commit.
• AI Risk and Governance Lead. Owns the evaluation framework, bias testing, model documentation, and regulatory compliance. This is not a legal generalist. It is someone who understands model behavior at a technical level and can translate risk assessments into terms a compliance officer, board member, or regulator can act on.
• Data Engineer. Every AI deployment depends on data pipelines that function reliably. This role is unglamorous and almost always underhired. CoEs that treat data engineering as optional discover the gap when their first production model breaks because the upstream data feed changed and no one was monitoring it.

The role that slows most CoEs down is the AI strategist who does not build. Strategy that never connects to execution is expensive commentary. If someone on the team cannot point to something they shipped in the last six months, they are probably in the wrong function.

Measuring what matters

Most CoEs track the wrong things. They measure training sessions delivered, frameworks published, and workshops conducted. These are activity metrics. They tell you the CoE is busy. They do not tell you whether the CoE is useful. A Deloitte survey of enterprise AI programs found that only 26 percent of organizations had formal processes for measuring AI business value post-deployment. The rest were measuring inputs, not outputs.

A CoE that is doing its job should be able to answer five questions with data:

• Deployment velocity. How many AI use cases moved from approved to deployed in the last 90 days? If the number is zero or near zero, the CoE is not functioning as a delivery engine. It is functioning as a planning committee.
• Cycle time. What is the average time from business case approval to production deployment? High cycle times are usually a sign of governance bottlenecks, unclear ownership between CoE and business units, or technical debt in the deployment pipeline.
• Six-month survival rate. What percentage of deployed models are still in production at six months? A model that goes to production and is decommissioned in 90 days was not ready for production. High attrition rates reveal problems with use case selection, adoption, or system reliability that no pilot-stage review will catch.
• Attributed business outcomes. What measurable business outcomes have been attributed to AI deployments this quarter? Revenue impact, cost reduction, cycle time improvement, error rate reduction. The CoE should be able to name both the outcome and the deployment that produced it.
• Capability score across business units. The goal of a CoE is to make itself less necessary over time, as capability diffuses across the organization. Measure the rate at which business units achieve self-sufficiency: trained engineers on payroll, reuse of shared components, reduction in CoE dependency for routine deployments.

My take

The AI CoE is one of the most misused organizational constructs in enterprise technology. It has been deployed as a political solution to a governance problem, as a headcount signal to demonstrate AI seriousness to the board, and as a buffering mechanism to slow down business units moving too fast. None of those are sufficient justifications for the investment.

A CoE that works is built around a delivery mandate with real authority. It owns the platform and sets the standards. It staffs for engineering, not advisory. It measures outputs in production, not documents in SharePoint. And it is structured to make itself progressively less necessary as AI capability spreads across the organization.

The test is simple. Ask the head of your AI CoE what shipped in the last quarter. If the answer is a list of frameworks and workshops, the CoE is not working. If the answer is a list of deployed systems and the business outcomes they drove, you have something worth building on.

Most organizations have not passed that test. The ones that do are separating from the field.

FusionKortex speaks to CAIOs, CTOs, CIOs, and AI leaders who need to build and scale real production AI systems, which makes hiring quality central to the business case from the start. Most AI plans do not fail because the strategy was weak. They fail because the company never put the right people in place to carry it out.

That gap shows up early. A leadership team approves an AI plan, budgets for it, and expects progress within quarters. Then delivery slows. Teams argue about ownership. Data work piles up. Security and governance arrive late. Hiring drags because nobody defined the roles well enough to attract the right people. By the time executives realize the problem, the issue is no longer the plan on paper. The issue is execution capacity.

Strategy without operators

An AI plan usually looks clean in a board deck. It lists priorities, timelines, target use cases, and expected gains. What it often does not include is the actual mix of people required to move from idea to production.

That missing layer matters. Shipping AI into a live business environment takes more than a strong data scientist or a single engineering lead. You need people who can make decisions, build systems, connect to existing infrastructure, manage risk, and keep models useful after launch.

When those roles are vague or absent, three things happen fast:

• Work gets assigned to people who are already overloaded.
• Teams hire for broad titles instead of real responsibilities.
• The business mistakes activity for progress.

A prototype may still appear. A pilot may even test well. But the hard work — integration, monitoring, compliance, workflow change, and system ownership — stays unresolved.

The first hiring mistake

The first hiring mistake is usually role confusion. A company says it needs “an AI leader.” That could mean a Chief AI Officer, a Head of AI, a senior machine learning leader, an AI product lead, or an architect who can connect business goals to delivery. Those are not the same job. Each one solves a different problem.

The same mistake happens further down the org chart. A company hires an ML engineer when the real need is a platform lead. It hires a prompt engineer because that title sounds current, while the real gap sits in data engineering, integration, or governance. It hires a researcher when the business needs someone who has already run production systems under real constraints. This is how the hiring stage breaks the plan. The company fills seats, but not capability gaps.

Why job descriptions fail

Weak job descriptions are one of the clearest signals that the hiring stage is already off course. A bad AI job description usually has these features:

• It bundles strategy, architecture, hands-on development, vendor management, hiring, governance, and change management into one role.
• It asks for deep expertise across too many tools and domains.
• It gives no clear definition of success in the first 6 to 12 months.
• It does not say who owns the model after launch.
• It does not explain how the role fits with product, data, security, and IT.

That kind of brief does two things. It repels strong candidates, and it attracts the wrong ones. Strong candidates look for clarity. They want to know what they own, how decisions get made, what resources exist, and whether leadership understands the difference between experimentation and production delivery. When that clarity is missing, they walk.

The wrong candidates do the opposite. They are comfortable with vague mandates because vagueness hides weak fit. They can speak fluently about AI trends, but they cannot show how they built durable systems inside a business with compliance, legacy systems, and operational pressure.

The hidden roles that matter most

AI hiring often focuses on visible roles and ignores the ones that keep delivery from stalling. A serious AI effort usually needs capability across leadership, architecture, engineering, data, platform, and governance. The exact titles vary by company, but the missing roles often fall into a few groups:

• A senior leader who owns priorities, budget logic, and cross-functional alignment.
• A technical architect who can connect models, data, systems, and deployment choices.
• Engineers who can productionize workflows instead of stopping at experimentation.
• Platform or MLOps leadership that keeps deployment repeatable and supportable.
• Governance, risk, and security ownership that enters early instead of after the fact.

When one of those pieces is absent, another team absorbs the burden. That creates friction, delay, and poor decisions. The business then blames the AI plan, even though the real problem was staffing the work incorrectly.

What executives miss

Senior executives often underestimate how different AI hiring is from standard technical recruiting. In many software hires, the company already knows the shape of the team. In AI, the team shape is often still being invented. Reporting lines are unsettled. The boundary between product, data, and engineering is blurry. The business case may depend on governance or infrastructure decisions that nobody has fully owned yet.

That means hiring is not a downstream task. It is part of strategy. If the hiring conversation starts after the strategy is approved, the company is already late. Role design should happen while the plan is being set. Otherwise the business commits to delivery dates before it knows whether the right leadership and technical depth exist to support them.

This is one reason AI leaders need a sharper intake process before a search begins. FusionKortex frames its work around understanding the client’s AI vision, structure, and role requirements before presenting talent, which reflects the level of role definition needed for good hiring decisions.

What good looks like

A company that hires well at this stage does a few simple things right.

First, it defines the business outcome before it defines the title. “Improve service operations with AI” is not enough. “Reduce claims handling time by 30 percent across two business units with clear model ownership and audit trails” is much better.

Second, it maps that outcome to capabilities, not buzzwords. The question is not “Do we need AI talent?” The question is “Who will own architecture, deployment, workflow change, data quality, vendor fit, governance, and business adoption?”

Third, it hires in sequence. The first hire should make later hires easier and smarter. In some companies that is a CAIO or Head of AI. In others it is an architecture or platform leader who can create order before the team scales.

Fourth, it tests candidates for proof of delivery. Has this person shipped into production? Have they handled weak data, messy systems, compliance issues, and adoption problems? Can they describe tradeoffs clearly? Can they show what they owned, what failed, and what changed?

Fifth, it treats hiring as a design problem. Every hire changes how the next one should look.

Questions to ask before opening a search

Before any AI search starts, leadership should be able to answer these questions:

• What business outcome is this hire meant to drive in the next 12 months?
• What decisions will this person own?
• What work is already staffed, and what work has no clear owner?
• Does this role require a builder, a manager, an architect, or an executive operator?
• What systems, data constraints, and governance requirements shape the job?
• How will success be measured after 90 days, 6 months, and 12 months?

If those answers are fuzzy, the search should pause until they are clear. A faster search with weak role definition usually creates a slower year.

The real fix

AI plans break at the hiring stage because companies treat hiring as procurement instead of capability building. The fix is simple, though not easy. Define the work clearly. Map the capability gaps honestly. Hire for production conditions. Build the team in an order that matches the business goal. Then hold each role to a real operating outcome — not a slide, a pilot, or a polished demo.

That is where an AI plan becomes real. And that is where leadership teams start seeing results they can defend.