Last week I was asked to take part in a Cambridge Judge Business School panel on AI transformation. It was a pretty wide ranging discussion but there was a useful lens on this given by the latest McKinsey State of AI 2025 report which had just come out. There were some interesting findings about enterprise adoption of AI including (predictably perhaps) that whilst adoption is near universal it is also shallow, and that many businesses remain stuck in the pilot phase. 88% of businesses for example, now report regular AI use in at least one business function, but usage is often concentrated in a few functions, and many organisations are still piloting use cases.
The ‘bottom-up trap’ and ‘top-down fantasy’
I think it’s easy to get caught up in the hype and believe that every business will be left behind if they don’t move faster with AI adoption. Talking a deliberate, steady approach to integrating potentially transformational technologies is not entirely a bad thing. But I do want to call out one disconnect which I think is a bigger potential challenge than the speed at which organisations are going. This is what Karl Yeh has called the ‘bottom-up trap’ and the ‘top-down fantasy’ problem. Companies, he says, often fall foul of a ‘bottom-up trap’ where local experiments stay siloed and struggle to scale beyond pockets of enthusiasm, efforts lack alignment with enterprise priorities limiting impact and investment, early adopters risk burnout without visible leadership support and so momentum fizzles as pilots fail to embed into workflows or decision-making. Equally the ‘top-down fantasy’ happens when senior leaders announce grand visions without a grounding in practical use cases, strategies assume compliance and overlook frontline realities and adoption barriers, cultural resistance grows when employees feel imposed upon rather than involved, and transformation plans stall when rhetoric outpaces capability building.
This disconnect between senior leadership intent, organisational strategy and the reality of how stuff actually gets done and what will actually help it to be done better is perhaps one of the key reasons why so many organisations are struggling to see a return on AI investments. That MIT study from a few months back that showed that 95% of generative AI pilot projects fail to produce measurable return or impact has come in for some criticism (around methodology) but there are plenty of other studies that show just how challenging it is to prove ROI on AI investments. The McKinsey survey found that over 80% of businesses were seeing no EBIT impact despite some benefits being seen in business units. Interestingly, of 25 attributes that were tested, workflow redesign had the biggest single EBIT impact and yet less than a quarter of businesses in the survey (21%) have fundamentally redesigned workflows and less than 20% had actually tracked GenAI KPIs at all (the top correlated practice for impact on the bottom-line).
Johnson and Johnson’s AI transformation is an interesting counterpoint to this (I’ve written up a longer case study on this here if you’re interested). They began in 2022 with a ‘thousand flowers bloom’ approach which invited ideas for where AI could bring benefit from right across its global business, and then facilitated open, safe-to-fail experimentation, overseen by a centralised AI governance board which ensured feasibility and alignment. This led to a burst of curiosity and creativity that resulted in 900 AI projects, supporting ground-up buy in, aligned experimentation and capability building as teams learned how AI could be (and could not be) applied to their domains.
The challenge then of course was fragmentation and duplication. So last year J & J pivoted to focus on the 10-15% of these use cases which were generating around 80% of the measurable value, concentrating investment in high-impact areas. They disbanded the central AI governance board, distributing the governance to core business units (commercial, R & D, supply chain) ensuring contextual and domain accountability, resource optimisation, and alignment to specific business goals such as accelerated drug discovery, and enhancing sales effectiveness. This new approach meant that they could focus on scaling the highest-value use cases with specific propositions including AI sales copilots, predictive models and simulations in drug discovery and supply chain risk management.
The J & J case study is a good example of starting broad, then narrowing to bring focus and impact, starting with central governance which then devolves as local initiatives scale, creating learning loops involving learning from failed initiatives, and treating it as an evolving capability, moving from AI projects to AI-powered processes.
I’d like to finish this reflection with a few thoughts on assessing ROI, which seems to be an area which many businesses are particularly struggling with. I think it can be useful here to focus on both hard and soft measures (as articulated here by IBM). The former are perhaps the more obvious measures that are easier to get to. But there is real value in also tracking soft measures relating more to things like decision-quality, customer and employee engagement and innovation velocity. Here’s my initial list:
Hard Measures (quantifiable financial and operational impact)
- Cost reduction: Track savings from automation, reduced labour, or lower error rates compared with baseline costs.
- Productivity gains: Measure increased output per employee or reduced cycle time using process metrics.
- Revenue uplift: Attribute incremental sales or cross-sell opportunities driven by AI-powered recommendations or dynamic pricing.
- Process efficiency: Compare throughput, defect rates, and operational downtime before and after AI implementation.
- Customer acquisition cost: Evaluate change in marketing efficiency or conversion rates due to AI targeting or optimisation.
- Resource utilisation: Quantify improvements in equipment uptime, inventory optimisation, or logistics efficiency.
- Time to insight: Measure speed of data analysis or decision turnaround relative to prior benchmarks.
- Model accuracy and impact: Track model performance improvements that drive measurable business outcomes (e.g., fraud detection rate).
Soft Measures (qualitative or more intangible benefits)
- Decision quality: Conduct pre- and post-implementation surveys to assess confidence, speed, and evidence-based reasoning in leadership decisions.
- Customer satisfaction: Use NPS, CSAT, or sentiment analysis to detect improvement in perceived experience due to AI-enabled personalisation.
- Employee engagement: Track adoption rates, satisfaction surveys, and task satisfaction when AI tools are introduced.
- Innovation velocity: Measure frequency of new product ideas, time-to-prototype, or new insights generated using AI.
- Brand perception: Use social listening or brand tracking studies to gauge sentiment shifts linked to AI-enhanced services.
- Risk reduction: Assess improved compliance, reduced incidents, or more accurate risk scoring as proxies for AI-driven foresight.
- Decision traceability: Track auditability and transparency in decision logs to measure governance maturity.
The softer, more intangible measures are harder to quantify, so useful techniques may include the use of balanced scorecards (combining quant KPIs with qual assessments), outcome mapping (attempting to define a cause and effect between AI use cases and strategic goals), decision reviews (using controlled before-and-after case comparisons), AI adoption metrics and qual feedback, and other proxy measures (e.g. speed-to-insight).
Back in May I wrote about using the classic Design Thinking framework of Desirability, Viability and Feasibility (DVF) to assess the value of AI initiatives. Feasibility relates to how easily something can be created, Viability meaning sustainable profitability or benefit to the business, and desirability relating to whether end users actually want what you’re creating. I mentioned then that the main focus seems to be going on feasibility (‘let’s do something with AI, what can we do?’) and viability (‘how can we use AI to maximise benefit to the business?’). It’s remarkably easy to forget about the ‘desirability’ of initiatives, or what will actually work for and be attractive to employees and/or customers. If we are to solve the bottom-up trap and top-down fantasy we need to be able to experiment widely but also scale in a focused, aligned and inclusive way to ensure that we’re bringing employees and customers on the journey with us.
A version of this post appeared on my weekly Substack of AI and digital trends, and transformation insights. To join our community of over ten thousand subscribers you can sign up to that here.
To get posts like this delivered straight to your inbox, drop your email into the box below.
Photo by Nick Fewings on Unsplash
Leave a Reply