The Lindy Effect, and the optimisation trap

A while back I read Shaw Talebi’s reflections from doing Nassim Taleb’s Real World Risk Institute course. One of the concepts featured in the course is The Lindy Effect – a principle derived from the observation that the future life expectancy of non-perishable entities, such as technologies, ideas, or institutions, is proportional to their current age. Put simply, the longer something has lasted, the longer it is likely to endure (sidenote: the concept was named after a New York deli and originally referenced the career prospects of comedians).

The Lindy Effect and corporate longevity

When we apply this idea to corporations, the Lindy Effect suggests that older companies have proven their resilience to market fluctuations, competitive pressures, and shifting consumer demands, which in turn increases their odds of future survival. And studies have indeed shown this to be true. The chart below (source) shows the survival rate over time of businesses that were established in 1994 right up until 2023. 20% of businesses had closed down within a year, 50% after five years and so on. The fact that this is a curve rather than a linear decline is the Lindy Effect – the older a business gets the higher its marginal survival percentages get.

Yet the Lindy Effect does not guarantee longevity. Instead it also highlights a key dichotomy: while enduring companies benefit from established brand equity, institutional knowledge, and operational stability, they are also at risk of becoming complacent, resistant to change, or vulnerable to disruption by more agile competitors.

A comprehensive study by the Telfer School of Management at the University of Ottawa into the implosion of Canadian Telecoms giant Nortel (the once market-leading company filed for bankruptcy in 2009) tells us a lot about how this can happen. The study’s lead author, Jonathan Calof, said:

‘There were three major factors that caused the failure. When Nortel was a market leader in the ’70s, it developed an arrogant culture, which led to poor financial discipline. Then in the ’90s, it focused so intensely on growth that it broke its ability to innovate and read the market. And after the tech bubble popped, it turned inward and cut costs to the point where it alienated customers.’

Arrogance, pursuing growth at all costs, turning inwards, efficiency at the expense of innovation. It’s a familiar story.

The Optimisation Trap

There are, let’s not forget, huge benefits that can come from continuous incremental improvement but there are also limits to the gains that can be secured by optimising an existing system. Sometimes we need to think bigger and redesign the system itself in order to take a leap forwards. And this is where problems can arise.

Optimisation, by its nature, reinforces the current way of doing things. It focuses on extracting maximum value from existing systems, products, or processes, often at the expense of exploring new paradigms. While this can enhance operational efficiency, it also narrows an organisation’s focus, creating blind spots to disruptive opportunities and external threats. Over time, the more optimised a company becomes around its existing model, the harder it becomes to envision, let alone embrace, transformative change.

We might call this the ‘optimisation trap’ – a relentless focus on improving existing advantages combined with a hubris that results in an unwillingness to think differently, leading to diminishing returns and a dangerous myopia that stifles innovation and long-term growth. There’s several signs of this happening:

1. Cognitive Inertia: As teams and leadership become deeply familiar with optimised systems, their thinking aligns more tightly with existing paradigms. This creates a bias toward incremental improvements over bold, untested ideas.
2. Resource Allocation: Optimisation often demands significant resources—time, talent, and capital—leaving little bandwidth for exploratory or disruptive initiatives.
3. Cultural Resistance: Organisations optimised for one model often develop cultures that reward stability and predictability. This can result in skepticism or outright resistance to disruptive ideas that challenge the status quo.
4. Success Reinforcement: Ironically, success exacerbates the problem. The more an optimized model delivers results, the harder it is to justify deviating from it. This creates a feedback loop that makes transformative thinking feel unnecessary or excessively risky.

I’ve written before about how companies can break out of optimisation. But today I want to leave you with one thought on this.

One aspect of the optimisation trap is that we optimise for the wrong timescale. And this can apply to people as well as organisations. Graham Weaver phrases this well in this short Reel. As we progress through life, working to make our situation better, sometimes we will be unwilling to make a bigger change because of the short term challenges that this brings. In other words, as the chart below shows, we go down before we go up.

Graham’s point is that we’re often continuously optimising for the short term, and are therefore unlikely to do anything that will put us in a worse situation, even if that is temporary. So we don’t make the move. But, in our personal lives, as in business, this can mean we end up climbing the wrong hill. Optimising on a longer timescale however, means that we recognise that short term challenges can lead to long term benefits and progress. It frees us up to make bigger, more breakthrough decisions and changes. And it helps us to get out of the optimisation trap.

Image: Shaw Talebi. Data source: BLS.

A version of this post was also published on my weekly Substack – To join our community of thousands of subscribers you can sign up to that here.

To get posts like this delivered straight to your inbox, drop your email into the box below.