Early morning on the 16th October 1962 and US National Security advisor McGeorge Bundy has just handed a folder of photographs to President John F. Kennedy. Inside were pictures taken by a U-2 plane that had flown over Western Cuba two days earlier. Analysts who had studied the pictures believed that they showed launch sites for Soviet medium-range ballistic missiles, just ninety miles from the Florida coast. By nine o’clock that morning Kennedy had pulled together his wider group of trusted advisers, called ExComm. The intelligence said that the sites were days away from being operational. The clock was already ticking.
The Joint Chiefs wanted to enact air strikes on the sites, followed by a full invasion of Cuba. The Air Force couldn’t guarantee that an airstrike could disable every launcher and anything left operational could turn this into the start of a war rather than the end of a crisis. Once a commitment had been made to this course of action there would be no turning back. The decision had an extra dimension to it, given that eighteen months earlier pretty much the same group of people had approved the disastrous invasion of Cuba known as the Bay of Pigs.
Under significant pressure to make a decision and to make the right one quickly, Kennedy did something interesting. He slowed down. He kept the ExComm running for the best part of a week, splitting the group into smaller sub-groups, bringing in outsiders and even absenting himself from some of the sessions on the grounds that his presence would lead the discussion in a certain direction. On the 22nd October he went on television to announce the naval quarantine.
The Kennedy story is a good example of a leader who is situationally aware enough to avoid what often happens in high-pressure decision-making in teams – groupthink (in fact the term ‘groupthink’ was inspired by research done by Yale psychologist Irving Janis into the Bay of Pigs decision). Every decision under pressure carries with it two costs – the cost of being wrong and the cost of delay. The real question to ask when under pressure is which is the larger cost.
When being wrong is expensive and being late is less so, rigour is affordable. But when being late is costly and being wrong is cheap it’s often better to move forwards now and then course correct if you need to. When both are high, the answer is not to pick one but to change the terms of the bet. To make it smaller, or reversible, or to spend a little to change the odds in your favour. Kennedy chose to take his time because the cost of being wrong was extremely high. But he also went for one of the few choices that had a way back out of it.
Sixty years on from the Cuban Missile Crisis, AI has changed the rules of decision-making. Where analysis once meant a heavy investment in time, in many cases that constraint has collapsed. Time improves decisions but being given the gift of time is often being taken as gains in speed or productivity rather than enabling more rigour. Meanwhile the cost of being wrong has become harder to see. Misleading documentation drafted by AI. A biased prompt which leads to unreliable insights in data analysis. An assumption that appeared in one deck unchecked and now appears in all of them. Code that works, written by someone who can’t debug it. A contested question that is presented as resolved and then quoted as fact. A strategy built from a consensus view because that’s what the model surfaced first. A precise forecast given confidently that ignores the uncertainty, meaning that every downstream decision is made against a level of accuracy that was never there. None of these seem like a bad decision on the day which is why they never get course corrected or reversed, but lots of small but poor quality decisions can accumulate over time.
So the trouble with small errors is that they don’t stay small, they can compound, especially when they’re not even seen. On 28th November 1979 a McDonnell Douglas DC-10 with over 200 passengers on board took off from Auckland airport on an 11-hour non-stop sightseeing trip to fly over Antarctica. Around noon that same day Captain Jim Collins brought the plane down out of the clouds to a height of around 2,000 feet so that the passengers could see the icy continent and take their pictures. As the plane emerged from the cloud instead of seeing McMurdo Sound and the coastline of Antarctica as expected, the pilots saw that they were actually about to fly into a mountain. They frantically tried to pull the plane up but tragically it was too late and flight TE901 crashed into the slopes of Mount Erebus killing all those on board. The enquiry that followed determined that the coordinates had been entered incorrectly two years earlier, routing the flight down McMurdo Sound. They were corrected on the morning of the flight, shifting the track two degrees east and directly over Mount Erebus but nobody told the crew. By the time they reached Antarctica this put the plane 28 miles east of where they thought they were. The pilots were not to blame but a couple of degrees change that nobody communicated had resulted in a terrible tragedy.
The ‘1 in 60’ rule of thumb in air navigation means that for every one degree that a plane veers from its intended course it misses its planned destination by 1 mile for every 60 miles that are flown. Fly 60 miles and you might still be able to see your destination and fix the error, but on a flight from London to Sydney (just over 10,500 miles) a one degree difference could mean that you arrive in Canberra 177 miles away. What can start out as a relatively minor error can compound over time and lead to significant divergence from the intended course.
When we’re under pressure it can be tempting to default to lazy use of AI which can then result in a cascade of poor decisions that may not be individually consequential but which can compound to lead us miles off course. So it’s worth thinking about four error mechanisms, two of which introduce the error and two of which prevent us from course correcting: compression, anchoring, fluency and erosion.
Starting with compression, every time we use AI for assimilation or summary it’s a judgement about what matters. Leave that judgement to an AI and it will flatten dissent, uncertainty and outliers in a way that makes it much harder to bring them back in. Then there’s anchoring. Leave the AI to frame the situation or challenge and it’s already set the parameters and the reference points which will dictate the discussion. The decision has effectively already been made and no-one even realises it.
The next two turn the error into a misaligned course. The first one of these is a fluency issue. AI-produced outputs can easily read as well-structured, well-sourced work that someone has put together with assurance. The register of AI output is confidence, making it potentially harder to diagnose error. The second is an erosion in our capacity to identify omissions, misdirection, or flaws, particularly when under pressure. It worries me that the more we use AI tools the more accepting we may become of their tone, focus, and quirks in outputs.
Compounding drift kills your ability to answer the two-costs question. You can’t weigh being wrong against being late when what your looking at has already been flattened, the framing already set, and your feel for when something isn’t quite right has been recalibrated.
This is different from the usual argument for human judgement in the use of AI. That argument is often about catching wrong answers but the pilots of flight TE901 had no reason to doubt the flight computer until they emerged from the clouds and finally saw where they actually were. Rather than being off course, they were flying an accurate course to the wrong place.
Kennedy’s real skill in 1962 was in creating a process and an environment where the trade-off could be argued openly and honestly. Splitting the group, bringing in outsiders, staying out of the discussion so that it didn’t become swayed by him. The question with AI is whether it is genuinely helping us to see the real trade-offs or whether we’re consistently two degrees out without even realising it.
A version of this post appeared on my weekly Substack of AI and digital trends, and transformation insights. To join our community of over thirteen thousand subscribers you can sign up to that here.
To get posts like this delivered straight to your inbox, drop your email into the box below.
Leave a Reply