I recently came across a news article which reported that Health New Zealand, the primary publicly funded body set up by the New Zealand government to oversee their healthcare system was using a single Excel spreadsheet to track $28 Billion of public money. An independent report by Deloitte found that the reliance on a single spreadsheet to manage their finances was a ‘major issue’. No kidding.
You might think that this kind of technological overreach is rare but I suspect that it’s far more common than we think, and it says a lot about humans, organisations and their relationship with technology – a topic worth exploring more here.
Let’s start with sunk cost fallacy and the tendency for organisations to cling on to legacy, outdated systems because of the considerable sums already spent on their implementation and maintenance. Even when newer, more efficient solutions are available. A common impulse here is to evolve software over time through bolt on solutions and end up with ‘spaghetti systems’, with large proportions of technology spend going on maintenance of core, outdated systems and hundreds of workarounds and bolt-ons, creating a highly complex house of cards.
For many years this was the challenge faced by the big banks, who relied on legacy mainframe systems and needed to spend up to 80% of their not insignificant IT budgets to undertake expensive system updates rather than investing in new, more agile technology solutions. These short-term fixes acted as a brake on innovation, giving a head-start to new and nimble fintech banks like Monzo, Revolut and Starling. As HSBC’s chief technology architect once said, these spaghetti systems can generate their own technology dependencies: ‘Everything is connected to everything else, pull on one thread and everything comes with it.’
Then there’s Gourville’s rule of thumb (see Harvard economist John Gourville’s original research paper) which states that companies tend to overvalue the benefits of their new products or technologies (by a factor of 3). Consumers, on the other hand, tend to overvalue the benefits of their existing habits and products (by a factor of 3). Substitute users in an organisation for consumers here and what this effectively means is that a new system or technology needs to be 10 times better than the last one to overcome this value gap and for it to be see as a viable replacement (I wrote more about technology acceptance models here if you’re interested).
Combine this with availability bias (familiar technology is readily available in our minds, making it seem like the best or most common option) and familiarity bias (we tend to see things that we are familiar with as being safer, and more reliable), and it’s not hard to see how people return to familiar technology (like Excel) to simplify working with complexity.
This, of course, can have serious consequences. In the autumn of 2020 at the height of the COVID-19 pandemic almost 16,000 COVID cases went unreported due to a poorly thought through use of Microsoft Excel. Public Health England (PHE) had set up an automatic process to pull data from commercial companies that were paid to analyse swab tests of the public into Excel templates. The data in these Excel files was then uploaded and made available to the NHS Test and Trace team and government COVID dashboards. Unfortunately the PHE developers had selected an old XLS file format to do this which meant that each template was limited to handling only 65,000 rows of data rather than the one million rows that Excel is actually capable of dealing with. As each test result created several rows of data this meant that each template had an upper limit of around 1400 cases that it could record and any additional cases that came in after that were simply not recorded. The mistake of using outdated software meant that there were eight days of incomplete data and thousands of cases that were not reported or passed on, with potentially very serious consequences. As the BBC noted at the time, Excel’s XLS file format goes back to 1987 and was superseded by XLSX in 2007 which would have been able to handle sixteen times the number of cases had it been used.
Excel is, of course, not an inherently bad tool (in fact quite the opposite), but the problems created by technology application creep (the growing use of comparatively basic technology for increasingly complex tasks) can be compounded by the use of the tools themselves. And spreadsheets are the perfect environment for mistakes to compound and grow unchecked.
Matt Parker’s book ‘Humble Pi: A Comedy of Maths Errors’ is brilliant on this. He notes how The European Spreadsheet Risks Interest Group, an organisation set up to look at this problem, estimates that 90 per cent of all spreadsheets contain errors. The ‘horror stories’ page of their website contains a whole series of spreadsheet challenges and errors. These include the scientific body that is in charge of standardising the names of genes (the HUGO Gene Nomenclature Committee) needing to provide guidelines to scientists for originating names for newly identified genes that avoid problems created by Excel auto formatting (for example Excel changing a gene’s alphanumeric symbol of MARCH1 into the date 1-Mar).
Then there’s the spreadsheet mistake that delayed the opening of a new £150 million Scottish hospital and led to £16 million of remedial work needing to be done to make the critical care rooms fit for use. Or how about the spreadsheet input error that lost a state fund set up in Ireland to support jobs a total of 750,000 Euros because a number was wrongly input as Euros rather than Dollars. Or the errors on a couple of spreadsheets from a County Sheriff’s Office in the US that cost that particular county almost half a million dollars. The County Sheriff is quoted as saying at the time that ‘the spreadsheets were emailed back and forth…Because of some cutting and pasting, not all the formulas were pasted correctly. It was an unintended error’. The list goes on.
I’d find it hard to believe that everyone that was working with the Health New Zealand spreadsheet thought that it was a good idea that they did things in that way. But what often happens is that it begins with solving a particular need in haste with a under-qualified system and as more and more dependency builds on that system it becomes harder and harder to unpick.
Unintended application creep becomes emergent, leading to a kind of functional drift in which technologies are applied in ways that they were never intended. As we move at pace into the wide and deep application of AI, this is something we should all be wary of.
A version of this post appeared on my weekly Substack of AI and digital trends, and transformation insights. To join our community of over ten thousand subscribers you can sign up to that here.
To get posts like this delivered straight to your inbox, drop your email into the box below.
Leave a Reply