‘Falling asleep at the wheel’ – a cautionary tale about good AI

I’ve been away for a break up in the French Alps – a wonderful time and place, and a great chance to mulch while hiking from bright sunshine up over the snow line about Getting High, and God-Like and the connections between them.

So I was delighted when my good friend Jonathan got in touch about an episode of ‘Cautionary Tales’ that he’d heard, which details the tragic story of a French airliner that crashed en route to Paris from Rio… and the small errors that the pilots made that turned out to be fatal.

It’s a brilliant exploration of the ‘paradox of automation’: when systems get better and better, and more and more reliable, they fail more and more rarely… and yet – because of this rare failure of a super-competent system – the humans who have to step in and take over paradoxically have to be more skilled than ever… even though their skills have been atrophied by lack of practice.

As Tim Harford puts it,

‘there is huge danger in abandoning our own judgement and letting the computers do the thinking.’

In fact, studies have shown (the list of sources for the episode is here) that HR recruiters who used ‘good AI’ made worse decisions on hiring as they ‘fell asleep at the wheel’:

And those who used ChatGPT in brainstorming sessions – despite having one of the greatest idea-creation tools at their disposal – failed to consistently come up with good ideas.

This is something I discuss in some depth in the final chapter of the book, an excerpt of which you can read here:

The more we use advanced systems, the more we are likely to become deskilled… the narrower our field of problem-solving will become, and the more reliant we will be on those machines… and those who control them.

That is a plane I don’t want to be flying in. Instead, we need to emphasise – as Harford does here – that it’s not about machines becoming better, or humans becoming better, but human-machine interaction becoming better. It’s only when we work better together with our technologies – these remarkable creations – that we will get the best out of them, and out of one another too.

You can buy a copy of the book here.

And do subscribe to the brilliant Process This series that Tripp Fuller and I have been putting together, featuring experts on AI from economics, theology, philosophy, politics and tech talking about the issues in play.