en.killbait.com
The hidden dangers of relentless efficiency: how over-optimisation weakens systems
The article explores the concept of 'efficiency' and how the relentless pursuit of optimisation can lead to unintended consequences. It begins with the example of hospital bed occupancy in England, where 95% of beds were occupied during flu season, leaving no room for unexpected surges in patients. This illustrates the first trap of optimisation: removing slack from systems, which makes them vulnerable to disruptions. The second trap is having a narrow definition of 'best', as seen in Milton Keynes' roundabout-heavy road system that prioritised traffic flow over passenger comfort. The third trap involves conflicting optimisations, demonstrated by the 2010 Flash Crash in US financial markets where automated trading algorithms caused extreme volatility. The article then connects these traps to artificial intelligence, warning that AI systems might remove human oversight, focus narrowly on measurable metrics while ignoring judgment or empathy, and interact in unpredictable ways when multiple systems collaborate. The authors argue that the challenge isn't just finding the best answer but deciding what 'best' really means and knowing what not to optimise away. The piece concludes by emphasising that while optimisation has brought significant societal benefits, avoiding these traps is crucial as optimisation becomes more powerful.