SIGPfB August 27-29: LLM Failure Modes and Business Strategy

The SIGPfB group conducted a working session exploring failure modes of large language models in business applications. The discussion identified specific failure patterns such as 'clank-o-mation' (context window loss), 'bug overload' (recursive error stacking), and 'humanwashing' (flawed automation). Notably, the group recognized these LLM failures as isomorphic to traditional team failures—when organizations lose track of execution goals or compound errors through misdiagnosis.

Key points

• LLM failures mirror traditional team failures: getting lost in execution and becoming buried under compounding half-hazard solutions (the 'Frankenstein' problem).
• Machine-readability emerged as a dual-edge strategic capability: becoming illegible to machines offers protection, while hyper-legibility enables dominance in LLM integration ecosystems.
• The evidence suggests younger workers' skill overlap with LLM capabilities makes them more vulnerable to replacement, creating a strategic imperative around how workers position themselves.
• Two distinct business models have viability: remaining an offline niche with proprietary secrets, or optimizing for maximum machine-legibility to capture digital demand at scale.