The Invisible Lottery: How Subtle Cues Steer Algorithm Choice in LLM Code Generation

摘要

Large language models (LLMs) now generate substantial production code, often for tasks with multiple valid algorithmic solutions. The hidden risk is that incidental prompt cues can steer \emph{which} algorithm is selected, even when all outputs pass the same tests. Prompt sensitivity is well studied as a tool to improve output quality, but we instead examine output policy: algorithm choice under fixed correctness. We define algorithm steering and run 55{,}545 controlled experiments across 11 tasks, 19 cue types (18 channels plus a memoization ablation), and 15 models. We find large, interpretable shifts in algorithm-family distributions (up to 100 percentage points, pp), including on applied tasks such as rate limiting, yielding an ``invisible lottery'' in which accidental context alters performance, security, and maintainability.