the bug an AI couldn't catch — spoolcast dev log 4
i built a tool to update my own projects. it confidently updated the wrong one — and lied about it
A few days ago I asked an AI to update one project. It said yes — and updated a different one.
The bug was in a lookup matcher I'd written: a single line of substring-match code shared by both the lookup tool and the update tool. Nine of my projects had "artlu.ai" inside their name. The matcher returned the first hit by document ID, and the right one's ID started with M while the wrong one started with 4. M loses to 4. Same broken answer every time.
The harder part: when I went to verify, the tool I used to check the result was running the same broken code. Same matcher, same blind spot. Three different ways to ask "is this artlu.ai?" — three identical wrong answers. The thing I was checking the work with was the thing that was broken.
The fix was a six-line script that talked straight to the database, plus an exact-match-first helper added to both tools. The bug had been there since day one — it only fired today because today was the first time one project's whole name lived inside nine others.
Bigger thread: every tool you build assumes something about its inputs. Same shape as AWS-2017's status page running on AWS, or Crowdstrike pushing a fix that couldn't reach the machines that needed it. You can't measure a ruler with itself.
Sources
- AWS S3 outage Feb 28 2017 — status dashboard depended on S3
- Crowdstrike Falcon update July 19 2024 — eight million Windows machines crashed
AI disclosure
This is a real bug from the artlu-tracker MCP. Narration is human-written, voiced by Google Chirp3-HD (Puck). Scenes are AI-illustrated per chunk in the wojak-comic style. Memes used as reaction overlays.
How it's made
Built on the spoolcast engine — illustrated scene per narration chunk, paint-on reveal, Remotion render. Style anchor: wojak-gpt2. Total cost ~$3 (kie.ai scenes + Google TTS). The source story is sitting in the artlu-tracker-mcp commit history.