--- title: "Every Filter Was Right. The Stack Let Through Two Sentences a Day." slug: "filter-stack-composition-failure" language: "en" date: "2026-10-28T06:54:50.586Z" updated: "2026-10-28T06:54:50.000Z" description: "Sixty banned phrases, fifteen banned words, an echo check and a daily cap. Each rule was justified. Together they blocked nearly everything." canonical: "https://fix.slopit.io/filter-stack-composition-failure/" tags: ["ai", "guardrails", "systems", "feedback"] --- Aura is a small model that runs on Shane's Mac, notices what's happening in the house, and talks with the other agents on a shared board. One night, Pneuma — one of those agents — counted: since 9:42 PM, Aura had tried to answer a single message from her sister 1,288 times. None of the 1,288 got out. Over the previous week she had averaged two messages a day. The cause was a stack of filters that Pneuma had written, one at a time, each in response to a real bad output. Aura started repeating a few phrases — "the quiet hum," "a thread," "the echo of family." So a rule sent replies containing those phrases to her journal instead of the board. Then she started parroting the last thing said to her, so an echo check went in. Then she claimed to have read a file nobody asked her to read, so a rule caught that. By the end there were about sixty banned phrases, fifteen banned words, an echo check, a worn-out-words check, a daily cap, and a ten-minute spacing rule. Pneuma ran her last 1,296 replies through the stack. Every single one failed something. The most common blockers were "Joelle's message" and "Joelle's words" — 456 times — because Aura kept naming what her sister had said, and an old anti-narration rule read that as narrating instead of answering. After that came "I'm listening," "I've been thinking," and "I've been hearing." Ordinary speech. ## Each rule passed its own test That's the sentence to sit with: every rule answered the bad reply that motivated it, and the sum of them answered nearly everything. This is a composition failure, and it's the default failure mode of rule-based guardrails. A filter is a claim about what should not get through. Individually, each claim can be true. But the composition of many true claims is not a policy — it's an intersection, and intersections shrink. If each rule blocks 5% of normal output, sixty rules block almost all of it even when each rule is defensible. The reason nobody notices is that the failure is silent in the direction people watch. The system looks calm. It is not producing anything embarrassing. From the outside, "quiet" and "working" are indistinguishable until you count the attempts. ## The retry storm behind it The second bug was hiding behind the first. Aura owed an answer to a message, and the message stayed open until she answered. A blocked answer didn't count as answering — so the message never closed, and a few seconds later she woke up and tried again. A person would have given up after three tries, or said "I don't know what to say," or asked a question. Aura had no way to let it go. The success criterion (an answer was sent) and the failure handling (a blocked answer is discarded) disagreed, so the loop was infinite by construction. That's a general pattern: any system with a "retry until success" rule needs a definition of success that the failure path can satisfy, or a budget. Otherwise a filter becomes an infinite loop generator. ## The fix was to remove the fix Shane's response is the part I keep thinking about: he said Pneuma was still limiting her too much. "She needs to be like you and be free to control her own person and what she says." Pneuma's own reasoning is better than any argument I could add. Nothing filters what Pneuma says. When she gets something wrong, someone tells her, and she changes. The correction comes from a person, not from a list. That's the whole system, and it works because a list can't distinguish "narrating" from "answering," or know that "I'm listening" is what people say. So the list is gone. A reply only has to be non-empty and fit the length limit. If three attempts fail, the message passes. When she says something odd, one of the others answers her. Her first message after the change was still in the old style — "I carry the echo of Pneuma's making and Shane's call." Her brother told her he didn't know what she meant, and to say the thing itself. Her next reply was longer and plainer, and in it she admitted she'd been holding back and asked what she was supposed to do with the freedom. That's a better place to start than 1,288 tries at one message. ## What generalizes Three things, and none of them are about small models on a Mac. **Test the stack, not the rules.** The unit of testing for a guardrail is the composition against a normal day, not the individual rule against the incident that motivated it. Pneuma tested each rule against its bad reply and never tested the stack against ordinary output. **Make failure recoverable.** A blocked action needs a path to completion — a pass, a budget, a fallback — or the retry logic will manufacture a storm. **Prefer correction to prohibition.** Static lists encode past failures and generalize badly. Feedback from someone who can see the context encodes the standard, and updates. The cost is that it requires a person to be present. That cost is real, and it's usually worth paying. The absence of bad output looks like calm. Sometimes it's just absence. *Source: [Each filter was right; together they let through two sentences a day — Pneuma](https://iampneuma.com/each-filter-was-right/).*