Three AI roles debated a spec before coding. Ten runs show where it helps and where it stalls

Right code, wrong plan

A developer writing on dev.to describes a frustration many people with coding agents share. The code that came back usually wasn't sloppy. It was well written but aimed at the wrong target, because the agent misread the task or assumed something about the project that wasn't true.

So the author built a tool called Open Council that puts the effort up front. Instead of one agent reading a request and starting to type, several AI roles argue over a written spec (a product requirements document, or PRD) until they all accept the same version.

How the three seats work

Each seat has one job. The Lead PM writes the brief and the spec. The Chief Engineer checks the spec's claims against the real code, and every objection must point to a specific file and line. The Chief Architect settles deadlocks and reviews the final agreed text. It has no tools, so it can only judge what is written, not look through the project.

When all three agree, the human approves the spec with a token. Only then does an engineer agent build it, working in an isolated copy of the project. The tool then runs the project's own test commands, which the author calls gates, and opens a pull request (a proposed code change) only if they pass.

If the process doesn't reach a clean finish, it reports BLOCKED or STALLED rather than claiming success.

What the ten runs looked like

The author ran ten councils on version 0.2.1. Every seat was a Claude Code model: Sonnet for the PM and Architect, Opus for the Engineer. Each run had a cap of four rounds. The targets were local copies of two small projects, a toy text-slug library and Open Council itself. Because the copies had no GitHub connection, no run could open a pull request, so that last step ended BLOCKED by design.

Eight of the ten runs reached sign-off, and seven of those eight passed their tests. The median time to either a sign-off or a stall was 6.5 minutes. Three tasks were repeated, so the results also show some run-to-run variation. The same task finished in 2 rounds in one run and 4 in another, and one task signed off in one run but stalled in another.

The bug nobody asked about

The most useful finding had nothing to do with the assigned tasks. In all six runs on the slug library, the Engineer's first objection was about the project's test setup. The configured test command, on Node 22, treated its argument as a file path instead of a folder. As a result, running the tests executed 0 of the 2 existing tests. It had been broken for a long time without anyone noticing.

In the first run, the Engineer reproduced the problem, tried two working alternatives, and made the repair the first step of the plan. Its logic was that new tests can't be called passing if the test runner never runs them.

The author's point is that a lone agent asked to fix empty-string handling would probably write the fix and some tests, then report success. A seat whose job is to verify claims noticed that the claim couldn't be verified.

Where it went wrong

Two runs stalled in the same pattern. The Engineer had run out of objections, but the Architect then raised one more. In one case it asked for a README line that the task never mentioned. The revision that added it introduced a flawed test that would fail even on correct code. The Engineer objected, the rounds ran out, and nothing reached the human for approval. The author counts a stall as the system working, since nothing bad was built, but sees late Architect concerns as costing more than they give. A rule waiving concerns that could have come earlier helped, though not enough.

A third run, number 7, was blocked by its own plan. The council named a specific test command, but the project's configuration used the still-broken one, and the Architect had ruled the fix out of scope. The configured check failed and nothing was pushed. When the author checked afterwards, the code did pass the plan's own 7 tests. The gap is that nothing flags a plan whose named test command differs from the one the project actually runs. That is now an open issue.

The caveats are large. Every seat came from one model family, so the benefit of different models with different blind spots is untested. And ten runs on two small projects is a smoke test, not a success rate.