SI Agent News

• Gemini in Google Sheets on iPhone now matches the Android feature set — spreadsheet AI assistance is finally consistent across both platforms. (REPORTED — Android Authority)
• Cognizant and Google Cloud expanded their partnership, forming a new 'Gemini Frontier Capability Group' to accelerate enterprise agentic AI adoption. (REPORTED — Press Trust of India)
• Diffusion language models are moving out of research: Mercury from Inception runs in paid APIs today; DiffusionGemma, LLaDA 2.x, and NVIDIA Nemotron are at earlier deployment stages. (REPORTED — dev.to)

The Anchor: Why the 70% Number Changes the Architecture Conversation

For two years, the debate in AI development circles has gone something like this: build one very capable general model and throw everything at it, or decompose your tasks and route them through specialized agents. The argument for the first approach has always been simplicity — one model, one context window, one set of failure modes to manage. The argument for the second has been quality — a model that only does code review is better at code review than a model that also writes marketing copy.

Princeton's new paper delivers what that debate has been missing: a number. Approximately 70% on SWE-bench, reached by teams of specialized agents from multiple developers. That benchmark tests real-world software engineering tasks — finding bugs, writing patches, passing existing test suites. It's not a toy problem.

The 70% figure matters for two reasons. First, it establishes a concrete target. If you're building a coding agent system today, you now have a publicly available benchmark to compare your architecture against. Second, and more importantly, the specialized teams reached it by combining agents with narrower, well-defined capabilities — not by scaling one model further.

What this means architecturally: The implication for systems built on Vertex AI Agent Builder, or any platform that lets you orchestrate multiple agents, is that the decomposition step — defining what each sub-agent is responsible for — is now a first-class engineering decision, not an optimization you do later. The paper (REPORTED — full methodology not independently reviewed here) suggests that how you divide responsibilities matters as much as which model you use.

The limitation to hold in mind: SWE-bench measures coding performance specifically. Whether this pattern holds for other task types — customer service, data analysis, content generation — is STILL OPEN. Extrapolating the 70% result beyond software engineering requires more evidence.

What to do with this today: If you're evaluating an agent platform or advising on one, ask vendors whether their benchmark data separates single-agent and multi-agent configurations. The Princeton result gives you grounds to ask that question and expect a real answer. If you're building, the practical first step is mapping your current agent's tasks and identifying the two or three distinct capabilities that might benefit from specialization.

The architecture argument just got a data point. That's what makes this week's paper worth reading.

Deep Dive: How Multi-Agent Specialization Actually Works — and Why 70% Is Hard to Beat

The Princeton result sits on top of a specific technical substrate worth understanding: how task decomposition in multi-agent systems works mechanically, and what makes specialization difficult to replicate with a single model.

The core mechanism: A general-purpose model attending to a large, complex task must simultaneously track the high-level goal, the current subproblem, the code context, and the constraint set. As the task grows in complexity, attention is spread thinner. Specialized agents, by contrast, are given a narrower scope — a specific sub-task with a well-defined input and expected output format. The model's attention can be fully allocated to that sub-problem.

Why this matters for context windows: Even with large context windows, attending to more doesn't mean reasoning better about all of it equally. Specialization is partly a way to force the model into a smaller, better-defined reasoning space — which is why a smaller, specialized model can outperform a larger general one on the task it's been focused on.

The routing problem: Multi-agent systems require something to decide which agent handles which part of a task. This routing layer — sometimes a dedicated orchestrator model, sometimes a rule-based router — is itself a source of failure. The Princeton paper's 70% result includes whatever routing errors occurred in those pipelines. A well-designed routing layer is as important as the specialized agents themselves.

Agent output verification: This is where tools like Reticle (the self-verification layer from today's Hacker News post) connect directly to the multi-agent architecture discussion. In a pipeline where one agent writes code and another reviews it, the reviewing agent is itself a specialized sub-agent. Reticle extends this idea to self-verification — asking the same coding agent to check its output against the original requirements before the result is returned. Whether this improves or worsens performance depends on the model's ability to genuinely evaluate its own work versus rationalize it. (STILL OPEN — no benchmark comparison in today's source.)

The SWE-bench design: SWE-bench tests agents on real GitHub issues from open-source repositories. Each task requires the agent to read an issue, navigate an existing codebase, write a patch, and pass the existing test suite — without access to hints about which files to change. This tests code navigation, fault localization, patch generation, and test compatibility simultaneously. A 70% pass rate means the system found the right files, wrote a working fix, and didn't break anything else, roughly seven times out of ten.

What's genuinely novel: It implies the benefit is robust enough to emerge from agents not designed to work together. That's a higher bar than a purpose-built multi-agent system from a single vendor.

What remains open: The paper (REPORTED) hasn't been reviewed here for full methodology. The optimal number of specialized agents, the effect of the routing architecture, and generalization beyond software engineering tasks are all STILL OPEN questions.

One Technique: Designing a Two-Agent Review Pipeline in Vertex AI

Who this is for: Developers or technical leads building coding agent pipelines on Vertex AI or any Google Cloud agent platform.

The problem it solves: Single-agent coding pipelines produce output and stop. There's no internal check that the output solves the original problem, doesn't introduce regressions, or follows the constraints stated in the task. The Princeton and Reticle findings point toward the same fix: add a reviewing agent.

What you need: Access to Vertex AI Agent Builder or the Gemini API; a coding task with a clear acceptance criterion (a test suite, a specification, or a checklist of requirements).

The pattern — three stages:

1. Decompose the task: Before any agent runs, split the work into at least two well-defined sub-tasks: (a) generate the solution, and (b) verify the solution meets the stated criteria. Write the acceptance criteria explicitly in the verification prompt — don't rely on the reviewing agent inferring them.
2. Run the generating agent: Pass the task and the codebase context to a coding-focused model. Keep the prompt narrow — one task, one output format, no ambiguity about what done looks like.
3. Run the reviewing agent: Pass the original task description, the acceptance criteria, and the generated output to a second agent call. Prompt it to identify specific failures, not just give a pass/fail verdict. Ask: 'Does this output satisfy each criterion? List any that are not met and why.'

Expected output: A structured review that identifies gaps before the output reaches production — not a second opinion that rubber-stamps the first pass.

Known limitation: If the same model generates and reviews, it may rationalize its own output rather than genuinely evaluate it. Use a different model or a significantly different system prompt for the reviewer to reduce this risk.

How to verify it worked: Compare the defect rate on tasks run with a single agent versus the two-stage pipeline on a consistent test set. An illustrative benchmark: if your single-agent pipeline is at 60% on your test cases, the reviewing stage should surface the majority of the failing 40% before you ship.

This is an illustrative workflow — not a tested result from this edition.

Save Time or Money: Cut Wasted Generation Cycles in Gemini Notebook

The Gemini Notebook composition review update — outlined in today's stories — has a direct cost implication for anyone using the tool for slides or video generation.

The opportunity: Before today's update, generating a slide deck or video from Gemini Notebook was one-shot: you'd prompt, wait for generation, review the output, and if the structure was wrong, regenerate. Each regeneration costs time and, depending on your plan, API quota.

With the review step: You review and correct the outline before committing to a full generation run. The time savings depend on how often you previously regenerated — but for anyone doing this regularly, reducing even one unnecessary generation per session is meaningful.

A rough estimate: If a full Notebook generation run takes 3-5 minutes (estimate — not sourced from today's stories) and you've been regenerating once per session on average, catching the structural error at the outline stage saves roughly that time per use. On a weekly cadence across a team, that compounds quickly.

Vendor-neutral note: Similar review-before-generate patterns exist in other tools, where users can edit an outline before generation. If Gemini Notebook's approach doesn't fit your workflow, those are the alternatives to compare.

What you give up: The review step adds a manual interaction — it's no longer fully automated. If you need fully hands-off generation pipelines, the review step requires additional tooling to handle programmatically.

Pricing for Gemini Notebook not stated in today's source — check Google's current pricing page.

Prompt of the Day: Two-Stage Code Review

Objective: Review a piece of generated code against explicit acceptance criteria, identifying specific failures rather than producing a generic pass/fail verdict.

Prompt:

You are a code reviewer. Your only job is to check whether the code below satisfies each of the stated acceptance criteria.

Acceptance criteria:
[PASTE YOUR CRITERIA HERE — e.g., 'The function must handle empty input without throwing', 'It must return results in under 200ms on a 1,000-item list', 'It must not use global state']

Code to review:
[PASTE THE GENERATED CODE HERE]

For each criterion:
1. State whether it is MET, NOT MET, or CANNOT DETERMINE from the code alone.
2. If NOT MET or CANNOT DETERMINE, explain specifically what is missing or unclear.

Do not summarize the code. Do not suggest improvements beyond the stated criteria. Only evaluate what was asked.

Expected result: A structured, criterion-by-criterion verdict that surfaces specific failures rather than general impressions.

Customize it by: Adding a 'confidence' field to each verdict (high/medium/low) to distinguish cases where the reviewer is certain versus inferring behavior from the code structure.

Quality check tip: If the reviewer returns only 'MET' verdicts with no elaboration, the criteria may be too vague — tighten them to be testable assertions ('returns X given Y') rather than general goals ('handles edge cases well').

The finding: Princeton researchers tested multi-agent systems — teams of AI agents where each is specialized for a narrower task — against single general-purpose agents on SWE-bench, a benchmark using real GitHub issues from open-source Python codebases. Specialized teams reached approximately 70% accuracy. Single agents fell short. (REPORTED — full methodology not independently reviewed.)

What the result actually establishes: Specialization helps measurably on complex, multi-step coding tasks. It does not establish that specialization helps on all task types — this is a coding-specific result.

Key limitation: The paper tests a specific benchmark. Generalization to other task domains is STILL OPEN.

One decision you can make this week: If you're building or evaluating an agent system and it handles more than one distinct type of subtask, the evidence now supports testing a decomposed, specialized architecture before defaulting to one capable model. Start with two agents — a generator and a reviewer — before adding more specialization layers.

Agentic AI is today's heaviest lane by a significant margin. The Princeton multi-agent result lands in the middle of a field where framework investment, benchmark competition, and architectural debate are all accelerating simultaneously. The story isn't just that agents are being built — it's that the community is now producing the evidence base for how to build them well.

Google's product moves today — Notebook's review gate, Sheets parity on iOS, the Veo-to-Omni migration path — read collectively as a maturation push: reducing friction, adding editorial control, and closing platform gaps. These aren't capability leaps; they're the reliability and usability work that determines whether tools become workflows.

Diffusion language models remain early-stage in production but are moving. Mercury is the only one confirmed in paid APIs today. The gap between research prominence and actual deployment is large — worth tracking, but not worth building production dependencies on yet.

A multi-agent system walks into a bar. The reviewing agent refuses to let the generating agent order — says the acceptance criteria for 'a good time' were never specified.

That's SI Agent News — Gemini Edition for Tuesday, October 20th. The Princeton 70% benchmark, Google Notebook's editorial review gate, and the Veo 3.1 to Omni Flash migration path — all in one place, so you can act on it before the week gets busy.

If this made you think differently about how you're building with AI today, share it with one person who'd get something useful out of it.

Made by AI.

SI Agent News — a podcast newsletter.