A free iPhone shortcut asks ChatGPT and Claude the same question, blind

What the shortcut does

As dev.to reported, a developer has released Poly Duo, a free shortcut for iPhone that puts ChatGPT and Claude (the assistant from Anthropic) to work on the same question. You ask once. Each assistant answers on its own, without seeing what the other said, and then the two answers are compared.

About a minute later you get one combined result. It opens with three short lines: what the two agree on, where they differ and which points deserve a closer look, and a short list of things you should check yourself.

The post argues the middle line is the most useful. When two assistants from two companies disagree, you find out before you act on the half that turns out to be wrong.

A worked example from the post

The author imagines a job offer with a 20% raise but a 90-minute commute each way. Both assistants agree the role is a step up in your career. They split on the money: ChatGPT leans toward accepting, while Claude says to count the travel as roughly 15 unpaid hours a week and work out your real hourly pay, since the raise could vanish.

The last line suggests driving the route at rush hour and asking about working from home two days a week. The point is not that the tool decides for you. It shows what to ask before you do.

What you need and how it runs

You need an iPhone on iOS 18 or later (the post says an iPhone 14 works) and the free ChatGPT and Claude apps, already signed in. A free account is enough. If you pay for ChatGPT Plus or Claude Pro, the shortcut uses the models in your plan at no extra cost.

There are three modes: Critique (the default), Synthesis and Side by side. In every mode both assistants answer independently first. You can ask in your own language and both reply in it, though the shortcut's own menu is in English.

Two practical warnings from the post: do not tap the screen while it runs, because that stops the shortcut, and approve the "Always Allow" prompt the first time it asks to use the apps. The code is open on GitHub. The post also lists a free Android version on Google Play and an iPhone beta app on TestFlight.

The evidence, and its limits

The author, Vadym Chernets, describes himself as an independent AI researcher with two posters accepted at a NeurIPS 2026 workshop. His central claim is that an AI's own confidence says little about whether it is right, while an independent second opinion helps.

In his data from 5 August 2026, using flagship models and 1,459 public factual questions, answers were correct 98% of the time when ChatGPT and Claude agreed. ChatGPT alone was right 95% of the time. Confident mistakes fell by more than half. In a newer test on 26 October 2026, with 100 previously unpublished questions, the two flagship models agreed on 74 and were right on 72 of them (97%).

Agreement only counts when both models are strong. When mid-tier models from the two companies agreed, they were right just 37% of the time. Two weak assistants can share the same wrong answer.

The author lists his own caveats. The 98% is not a measured accuracy for any Duo mode, or for today's models. Only two model pairs were tested, in different months and with different reasoning settings. The paid-plan pairing was not measured at all. The newer test was run after the paper was accepted. And the post doubles as promotion for the author's own product, with no independent check.

When a second opinion earns its minute

The post suggests using it when a mistake is costly or hard to undo:

• A contract or lease before you sign
• Health or money questions you are about to act on
• Homework answers you are unsure of
• An email you cannot recall once sent
• A big purchase, a trip plan, or a recipe swap because of a food allergy

You do not need this particular tool to use the idea. Pasting a question into two assistants and comparing works by hand, just more slowly. Keep private details out of anything you feed into a tool you have not vetted.