Apple lets app makers plug Claude, Gemini and other AI models into its toolkit

One toolkit, more than one brain

Apple has updated its Foundation Models framework, the toolkit that lets developers add large language models (the AI behind chatbots) to apps. As news.lavx.hu reported, the framework is no longer limited to Apple's own AI. It can now connect to server-based models from other companies, with Claude and Gemini named as examples.

Think of a kitchen that used to have one house cook. Now other cooks can work in it, using the same stoves and the same ordering system. The app's menu does not have to change just because a different cook is in charge.

The framework is a native Swift API. Swift is Apple's programming language for its devices, and an API is the set of controls developers use to talk to a service.

How the plug-in system works

Apple says developers can call its own models in two places: directly on the device, or in Private Cloud Compute, Apple's cloud option for bigger jobs. They can also connect outside server models through the same interface. Apple points out that the on-device models can power features without an internet connection.

The key mechanism is a protocol, meaning an agreed set of rules. Apple says any provider can publish a Swift package that follows its Language Model protocol. Apple lists four supported third-party options: Core AI Language Models, MLX Language Models, Claude and Gemini.

If the packages work as described, a developer could switch models without rewriting the parts of the app that handle features like tool calling, where the model uses other functions in the app, and guided generation, which Apple lists among the built-in features.

Images, agents and testing

The framework now accepts prompts that mix images and text. Apple ties this to its Vision framework, so a model can read text from a picture with optical character recognition and scan barcodes, with both running on the device. Developers do not have to build that pipeline themselves.

For agents, meaning AI that carries out multi-step tasks, Apple introduces Dynamic Profiles. A profile can set the model, the temperature (how adventurous the answers are), the reasoning level and the available tools. A developer can change profiles in the middle of one session, and the conversation record stays shared.

• Evaluations framework: tests how prompts and behavior hold up when instructions change or a different model is swapped in.
• fm command line tool: lets people script model workflows without writing app code, and it needs macOS 27.
• Open source: a utilities package with experimental pieces for skills, context management and a Chat Completions client, plus a Python SDK.

Who gets what, and what is missing

Apple includes a cost incentive. Developers in the App Store Small Business Program with fewer than 2 million first-time App Store downloads can use Apple's models on Private Cloud Compute with no cloud API cost. Larger developers are not covered by that offer.

Several gaps remain. The announcement describes features but offers no benchmark results, speed measurements or accuracy comparisons. Pricing, usage limits and data handling for Claude, Gemini and other outside providers are left to those providers and to how each app is built.

Apple also points developers to its Acceptable Use Requirements and to documentation for its AFM 3 Cloud model, which set limits on how the models can be used. The open source utilities carry an experimental label, so their interfaces may change.

What to expect next

Our newsletter noted that dates and the specific apps are not stated, and that remains true. Nothing here tells you when your favorite app will offer a model choice, or whether it will.

The practical test will come from developers. They will need to try the framework on their own workloads to judge answer quality, on-device speed, cost and the ways it fails. Until they report back, this is a promising set of building blocks rather than a proven upgrade.