SIAgentNews.com
← All news

Apple is hiring engineers to build a multi-app AI agent

A new job listing reveals Apple is developing an agentic assistant capable of performing complex, multi-step tasks across various apps and services.

· First reported by The Verge

Papercut shadow box: picture for the story 'Apple is hiring engineers to build a multi-app AI agent'
AI-made papercut shadow box for this story. No real people or products are shown.

The short version

  • The new AI agent will be able to perform multi-step tasks across browsers, command lines, code, and APIs.
  • Apple is hiring for a lead role to coordinate with Siri, UI Frameworks, and Apple Foundation Models teams.
  • The assistant aims to mimic how humans move fluidly between different tools like document editors and terminals.
  • Apple is already rolling out an AI-upgraded Siri that can answer questions about what is on a user's screen.

Why it matters

This signals a shift from simple voice commands to 'agentic' AI that can actually execute workflows for you. It means your phone or Mac could soon handle complex chores, like organizing files or editing photos, without you clicking every button.

The shift toward agentic AI

Apple is moving beyond simple voice commands to develop a more sophisticated AI agent. According to a job listing spotted by Bloomberg, the company is looking for an engineer to lead the creation of an assistant that can perform a wide range of tasks. Unlike current systems that might just answer a question, this agent is designed to take action.

The goal is to create a system that can browse the web, write code, and call various services autonomously. This represents a move toward 'agentic' AI, which refers to systems capable of planning and executing multi-step goals. These agents do not just provide information; they interact with the software environment to complete a user's request from start to finish.

By building this technology, Apple intends to compete with existing AI products from major tech rivals. The company is looking to match capabilities found in offerings from Meta, OpenAI, Anthropic, and Google. These competitors are already beginning to offer tools that can take control of computers or perform specific tasks through dedicated apps.

How the agent will function

The job listing describes an AI that can move fluidly between different tools, much like a human user does. It is intended to work across browsers, document editors, terminals, and various APIs. The system will be designed to handle complex workflows that require switching between multiple different applications to reach a final result.

To achieve this, the new AI agent will be able to coordinate specialized sub-agents along the way. This means a large task could be broken down into smaller pieces, with different parts of the AI handling specific actions like searching for data or formatting a file. This modular approach allows the system to manage complicated instructions more effectively.

The engineering team will work closely with several internal divisions to make this a reality. This includes the teams responsible for Siri, the company's UI Frameworks, and its Apple Foundation Models. By integrating these departments, Apple aims to ensure the agent feels native to the user interface while remaining powered by its core machine learning research.

Expanding Siri's capabilities

Apple has already begun the process of upgrading Siri with new AI capabilities. In September, the company started rolling out an improved version that can answer questions about what is currently visible on a user's screen. This is a foundational step toward the more autonomous behavior described in the new job listing.

Reports suggest that the company may soon add specific actions to this upgraded Siri. These could include the ability to rotate and crop images within the Photos app or organize tabs within the Safari browser. These features would allow the assistant to perform routine maintenance and editing tasks that currently require manual user input.

The company is also providing developers with new ways to integrate these AI features. With the upcoming iOS 27.3 update, developers will be able to connect Siri AI into all apps. This is currently being offered as part of a Developer Preview, which will eventually allow Siri to perform tasks across a much wider variety of third-party software.

Competing in the AI landscape

Apple is entering a crowded market where several companies are racing to build functional AI assistants. For example, Muse from Meta and Dots from OpenAI are already being used to interact with computer systems. Google has also recently started allowing its Gemini AI to perform similar actions on computers.

The job listing highlights that people do their most valuable work by moving between different tools. Apple wants its AI agent to replicate this behavior by navigating between a browser, a terminal, and an editor. By focusing on this fluidity, Apple aims to make its AI feel like a natural extension of the user's existing workflow.

The move suggests that Apple views the future of personal computing as one where the AI acts as a coordinator. Instead of the user opening five different apps to complete a project, the agent will be the primary interface. This strategy aims to keep users within the Apple ecosystem by making the transition between different services seamless and automated.

What to expect next

The next phase for Apple involves refining the underlying models that will power these agents. Because the assistant needs to work across many different apps, the company must ensure it can understand various user interfaces and command structures. This requires a high level of precision to ensure the AI doesn't make mistakes when interacting with sensitive data or complex software.

Users can expect to see these capabilities trickle out as the company tests different use cases. The current focus on Siri's ability to understand the screen is a precursor to the agent's ability to take over the screen. As the engineering team builds out the framework, the scope of what Siri can do will likely expand from simple queries to complex, multi-step actions.

The integration of these agents will likely happen gradually across the iPhone and Mac platforms. By giving developers the tools to connect Siri to all apps, Apple is creating a path for the agent to become a universal assistant. This could eventually change how people interact with their devices, moving from manual navigation to high-level goal setting.

The goal of the project

The job listing explicitly states that the goal is to build agents that can plan and carry out multi-step tasks. This is a significant technical hurdle because it requires the AI to understand the sequence of events needed to reach a goal. For instance, if a user asks to 'prepare a report,' the agent must know how to find the data, write the text, and format the final document.

To do this, the agent must be able to interact with APIs, which are sets of rules that allow different software programs to talk to each other. By mastering these connections, the AI can bridge the gap between different apps. This allows it to pull information from a database and put it into a spreadsheet or a web browser without human intervention.

Ultimately, Apple wants to create a tool that understands the context of a user's work. By coordinating specialized sub-agents, the system can handle different parts of a project simultaneously. This approach aims to provide a more powerful and capable assistant that can handle the heavy lifting of digital organization and content creation.

Impact on user experience

For the average user, these changes mean that the smartphone or computer could become much more proactive. Instead of searching for a specific setting or app, a user might simply tell the AI what they want to achieve. The AI would then navigate the menus and buttons on behalf of the user to get the job done.

This shift could significantly reduce the time spent on repetitive digital tasks. Whether it is organizing a cluttered photo library or managing multiple browser tabs, the AI agent is designed to handle the 'busy work.' This allows users to focus on the creative or analytical parts of their tasks while the agent manages the logistics of moving data between tools.

However, this also means that the AI will need to have a deep understanding of the user's habits and preferences. As the agent gains the ability to work across apps and call services, it will need to be highly reliable. Apple's focus on UI Frameworks suggests they are prioritizing a smooth and consistent experience as these new agentic capabilities are rolled out to the public.

Sources and links

First reported by The Verge. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.

Liked it? Say why

Add a line for other readers.

No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.

More AI news