Kosa8 Labs releases security benchmark for regulated AI agents

Benchmarking the AI frontier

Kosa8 Labs recently released a 46-page technical report titled "Sealed Machines for Autonomous Agents." The document evaluates 11 different AI agent infrastructure platforms to see how they hold up against the needs of highly regulated industries. These sectors include banks, hospitals, government agencies, and defense suppliers.

The study specifically looks at platforms such as Docker, E2B, Modal, Cloudflare, AWS, and Google. Kosa8 Labs claims that its own platform is the only one to meet all 13 of the security requirements it established for these environments. However, the company conducted the research itself, which is a notable limitation for the report's objectivity.

The findings suggest a significant gap in the current market. According to the report's own scoring system, none of the competing systems managed to satisfy more than four of the 13 criteria for regulated environments. This suggests that while many tools exist for running AI agents, few are currently built to handle the strict security demands of the world's most sensitive organizations.

Assumptions of a compromised system

The framework used by Kosa8 Labs starts with a deliberately pessimistic assumption. It assumes that an attacker has already gained root access inside the agent's execution environment. From this baseline, the report measures what a compromised agent can still reach and whether an organization can reconstruct the agent's actions.

The report focuses on specific controls that the company argues most current platforms ignore. These include requiring a second person to authorize dangerous tasks while the agent is running and network protections that block traffic to unauthorized locations. It also looks for audit logs that are resistant to tampering and can be checked by a third party offline.

To test these theories, the company ran eight different attack scenarios. These included a poisoned GitHub issue used for data theft, a malicious npm package designed to find SSH keys, and an MCP tool "rug pull." Kosa8 Labs reported that its platform contained or fully handled all eight of these specific attack scenarios.

Performance and technical hurdles

While the security features are a priority, the report also tracks the performance of the running agent machines. Kosa8 Labs noted that the system reached a middle checkpoint speed of 638 milliseconds and a rewind speed of 520 ms. These numbers represent the time it takes to save the state of a running agent, including its memory and active processes.

The company is transparent about some of the trade-offs involved in high security. For instance, creating a sandbox takes about four seconds, which is slower than the warm-pool starts offered by hosted cloud services. Additionally, the report notes that one specific restore path currently suffers from an unexplained slow mode.

Despite these performance hurdles, the company argues that the current industry response to AI risks is insufficient. Founder and CEO Dhruvit Talati noted that while the industry uses better prompts and filters, regulated organizations need better control. He stated, "What regulated organisations are missing is control and evidence on their own hardware."

Regulatory and market context

The timing of this report is significant as financial regulators begin to define expectations for agentic AI. The Bank of England and the FCA are both emphasizing explainability and auditability in their guidance. These requirements align directly with the audit logs and rollback features that Kosa8 Labs is highlighting as key differentiators.

In the European Union, the DORA framework places binding requirements on ICT risk management for financial institutions. These rules can extend to third-party AI tools, making the security of the underlying infrastructure a legal necessity. This creates a high barrier to entry for any AI tool that cannot provide a verifiable audit trail.

The competitive landscape is currently split between established cloud providers and specialist startups. While giants like AWS and Google are extending their sandbox services, startups are building purpose-built environments. Kosa8 Labs positions itself for firms that want to avoid shared public clouds by offering hardware-level isolation and verifiable trails on private infrastructure.

The road to credibility

Kosa8 Labs is preparing for a public launch on Product Hunt on 20 October 2026. While the technical report establishes a baseline for their claims, the company still faces several hurdles to gain full trust from enterprise clients. Specifically, they do not yet have a SOC 2 certification or a third-party security audit.

For many regulated financial institutions, these two certifications are standard requirements before any procurement sign-off can occur. Without them, the company's self-conducted research remains the primary evidence of their security claims. The market will likely look for independent verification of the benchmark methodology as the next major milestone.

As the industry moves toward more autonomous AI, the demand for "sealed machines" will likely increase. Organizations will need to balance the speed of AI execution with the need for hard boundaries. Kosa8 Labs is betting that the next phase of AI adoption will be defined by who can provide the most rigorous evidence of safety and control.