SI Agent News
DevelopingOpen modelsChips

Thinking Machines releases Inkling 2 Mini for local offline use

The new 26B parameter model uses a looped architecture to perform extra reasoning on consumer hardware like MacBooks.

20GBModel memory footprint

The short version

Why it matters

This model allows users to run high-performing AI agents locally on personal computers rather than relying on expensive cloud infrastructure. It demonstrates how clever architecture can provide 'smarter' reasoning without requiring massive amounts of hardware memory.

A smaller, smarter model

Thinking Machines Lab recently introduced Inkling 2 Mini, a new open-weight AI model designed to balance high performance with hardware accessibility. While the company also offers a much larger 594-billion-parameter model, the Mini version is built to be usable on consumer-grade devices. It aims to handle complex tasks like coding, reasoning, and agentic workloads without needing a massive server farm.

As Wccftech reported, the model achieved an Artificial Analysis Intelligence Index score of 20.1. This suggests it can compete with models that are significantly larger in size. By focusing on efficiency, the developers have created a tool that provides a high level of intelligence while remaining small enough for individual users to host privately.

The model is now available under the Apache 2.0 license, which allows for broad use and modification. Users can find the weights on Hugging Face or deploy it locally using Ollama. It also supports both text and image inputs, making it versatile for different types of creative and technical projects.

The looped transformer logic

The core innovation of Inkling 2 Mini lies in its 24-layer Mixture of Experts (MoE) transformer architecture. In a standard model, every token passes through each layer once. However, Thinking Machines designed this model to execute its middle 12 layers three times while the first and last six layers operate normally. This means the model performs 48 layer applications per token despite only storing the weights for 24 layers.

This specific design allows the model to spend more time reasoning internally before it produces its next piece of text. By looping the internal layers, the model effectively increases its active parameter count to 5.4 billion per token. This gives it the computational power to think more deeply without the memory overhead that usually comes with adding more parameters to a model.

The training process also utilized specific hardware-accelerated capabilities from NVIDIA's GB300 Grace Blackwell Ultra platform. Most weights were trained using 4-bit NVFP4 precision, while the final four layers used 8-bit MXFP8 precision. Because the weights are deployed in the same precision they were trained in, the model avoids the performance losses that often happen during post-training quantization.

Performance in coding and reasoning

The developers claim that this looping approach provides significant improvements for agentic coding tasks. In testing on DeepSWE v1.1, Inkling 2 Mini reached a score of 48.2%, which outperformed the Qwen3.8-27B model's score of 42.2%. It also secured a 77.9% score on the SWE-bench Pro v2 Hard test, surpassing the 73.8% score of its competitor. These results highlight the model's ability to handle complex software engineering problems.

Beyond coding, the model showed strong results on other benchmarks like the Vals Index v2.2, where it scored 46.5%. It also achieved a 79.2% score on SWE-bench Verified. While these are company-reported results and may vary based on specific testing configurations, they indicate that the model is highly capable for its size class. It is particularly well-suited for users who need a reliable assistant for technical work.

Despite these strengths, the model does have some limitations that users should keep in mind. The researchers noted that it can struggle with complex multi-turn conversations. It also faces challenges when given instructions that involve a very large number of specific constraints. These trade-offs are common when optimizing a model to run efficiently on local hardware.

Local execution and privacy

One of the most practical features of Inkling 2 Mini is its 20GB memory footprint. This size allows it to run entirely offline on Apple silicon Macs that have at least 32GB of unified memory. To demonstrate this, the company showed the model running on an M5 Pro MacBook Pro with 48GB of memory during a flight. In that scenario, the model was able to generate a playable flight simulator in just a few minutes.

For users who do not use Apple hardware, the model is also officially supported on NVIDIA's DGX Spark desktop AI computer. Other GPU configurations are also possible, though 32GB or more of VRAM is recommended for a smooth experience. Running the model locally means that data does not have to be sent to the cloud, which is a major benefit for privacy and security.

This accessibility makes the model an attractive option for developers who want to build AI agents without relying on external APIs. By hosting the model on their own machine, they can maintain complete control over their data and costs. It represents a shift toward making high-quality, reasoning-heavy AI available to everyone with a capable personal computer.

Safety and transparency concerns

The use of looped transformers introduces a complex question regarding how we monitor AI behavior. Because these models can perform extra internal computation without writing down every single step, some researchers worry about transparency.

Sources and links

Liked it? Say why

Add a line for other readers.

No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.

Listen to the AI Wire headlines

Listen to the AI Wire

Listen to our daily AI shows every morning: on your commute, in the shower, wherever you are

Listen to the daily shows

First reported by Wccftech · Read the original. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.

More AI news