SIAgentNews.com
← All news
DevelopingOpen models

How to give local AI models live web search for free

Users can bypass monthly subscriptions by connecting local LLMs to open-source search engines like SearXNG.

· First reported by xda-developers.com

57tokens per second generation speed

The short version

  • Local models typically have knowledge cutoffs of one to two years, making them less useful for current events.
  • A DIY setup using SearXNG, mcp-searxng, and llama-server allows models to perform live searches without API keys.
  • The configuration enables models to detect when a query requires new data and call search tools automatically.
  • This method removes the need for paid cloud subscriptions like ChatGPT Plus or Gemini Advanced for web-enabled tasks.

Why it matters

This setup allows privacy-conscious users to maintain a local AI workflow without sacrificing the ability to access real-time data. It effectively levels the playing field between free local models and paid cloud services.

The limitation of local AI

One of the primary hurdles for people using local large language models (LLMs) is the lack of native web search. When you download a model's weights from a repository like Hugging Face, the information it contains is frozen at a specific point in time. This creates a significant knowledge gap for users who need up-to-date facts.

Most local models have training cutoffs that are one to two years old. Because the weights themselves cannot cross-reference new information, the model will either admit it does not know the answer or provide a confident but incorrect guess. This lack of recency is a major reason why many people revert to cloud services like Gemini or ChatGPT.

For many workflows, the inability to access live information is a deal-breaker. However, these limitations do not have to be permanent. As reported by xda-developers.com, it is possible to bridge this gap by providing the model with specific tools to browse the internet on its own.

A DIY search architecture

To solve the recency problem without paying for a subscription, users can build a three-part bridge between their local model and the internet. The first component is SearXNG, which is an open-source metasearch engine. It can be hosted in a container to route queries to major search engines like Google, Bing, and DuckDuckGo without requiring an API key or a personal account.

The second component is a small program called mcp-searxng. This acts as a translator that runs on Node.js. It takes the search results from SearXNG and turns them into

Hardware and model performance

Running a capable local model requires a balanced hardware configuration. The setup described uses a mid-range gaming PC featuring an RTX 4070 Ti Super with 16GB of GDDR6X memory, a Ryzen 5 7600X processor, and 32GB of DDR5-6000 memory. This hardware allows for efficient processing of a mixture-of-experts model like Qwen3.6-35B-A3B.

Because the model weights exceed the available video memory, the system offloads a portion of the expert weights to the system RAM. This configuration achieves a generation speed of about 57 tokens per second, which is faster than a human can read, while maintaining a large context window of 64K tokens.

Configuring the software bridge

Setting up the connection involves specific commands to link the components. After installing Node.js, a PowerShell instance is used to set environment variables for the SearXNG URL, the MCP HTTP port, and the stateless mode. These commands then run the mcp-searxng program to prepare the bridge for the model's interaction with the search engine.

The final piece of the puzzle is the llama-server, which hosts the model. To enable the necessary functionality, the user must include specific flags in the launch command, such as --jinja for tool calling and --webui-mcp-proxy. These instructions allow the model to recognize and execute the search tools provided by the bridge.

Activating the web search tool

Once the backend is running, the user can activate the features through the web interface. By navigating to the MCP servers page, the user adds the local address for the bridge as a new server. They then toggle the "Use llama-server proxy" option and enable SearXNG for the chat using a plus button in the message box.

To verify the system works, the user tested the model by asking about the date of CES 2027 without providing any specific instructions to search the web. The model correctly identified that the question required current information, performed the search autonomously, and returned the correct answer. The web UI displays each tool call, allowing the user to see exactly when the model reaches out for data.

What to watch

  • How easily other open-source search engines can be integrated into this bridge.
  • Whether more users adopt these DIY setups to avoid monthly cloud AI fees.

Try it

If you have a local setup, look for 'mcp-searxng' on GitHub to see how to connect your model to a metasearch engine.

Sources and links

First reported by xda-developers.com. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.

Liked it? Say why

Add a line for other readers.

No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.

More AI news