SI Agent News
DevelopingOpenAIOpen models

datasette-llm adds support for OpenAI-compatible endpoints, so local models can plug in

A new alpha of Simon Willison's Datasette plugin lets you point Datasette at models hosted anywhere, including ones running on your own computer.

The short version

Why it matters

Until now, using a language model with Datasette plugins meant choosing from the models the plugin supported. Support for the OpenAI-style protocol widens that to almost any model server, including one on your own laptop. For people who want to keep data private, that makes experimenting more practical.

What changed in this release

As simonwillison.net reported, datasette-llm 0.1a11 adds support for OpenAI-compatible endpoints. An endpoint is simply a web address that a program sends requests to, and "OpenAI-compatible" means the server accepts requests in the same format OpenAI's service uses. The source calls this a pseudo-standard, meaning it is widely copied rather than formally agreed.

datasette-llm is a middle layer. It is a plugin that handles model configuration so that other Datasette plugins don't each need their own settings. Datasette itself is a tool for exploring and publishing data.

Why compatibility matters

Many programs that run models on your own machine copy that same request format. The source names LM Studio, Ollama and llama.cpp, which it notes has just reached v1.0.0. Because they all speak the same protocol, one kind of setting can cover all of them.

In practice you tell Datasette three things: a nickname for the model, the model's name, and the address where it is running. Datasette then sends its requests there instead of to a distant company's servers.

The example setup

The release notes show a command-line recipe. It downloads a sample database from datasette.io, then uses uvx, a tool that runs Python programs without a separate install step. The command launches Datasette with the datasette-agent plugin.

The model setting points at a local server on port 1234, running a Qwen model called qwen/qwen3.8-27b through LM Studio. The result, in the source's words, is a local server running a Datasette Agent instance powered by your local model.

Caveats and what's next

The "a11" in the version number marks an alpha, meaning a pre-release. Expect rough edges and changes between versions. The source doesn't say how well any particular local model performs inside Datasette.

If you already use Datasette and run local models, this is a low-cost experiment. Keep it away from important data until the plugin matures.

What to watch

Try it

If you already run a local model in LM Studio, Ollama or llama.cpp, try the source's uvx recipe with your own model name and address. Use the sample database, not real data.

Sources and links

Liked it? Say why

Add a line for other readers.

No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.

Listen to the AI Wire headlines

Listen to the AI Wire

Listen to our daily AI shows every morning: on your commute, in the shower, wherever you are

Listen to the daily shows

First reported by simonwillison.net · Read the original. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.

More AI news