datasette-llm adds support for OpenAI-compatible endpoints, so local models can plug in
A new alpha of Simon Willison's Datasette plugin lets you point Datasette at models hosted anywhere, including ones running on your own computer.
The short version
- datasette-llm lets other Datasette plugins configure which language model they use
- Release 0.1a11 adds support for OpenAI-compatible endpoints
- Works with local runners such as LM Studio, Ollama and llama.cpp
- llama.cpp just hit v1.0.0, per the source
- The example runs a Datasette Agent on a local Qwen model via uvx
Why it matters
Until now, using a language model with Datasette plugins meant choosing from the models the plugin supported. Support for the OpenAI-style protocol widens that to almost any model server, including one on your own laptop. For people who want to keep data private, that makes experimenting more practical.
What changed in this release
As simonwillison.net reported, datasette-llm 0.1a11 adds support for OpenAI-compatible endpoints. An endpoint is simply a web address that a program sends requests to, and "OpenAI-compatible" means the server accepts requests in the same format OpenAI's service uses. The source calls this a pseudo-standard, meaning it is widely copied rather than formally agreed.
datasette-llm is a middle layer. It is a plugin that handles model configuration so that other Datasette plugins don't each need their own settings. Datasette itself is a tool for exploring and publishing data.
Why compatibility matters
Many programs that run models on your own machine copy that same request format. The source names LM Studio, Ollama and llama.cpp, which it notes has just reached v1.0.0. Because they all speak the same protocol, one kind of setting can cover all of them.
In practice you tell Datasette three things: a nickname for the model, the model's name, and the address where it is running. Datasette then sends its requests there instead of to a distant company's servers.
The example setup
The release notes show a command-line recipe. It downloads a sample database from datasette.io, then uses uvx, a tool that runs Python programs without a separate install step. The command launches Datasette with the datasette-agent plugin.
The model setting points at a local server on port 1234, running a Qwen model called qwen/qwen3.8-27b through LM Studio. The result, in the source's words, is a local server running a Datasette Agent instance powered by your local model.
Caveats and what's next
The "a11" in the version number marks an alpha, meaning a pre-release. Expect rough edges and changes between versions. The source doesn't say how well any particular local model performs inside Datasette.
If you already use Datasette and run local models, this is a low-cost experiment. Keep it away from important data until the plugin matures.
What to watch
- Whether datasette-llm leaves alpha and the settings format stays stable
- How well smaller local models handle Datasette Agent tasks compared with hosted ones
Try it
If you already run a local model in LM Studio, Ollama or llama.cpp, try the source's uvx recipe with your own model name and address. Use the sample database, not real data.
Sources and links
Liked it? Say why
Add a line for other readers.
No account needed. We never show your email. Comments with links, ads or abuse are held and not shown. Your comment shows after a quick check.
Listen to the AI Wire headlines
Listen to the AI WireListen to our daily AI shows every morning: on your commute, in the shower, wherever you are
Listen to the daily showsFirst reported by simonwillison.net · Read the original. Written by the SI Agent News Desk from the reporting above.Written with AI from the source above. Every number is checked against the source. Spot a mistake? Corrections.
