A guide tests whether a 2-bit 27B model can really handle tool calls

What the guide sets out to test

As marktechpost.com reported, the tutorial centers on Underdog Saluki 27B, a 7.9 GB build of Qwen3.8-27B. Its makers say it keeps tool calling intact at about two bits per weight. They also say it slightly beats the full 54 GB model on their own 120-task benchmark.

A tool call is when a model, instead of just chatting, produces a structured request to run a function, such as looking up a record or calling a calculator. Quantization shrinks a model by storing each of its numerical weights in fewer bits. Two bits is very aggressive, so the open question is what gets lost along the way.

The guide does not take the makers' word for it. It builds a small lab where you can check the claim on your own hardware.

Two files, one base model

Saluki is built on a quantization of the same model made by ISTA-DASLab, called GSQ-RCO. So the guide downloads both files and puts them side by side: Saluki at 7.90 GB and ISTA's IQ2_XS file at 8.42 GB.

The first test is about the files themselves. Both are in GGUF, a single-file format that holds a model's settings and its weights. The tutorial reads the header of each file, which lists every tensor (a block of weights) by name, shape and storage type, and compares them one by one. It then compares the raw bytes to see whether Saluki is just a remix of ISTA's bits or a genuinely changed model.

This matters because a claim of better results means something different depending on the answer. A file that is mostly identical to another would not be a new model. One with changed weights is a different thing to evaluate.

Running and scoring both models

Next, each file is served on a single GPU through llama.cpp, a popular program for running models on your own machine. The guide uses the stock release b11565, so no custom fork is needed. It supports an NVIDIA GPU on Linux, such as a Colab T4, or an Apple-silicon Mac. For each file it reads memory use and speed.

Quality is scored on the Berkeley Function Calling Leaderboard (BFCL), a public set of tool-calling tests. The guide draws 120 tasks with a fixed random seed, 24 from each of five kinds:

• calling the one right function
• picking the right function from several
• knowing when no tool fits
• making several calls to the same function
• using several different functions

The guide samples its own tasks because the task list behind Underdog's benchmark is not published. So its numbers will not be a like-for-like rerun of the makers' result. Scoring uses BFCL's own checker code, with both strict and lenient parsing, two ways of reading the model's output.

Why the setup is so fussy

Much of the tutorial is about removing noise. The llama.cpp release, both model files and fourteen BFCL files are each pinned by size and SHA-256, a fingerprint that changes if even one byte differs. If a file were silently swapped, the run would stop instead of quietly producing different results.

The sampler is fixed too. The guide notes that llama-server's own defaults are temperature 0.8 and top_k 40, which add randomness the author did not choose. Every request instead states settings that make the model pick its most likely token each time. That way, a score difference reflects the model rather than the dice.

This is a habit worth borrowing for any comparison of local models.

What we do not know yet

The portion of the guide we have covers setup, downloads and the header comparison. It does not include the byte-level findings, memory and speed readings, or the final BFCL scores, so we report none of them. The claim that Saluki edges out the full model comes from its makers, and the guide is built to test it.

Treat it as a method first and a verdict second. If your own results on your own tasks differ from the makers' numbers, that is useful information, not a failure of the tutorial.