The problem
How it works
llama.cpp’s function-calling guide explains how llama-server handles tools. Started with --jinja and a model whose chat template supports tools, it parses tool calls in the native formats of model families such as Llama 3.x, Qwen 2.5, Hermes, Mistral Nemo, Functionary and GPT-OSS, and falls back to a generic format for templates it does not recognise. The guide lists example commands that download a GGUF model from Hugging Face and start the server, shows how to override a broken or missing template with --chat-template-file, and gives a curl request with a tools array to check the response. Parallel tool calls can be switched on per request.
Results and limitations
This is project documentation with examples but no reported agent project, so it is marked planned.
Before you try it
Check the /props endpoint for a tool-capable chat template before blaming the agent; the generic fallback works but uses more tokens. The guide warns that heavy KV-cache quantisation, such as -ctk q4_0, can badly hurt tool calling, and notes that DeepSeek R1 support is still rough. Small local models call tools less reliably than large hosted ones, so test on your real tasks. Results depend on your setup and were not independently reproduced by Top 200 Repos.
Tools and repositories
- Tools
- llama.cpp (llama-server) · A GGUF model from Hugging Face
- Prerequisites
- A machine that can run the chosen model, ideally with a GPU or Apple silicon.
Try it yourself: Check whether a local model is ready for agent work
Fill the fields to adapt the prompt. Empty fields stay as visible [placeholders].
I am serving [Model] with llama-server at [Server URL]. Write a short test script in [Language] that sends three chat requests with a tools array, using these tools: [Tools to test]. One request should need no tool, one should need a single tool call, and one should need two calls in sequence. Print whether each response used the tools correctly and with valid JSON arguments.Run it again after changing the model, quantisation or template, and keep the results.
Sources
- Function Calling (opens in a new tab)
Project documentation · github.com · Undated · Source checked 12 Nov 2026, 18:03 UTC
What this source supports
- OpenAI-style function calling is supported in llama-server when started with the --jinja flag
- Native tool call formats include Llama 3.x, Functionary, Hermes 2/3, Qwen 2.5, Mistral Nemo, Firefunction, Command R7B, DeepSeek R1 and GPT-OSS, with a generic fallback
- Parallel tool calling is disabled by default and enabled per request with parallel_tool_calls
- Template overrides with --chat-template-file, and checking chat_template in /props
- Extreme KV quantisations such as -ctk q4_0 can substantially degrade tool calling