llama.cpp and Unsloth

Serve a local GGUF model with llama.cpp and connect it to Cua Driver.

llama.cpp gives you direct control over local model serving. Unsloth publishes quantized GGUF variants that can reduce the memory required to run supported models.

The recorded Muse Glimmer demos use:

Unsloth UD-Q4_K_XL GGUF → llama.cpp → Claude Code → filtered Cua Driver MCP → macOS Lume guest

Follow Run a local model with Cua Driver for the tested llama-server flags and harness configuration.

The local-model guide also covers the main optimization: reduce the MCP tool catalog before the model sees it. Tool schemas, screenshots, and accessibility trees all consume context. Keep state reads bounded and expose only the actions required by the task.

Upstream: llama.cpp, its server documentation, function-calling guide, and Unsloth's Muse Glimmer guide.