
faq-general
What is a token budget and why is it mandatory?
General FAQ · The daemon
Tokens returned for one query
lower is better
naive top-K · no budget
token_budget=4000 · server-enforced
token_budget=2000 · server-enforced
token_budget=500 · server-enforced
Budgets are enforced server-side: metadata first, content only when it earns its tokens. Same corpus, far fewer tokens than top-K stuffing. Reproducible from docs/benchmarks.md.
Every retrieval call to the daemon takes a hard token_budget, and the daemon trims the result to fit. There is no unbudgeted retrieval path.
Why mandatory? Because the failure mode it prevents is the most common one in agent systems: a retrieval tool that returns "everything relevant" and quietly floods the context window, degrading the model's reasoning and inflating cost. With budgets, the ceiling is explicit per call: the caller decides what a lookup is worth and the daemon honours it exactly.
The economics hold up under measurement: on the ScoreCrux benchmark, budgeted retrieval returns far fewer tokens than naive top-K, with answer quality essentially unchanged.
Practical guidance: keep budgets tight by default and raise them deliberately for calls that genuinely need depth. A budget you raised on purpose is a decision; a missing budget is an outage waiting for scale.