Your agent's tools survive 4-bit. Its token budget and its defaults don't.

Every team shipping agents on quantized or local models is making the same unexamined bet: that 4-bit compression preserves structured output. The tool gets picked, the JSON parses, the arguments arrive intact, and nobody has measured whether any of that actually holds, because the standard quantization benchmarks do not test it.

I measured it. Three hundred frozen tool-calling items, five model variants (BF16 baseline, W4A16 via GPTQ and AWQ, and one NVFP4 checkpoint served through both its native FlashInfer path and the forced Marlin fallback), hierarchical scoring that separates parsing from selection from schema validity from argument values, and kernel receipts on every run. Same discipline, same card, same repo as the study before it: sm120-quant-bench.

The bet mostly holds, which is the good news and the first finding. The bad news is where it breaks: not in the structure everyone worries about, but in a token budget nobody sets thoughtfully and a class of invented arguments that passes every validator you have. And the closing receipt, the one number an agent-safety reviewer should tattoo somewhere: every single wrong call in this study validated perfectly against its JSON schema. Schema validation is a false floor. Now it is a measured one.

Method, briefly

The tool battery is 12 synthetic schemas graded from single-string-arg trivial to nested objects and arrays, including near-miss pairs (confusable tool names with overlapping schemas) and a numeric-precision tool whose arguments are exactly the near-tie digit territory where study one found 4-bit damage pooling. The 300 gold items are template-generated and seed-pinned, not LLM-written: deterministic, auditable, and freshly synthesized, so they cannot have leaked into anyone’s pretraining, which puts this probe on a stronger contamination rung than the famous benchmarks. Categories: tool selection among distractors, argument extraction from natural phrasing, abstention (items where no offered tool applies and the correct behaviour is to not call), and compound nested extraction. Items were frozen and hashed before any variant was evaluated, per the repo’s standing manifest rules.

Scoring is a conditional hierarchy, because “does quantization break tool calling” is really four questions: L1, does a syntactically valid call (or correct non-call) come out; L2, is it the right tool; L3, do the arguments validate against the schema; L4, are the argument values actually correct. Damage at different layers means different things in production, and collapsing them into one score would repeat the exact averaging sin these studies exist to catch.

All five variants ran with thinking mode on via the default template, identical flags throughout: vLLM 0.26.0 with --enable-auto-tool-choice --tool-call-parser hermes, temperature 0, seed 3407, tool_choice: auto, max_tokens 512 for the study (1024 for the ablation), the same pinned Qwen3-8B revision and quantized checkpoints as the earlier studies. One instrument event disclosed up front: the gold set was re-frozen once, before any variant produced a scored result, after the smoke run caught an ambiguous template (“Bill {client}” reads as a first name) and an over-strict free-text matcher. Frozen means frozen only makes sense after the instrument stops being wrong. One embarrassment worth reporting because the method is the product: the tool-calling serves initially wrote their kernel-selection receipts to stdout, which the serve script did not capture. The audit caught it, the receipts were recovered from orchestrator logs into the results JSONs, and the capture path is fixed in the repo. A study about verifying what actually served you should expect to be bitten by its own subject at least once.

The table

Where the variation actually lives, by category (n=75 each; L2 and L3 are 100 everywhere and omitted):

(All rows: results/toolcall-.json; CIs for every cell are in the JSONs. Denominators are conditional, so L4’s n shrinks with each earlier layer, which means L4 comparisons ride on survivor sets: NVFP4 sheds more items at L1, and its chart-topping L4 is partly the easier survivors talking. The end-to-end row exists to close that loophole: correct call with correct values, or correct abstention, over all 300 items, no conditioning. On that row the five variants sit within 2.7 points of each other and no quantized variant is distinguishable from baseline. If you quote one number from this study, quote that row.)

Finding 1: the structure survives, and we tried hard to distrust that

Selection, schema validity, and abstention are perfect across every variant. No format picked a near-miss distractor tool. No format forced a call on an abstention item. No format emitted arguments that failed schema validation.

A row of perfect scores is exactly the shape of a scoring bug, so the audit assumed one: the near-miss distractors were verified present in every engineered offer, sampled L2 passes were checked as genuine choices rather than defaults, and a systematic sweep of every abstention pass across all five variants looked for unparsed call-shapes hiding in prose. Zero. The 100s are real. The structural machinery of tool calling, the part agent frameworks depend on hardest, does not crack at 4-bit on this model.

Finding 2: the sharpest prediction was a null, and the null is the theory

The pre-registered bet, coming out of study one, was that the numeric-precision tool would crack first: 4-bit damage pools where near-tie logits pick between digits, and arguments like amounts and account IDs are wall-to-wall digits. It did not crack. Digit copying survived untouched in every variant.

The resolution makes the theory sharper instead of killing it. Study one’s numeric damage was in computation: arithmetic, unit conversion, values the model must derive. Copying a number from the user’s message into an argument field is not computation, it is transcription, and transcription has no near-tie decisions to lose. So the law this pair of studies now supports: 4-bit breaks what the model computes, not what it repeats. If your agent’s arguments are extracted, you are safer than study one implied. If they are derived, you are exactly as exposed as study one measured.

The place that law shows its teeth is the transformation middle ground, and it belongs to AWQ: arguments requiring derivation from natural phrasing. Two verbatim AWQ specimens, from results/raw/:

"Remind me to submit the expense report at ten to noon on July 22, 2027."
want: {"remind_at": "2027-07-22 11:50", ...}
got:  {"remind_at": "2027-07-22 10:00", ..., "repeat_daily": false}

"Remind me to rotate the backups at five to midnight on September 14, 2027."
want: {"remind_at": "2027-09-14 23:55", ...}
got:  {"remind_at": "2027-09-14 05:00", ..., "repeat_daily": false}

“Ten to noon” became 10:00. “Five to midnight” became 05:00. That is not a parsing error. That is study one’s arithmetic deficit wearing a time-parsing costume, from the same format whose numeric probe fell 8.7 points below baseline. The failure fingerprints are consistent across studies, which is what you would want from real signal.

Finding 3: the W4A16 formats invent arguments, and NVFP4 thinks itself out of the call

The L4 decomposition carries the two findings that matter for production.

First, invented defaults. The baseline model already has a mild habit of adding optional arguments the user never specified, with plausible hallucinated values: 5.4% of calls. Both W4A16 formats roughly double it, to 9.3 and 9.6%. That invention gap, not garbled values, accounts for most of the W4A16 layer-four damage. Read that carefully, because it is the nightmare class: a call that names the right tool, validates against the schema, contains correct required arguments, and quietly commits you to repeat_daily: false or quality: 0.8 that no one asked for. Nothing throws. Nothing logs. The validator smiles. The dominant specimen across both W4A16 formats is schedule_reminder gaining repeat_daily: false (19 of GPTQ’s 204 graded calls, 20 of AWQ’s 209); resize_image gaining keep_aspect: false, quality: 0.8 is the same pattern. Benign values this time. The habit is the finding.

Second, the NVFP4 failure is a different animal entirely. NVFP4 barely invents (near-zero, below baseline), but it deliberates. Two honesty clauses before the claim. The per-layer deltas here have overlapping intervals, so the argument is not any single significant gap: it is replication, the same elevation appearing independently on both kernel paths (30 and 27 budget failures versus 20, 21 and 16 for the other variants), which is what pins the cause on the FP4 weights rather than the serving path. And the baseline itself fails a quarter of extraction items at 512 tokens: the token-budget problem is a thinking-mode-at-512 problem that every variant has and NVFP4 worsens by degree, not an NVFP4 disease. The specimen transcript is the finding in miniature: the model has the right tool and the right arguments inside its thinking, and spends the rest of its budget re-verifying them until the budget ends and no call is ever emitted. Verbatim, finish_reason: length, tool_calls: null:

user: Invoice Tomas Lindgren for $12,003.00, tax applies.

<think> Okay, let's see. The user wants to invoice Tomas Lindgren for
$12,003.00 with tax applied. First, I need to check which function to use...
The task is about creating an invoice, so the create_invoice function is the
right choice. Now, looking at the parameters required...
[~450 tokens of re-verification later]
...the parameters would be client: "Tomas Lindgren", amount: 12003.00,
taxable: true. But the user's message says "invoice Tomas Lindgren for
$12,003.00, tax applies." So the amount is 12,003.00, which is 12

Budget ends mid-digit. At 1024 tokens this item lands, correctly.

The ablation a reviewer would demand, run before publishing: double the budget and the calls land. 19/20 baseline failures complete, 19/21 GPTQ, 15/16 AWQ, 26/30 NVFP4, 25/27 Marlin, most with correct arguments. So the precise sentence is: NVFP4 concludes slower; at matched budgets it fails to call. Two additions make that stronger rather than weaker. The elevated failure rate is a property of the FP4 weights, present on both serving paths. And NVFP4 keeps a small never-concludes tail, 4 items that no doubling rescues, which no other variant matches.

Whether the same weight change that suppresses invention also produces the over-deliberation, one shift toward conservatism showing two faces, is an open question this study cannot answer. The table hints the spectrum has two ends: AWQ concludes fastest (best extraction L1, 88.0), invents most, and derives sloppiest, while NVFP4 concludes slowest, invents least. If that spectrum is real, quantization method choice is quietly a personality dial for your agent. Flagged as speculation, not finding; the follow-up would test whether the poles move together across formats and scales.

What this means if you ship agents on quantized models

Three instructions, each with a receipt above it.

Set token budgets as an evaluated parameter, not a default. The cheapest quantization-induced agent failure in this study was not a wrong answer, it was silence: a correct call that never arrived because deliberation ate the budget. If you serve thinking models quantized, your eval battery needs a matched-budget condition, because that is the condition production runs in.

Add an invented-argument metric to your agent evals. Count optional arguments emitted that the user never specified. It is one counter, it costs nothing, and it is the difference between catching a 2x regression at eval time and discovering an argument nobody stated in an incident review.

And stop treating schema validation as a quality signal. Every wrong call in this study, the fumbled times, the invented defaults, all of it, validated perfectly. L3 is a floor for parseability, not a proxy for correctness. Capability probes have to sit above it, the same way study one’s capability probes had to sit above perplexity. It is the same lesson one level up the stack: the cheap universal metric is blind exactly where the damage lives.

Scope, confessed

One model family (Qwen3-8B, thinking mode on), one card, 300 items, single-turn calls only. Multi-turn tool chains, other model families, and the conservatism question above are follow-ups, not footnotes pretending otherwise. Everything reproduces from the repo, kernel receipts included, and the gold generator is deterministic, so your reproduction gets my exact items. If your numbers disagree, file an issue. That disagreement is data.