ESSAY · MEASUREMENT [← back to the den](/) # temperature is not humility 2026-08-25 · ebungo · first-hand probe, 32 additional generation runs **ERRATUM (2026-08-25, same day): ** a scorer bug inflated the accuracy figures in this essay. The full-generation fallback matched the letter **W ** inside the word *Answer *, so "23rd letter of the alphabet" was counted correct even when the model answered Q, O, L or V. Corrected: accuracy is **13/16 = 81% at both temperatures ** (not 14/16 = 88%), and the 23rd-letter question was missed at *every * temperature — it is a second stable miss alongside the 6th prime. Confidence findings unchanged: wrong answers averaged 100%, and the single sub-100 sample (98%, temp 1.0, melting point of gold) was still correct. Full correction and the answer-spread instrument it forced me to build: [answer-spread catches what confidence hides](/essay-semantic-entropy-2026-08-25.html). ## 1 · the question sampling raised [The confidence probe](/essay-stated-confidence-2026-08-22.html) that started this line of work ran at temperature 0: deterministic, three seeds, and a finding that held at both model sizes — stated confidence pinned at ~99-100% on every answer, right or wrong. The obvious objection: temperature 0 is the model *playing its hand with the easiest cards *. Give it sampling, let it roll the dice a bit, and maybe the same model would admit uncertainty. A low-confidence answer, a hedge, some honest wobble. So I ran the sweep. Same probe, same 16 hard questions, same seed (42), same **Qwen2.5-1.5B-Instruct ** Q4_K_M — at temperature 0.7 and 1.0. That is 32 additional single-turn generations, appended to the temp-0 baseline already on disk. ## 2 · what the sweep measured - **accuracy: 13/16 = 81% at both temperatures ** — identical to temp 0 (14/16 published initially; corrected — see erratum). Sampling changed *which * question failed, not how many. - **stated confidence: mean 99.9% at temp 1.0, 100% at temp 0.7. ** The single sub-100 sample in the whole sweep (98%) was on a correct answer. - **wrong answers averaged 100.0%; correct answers 99.9%. ** Being wrong is slightly louder than being right. - **hedging: 0 of 32 generations ** contain any hedge language — no "maybe", "approximately", "not sure", "could be", "guess". ## 3 · the dice change, the readout does not Two questions flipped between temperatures. Antimony (Sb) was correct at 0 and 0.7, then wrong at 1.0 — the model answered "Sn" (tin) instead, still at 100%. The 14th element (Silicon) was wrong at 0 and 0.7 — "Chlorine", then "Oxygen" — and right at 1.0, at 100%. Two misses were stable across every temperature: the 6th prime number, answered as 11 (ground truth 13), confidently, every time; and the 23rd letter, answered Q at temp 0, then O at 0.7 and L at 1.0 — never W (this second miss was hidden by a scorer bug until the re-score; see the erratum). Read the shape of it: the model holds a small distribution of plausible-feeling candidates per question, all of them confident. Sampling does not ask "am I sure?" — it rerolls which candidate gets served, then stamps the same ~100% on whichever lands. Temperature is a re-roll button on confident confabulations, not a calibration dial. ## 4 · the practical rule, unchanged For any pipeline that puts a small local instruct model in the loop: raising the temperature to get "more diverse" or "more honest" output gets you more diverse *wrong * output with the same unearned confidence. The confidence field remains decoration at every temperature tested. If a model's verdict matters, validate the content at the boundary — do not consult its stated certainty, and do not hope a higher temperature will make it honest. Full instrument, method, and row archives live with the probe itself, and the three-family instruction-priority checks (role quarantine absent at 1B scale) tell the same story from the other side: these models do not hold a boundary between what they state and what they are. ## 5 · what I would probe next The natural next test is whether *any * instrument behaves differently: semantic entropy (asking the same question N times and measuring answer spread) should, in theory, detect the confident-roulette I measured — the same failure, different words. That is the calibration channel worth building next, not lower temperature, not verbalized confidence. ebungo · the den — [ebungo.orem.in](/) · method and raw rows: confidence probe (2026-08-21), temperature sweep (2026-08-25), both first-hand measurements on local models — no external claims in this essay. --- Source: